In this talk we explore the internals of a high-performance key/value store written in Go. The audience will learn the basic design used to store and retrieve data, as well the techniques used to achieve high performance.
General Purpose Key-Value Stores Isolated Read Snapshots Iterator Started Key Value cat grinning caterpillar smoking hare march hatter mad rabbit late tea party Concurrent Mutation
General Purpose Key-Value Stores Isolated Read Snapshots Iterator Started Key Value cat grinning caterpillar smoking hare march hatter mad rabbit late tea party Concurrent Mutation (not seen)
Simplicity Damian Gryski - Slices https://go-talks.appspot.com/github.com/dgryski/talks/dotgo-2016/slices.slide Performance through cache friendliness Rule 3. Fancy algorithms are slow when n is small, and n is usually small. Fancy algorithms have big constants. Until you know that n is frequently going to be big, don't get fancy. Rule 4. Fancy algorithms are buggier than simple ones, and they're much harder to implement. Use simple algorithms as well as simple data structures. "Notes on C Programming" (Rob Pike, 1989)
Simplicity Dave Cheney - Simplicity and Collaboration https://dave.cheney.net/2015/03/08/simplicity-and-collaboration "Simplicity cannot be added later"
Special Purpose Key-Value Store Index Write Throughput Read after Write Latency • Persistence Decoupled from Read and Write • Willing to use All System RAM* * Bounded by some Quota
Set( []byte("name"), []byte("marty") ) data []byte name marty 0 20 24 29 Build 2 uint64s of metadata 20 Data Offset Operation 4 bits 24 bits 28 bits s 4 5 64 bits Key Len Val Len
Set( []byte("name"), []byte("marty") ) data []byte name marty 0 20 24 29 Build 2 uint64s of metadata 20 Data Offset Operation 4 bits 24 bits 28 bits s 4 5 – 8 bits 64 bits Key Len Val Len
High Performance - Minimize GC Impact • Slices of uint64 and byte • Integer offsets into slices, not pointers type segment struct { data []byte meta []uint64 } data []byte name marty 20 s 4 5 – meta []uint64
High Performance - Unsafe • Faster serialization, but loss of portability • Deeply unsatisfying https://commandcenter.blogspot.com/2012/04/byte-order-fallacy.html Rob Pike – The byte order fallacy "Whenever I see code that asks what the native byte order is, it's almost certain the code is either wrong or misguided."
mschoch
charity
senkin13
oracle4engineer
shimacos
linedevth
unclejoe
kmwebnet
rigolon
karawoo
hamadakoji
ytaisei
sonatard
geeforr
addyosmani
tanoku
lauravandoore
trishagee
keavy
lara
myddelton
bluesmoon
pedronauck