Talk given at the October 2015 Denver Go meetup.
Why You Shouldn’tWrite a DatabaseBen Johnson
View Slide
Let’s define“database”
low-levelLevelDB, BerkeleyDB, LMDB
Direct interface to OS Files, pages, & blocksResponsible for data integrityTypically key/value
high-levelSQLite, MySQL, PostgreSQL, Mongo, InfluxDB, etc
Builds on low-level stores Rows, tables, indexesInterfaces with end userRelational, document, time series
Me
SkyDBReportifyDBBoltDBInfluxDB
There are n+1 reasonsnot to write a database
#1. High barrier to entry
How to write a database
legit
sorta,but not quite
seriously?
suggestsusing XML
nope
Available resources?
Research papers!
Research papers!Narrowly focused (indexing, storage, locks)
Research papers!High level (little or no code)
Research papers!Assumes a Ph.D
Read source code!
Read source code!MySQL >1M SLOC
Read source code!Even small databases are 10KLOC+
There is noWriting Databases 101
#2. Debugging Sucks
Bugs are catastrophicCause corruption, loss of data integrity
Bugs are catastrophicUsers with data loss are very unhappy
Find a good hex editor
Find a good hex editor(I use Hex Fiend)
This is not what adatabase looks like
THIS is what adatabase looks like
Debugging w/o dataUsers usually can’t release their data
#3. Tradeoffs
Users don’tunderstand tradeoffs
Read OptimizedvsWrite Optimized
CPU Bound vs IO Bound
https://gist.github.com/jboner/2841832
Features are liabilitiesIt’s not if you have bugs, it’s how many
#4. Limited Community
Very few people whohave written aproduction database
TONS of people whowill tell you whyyour database sucks
Databases are hard
The End
The End(Just kidding!)
Not scared off yet?
Only 2 reasons towrite a database:
#1. To learn
Data IntegrityIndexingIsolation levelsRecoveryPrefetchingParser & LexersReplicationSnapshottingTransactionsMaterialized ViewReferential IntegrityQuery CompilationQuery PlanningQuery OptimizationSerializabilityWrite Ahead LogMemory ProfilingRedo/Undo LogSnapshot IsolationRecoveryTwo-Phase CommitQuorums
#2. Gain efficiency(For a specific use case)
The more generic your database,the fewer assumptions you can make
Go write a database!
The End@benbjohnson
Questions?@benbjohnson