Link
Embed
Share
Beginning
This slide
Copy link URL
Copy link URL
Copy iframe embed code
Copy iframe embed code
Copy javascript embed code
Copy javascript embed code
Share
Tweet
Share
Tweet
Slide 1
Slide 1 text
Why You Shouldn’t Write a Database Ben Johnson
Slide 2
Slide 2 text
Let’s define “database”
Slide 3
Slide 3 text
low-level LevelDB, BerkeleyDB, LMDB
Slide 4
Slide 4 text
Direct interface to OS Files, pages, & blocks Responsible for data integrity Typically key/value
Slide 5
Slide 5 text
high-level SQLite, MySQL, PostgreSQL, Mongo, InfluxDB, etc
Slide 6
Slide 6 text
Builds on low-level stores Rows, tables, indexes Interfaces with end user Relational, document, time series
Slide 7
Slide 7 text
Me
Slide 8
Slide 8 text
SkyDB ReportifyDB BoltDB InfluxDB
Slide 9
Slide 9 text
There are n+1 reasons not to write a database
Slide 10
Slide 10 text
#1. High barrier to entry
Slide 11
Slide 11 text
How to write a database
Slide 12
Slide 12 text
No content
Slide 13
Slide 13 text
legit
Slide 14
Slide 14 text
sorta, but not quite
Slide 15
Slide 15 text
seriously?
Slide 16
Slide 16 text
suggests using XML
Slide 17
Slide 17 text
nope
Slide 18
Slide 18 text
Available resources?
Slide 19
Slide 19 text
Research papers!
Slide 20
Slide 20 text
Research papers! Narrowly focused (indexing, storage, locks)
Slide 21
Slide 21 text
Research papers! High level (little or no code)
Slide 22
Slide 22 text
Research papers! Assumes a Ph.D
Slide 23
Slide 23 text
Read source code!
Slide 24
Slide 24 text
Read source code! MySQL >1M SLOC
Slide 25
Slide 25 text
Read source code! Even small databases are 10KLOC+
Slide 26
Slide 26 text
There is no Writing Databases 101
Slide 27
Slide 27 text
#2. Debugging Sucks
Slide 28
Slide 28 text
Bugs are catastrophic Cause corruption, loss of data integrity
Slide 29
Slide 29 text
Bugs are catastrophic Users with data loss are very unhappy
Slide 30
Slide 30 text
Find a good hex editor
Slide 31
Slide 31 text
Find a good hex editor (I use Hex Fiend)
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
This is not what a database looks like
Slide 34
Slide 34 text
No content
Slide 35
Slide 35 text
THIS is what a database looks like
Slide 36
Slide 36 text
Debugging w/o data Users usually can’t release their data
Slide 37
Slide 37 text
#3. Tradeoffs
Slide 38
Slide 38 text
Users don’t understand tradeoffs
Slide 39
Slide 39 text
No content
Slide 40
Slide 40 text
Read Optimized vs Write Optimized
Slide 41
Slide 41 text
CPU Bound vs IO Bound
Slide 42
Slide 42 text
https://gist.github.com/jboner/2841832
Slide 43
Slide 43 text
Features are liabilities It’s not if you have bugs, it’s how many
Slide 44
Slide 44 text
#4. Limited Community
Slide 45
Slide 45 text
Very few people who have written a production database
Slide 46
Slide 46 text
TONS of people who will tell you why your database sucks
Slide 47
Slide 47 text
Databases are hard
Slide 48
Slide 48 text
The End
Slide 49
Slide 49 text
The End (Just kidding!)
Slide 50
Slide 50 text
Not scared off yet?
Slide 51
Slide 51 text
Only 2 reasons to write a database:
Slide 52
Slide 52 text
#1. To learn
Slide 53
Slide 53 text
Data Integrity Indexing Isolation levels Recovery Prefetching Parser & Lexers Replication Snapshotting Transactions Materialized View Referential Integrity Query Compilation Query Planning Query Optimization Serializability Write Ahead Log Memory Profiling Redo/Undo Log Snapshot Isolation Recovery Two-Phase Commit Quorums
Slide 54
Slide 54 text
#2. Gain efficiency (For a specific use case)
Slide 55
Slide 55 text
The more generic your database, the fewer assumptions you can make
Slide 56
Slide 56 text
Go write a database!
Slide 57
Slide 57 text
The End @benbjohnson
Slide 58
Slide 58 text
Questions? @benbjohnson