Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Why You Shouldn't Write a Database
Search
benbjohnson
October 22, 2015
Technology
2
540
Why You Shouldn't Write a Database
Talk given at the October 2015 Denver Go meetup.
benbjohnson
October 22, 2015
Tweet
Share
More Decks by benbjohnson
See All by benbjohnson
Static Code Analysis Using SSA
benbjohnson
6
1.7k
Writing a High Performance Database in Go
benbjohnson
11
6.7k
Raft: The Understandable Distributed Consensus Protocol
benbjohnson
106
28k
Behavioral Analytics - Understanding the "why" and "how" of your users.
benbjohnson
9
2k
New Tech - Sky
benbjohnson
2
370
Behavioral Databases
benbjohnson
16
2.5k
Sky Analytics Pitch Deck
benbjohnson
0
580
Other Decks in Technology
See All in Technology
株式会社ログラス − エンジニア向け会社説明資料 / Loglass Comapany Deck for Engineer
loglass2019
3
33k
20241125 - AI 繪圖實戰魔法工作坊 @ 實踐大學
dpys
1
440
知っててうれしい HTTP Cookie を使ったセッション管理について
greendrop
0
100
Fearsome File Formats
ange
0
550
型情報を用いたLintでコード品質を向上させる
sansantech
PRO
2
210
能動的ドメイン名ライフサイクル管理のすゝめ / Practice on Active Domain Name Lifecycle Management
nttcom
0
310
DUSt3R, MASt3R, MASt3R-SfM にみる3D基盤モデル
spatial_ai_network
3
480
エンジニアリングマネージャー視点での、自律的なスケーリングを実現するFASTという選択肢 / RSGT2025
yoshikiiida
4
2.3k
I could be Wrong!! - Learning from Agile Experts
kawaguti
PRO
8
1.9k
React Routerで実現する型安全なSPAルーティング
sansantech
PRO
4
840
テストを書かないためのテスト/ Tests for not writing tests
sinsoku
1
150
ソフトウェア開発における「パーフェクトな意思決定」/Perfect Decision-Making in Software Development
yayoi_dd
2
2.6k
Featured
See All Featured
Statistics for Hackers
jakevdp
797
220k
Thoughts on Productivity
jonyablonski
68
4.4k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
3
330
Mobile First: as difficult as doing things right
swwweet
222
9k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
33
2k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
10
850
The Cult of Friendly URLs
andyhume
78
6.1k
BBQ
matthewcrist
85
9.4k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
230
52k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
132
33k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
Gamification - CAS2011
davidbonilla
80
5.1k
Transcript
Why You Shouldn’t Write a Database Ben Johnson
Let’s define “database”
low-level LevelDB, BerkeleyDB, LMDB
Direct interface to OS Files, pages, & blocks Responsible for
data integrity Typically key/value
high-level SQLite, MySQL, PostgreSQL, Mongo, InfluxDB, etc
Builds on low-level stores Rows, tables, indexes Interfaces with end
user Relational, document, time series
Me
SkyDB ReportifyDB BoltDB InfluxDB
There are n+1 reasons not to write a database
#1. High barrier to entry
How to write a database
None
legit
sorta, but not quite
seriously?
suggests using XML
nope
Available resources?
Research papers!
Research papers! Narrowly focused (indexing, storage, locks)
Research papers! High level (little or no code)
Research papers! Assumes a Ph.D
Read source code!
Read source code! MySQL >1M SLOC
Read source code! Even small databases are 10KLOC+
There is no Writing Databases 101
#2. Debugging Sucks
Bugs are catastrophic Cause corruption, loss of data integrity
Bugs are catastrophic Users with data loss are very unhappy
Find a good hex editor
Find a good hex editor (I use Hex Fiend)
None
This is not what a database looks like
None
THIS is what a database looks like
Debugging w/o data Users usually can’t release their data
#3. Tradeoffs
Users don’t understand tradeoffs
None
Read Optimized vs Write Optimized
CPU Bound vs IO Bound
https://gist.github.com/jboner/2841832
Features are liabilities It’s not if you have bugs, it’s
how many
#4. Limited Community
Very few people who have written a production database
TONS of people who will tell you why your database
sucks
Databases are hard
The End
The End (Just kidding!)
Not scared off yet?
Only 2 reasons to write a database:
#1. To learn
Data Integrity Indexing Isolation levels Recovery Prefetching Parser & Lexers
Replication Snapshotting Transactions Materialized View Referential Integrity Query Compilation Query Planning Query Optimization Serializability Write Ahead Log Memory Profiling Redo/Undo Log Snapshot Isolation Recovery Two-Phase Commit Quorums
#2. Gain efficiency (For a specific use case)
The more generic your database, the fewer assumptions you can
make
Go write a database!
The End @benbjohnson
Questions? @benbjohnson