Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Why You Shouldn't Write a Database
Search
benbjohnson
October 22, 2015
Technology
2
560
Why You Shouldn't Write a Database
Talk given at the October 2015 Denver Go meetup.
benbjohnson
October 22, 2015
Tweet
Share
More Decks by benbjohnson
See All by benbjohnson
Static Code Analysis Using SSA
benbjohnson
6
1.8k
Writing a High Performance Database in Go
benbjohnson
11
6.8k
Raft: The Understandable Distributed Consensus Protocol
benbjohnson
106
28k
Behavioral Analytics - Understanding the "why" and "how" of your users.
benbjohnson
9
2k
New Tech - Sky
benbjohnson
2
380
Behavioral Databases
benbjohnson
16
2.5k
Sky Analytics Pitch Deck
benbjohnson
0
610
Other Decks in Technology
See All in Technology
AWSを利用する上で知っておきたい名前解決のはなし(10分版)
nagisa53
10
3.2k
Aurora DSQLはサーバーレスアーキテクチャの常識を変えるのか
iwatatomoya
1
1.1k
会社紹介資料 / Sansan Company Profile
sansan33
PRO
6
380k
5年目から始める Vue3 サイト改善 #frontendo
tacck
PRO
3
230
研究開発と製品開発、両利きのロボティクス
youtalk
1
530
普通のチームがスクラムを会得するたった一つの冴えたやり方 / the best way to scrum
okamototakuyasr2
0
100
【NoMapsTECH 2025】AI Edge Computing Workshop
akit37
0
220
エンジニアリングマネージャーの成長の道筋とキャリア / Developers Summit 2025 KANSAI
daiksy
1
410
ハードウェアとソフトウェアをつなぐ全てを内製している企業の E2E テストの作り方 / How to create E2E tests for a company that builds everything connecting hardware and software in-house
bitkey
PRO
1
160
「何となくテストする」を卒業するためにプロダクトが動く仕組みを理解しよう
kawabeaver
0
420
[ JAWS-UG 東京 CommunityBuilders Night #2 ]SlackとAmazon Q Developerで 運用効率化を模索する
sh_fk2
3
450
プラットフォーム転換期におけるGitHub Copilot活用〜Coding agentがそれを加速するか〜 / Leveraging GitHub Copilot During Platform Transition Periods
aeonpeople
1
200
Featured
See All Featured
Facilitating Awesome Meetings
lara
55
6.5k
Java REST API Framework Comparison - PWX 2021
mraible
33
8.8k
BBQ
matthewcrist
89
9.8k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.9k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
3k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.1k
Statistics for Hackers
jakevdp
799
220k
Large-scale JavaScript Application Architecture
addyosmani
513
110k
Building Better People: How to give real-time feedback that sticks.
wjessup
368
19k
Building Flexible Design Systems
yeseniaperezcruz
329
39k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
131
19k
Transcript
Why You Shouldn’t Write a Database Ben Johnson
Let’s define “database”
low-level LevelDB, BerkeleyDB, LMDB
Direct interface to OS Files, pages, & blocks Responsible for
data integrity Typically key/value
high-level SQLite, MySQL, PostgreSQL, Mongo, InfluxDB, etc
Builds on low-level stores Rows, tables, indexes Interfaces with end
user Relational, document, time series
Me
SkyDB ReportifyDB BoltDB InfluxDB
There are n+1 reasons not to write a database
#1. High barrier to entry
How to write a database
None
legit
sorta, but not quite
seriously?
suggests using XML
nope
Available resources?
Research papers!
Research papers! Narrowly focused (indexing, storage, locks)
Research papers! High level (little or no code)
Research papers! Assumes a Ph.D
Read source code!
Read source code! MySQL >1M SLOC
Read source code! Even small databases are 10KLOC+
There is no Writing Databases 101
#2. Debugging Sucks
Bugs are catastrophic Cause corruption, loss of data integrity
Bugs are catastrophic Users with data loss are very unhappy
Find a good hex editor
Find a good hex editor (I use Hex Fiend)
None
This is not what a database looks like
None
THIS is what a database looks like
Debugging w/o data Users usually can’t release their data
#3. Tradeoffs
Users don’t understand tradeoffs
None
Read Optimized vs Write Optimized
CPU Bound vs IO Bound
https://gist.github.com/jboner/2841832
Features are liabilities It’s not if you have bugs, it’s
how many
#4. Limited Community
Very few people who have written a production database
TONS of people who will tell you why your database
sucks
Databases are hard
The End
The End (Just kidding!)
Not scared off yet?
Only 2 reasons to write a database:
#1. To learn
Data Integrity Indexing Isolation levels Recovery Prefetching Parser & Lexers
Replication Snapshotting Transactions Materialized View Referential Integrity Query Compilation Query Planning Query Optimization Serializability Write Ahead Log Memory Profiling Redo/Undo Log Snapshot Isolation Recovery Two-Phase Commit Quorums
#2. Gain efficiency (For a specific use case)
The more generic your database, the fewer assumptions you can
make
Go write a database!
The End @benbjohnson
Questions? @benbjohnson