Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Why You Shouldn't Write a Database
Search
benbjohnson
October 22, 2015
Technology
2
530
Why You Shouldn't Write a Database
Talk given at the October 2015 Denver Go meetup.
benbjohnson
October 22, 2015
Tweet
Share
More Decks by benbjohnson
See All by benbjohnson
Static Code Analysis Using SSA
benbjohnson
6
1.6k
Writing a High Performance Database in Go
benbjohnson
11
6.6k
Raft: The Understandable Distributed Consensus Protocol
benbjohnson
106
27k
Behavioral Analytics - Understanding the "why" and "how" of your users.
benbjohnson
9
1.9k
New Tech - Sky
benbjohnson
2
350
Behavioral Databases
benbjohnson
16
2.4k
Sky Analytics Pitch Deck
benbjohnson
0
550
Other Decks in Technology
See All in Technology
DDDにおける認可の扱いとKotlinにおける実装パターン / authorization-for-ddd-and-kotlin-implement-pattern
urmot
4
390
Android研修【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
100
AOAI Dev Day - Opening Session
yoshidashingo
2
480
スタートアップにおける組織設計とスクラムの長期戦略 / Scrum Fest Kanazawa 2024
yoshikiiida
13
3.7k
CEL(Common Expression Language)で書いた条件にマッチしたIAM Policyを見つける / iam-policy-finder
fujiwara3
0
720
テスト・設計研修【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
170
ABEMAにおけるLLMを用いたコンテンツベース推薦システム導入と効果検証
cyberagentdevelopers
PRO
1
790
技術負債による事業の失敗はなぜ起こるのか / Why do business failures due to technical debt occur?
i35_267
0
190
What if...? 처음부터 다시 LLM 어플리케이션을 개발한다면
huffon
0
1.1k
DevIO2024_レガシー運用からの脱却 -クラウド活用の実践事例とベストプラクティス-
jun2882
0
210
地理情報とAPIのトレンド
nagix
0
160
データベース研修 DB基礎【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
230
Featured
See All Featured
What's in a price? How to price your products and services
michaelherold
239
11k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
17
1.5k
Principles of Awesome APIs and How to Build Them.
keavy
124
16k
Web Components: a chance to create the future
zenorocha
307
41k
A designer walks into a library…
pauljervisheath
201
24k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
129
32k
How to Think Like a Performance Engineer
csswizardry
4
590
Being A Developer After 40
akosma
72
580k
Building Adaptive Systems
keathley
34
2k
Fantastic passwords and where to find them - at NoRuKo
philnash
42
2.7k
The Cost Of JavaScript in 2023
addyosmani
31
4.7k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
662
120k
Transcript
Why You Shouldn’t Write a Database Ben Johnson
Let’s define “database”
low-level LevelDB, BerkeleyDB, LMDB
Direct interface to OS Files, pages, & blocks Responsible for
data integrity Typically key/value
high-level SQLite, MySQL, PostgreSQL, Mongo, InfluxDB, etc
Builds on low-level stores Rows, tables, indexes Interfaces with end
user Relational, document, time series
Me
SkyDB ReportifyDB BoltDB InfluxDB
There are n+1 reasons not to write a database
#1. High barrier to entry
How to write a database
None
legit
sorta, but not quite
seriously?
suggests using XML
nope
Available resources?
Research papers!
Research papers! Narrowly focused (indexing, storage, locks)
Research papers! High level (little or no code)
Research papers! Assumes a Ph.D
Read source code!
Read source code! MySQL >1M SLOC
Read source code! Even small databases are 10KLOC+
There is no Writing Databases 101
#2. Debugging Sucks
Bugs are catastrophic Cause corruption, loss of data integrity
Bugs are catastrophic Users with data loss are very unhappy
Find a good hex editor
Find a good hex editor (I use Hex Fiend)
None
This is not what a database looks like
None
THIS is what a database looks like
Debugging w/o data Users usually can’t release their data
#3. Tradeoffs
Users don’t understand tradeoffs
None
Read Optimized vs Write Optimized
CPU Bound vs IO Bound
https://gist.github.com/jboner/2841832
Features are liabilities It’s not if you have bugs, it’s
how many
#4. Limited Community
Very few people who have written a production database
TONS of people who will tell you why your database
sucks
Databases are hard
The End
The End (Just kidding!)
Not scared off yet?
Only 2 reasons to write a database:
#1. To learn
Data Integrity Indexing Isolation levels Recovery Prefetching Parser & Lexers
Replication Snapshotting Transactions Materialized View Referential Integrity Query Compilation Query Planning Query Optimization Serializability Write Ahead Log Memory Profiling Redo/Undo Log Snapshot Isolation Recovery Two-Phase Commit Quorums
#2. Gain efficiency (For a specific use case)
The more generic your database, the fewer assumptions you can
make
Go write a database!
The End @benbjohnson
Questions? @benbjohnson