Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Why You Shouldn't Write a Database
Search
Sponsored
·
SiteGround - Reliable hosting with speed, security, and support you can count on.
→
benbjohnson
October 22, 2015
Technology
600
2
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Why You Shouldn't Write a Database
Talk given at the October 2015 Denver Go meetup.
benbjohnson
October 22, 2015
Transcript
58 slides
Why You Shouldn’t Write a Database Ben Johnson
Let’s define “database”
low-level LevelDB, BerkeleyDB, LMDB
Direct interface to OS Files, pages, & blocks Responsible for
data integrity Typically key/value
high-level SQLite, MySQL, PostgreSQL, Mongo, InfluxDB, etc
Builds on low-level stores Rows, tables, indexes Interfaces with end
user Relational, document, time series
Me
SkyDB ReportifyDB BoltDB InfluxDB
There are n+1 reasons not to write a database
#1. High barrier to entry
How to write a database
None
legit
sorta, but not quite
seriously?
suggests using XML
nope
Available resources?
Research papers!
Research papers! Narrowly focused (indexing, storage, locks)
Research papers! High level (little or no code)
Research papers! Assumes a Ph.D
Read source code!
Read source code! MySQL >1M SLOC
Read source code! Even small databases are 10KLOC+
There is no Writing Databases 101
#2. Debugging Sucks
Bugs are catastrophic Cause corruption, loss of data integrity
Bugs are catastrophic Users with data loss are very unhappy
Find a good hex editor
Find a good hex editor (I use Hex Fiend)
None
This is not what a database looks like
None
THIS is what a database looks like
Debugging w/o data Users usually can’t release their data
#3. Tradeoffs
Users don’t understand tradeoffs
None
Read Optimized vs Write Optimized
CPU Bound vs IO Bound
https://gist.github.com/jboner/2841832
Features are liabilities It’s not if you have bugs, it’s
how many
#4. Limited Community
Very few people who have written a production database
TONS of people who will tell you why your database
sucks
Databases are hard
The End
The End (Just kidding!)
Not scared off yet?
Only 2 reasons to write a database:
#1. To learn
Data Integrity Indexing Isolation levels Recovery Prefetching Parser & Lexers
Replication Snapshotting Transactions Materialized View Referential Integrity Query Compilation Query Planning Query Optimization Serializability Write Ahead Log Memory Profiling Redo/Undo Log Snapshot Isolation Recovery Two-Phase Commit Quorums
#2. Gain efficiency (For a specific use case)
The more generic your database, the fewer assumptions you can
make
Go write a database!
The End @benbjohnson
Questions? @benbjohnson
More Decks by benbjohnson
See All by benbjohnson
Static Code Analysis Using SSA
benbjohnson
6
1.9k
Writing a High Performance Database in Go
benbjohnson
11
6.9k
Raft: The Understandable Distributed Consensus Protocol
benbjohnson
105
29k
Behavioral Analytics - Understanding the "why" and "how" of your users.
benbjohnson
9
2.1k
New Tech - Sky
benbjohnson
2
430
Behavioral Databases
benbjohnson
16
2.6k
Sky Analytics Pitch Deck
benbjohnson
0
710
Other Decks in Technology
See All in Technology
[NW-JAWS DAY]「うちのAWS、インターネット出られないんで」― AWS外の自社サービスをPrivateLinkでシュッと公開する方法
ykimi
3
900
攻撃と守りは何が変わり、何が変わらないのか~最近の不正アクセスから考える、リスク評価と標準の使いどころ
oidfj
2
250
統制されたID・権限をどう利用するか : DPoP/FAPIによるAPIセキュリティ強化
oidfj
1
230
AI-Ready なデータ基盤ってなに? & 新卒未経験が、いま考えていること
imuta
0
1k
エンタープライズ事業未経験のデータ屋、 AI時代にN回撃沈する.pdf
ryoskdara_
0
170
マネージャーをエスパーにしない マネジメントされる側から始める、働きやすい現場づくり
comi190327
1
830
個人ASを取得して、自宅Kubernetes上のVMをグローバルIPで公開した話
munenick
0
210
農家は秘伝のタレを知らない - NOBODY KNOWS THE HIDDEN SOURCE - Kaggriculture 3rd Place Solution
masspeaks
3
620
Re:ゼロから始めるシェアハウスのペットな鯖
silmin_
0
260
E2E Test というハーネス — AI時代の自動テスト民主化
daishukojima
10
5.9k
Kansai WordPress Meetup@大阪『ブロックテーマを作る上で知っておくべきこと
torounit
0
260
マネジメントされる側の仕事は「文句」を言うこと
riyaamemiya
7
5.9k
Featured
See All Featured
Bootstrapping a Software Product
garrettdimon
PRO
306
120k
How to Talk to Developers About Accessibility
jct
2
620
Primal Persuasion: How to Engage the Brain for Learning That Lasts
tmiket
0
520
Groundhog Day: Seeking Process in Gaming for Health
codingconduct
0
490
Hiding What from Whom? A Critical Review of the History of Programming languages for Music
tomoyanonymous
3
1.5k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
56
3.5k
The Cult of Friendly URLs
andyhume
79
7.1k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
12
1.4k
We Have a Design System, Now What?
morganepeng
55
8.4k
Measuring Dark Social's Impact On Conversion and Attribution
stephenakadiri
2
330
The agentic SEO stack - context over prompts
schlessera
0
1k
[Rails World 2026] Durable orchestration on Rails: from continuation to workflow
palkan
4
1.3k