Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
2
530
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
290
Designing and implementing GraphQL API
mariuszgil
1
46
Discovering unknown with EventStorming ConFoo
mariuszgil
0
260
Game of Developer Life... Deconstructed
mariuszgil
1
150
Back to forgotten roots
mariuszgil
1
370
Go micro with microservices
mariuszgil
5
580
Machine Learning for the rescue
mariuszgil
0
360
Discovering graph structures
mariuszgil
3
530
Introduction to Aerospike with PHP
mariuszgil
8
760
Other Decks in Programming
See All in Programming
Devin入門 〜月500ドルから始まるAIチームメイトとの開発生活〜 / Introduction Devin 〜Development With AI Teammates〜
rkaga
5
1.6k
Google Cloudとo11yで実現するアプリケーション開発者主体のDB改善
nnaka2992
1
160
Go 1.24でジェネリックになった型エイリアスの紹介
syumai
2
320
未経験でSRE、はじめました! 組織を支える役割と軌跡
curekoshimizu
1
230
バイセルでの AI を用いた開発の取り組み ~ Devin, Cursor の活用事例・知見共有 ~
umaidashi
0
130
Generating OpenAPI schema from serializers throughout the Rails stack - Kyobashi.rb #5
envek
1
450
Introduction to C Extensions
sylph01
3
130
PromptyによるAI開発入門
ymd65536
1
150
The Clean ArchitectureがWebフロントエンドでしっくりこないのは何故か / Why The Clean Architecture does not fit with Web Frontend
twada
PRO
62
21k
仕様変更に耐えるための"今の"DRY原則を考える
mkmk884
9
3.3k
自力でTTSモデルを作った話
zgock999
0
140
ナレッジイネイブリングにAIを活用してみる ゆるSRE勉強会 #9
nealle
0
170
Featured
See All Featured
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
4 Signs Your Business is Dying
shpigford
183
22k
RailsConf 2023
tenderlove
29
1k
Designing for Performance
lara
605
68k
Music & Morning Musume
bryan
46
6.4k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
11
1.3k
The Power of CSS Pseudo Elements
geoffreycrofte
75
5.5k
GitHub's CSS Performance
jonrohan
1030
460k
Documentation Writing (for coders)
carmenintech
69
4.6k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Code Reviewing Like a Champion
maltzj
521
39k
BBQ
matthewcrist
87
9.5k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil mariusz@mariuszgil.pl