Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
2
530
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
300
Designing and implementing GraphQL API
mariuszgil
1
50
Discovering unknown with EventStorming ConFoo
mariuszgil
0
270
Game of Developer Life... Deconstructed
mariuszgil
1
150
Back to forgotten roots
mariuszgil
1
380
Go micro with microservices
mariuszgil
5
600
Machine Learning for the rescue
mariuszgil
0
370
Discovering graph structures
mariuszgil
3
530
Introduction to Aerospike with PHP
mariuszgil
8
770
Other Decks in Programming
See All in Programming
ウォンテッドリーの「ココロオドル」モバイル開発 / Wantedly's "kokoro odoru" mobile development
kubode
1
290
20250426 GDGoC 合同新歓 - GDGoC のススメ
getty708
0
110
GitHub Copilot for Azureを使い倒したい
ymd65536
1
330
Optimizing JRuby 10
headius
0
580
スモールスタートで始めるためのLambda×モノリス(Lambdalith)
akihisaikeda
2
400
Bedrock × Confluenceで簡単(?)社内RAG
iharuoru
1
120
カウシェで Four Keys の改善を試みた理由
ike002jp
1
130
Instrumentsを使用した アプリのパフォーマンス向上方法
hinakko
0
240
M5UnitUnified 最新動向 2025/05
gob
0
130
Storybookの情報をMCPサーバー化する
shota_tech
2
850
Jakarta EE Meets AI
ivargrimstad
0
850
ComposeでのPicture in Picture
takathemax
0
130
Featured
See All Featured
VelocityConf: Rendering Performance Case Studies
addyosmani
329
24k
A designer walks into a library…
pauljervisheath
205
24k
Writing Fast Ruby
sferik
628
61k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Adopting Sorbet at Scale
ufuk
76
9.3k
Thoughts on Productivity
jonyablonski
69
4.6k
Side Projects
sachag
453
42k
Product Roadmaps are Hard
iamctodd
PRO
53
11k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
700
The Power of CSS Pseudo Elements
geoffreycrofte
75
5.8k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
47
5.4k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
12k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil mariusz@mariuszgil.pl