Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Sponsored
·
SiteGround - Reliable hosting with speed, security, and support you can count on.
→
Mariusz Gil
April 12, 2013
Programming
2
550
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
320
Designing and implementing GraphQL API
mariuszgil
1
85
Discovering unknown with EventStorming ConFoo
mariuszgil
0
310
Game of Developer Life... Deconstructed
mariuszgil
1
180
Back to forgotten roots
mariuszgil
1
400
Go micro with microservices
mariuszgil
5
680
Machine Learning for the rescue
mariuszgil
0
420
Discovering graph structures
mariuszgil
3
550
Introduction to Aerospike with PHP
mariuszgil
8
830
Other Decks in Programming
See All in Programming
生成AIを使ったコードレビューで定性的に品質カバー
chiilog
1
250
AI巻き込み型コードレビューのススメ
nealle
0
120
AI時代のキャリアプラン「技術の引力」からの脱出と「問い」へのいざない / tech-gravity
minodriven
20
6.8k
Honoを使ったリモートMCPサーバでAIツールとの連携を加速させる!
tosuri13
1
170
OCaml 5でモダンな並列プログラミングを Enjoyしよう!
haochenx
0
140
Oxlint JS plugins
kazupon
1
740
カスタマーサクセス業務を変革したヘルススコアの実現と学び
_hummer0724
0
630
MUSUBIXとは
nahisaho
0
130
dchart: charts from deck markup
ajstarks
3
990
AI Schema Enrichment for your Oracle AI Database
thatjeffsmith
0
250
コマンドとリード間の連携に対する脅威分析フレームワーク
pandayumi
1
450
AI時代の認知負荷との向き合い方
optfit
0
150
Featured
See All Featured
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
9
1.2k
New Earth Scene 8
popppiees
1
1.5k
Building Applications with DynamoDB
mza
96
6.9k
JAMstack: Web Apps at Ludicrous Speed - All Things Open 2022
reverentgeek
1
320
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.6k
Build your cross-platform service in a week with App Engine
jlugia
234
18k
How to Grow Your eCommerce with AI & Automation
katarinadahlin
PRO
0
100
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
12
1k
End of SEO as We Know It (SMX Advanced Version)
ipullrank
3
3.9k
Neural Spatial Audio Processing for Sound Field Analysis and Control
skoyamalab
0
160
Avoiding the “Bad Training, Faster” Trap in the Age of AI
tmiket
0
72
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
287
14k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil
[email protected]