Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
2
540
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
310
Designing and implementing GraphQL API
mariuszgil
1
60
Discovering unknown with EventStorming ConFoo
mariuszgil
0
280
Game of Developer Life... Deconstructed
mariuszgil
1
160
Back to forgotten roots
mariuszgil
1
380
Go micro with microservices
mariuszgil
5
630
Machine Learning for the rescue
mariuszgil
0
380
Discovering graph structures
mariuszgil
3
540
Introduction to Aerospike with PHP
mariuszgil
8
790
Other Decks in Programming
See All in Programming
ゲームの物理
fadis
3
720
Constant integer division faster than compiler-generated code
herumi
2
390
Vibe Codingの幻想を超えて-生成AIを現場で使えるようにするまでの泥臭い話.ai
fumiyakume
21
10k
令和最新版手のひらコンピュータ
koba789
13
5.7k
『リコリス・リコイル』に学ぶ!! 〜キャリア戦略における計画的偶発性理論と変わる勇気の重要性〜
wanko_it
1
180
ワープロって実は計算機で
pepepper
2
1k
Claude Code で Astro blog を Pages から Workers へ移行してみた
codehex
0
180
変化を楽しむエンジニアリング ~ いままでとこれから ~
murajun1978
0
670
一人でAIプロダクトを作るための工夫 〜技術選定・開発プロセス編〜 / I want AI to work harder
rkaga
4
700
SwiftでMCPサーバーを作ろう!
giginet
PRO
2
220
kiroでゲームを作ってみた
iriikeita
0
140
Streamlitで実現できるようになったこと、実現してくれたこと
ayumu_yamaguchi
2
280
Featured
See All Featured
Thoughts on Productivity
jonyablonski
69
4.8k
We Have a Design System, Now What?
morganepeng
53
7.7k
Why Our Code Smells
bkeepers
PRO
337
57k
Stop Working from a Prison Cell
hatefulcrawdad
271
21k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
8
750
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Balancing Empowerment & Direction
lara
1
530
How GitHub (no longer) Works
holman
314
140k
Rebuilding a faster, lazier Slack
samanthasiow
83
9.1k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Producing Creativity
orderedlist
PRO
346
40k
Fashionably flexible responsive web design (full day workshop)
malarkey
407
66k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil
[email protected]