Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
580
2
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Database scaling patterns
Mariusz Gil
April 12, 2013
Transcript
39 slides
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil
[email protected]
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
360
Designing and implementing GraphQL API
mariuszgil
1
120
Discovering unknown with EventStorming ConFoo
mariuszgil
0
330
Game of Developer Life... Deconstructed
mariuszgil
1
210
Back to forgotten roots
mariuszgil
1
470
Go micro with microservices
mariuszgil
5
740
Machine Learning for the rescue
mariuszgil
0
470
Discovering graph structures
mariuszgil
3
570
Introduction to Aerospike with PHP
mariuszgil
8
890
Other Decks in Programming
See All in Programming
Lambda durable functionsでHuman in the Loopを楽に…楽に……
har1101
2
330
コード生成時代のAPI定義を支えるRequest Spec移行
pvcresin
0
200
社内AIエージェント「Askワンバン」における コストチューニングの取り組み
t0yohei
0
170
Spec-Driven Development - Why Vaadin Is the Perfect Fit
simas
PRO
0
250
AI Augmentation Techniques with Liberty
ivargrimstad
1
500
Kotlin Fest 2026: Kotlin コンパイラに Contribute してみよう! - Kotlin/Wasm で深い再帰を書けるようにした話
ternbusty
1
260
まずはコードより業務と向き合う:テーブル解体のためにPdMを呼んだ話
sakawe_ee
0
240
Kawaiicon: Kubernetes, it's no longer cool, but let's secure it anyway!
barnbarn
0
130
私なりのハーネスエンジニアリング ~独自の言語処理系の開発を通して~
seosoft
0
210
imagewebp()の向こう側: WebPはなぜ JPEG・PNGより軽いのか
yuzneri
0
110
Scalaの型でマルチテナントの認可を守る
nextbeatdev
0
150
[DroidKnights 2026] Workflow AX: From Auto-Documented Policies to E2E Testing
raeminkang
0
580
Featured
See All Featured
Art, The Web, and Tiny UX
lynnandtonic
304
22k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
32
3.1k
The SEO Collaboration Effect
kristinabergwall1
1
620
Producing Creativity
orderedlist
PRO
348
41k
Build your cross-platform service in a week with App Engine
jlugia
234
19k
Embracing the Ebb and Flow
colly
88
5.2k
Impact Scores and Hybrid Strategies: The future of link building
tamaranovitovic
0
480
Color Theory Basics | Prateek | Gurzu
gurzu
1
520
Balancing Empowerment & Direction
lara
6
1.4k
Crafting Experiences
bethany
1
440
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
Making Projects Easy
brettharned
120
6.9k