Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
2
540
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
310
Designing and implementing GraphQL API
mariuszgil
1
66
Discovering unknown with EventStorming ConFoo
mariuszgil
0
280
Game of Developer Life... Deconstructed
mariuszgil
1
170
Back to forgotten roots
mariuszgil
1
390
Go micro with microservices
mariuszgil
5
630
Machine Learning for the rescue
mariuszgil
0
390
Discovering graph structures
mariuszgil
3
540
Introduction to Aerospike with PHP
mariuszgil
8
800
Other Decks in Programming
See All in Programming
デミカツ切り抜きで面倒くさいことはPythonにやらせよう
aokswork3
0
180
ИИ-Агенты в каждый дом – Алексей Порядин, PythoNN
sobolevn
0
150
止められない医療アプリ、そっと Swift 6 へ
medley
1
110
AIエージェント時代における TypeScriptスキーマ駆動開発の新たな役割
bicstone
4
1.5k
Чего вы не знали о строках в Python – Василий Рябов, PythoNN
sobolevn
0
160
WebエンジニアがSwiftをブラウザで動かすプレイグラウンドを作ってみた
ohmori_yusuke
0
170
CSC305 Lecture 04
javiergs
PRO
0
250
複雑化したリポジトリをなんとかした話 pipenvからuvによるモノレポ構成への移行
satoshi256kbyte
1
760
iOS 17で追加されたSubscriptionStoreView を利用して5分でサブスク実装チャレンジ
natmark
0
560
Local Peer-to-Peer APIはどのように使われていくのか?
hal_spidernight
2
440
Go Conference 2025: Goで体感するMultipath TCP ― Go 1.24 時代の MPTCP Listener を理解する
takehaya
7
1.6k
大規模アプリのDIフレームワーク刷新戦略 ~過去最大規模の並行開発を止めずにアプリ全体に導入するまで~
mot_techtalk
0
380
Featured
See All Featured
Optimising Largest Contentful Paint
csswizardry
37
3.4k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
114
20k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
358
30k
RailsConf 2023
tenderlove
30
1.2k
The Invisible Side of Design
smashingmag
301
51k
A better future with KSS
kneath
239
17k
GraphQLの誤解/rethinking-graphql
sonatard
73
11k
Writing Fast Ruby
sferik
629
62k
Mobile First: as difficult as doing things right
swwweet
224
10k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.5k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
9
960
Building Applications with DynamoDB
mza
96
6.6k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil
[email protected]