Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
2
540
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
310
Designing and implementing GraphQL API
mariuszgil
1
63
Discovering unknown with EventStorming ConFoo
mariuszgil
0
280
Game of Developer Life... Deconstructed
mariuszgil
1
160
Back to forgotten roots
mariuszgil
1
390
Go micro with microservices
mariuszgil
5
630
Machine Learning for the rescue
mariuszgil
0
390
Discovering graph structures
mariuszgil
3
540
Introduction to Aerospike with PHP
mariuszgil
8
800
Other Decks in Programming
See All in Programming
Oracle Database Technology Night 92 Database Connection control FAN-AC
oracle4engineer
PRO
1
470
速いWebフレームワークを作る
yusukebe
5
1.7k
AI時代のUIはどこへ行く?
yusukebe
18
9k
為你自己學 Python - 冷知識篇
eddie
1
350
Azure SRE Agentで運用は楽になるのか?
kkamegawa
0
2.5k
FindyにおけるTakumi活用と脆弱性管理のこれから
rvirus0817
0
530
Introducing ReActionView: A new ActionView-compatible ERB Engine @ Rails World 2025, Amsterdam
marcoroth
0
710
個人軟體時代
ethanhuang13
0
330
250830 IaCの選定~AWS SAMのLambdaをECSに乗り換えたときの備忘録~
east_takumi
0
400
HTMLの品質ってなんだっけ? “HTMLクライテリア”の設計と実践
unachang113
4
2.9k
Compose Multiplatform × AI で作る、次世代アプリ開発支援ツールの設計と実装
thagikura
0
170
今だからこそ入門する Server-Sent Events (SSE)
nearme_tech
PRO
3
250
Featured
See All Featured
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.1k
Imperfection Machines: The Place of Print at Facebook
scottboms
268
13k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Intergalactic Javascript Robots from Outer Space
tanoku
272
27k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.9k
It's Worth the Effort
3n
187
28k
Raft: Consensus for Rubyists
vanstee
140
7.1k
Side Projects
sachag
455
43k
The Cost Of JavaScript in 2023
addyosmani
53
8.9k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
23
1.4k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
52
5.6k
Bash Introduction
62gerente
615
210k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil
[email protected]