Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
2
510
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
270
Designing and implementing GraphQL API
mariuszgil
1
39
Discovering unknown with EventStorming ConFoo
mariuszgil
0
240
Game of Developer Life... Deconstructed
mariuszgil
1
150
Back to forgotten roots
mariuszgil
1
350
Go micro with microservices
mariuszgil
5
500
Machine Learning for the rescue
mariuszgil
0
330
Discovering graph structures
mariuszgil
3
500
Introduction to Aerospike with PHP
mariuszgil
8
690
Other Decks in Programming
See All in Programming
RubyGems on ruby.wasm
kateinoigakukun
0
630
Module Harmony について
yosuke_furukawa
PRO
3
660
Kubernetes Code Contribution入門
bells17
3
700
Polarsの現状
daikikatsuragawa
0
1.4k
ライブラリをパブリッシュせずにすばやく試す
exoego
2
110
新年度から始めるJava言語のファーストステップ2024
tatsuya1bm
0
150
Svelte採用記 - 位置情報と可視化の会社で、全社の標準技術スタックに選ぶまで / Svelte Japan Online Meetup #3
sorami
2
210
The rollercoaster of releasing an Android, iOS, and macOS app with Kotlin Multiplatform | KotlinConf
prof18
0
110
Good intentions gone bad
evaferreira
0
100
コンパウンドプロダクト開発の質とスピードを支える Protobuf と Connect #アーキテクチャ_findy / Boosting Compound Product Development Efficiency with Protobuf and Connect
izumin5210
12
640
RuboCop: LSP and Prism
koic
3
1.6k
Upgrading Legacy to the Latest PHP Version
afilina
PRO
0
160
Featured
See All Featured
Why Our Code Smells
bkeepers
PRO
331
56k
10 Git Anti Patterns You Should be Aware of
lemiorhan
650
58k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
41
4.5k
Building Your Own Lightsaber
phodgson
100
5.8k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
4
190
Understanding Cognitive Biases in Performance Measurement
bluesmoon
12
1.1k
Atom: Resistance is Futile
akmur
260
25k
The Mythical Team-Month
searls
217
42k
The Illustrated Children's Guide to Kubernetes
chrisshort
32
47k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
501
140k
Designing Experiences People Love
moore
136
23k
Pencils Down: Stop Designing & Start Developing
hursman
117
11k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil
[email protected]