Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
2
510
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
260
Designing and implementing GraphQL API
mariuszgil
1
38
Discovering unknown with EventStorming ConFoo
mariuszgil
0
230
Game of Developer Life... Deconstructed
mariuszgil
1
150
Back to forgotten roots
mariuszgil
1
350
Go micro with microservices
mariuszgil
5
480
Machine Learning for the rescue
mariuszgil
0
330
Discovering graph structures
mariuszgil
3
490
Introduction to Aerospike with PHP
mariuszgil
8
670
Other Decks in Programming
See All in Programming
object-oriented-conference-2024
fuwasegu
6
1.8k
ここ1~2年くらいで 使えるようになった(主要ブラウザーの最新版 がすべて対応した ) ウェブの新機能について ランダムに喋る!
myzkyy
9
6.5k
Parallel Socket Communication in Swift
s_shimotori
0
220
ファイル先頭の use の意味、説明できますか? 〜PHP の namespace と autoloading の関係を正しく理解しよう〜 / namespace and autoloading in php
okashoi
2
470
PHPでOfficeファイルを取り扱う! PHP Officeライブラリを プロダクトに組み込んだ話
hirobe1999
0
840
フロントエンドパフォーマンス 入門
shouta2
7
1.5k
Open Source Swiftc Workshop
kitasuke
1
180
MySQL のインデックスの種類をおさらいしよう! / overviewing indexes in MySQL
okashoi
0
170
Open Source Swift Workshop - Foundation and first party libraries
ikesyo
0
180
TCAの Shared Stateって どういう仕組みになってんの?
yimajo
0
330
LPIXEL×CADDi_kaerururu
kaerururu
3
300
オブジェクト指向コードレビューの新しいアプローチ
akkie76
3
1.5k
Featured
See All Featured
The Power of CSS Pseudo Elements
geoffreycrofte
58
4.9k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
113
18k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
219
21k
Being A Developer After 40
akosma
56
580k
Mobile First: as difficult as doing things right
swwweet
215
8.5k
Build The Right Thing And Hit Your Dates
maggiecrowley
23
1.9k
Large-scale JavaScript Application Architecture
addyosmani
501
110k
Principles of Awesome APIs and How to Build Them.
keavy
119
16k
Unsuck your backbone
ammeep
660
56k
Why Our Code Smells
bkeepers
PRO
330
56k
Rails Girls Zürich Keynote
gr2m
91
13k
Web Components: a chance to create the future
zenorocha
304
41k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil
[email protected]