Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Database scaling patterns
Search
Mariusz Gil
April 12, 2013
Programming
2
540
Database scaling patterns
Mariusz Gil
April 12, 2013
Tweet
Share
More Decks by Mariusz Gil
See All by Mariusz Gil
Aspect Oriented Programming
mariuszgil
1
310
Designing and implementing GraphQL API
mariuszgil
1
58
Discovering unknown with EventStorming ConFoo
mariuszgil
0
280
Game of Developer Life... Deconstructed
mariuszgil
1
160
Back to forgotten roots
mariuszgil
1
380
Go micro with microservices
mariuszgil
5
620
Machine Learning for the rescue
mariuszgil
0
380
Discovering graph structures
mariuszgil
3
540
Introduction to Aerospike with PHP
mariuszgil
8
790
Other Decks in Programming
See All in Programming
ペアプロ × 生成AI 現場での実践と課題について / generative-ai-in-pair-programming
codmoninc
2
21k
テストから始めるAgentic Coding 〜Claude Codeと共に行うTDD〜 / Agentic Coding starts with testing
rkaga
15
5.4k
「テストは愚直&&網羅的に書くほどよい」という誤解 / Test Smarter, Not Harder
munetoshi
0
190
Flutterで備える!Accessibility Nutrition Labels完全ガイド
yuukiw00w
0
170
20250628_非エンジニアがバイブコーディングしてみた
ponponmikankan
0
710
PostgreSQLのRow Level SecurityをPHPのORMで扱う Eloquent vs Doctrine #phpcon #track2
77web
2
570
PHPで始める振る舞い駆動開発(Behaviour-Driven Development)
ohmori_yusuke
2
430
AI駆動のマルチエージェントによる業務フロー自動化の設計と実践
h_okkah
0
220
可変変数との向き合い方 $$変数名が踊り出す$$ / php conference Variable variables
gunji
0
120
型で語るカタ
irof
0
620
テスト駆動Kaggle
isax1015
1
520
テスターからテストエンジニアへ ~新米テストエンジニアが歩んだ9ヶ月振り返り~
non0113
0
120
Featured
See All Featured
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
108
19k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3.1k
Balancing Empowerment & Direction
lara
1
440
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.7k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Building Adaptive Systems
keathley
43
2.7k
Building an army of robots
kneath
306
45k
What's in a price? How to price your products and services
michaelherold
246
12k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
830
The Pragmatic Product Professional
lauravandoore
35
6.7k
Transcript
Database scaling patterns Mariusz Gil the we a way
ABOUT ME
EXPERIENCE from 1 to 100+ DB servers
EXPERIENCE from 1 to 100+ DB servers
What i it a ABOUT ?
DATABASE IS MAIN PROBLEM IN 99% OF WEB APPS
TYPICAL ISSUES? NORMALIZED TO xNF INEFFICIENT SCHEMA, BAD INDEXES AND
QUERIES, UNUSED SLOW-LOG
shared setup
!" # $ %% & ' !( !" # !" # shared server
cache all the things, wisely...
!" # $ %% & ' !( !" # !" # shared server
separated servers
!" # $ %% & ' !( !" # !" # DB server
scale up
scale up
denormalization
DB server
!" # $ %% & ' !( !" # !" # !" # $ %% & ' !( !" # !" # DB server
DB server
!" # $ %% & ' !( !" # !" # DB server !" # $ %% & ' !( !" # ) !" # )
DB server DB server
!" # $ %% & ' !( !" # ) !" # ) # $ %% & ' !( !" # !" # ID SENDER_ID RECEIVER_ID ... 1 10 1 2 20 2 3 10 1 4 20 3 5 10 1 6 20 4 7 10 1 USER_ID MESSAGE_ID 1 1 1 3 1 5 1 7 2 2 3 4 4 6 USER_ID MESSAGE_ID 10 1 10 3 10 5 10 7 20 2 20 4 20 6
replication
MASTER-SLAVE scale reads, but not writes
DB master
!" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) DB slave !" # $ %% & ' !( !" # ) !" # ) ... INSERT... UPDATE... DELETE... SELECT... SELECT...
MASTER-MASTER just for HA, not for scaling
DB master
!" # $ %% & ' !( !" # ) !" # ) DB master !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT...
sharding
DB shard $ %% &
' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) DB shard !" # $ %% & ' !( !" # ) !" # ) INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT... INSERT... UPDATE... DELETE... SELECT...
Sharding i HARD !
Resharding i HARDER !
JOINS FORGET ABOUT IT data could be splitted into many
shards
ALTERS THINK TWICE ABOUT IT altering many shards could be
a nice challenge
fault tolerant multi-master sharding
SCALE WHEN NECESSARY NOT FOR FUN
Alternative TOOLS ?
None
None
None
None
None
Thanks! Any questions? @mariuszgil mariusz@mariuszgil.pl