Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scaling MongoDB | Sergey Gavruk
Search
Minsk MongoDB User Group
October 04, 2012
Programming
2
180
Scaling MongoDB | Sergey Gavruk
Sergey Gavruk
Meetup #7
Minsk MongoDB User Group
October 04, 2012
Tweet
Share
More Decks by Minsk MongoDB User Group
See All by Minsk MongoDB User Group
MongoDB by Chef | Yauhen Artsiukhou
bymongo
0
130
MongoDB at IronMQ | Alexander Kolesen
bymongo
0
850
Event sourcing + CQRS + MongoDB | Alex Shkor
bymongo
1
650
How it works. Indexes | Kirill Duborenko
bymongo
5
280
Aggregation Framework | Mikhail Burtylev
bymongo
1
100
MongoDB 2.2: Release update + Roadmap | Alvin Richards
bymongo
1
100
Meetup#6 Intro | Alex Litvinok
bymongo
1
49
Deploying MongoDB on Amazon WS | Michael Karpitsky
bymongo
2
110
About the problem of DBMS choice & what to do if you have gone the wrong way | Roman Bugaev
bymongo
3
120
Other Decks in Programming
See All in Programming
なんでRustの環境構築してないのにRust製のツールが動くの? / Why Do Rust-Based Tools Run Without a Rust Environment?
ssssota
14
46k
実践Claude Code:20の失敗から学ぶAIペアプログラミング
takedatakashi
18
8.7k
monorepo の Go テストをはやくした〜い!~最小の依存解決への道のり~ / faster-testing-of-monorepos
convto
2
560
React Nativeならぬ"Vue Native"が実現するかも?_新世代マルチプラットフォーム開発フレームワークのLynxとLynxのVue.js対応を追ってみよう_Vue Lynx
yut0naga1_fa
2
1.5k
エンジニアインターン「Treasure」とHonoの2年、そして未来へ / Our Journey with Hono Two Years at Treasure and Beyond
carta_engineering
0
430
CSC509 Lecture 06
javiergs
PRO
0
270
Ktorで簡単AIアプリケーション
tsukakei
0
110
3年ぶりにコードを書いた元CTOが Claude Codeと30分でMVPを作った話
maikokojima
0
650
Developer Joy - The New Paradigm
hollycummins
1
370
Building, Deploying, and Monitoring Ruby Web Applications with Falcon (Kaigi on Rails 2025)
ioquatix
4
2.5k
他言語経験者が Golangci-lint を最初のコーディングメンターにした話 / How Golangci-lint Became My First Coding Mentor: A Story from a Polyglot Programmer
uma31
0
430
kiroとCodexで最高のSpec駆動開発を!!数時間で web3ネイティブなミニゲームを作ってみたよ!
mashharuki
0
920
Featured
See All Featured
Unsuck your backbone
ammeep
671
58k
Java REST API Framework Comparison - PWX 2021
mraible
34
8.9k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
359
30k
Music & Morning Musume
bryan
46
6.9k
Building Flexible Design Systems
yeseniaperezcruz
329
39k
Building an army of robots
kneath
305
46k
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.6k
Fireside Chat
paigeccino
41
3.7k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
253
22k
A Tale of Four Properties
chriscoyier
161
23k
Designing for humans not robots
tammielis
254
26k
The Straight Up "How To Draw Better" Workshop
denniskardys
238
140k
Transcript
Scaling Sergey Gavruk @gavruk
Scaling • Ver2cal • Horizontal • By op2miza2on – Op2mize
your queries, schema, indexes – Tune you file system – Choose right disks
Share nothing architecture • Michael Stonebraker First
implementa2on in 1983 Google calls this “Sharding”
Sharding goals • App doesn’t know about clusters • Cluster
should always be available for reads and writes • Cluster should grow easily
Sharding features • Range-‐based data par22oning • Automa2c
data volume distribu2on • Transparent query rou2ng
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”)
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”) [“d”, “g”) 100 GB 500 GB 100 GB 100 GB 100 GB 400 GB 200 GB 100 GB
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”)
[“a”, “d”) 300 [“g”, “k”) 300 [“m”,
“s”) [“s”, “z”) 400 GB 400 GB 100 GB 100 GB [“d”, “g”) 100 [“k”, “m”) 100
None
Chunks -‐∞ +∞
Chunks -‐∞ +∞
null Numbers Strings Objects Arrays
binary data ObjectIds booleans Dates regular expressions smaller bigger
Balancing mongos balancer Config server Config
server Config server Shard 1 Shard 2
Balancer goals • keep data distributed • minimize the amount
of data transfered
Balancing mongos balancer Config server Config
server Config server Shard 1 Shard 2 Move chunk X to shard 2
Balancing Number of chunks Migra:on threshold <
20 2 21-‐80 4 80+ 8
Balancing schedule db.seangs.update({ _id : "balancer" },
{ $set : { ac2veWindow : { start : "23:00", stop : "6:00" } } }, true )
Routed Request mongos Shard 1 Shard 2
Shard 3
mongos Shard 1 Shard 2 Shard 3
Request without shard key
Without shard key + sor2ng mongos Shard 1
Shard 2 Shard 3
Consider the shard cluster if: • Data exceeds the
storage capacity of a single node • Size of working set will soon exceed your RAM • Large amount of writes
Restric2ons • You cannot update a shard key
• You must use a shard key for a single update • Index on shard key
Ideal shard key • easily divisible. • will
distribute write opera2ons among the cluster • will make it possible for the mongos to return most query opera2ons directly from a single specific mongod instance
Choosing a shard key {
_id: "1", user_id: "2345652221", date_2me: "2012-‐10-‐04“, tweet_text: “Hello world” } Reliability
Choosing a shard key Ascending { TimeStamp:
12355232, … }
Choosing a shard key Low-‐cardinality key {
Con:nent: “Europe”, Name: “Tom”, … } Zip code?
Demo
Any questions? mailto:
[email protected]