Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scaling MongoDB | Sergey Gavruk
Search
Minsk MongoDB User Group
October 04, 2012
Programming
2
170
Scaling MongoDB | Sergey Gavruk
Sergey Gavruk
Meetup #7
Minsk MongoDB User Group
October 04, 2012
Tweet
Share
More Decks by Minsk MongoDB User Group
See All by Minsk MongoDB User Group
MongoDB by Chef | Yauhen Artsiukhou
bymongo
0
120
MongoDB at IronMQ | Alexander Kolesen
bymongo
0
840
Event sourcing + CQRS + MongoDB | Alex Shkor
bymongo
1
640
How it works. Indexes | Kirill Duborenko
bymongo
5
270
Aggregation Framework | Mikhail Burtylev
bymongo
1
79
MongoDB 2.2: Release update + Roadmap | Alvin Richards
bymongo
1
83
Meetup#6 Intro | Alex Litvinok
bymongo
1
43
Deploying MongoDB on Amazon WS | Michael Karpitsky
bymongo
2
110
About the problem of DBMS choice & what to do if you have gone the wrong way | Roman Bugaev
bymongo
3
110
Other Decks in Programming
See All in Programming
Semantic search with Django and pgvector
pauloxnet
0
240
Rails と人魚の話/rails-and-mermaid
sanfrecce_osaka
0
100
SwiftUIで使いやすいToastの作り方 / How to build a Toast system which is easy to use in SwiftUI
lovee
3
110
Front-end application development, Symfony-style(s)
dunglas
2
1.9k
デザインシステムで Tailwind CSSとCSS in JSに分散投資をしたら良かった話
fsubal
18
4.9k
コードレビューで学ぶ!Kotlinオブジェクト指向デザインパターン
akkie76
2
180
Elm Form Validation
bkuhlmann
0
510
雑に思考を整理する技術と効能
konifar
57
26k
SwiftUI Performance 不要なViewの再描画と更新を抑える
bigamitiongit
1
160
pixivアプリでマルチモジュールを実現するまで
gatosyocora
1
130
Rubyでたのしむクリエイティブコーディング/Enjoy Creative coding with Ruby
chobishiba
1
170
DMMプラットフォームがTiDB Cloudを採用した背景
pospome
8
3.7k
Featured
See All Featured
GraphQLの誤解/rethinking-graphql
sonatard
50
9.2k
Bash Introduction
62gerente
604
210k
Adopting Sorbet at Scale
ufuk
67
8.6k
Writing Fast Ruby
sferik
620
60k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
273
13k
The Cost Of JavaScript in 2023
addyosmani
14
3.8k
Stop Working from a Prison Cell
hatefulcrawdad
266
19k
The Illustrated Children's Guide to Kubernetes
chrisshort
29
46k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
226
51k
No one is an island. Learnings from fostering a developers community.
thoeni
14
2.1k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
352
28k
Building Flexible Design Systems
yeseniaperezcruz
318
37k
Transcript
Scaling Sergey Gavruk @gavruk
Scaling • Ver2cal • Horizontal • By op2miza2on – Op2mize
your queries, schema, indexes – Tune you file system – Choose right disks
Share nothing architecture • Michael Stonebraker First
implementa2on in 1983 Google calls this “Sharding”
Sharding goals • App doesn’t know about clusters • Cluster
should always be available for reads and writes • Cluster should grow easily
Sharding features • Range-‐based data par22oning • Automa2c
data volume distribu2on • Transparent query rou2ng
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”)
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”) [“d”, “g”) 100 GB 500 GB 100 GB 100 GB 100 GB 400 GB 200 GB 100 GB
[“a”, “g”) [“g”, “m”) [“m”, “s”) [“s”,
“z”)
[“a”, “d”) 300 [“g”, “k”) 300 [“m”,
“s”) [“s”, “z”) 400 GB 400 GB 100 GB 100 GB [“d”, “g”) 100 [“k”, “m”) 100
None
Chunks -‐∞ +∞
Chunks -‐∞ +∞
null Numbers Strings Objects Arrays
binary data ObjectIds booleans Dates regular expressions smaller bigger
Balancing mongos balancer Config server Config
server Config server Shard 1 Shard 2
Balancer goals • keep data distributed • minimize the amount
of data transfered
Balancing mongos balancer Config server Config
server Config server Shard 1 Shard 2 Move chunk X to shard 2
Balancing Number of chunks Migra:on threshold <
20 2 21-‐80 4 80+ 8
Balancing schedule db.seangs.update({ _id : "balancer" },
{ $set : { ac2veWindow : { start : "23:00", stop : "6:00" } } }, true )
Routed Request mongos Shard 1 Shard 2
Shard 3
mongos Shard 1 Shard 2 Shard 3
Request without shard key
Without shard key + sor2ng mongos Shard 1
Shard 2 Shard 3
Consider the shard cluster if: • Data exceeds the
storage capacity of a single node • Size of working set will soon exceed your RAM • Large amount of writes
Restric2ons • You cannot update a shard key
• You must use a shard key for a single update • Index on shard key
Ideal shard key • easily divisible. • will
distribute write opera2ons among the cluster • will make it possible for the mongos to return most query opera2ons directly from a single specific mongod instance
Choosing a shard key {
_id: "1", user_id: "2345652221", date_2me: "2012-‐10-‐04“, tweet_text: “Hello world” } Reliability
Choosing a shard key Ascending { TimeStamp:
12355232, … }
Choosing a shard key Low-‐cardinality key {
Con:nent: “Europe”, Name: “Tom”, … } Zip code?
Demo
Any questions? mailto:
[email protected]