Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Toronto MongoDB User Group
Search
Arthur Nogueira Neves
February 20, 2013
Technology
1
85
Toronto MongoDB User Group
Presetation given on Toronto MongoDB User group
Arthur Nogueira Neves
February 20, 2013
Tweet
Share
More Decks by Arthur Nogueira Neves
See All by Arthur Nogueira Neves
Using multiple connections in ActiveRecord
arthurnn
3
960
Rails talk - RubyLightningTalksTO
arthurnn
0
94
Abril Pro Ruby
arthurnn
0
330
Other Decks in Technology
See All in Technology
american airlines®️ USA Contact Numbers: Complete 2025 Support Guide
supportflight
1
110
Tokyo_reInforce_2025_recap_iam_access_analyzer
hiashisan
0
180
スタートアップに選択肢を 〜生成AIを活用したセカンダリー事業への挑戦〜
nstock
0
170
United airlines®️ USA Contact Numbers: Complete 2025 Support Guide
unitedflyhelp
0
310
KubeCon + CloudNativeCon Japan 2025 Recap
ren510dev
1
380
生成AI時代の開発組織・技術・プロセス 〜 ログラスの挑戦と考察 〜
itohiro73
1
460
20250705 Headlamp: 專注可擴展性的 Kubernetes 用戶界面
pichuang
0
270
第4回Snowflake 金融ユーザー会 Snowflake summit recap
tamaoki
1
280
CDKTFについてざっくり理解する!!~CloudFormationからCDKTFへ変換するツールも作ってみた~
masakiokuda
1
140
面倒な作業はAIにおまかせ。Flutter開発をスマートに効率化
ruideengineer
0
250
Core Audio tapを使ったリアルタイム音声処理のお話
yuta0306
0
190
「クラウドコスト絶対削減」を支える技術—FinOpsを超えた徹底的なクラウドコスト削減の実践論
delta_tech
4
170
Featured
See All Featured
Six Lessons from altMBA
skipperchong
28
3.9k
Bash Introduction
62gerente
613
210k
Rails Girls Zürich Keynote
gr2m
95
14k
Practical Orchestrator
shlominoach
189
11k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
138
34k
Building Better People: How to give real-time feedback that sticks.
wjessup
367
19k
Adopting Sorbet at Scale
ufuk
77
9.5k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
48
2.9k
How to Ace a Technical Interview
jacobian
278
23k
Why Our Code Smells
bkeepers
PRO
336
57k
Transcript
MongoDB Toronto User group 1 Saturday, 23 February, 13
Hi. 2 Saturday, 23 February, 13
Hi. @arthurnn 2 Saturday, 23 February, 13
Hi. @arthurnn 2 Saturday, 23 February, 13
Hi. @arthurnn 2 Saturday, 23 February, 13
3 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
RPC Tech 4 Saturday, 23 February, 13
Databases 5 Saturday, 23 February, 13
Databases 5 Saturday, 23 February, 13
Databases 5 Saturday, 23 February, 13
Databases Memcached 5 Saturday, 23 February, 13
Databases Memcached Sphinx -> ElasticSearch 5 Saturday, 23 February, 13
+ 6 Saturday, 23 February, 13
+ Activity Feed 6 Saturday, 23 February, 13
+ Activity Feed User Settings 6 Saturday, 23 February, 13
+ Activity Feed User Settings Push Notification 6 Saturday, 23
February, 13
+ Activity Feed User Settings Push Notification Device track 6
Saturday, 23 February, 13
+ Activity Feed User Settings Push Notification Device track Stats
6 Saturday, 23 February, 13
Servers 1.0 7 Saturday, 23 February, 13
Servers 1.0 mongod primary 7 Saturday, 23 February, 13
Servers 1.0 mongod primary mongod secondary 7 Saturday, 23 February,
13
Servers 1.0 mongod primary mongod secondary mongod secondary 7 Saturday,
23 February, 13
Servers 1.0 mongod primary mongod secondary mongod secondary mongod hidden
7 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod mongod Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod mongod mongod Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod mongod mongod Main 8 Saturday, 23 February, 13
User Settings { "_id" : 123, "stats_page_empty" : true, "statistics_page_order"
: [ 1, 4, 2, 3 ] } 9 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} index: {"user_id": 1, "updated_at": -1} index: {"user_id": 1} 10 Saturday, 23 February, 13
Activity Feed - Flow 200M+ { "_id" : ObjectId(""), "aid"
: 1127689, # Actor id "otype" : 9, # Object type for cast "v" : 7, # Verb of the action "oid" : 25000475, # Object id "uid" : 1127689 # User id } 11 Saturday, 23 February, 13
db.ensureIndex ??? { aid: 1 } { aid: 1, _id:
1 } { _id: 1, aid: 1 } { aid: 1, v: 1, _id: 1 } 12 Saturday, 23 February, 13
Data access 13 Saturday, 23 February, 13
Data access db.activities.find( aid: 22, v: {'$in': [2,3,4]}) .sort(_id: -1).limit(10)
13 Saturday, 23 February, 13
{ aid: 1 } 14 Saturday, 23 February, 13
{ aid: 1, v:1 } { aid: 1, v:1, _id:
-1 } 15 Saturday, 23 February, 13
{ aid: 1, _id: -1 } { aid: 1, _id:
-1, v: 1 } 16 Saturday, 23 February, 13
17 Saturday, 23 February, 13
db.activities.find( aid: {'$in': [1,10]}, v: {'$in': [2,3,4]}) .sort(_id: -1).limit(10) 17
Saturday, 23 February, 13
Shard key ??? 18 Saturday, 23 February, 13
Shard key ??? { aid: 1, v: 1, _id: 1
} 18 Saturday, 23 February, 13
Mistakes on shard 19 Saturday, 23 February, 13
Mistakes on shard Automatic shard 19 Saturday, 23 February, 13
Mistakes on shard Automatic shard SSDs on Primaries 19 Saturday,
23 February, 13
After sharding? 20 Saturday, 23 February, 13
After sharding? Good performance on writes 20 Saturday, 23 February,
13
After sharding? Good performance on User Flow reads Good performance
on writes 20 Saturday, 23 February, 13
After sharding? Good performance on User Flow reads Bad performance
on Friends Flow reads Good performance on writes 20 Saturday, 23 February, 13
Solutions: 21 Saturday, 23 February, 13
Solutions: #1: denormalize 21 Saturday, 23 February, 13
{ "_id" : ObjectId(""), "user" : 123, # viewer "page"
: 1, "feed" : [{activity}, {activity}] } Flow Feed by user 22 Saturday, 23 February, 13
{ "_id" : ObjectId(""), "user" : 123, # viewer "page"
: 1, "feed" : [{activity}, {activity}] } Flow Feed by user Fan out on write 22 Saturday, 23 February, 13
Solutions: #1: denormalize 23 Saturday, 23 February, 13
Solutions: #1: denormalize #2: cache 23 Saturday, 23 February, 13
key: user_id 24 Saturday, 23 February, 13
key: user_id prepend activity 24 Saturday, 23 February, 13
key: user_id prepend activity activity append 24 Saturday, 23 February,
13
key: user_id prepend activity activity append activity append 24 Saturday,
23 February, 13
Future Denormaziled solution. 25 Saturday, 23 February, 13
Thank you 26 Saturday, 23 February, 13