Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Toronto MongoDB User Group
Search
Arthur Nogueira Neves
February 20, 2013
Technology
1
85
Toronto MongoDB User Group
Presetation given on Toronto MongoDB User group
Arthur Nogueira Neves
February 20, 2013
Tweet
Share
More Decks by Arthur Nogueira Neves
See All by Arthur Nogueira Neves
Using multiple connections in ActiveRecord
arthurnn
3
960
Rails talk - RubyLightningTalksTO
arthurnn
0
94
Abril Pro Ruby
arthurnn
0
340
Other Decks in Technology
See All in Technology
MCP認可の現在地と自律型エージェント対応に向けた課題 / MCP Authorization Today and Challenges to Support Autonomous Agents
yokawasa
5
1.8k
マルチモーダル基盤モデルに基づく動画と音の解析技術
lycorptech_jp
PRO
4
510
AIのグローバルトレンド 2025 / ai global trend 2025
kyonmm
PRO
1
120
VLMサービスを用いた請求書データ化検証 / SaaSxML_Session_1
sansan_randd
0
220
Claude CodeでKiroの仕様駆動開発を実現させるには...
gotalab555
3
890
【2025 Japan AWS Jr. Champions Ignition】点から線、線から面へ〜僕たちが起こすコラボレーション・ムーブメント〜
amixedcolor
1
120
【CEDEC2025】ブランド力アップのためのコンテンツマーケティング~ゲーム会社における情報資産の活かし方~
cygames
PRO
0
230
バクラクによるコーポレート業務の自動運転 #BetAIDay
layerx
PRO
1
840
2025-07-31: GitHub Copilot Agent mode at Vibe Coding Cafe (15min)
chomado
2
370
ビジネス文書に特化した基盤モデル開発 / SaaSxML_Session_2
sansan_randd
0
260
LIFF CLIとngrokを使ったLIFF/LINEミニアプリのお手軽実機確認
diggymo
0
230
Claude Codeは仕様駆動の夢を見ない
gotalab555
13
2.3k
Featured
See All Featured
Balancing Empowerment & Direction
lara
1
530
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
283
13k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
656
60k
Embracing the Ebb and Flow
colly
86
4.8k
Bash Introduction
62gerente
614
210k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
3k
Faster Mobile Websites
deanohume
308
31k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3.1k
How STYLIGHT went responsive
nonsquared
100
5.7k
Documentation Writing (for coders)
carmenintech
73
5k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
Transcript
MongoDB Toronto User group 1 Saturday, 23 February, 13
Hi. 2 Saturday, 23 February, 13
Hi. @arthurnn 2 Saturday, 23 February, 13
Hi. @arthurnn 2 Saturday, 23 February, 13
Hi. @arthurnn 2 Saturday, 23 February, 13
3 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
RPC Tech 4 Saturday, 23 February, 13
Databases 5 Saturday, 23 February, 13
Databases 5 Saturday, 23 February, 13
Databases 5 Saturday, 23 February, 13
Databases Memcached 5 Saturday, 23 February, 13
Databases Memcached Sphinx -> ElasticSearch 5 Saturday, 23 February, 13
+ 6 Saturday, 23 February, 13
+ Activity Feed 6 Saturday, 23 February, 13
+ Activity Feed User Settings 6 Saturday, 23 February, 13
+ Activity Feed User Settings Push Notification 6 Saturday, 23
February, 13
+ Activity Feed User Settings Push Notification Device track 6
Saturday, 23 February, 13
+ Activity Feed User Settings Push Notification Device track Stats
6 Saturday, 23 February, 13
Servers 1.0 7 Saturday, 23 February, 13
Servers 1.0 mongod primary 7 Saturday, 23 February, 13
Servers 1.0 mongod primary mongod secondary 7 Saturday, 23 February,
13
Servers 1.0 mongod primary mongod secondary mongod secondary 7 Saturday,
23 February, 13
Servers 1.0 mongod primary mongod secondary mongod secondary mongod hidden
7 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod mongod Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod mongod mongod Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod mongod mongod Main 8 Saturday, 23 February, 13
User Settings { "_id" : 123, "stats_page_empty" : true, "statistics_page_order"
: [ 1, 4, 2, 3 ] } 9 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} index: {"user_id": 1, "updated_at": -1} index: {"user_id": 1} 10 Saturday, 23 February, 13
Activity Feed - Flow 200M+ { "_id" : ObjectId(""), "aid"
: 1127689, # Actor id "otype" : 9, # Object type for cast "v" : 7, # Verb of the action "oid" : 25000475, # Object id "uid" : 1127689 # User id } 11 Saturday, 23 February, 13
db.ensureIndex ??? { aid: 1 } { aid: 1, _id:
1 } { _id: 1, aid: 1 } { aid: 1, v: 1, _id: 1 } 12 Saturday, 23 February, 13
Data access 13 Saturday, 23 February, 13
Data access db.activities.find( aid: 22, v: {'$in': [2,3,4]}) .sort(_id: -1).limit(10)
13 Saturday, 23 February, 13
{ aid: 1 } 14 Saturday, 23 February, 13
{ aid: 1, v:1 } { aid: 1, v:1, _id:
-1 } 15 Saturday, 23 February, 13
{ aid: 1, _id: -1 } { aid: 1, _id:
-1, v: 1 } 16 Saturday, 23 February, 13
17 Saturday, 23 February, 13
db.activities.find( aid: {'$in': [1,10]}, v: {'$in': [2,3,4]}) .sort(_id: -1).limit(10) 17
Saturday, 23 February, 13
Shard key ??? 18 Saturday, 23 February, 13
Shard key ??? { aid: 1, v: 1, _id: 1
} 18 Saturday, 23 February, 13
Mistakes on shard 19 Saturday, 23 February, 13
Mistakes on shard Automatic shard 19 Saturday, 23 February, 13
Mistakes on shard Automatic shard SSDs on Primaries 19 Saturday,
23 February, 13
After sharding? 20 Saturday, 23 February, 13
After sharding? Good performance on writes 20 Saturday, 23 February,
13
After sharding? Good performance on User Flow reads Good performance
on writes 20 Saturday, 23 February, 13
After sharding? Good performance on User Flow reads Bad performance
on Friends Flow reads Good performance on writes 20 Saturday, 23 February, 13
Solutions: 21 Saturday, 23 February, 13
Solutions: #1: denormalize 21 Saturday, 23 February, 13
{ "_id" : ObjectId(""), "user" : 123, # viewer "page"
: 1, "feed" : [{activity}, {activity}] } Flow Feed by user 22 Saturday, 23 February, 13
{ "_id" : ObjectId(""), "user" : 123, # viewer "page"
: 1, "feed" : [{activity}, {activity}] } Flow Feed by user Fan out on write 22 Saturday, 23 February, 13
Solutions: #1: denormalize 23 Saturday, 23 February, 13
Solutions: #1: denormalize #2: cache 23 Saturday, 23 February, 13
key: user_id 24 Saturday, 23 February, 13
key: user_id prepend activity 24 Saturday, 23 February, 13
key: user_id prepend activity activity append 24 Saturday, 23 February,
13
key: user_id prepend activity activity append activity append 24 Saturday,
23 February, 13
Future Denormaziled solution. 25 Saturday, 23 February, 13
Thank you 26 Saturday, 23 February, 13