Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Toronto MongoDB User Group
Search
Arthur Nogueira Neves
February 20, 2013
Technology
1
85
Toronto MongoDB User Group
Presetation given on Toronto MongoDB User group
Arthur Nogueira Neves
February 20, 2013
Tweet
Share
More Decks by Arthur Nogueira Neves
See All by Arthur Nogueira Neves
Using multiple connections in ActiveRecord
arthurnn
3
960
Rails talk - RubyLightningTalksTO
arthurnn
0
94
Abril Pro Ruby
arthurnn
0
330
Other Decks in Technology
See All in Technology
強化されたAmazon Location Serviceによる新機能と開発者体験
dayjournal
2
210
AWS CDK 実践的アプローチ N選 / aws-cdk-practical-approaches
gotok365
6
740
変化する開発、進化する体系時代に適応するソフトウェアエンジニアの知識と考え方(JaSST'25 Kansai)
mizunori
1
210
フィンテック養成勉強会#54
finengine
0
180
生成AI活用の組織格差を解消する 〜ビジネス職のCursor導入が開発効率に与えた好循環〜 / Closing the Organizational Gap in AI Adoption
upamune
2
1.2k
Абьюзим random_bytes(). Фёдор Кулаков, разработчик Lamoda Tech
lamodatech
0
340
PHPでWebブラウザのレンダリングエンジンを実装する
dip_tech
PRO
0
200
Fabric + Databricks 2025.6 の最新情報ピックアップ
ryomaru0825
1
140
データプラットフォーム技術におけるメダリオンアーキテクチャという考え方/DataPlatformWithMedallionArchitecture
smdmts
5
630
How Community Opened Global Doors
hiroramos4
PRO
1
120
あなたの声を届けよう! 女性エンジニア登壇の意義とアウトプット実践ガイド #wttjp / Call for Your Voice
kondoyuko
4
440
2年でここまで成長!AWSで育てたAI Slack botの軌跡
iwamot
PRO
4
700
Featured
See All Featured
How to Think Like a Performance Engineer
csswizardry
24
1.7k
Gamification - CAS2011
davidbonilla
81
5.3k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
Building Applications with DynamoDB
mza
95
6.5k
Raft: Consensus for Rubyists
vanstee
140
7k
We Have a Design System, Now What?
morganepeng
53
7.7k
A better future with KSS
kneath
239
17k
Thoughts on Productivity
jonyablonski
69
4.7k
Build The Right Thing And Hit Your Dates
maggiecrowley
36
2.8k
Scaling GitHub
holman
459
140k
Why You Should Never Use an ORM
jnunemaker
PRO
57
9.4k
Making the Leap to Tech Lead
cromwellryan
134
9.3k
Transcript
MongoDB Toronto User group 1 Saturday, 23 February, 13
Hi. 2 Saturday, 23 February, 13
Hi. @arthurnn 2 Saturday, 23 February, 13
Hi. @arthurnn 2 Saturday, 23 February, 13
Hi. @arthurnn 2 Saturday, 23 February, 13
3 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
RPC Tech 4 Saturday, 23 February, 13
Databases 5 Saturday, 23 February, 13
Databases 5 Saturday, 23 February, 13
Databases 5 Saturday, 23 February, 13
Databases Memcached 5 Saturday, 23 February, 13
Databases Memcached Sphinx -> ElasticSearch 5 Saturday, 23 February, 13
+ 6 Saturday, 23 February, 13
+ Activity Feed 6 Saturday, 23 February, 13
+ Activity Feed User Settings 6 Saturday, 23 February, 13
+ Activity Feed User Settings Push Notification 6 Saturday, 23
February, 13
+ Activity Feed User Settings Push Notification Device track 6
Saturday, 23 February, 13
+ Activity Feed User Settings Push Notification Device track Stats
6 Saturday, 23 February, 13
Servers 1.0 7 Saturday, 23 February, 13
Servers 1.0 mongod primary 7 Saturday, 23 February, 13
Servers 1.0 mongod primary mongod secondary 7 Saturday, 23 February,
13
Servers 1.0 mongod primary mongod secondary mongod secondary 7 Saturday,
23 February, 13
Servers 1.0 mongod primary mongod secondary mongod secondary mongod hidden
7 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod mongod Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod mongod mongod Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod mongod mongod Main 8 Saturday, 23 February, 13
User Settings { "_id" : 123, "stats_page_empty" : true, "statistics_page_order"
: [ 1, 4, 2, 3 ] } 9 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} index: {"user_id": 1, "updated_at": -1} index: {"user_id": 1} 10 Saturday, 23 February, 13
Activity Feed - Flow 200M+ { "_id" : ObjectId(""), "aid"
: 1127689, # Actor id "otype" : 9, # Object type for cast "v" : 7, # Verb of the action "oid" : 25000475, # Object id "uid" : 1127689 # User id } 11 Saturday, 23 February, 13
db.ensureIndex ??? { aid: 1 } { aid: 1, _id:
1 } { _id: 1, aid: 1 } { aid: 1, v: 1, _id: 1 } 12 Saturday, 23 February, 13
Data access 13 Saturday, 23 February, 13
Data access db.activities.find( aid: 22, v: {'$in': [2,3,4]}) .sort(_id: -1).limit(10)
13 Saturday, 23 February, 13
{ aid: 1 } 14 Saturday, 23 February, 13
{ aid: 1, v:1 } { aid: 1, v:1, _id:
-1 } 15 Saturday, 23 February, 13
{ aid: 1, _id: -1 } { aid: 1, _id:
-1, v: 1 } 16 Saturday, 23 February, 13
17 Saturday, 23 February, 13
db.activities.find( aid: {'$in': [1,10]}, v: {'$in': [2,3,4]}) .sort(_id: -1).limit(10) 17
Saturday, 23 February, 13
Shard key ??? 18 Saturday, 23 February, 13
Shard key ??? { aid: 1, v: 1, _id: 1
} 18 Saturday, 23 February, 13
Mistakes on shard 19 Saturday, 23 February, 13
Mistakes on shard Automatic shard 19 Saturday, 23 February, 13
Mistakes on shard Automatic shard SSDs on Primaries 19 Saturday,
23 February, 13
After sharding? 20 Saturday, 23 February, 13
After sharding? Good performance on writes 20 Saturday, 23 February,
13
After sharding? Good performance on User Flow reads Good performance
on writes 20 Saturday, 23 February, 13
After sharding? Good performance on User Flow reads Bad performance
on Friends Flow reads Good performance on writes 20 Saturday, 23 February, 13
Solutions: 21 Saturday, 23 February, 13
Solutions: #1: denormalize 21 Saturday, 23 February, 13
{ "_id" : ObjectId(""), "user" : 123, # viewer "page"
: 1, "feed" : [{activity}, {activity}] } Flow Feed by user 22 Saturday, 23 February, 13
{ "_id" : ObjectId(""), "user" : 123, # viewer "page"
: 1, "feed" : [{activity}, {activity}] } Flow Feed by user Fan out on write 22 Saturday, 23 February, 13
Solutions: #1: denormalize 23 Saturday, 23 February, 13
Solutions: #1: denormalize #2: cache 23 Saturday, 23 February, 13
key: user_id 24 Saturday, 23 February, 13
key: user_id prepend activity 24 Saturday, 23 February, 13
key: user_id prepend activity activity append 24 Saturday, 23 February,
13
key: user_id prepend activity activity append activity append 24 Saturday,
23 February, 13
Future Denormaziled solution. 25 Saturday, 23 February, 13
Thank you 26 Saturday, 23 February, 13