Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Toronto MongoDB User Group
Search
Arthur Nogueira Neves
February 20, 2013
Technology
1
85
Toronto MongoDB User Group
Presetation given on Toronto MongoDB User group
Arthur Nogueira Neves
February 20, 2013
Tweet
Share
More Decks by Arthur Nogueira Neves
See All by Arthur Nogueira Neves
Using multiple connections in ActiveRecord
arthurnn
3
960
Rails talk - RubyLightningTalksTO
arthurnn
0
94
Abril Pro Ruby
arthurnn
0
340
Other Decks in Technology
See All in Technology
Evolution on AI Agent and Beyond - AGI への道のりと、シンギュラリティの3つのシナリオ
masayamoriofficial
0
190
あなたの知らない OneDrive
murachiakira
0
240
実践データベース設計 ①データベース設計概論
recruitengineers
PRO
4
430
そのコンポーネント、サーバー?クライアント?App Router開発のモヤモヤを可視化する補助輪
makotot
4
620
トヨタ生産方式(TPS)入門
recruitengineers
PRO
4
380
現場が抱える様々な問題は “組織設計上” の問題によって生じていることがある / Team-oriented Organization Design 20250827
mtx2s
6
1.5k
我々は雰囲気で仕事をしている / How can we do vibe coding as well
naospon
2
220
Yahoo!ニュースにおけるソフトウェア開発
lycorptech_jp
PRO
0
380
ZOZOTOWNフロントエンドにおけるディレクトリの分割戦略
zozotech
PRO
18
5.5k
JavaScript 研修
recruitengineers
PRO
4
340
認知戦の理解と、市民としての対抗策
hogehuga
0
370
攻撃と防御で実践するプロダクトセキュリティ演習~導入パート~
recruitengineers
PRO
3
340
Featured
See All Featured
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3.1k
Statistics for Hackers
jakevdp
799
220k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
Why You Should Never Use an ORM
jnunemaker
PRO
59
9.5k
Thoughts on Productivity
jonyablonski
69
4.8k
Why Our Code Smells
bkeepers
PRO
338
57k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
185
54k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
36
2.5k
Mobile First: as difficult as doing things right
swwweet
223
9.9k
Transcript
MongoDB Toronto User group 1 Saturday, 23 February, 13
Hi. 2 Saturday, 23 February, 13
Hi. @arthurnn 2 Saturday, 23 February, 13
Hi. @arthurnn 2 Saturday, 23 February, 13
Hi. @arthurnn 2 Saturday, 23 February, 13
3 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
Tech 4 Saturday, 23 February, 13
RPC Tech 4 Saturday, 23 February, 13
Databases 5 Saturday, 23 February, 13
Databases 5 Saturday, 23 February, 13
Databases 5 Saturday, 23 February, 13
Databases Memcached 5 Saturday, 23 February, 13
Databases Memcached Sphinx -> ElasticSearch 5 Saturday, 23 February, 13
+ 6 Saturday, 23 February, 13
+ Activity Feed 6 Saturday, 23 February, 13
+ Activity Feed User Settings 6 Saturday, 23 February, 13
+ Activity Feed User Settings Push Notification 6 Saturday, 23
February, 13
+ Activity Feed User Settings Push Notification Device track 6
Saturday, 23 February, 13
+ Activity Feed User Settings Push Notification Device track Stats
6 Saturday, 23 February, 13
Servers 1.0 7 Saturday, 23 February, 13
Servers 1.0 mongod primary 7 Saturday, 23 February, 13
Servers 1.0 mongod primary mongod secondary 7 Saturday, 23 February,
13
Servers 1.0 mongod primary mongod secondary mongod secondary 7 Saturday,
23 February, 13
Servers 1.0 mongod primary mongod secondary mongod secondary mongod hidden
7 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod mongod Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod mongod mongod Main 8 Saturday, 23 February, 13
Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod
mongoc mongod mongoc mongod mongoc mongod mongod mongod Main 8 Saturday, 23 February, 13
User Settings { "_id" : 123, "stats_page_empty" : true, "statistics_page_order"
: [ 1, 4, 2, 3 ] } 9 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} index: {"user_id": 1} 10 Saturday, 23 February, 13
Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"
: 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} index: {"user_id": 1, "updated_at": -1} index: {"user_id": 1} 10 Saturday, 23 February, 13
Activity Feed - Flow 200M+ { "_id" : ObjectId(""), "aid"
: 1127689, # Actor id "otype" : 9, # Object type for cast "v" : 7, # Verb of the action "oid" : 25000475, # Object id "uid" : 1127689 # User id } 11 Saturday, 23 February, 13
db.ensureIndex ??? { aid: 1 } { aid: 1, _id:
1 } { _id: 1, aid: 1 } { aid: 1, v: 1, _id: 1 } 12 Saturday, 23 February, 13
Data access 13 Saturday, 23 February, 13
Data access db.activities.find( aid: 22, v: {'$in': [2,3,4]}) .sort(_id: -1).limit(10)
13 Saturday, 23 February, 13
{ aid: 1 } 14 Saturday, 23 February, 13
{ aid: 1, v:1 } { aid: 1, v:1, _id:
-1 } 15 Saturday, 23 February, 13
{ aid: 1, _id: -1 } { aid: 1, _id:
-1, v: 1 } 16 Saturday, 23 February, 13
17 Saturday, 23 February, 13
db.activities.find( aid: {'$in': [1,10]}, v: {'$in': [2,3,4]}) .sort(_id: -1).limit(10) 17
Saturday, 23 February, 13
Shard key ??? 18 Saturday, 23 February, 13
Shard key ??? { aid: 1, v: 1, _id: 1
} 18 Saturday, 23 February, 13
Mistakes on shard 19 Saturday, 23 February, 13
Mistakes on shard Automatic shard 19 Saturday, 23 February, 13
Mistakes on shard Automatic shard SSDs on Primaries 19 Saturday,
23 February, 13
After sharding? 20 Saturday, 23 February, 13
After sharding? Good performance on writes 20 Saturday, 23 February,
13
After sharding? Good performance on User Flow reads Good performance
on writes 20 Saturday, 23 February, 13
After sharding? Good performance on User Flow reads Bad performance
on Friends Flow reads Good performance on writes 20 Saturday, 23 February, 13
Solutions: 21 Saturday, 23 February, 13
Solutions: #1: denormalize 21 Saturday, 23 February, 13
{ "_id" : ObjectId(""), "user" : 123, # viewer "page"
: 1, "feed" : [{activity}, {activity}] } Flow Feed by user 22 Saturday, 23 February, 13
{ "_id" : ObjectId(""), "user" : 123, # viewer "page"
: 1, "feed" : [{activity}, {activity}] } Flow Feed by user Fan out on write 22 Saturday, 23 February, 13
Solutions: #1: denormalize 23 Saturday, 23 February, 13
Solutions: #1: denormalize #2: cache 23 Saturday, 23 February, 13
key: user_id 24 Saturday, 23 February, 13
key: user_id prepend activity 24 Saturday, 23 February, 13
key: user_id prepend activity activity append 24 Saturday, 23 February,
13
key: user_id prepend activity activity append activity append 24 Saturday,
23 February, 13
Future Denormaziled solution. 25 Saturday, 23 February, 13
Thank you 26 Saturday, 23 February, 13