Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Toronto MongoDB User Group

Avatar for Arthur Nogueira Neves Arthur Nogueira Neves
February 20, 2013
Technology
1
86

Toronto MongoDB User Group

Presetation given on Toronto MongoDB User group

Avatar for Arthur Nogueira Neves

Arthur Nogueira Neves

February 20, 2013
Tweet

More Decks by Arthur Nogueira Neves

See All by Arthur Nogueira Neves
Using multiple connections in ActiveRecord
Avatar for Arthur Nogueira Neves arthurnn
3
980
Rails talk - RubyLightningTalksTO
Avatar for Arthur Nogueira Neves arthurnn
0
100
Abril Pro Ruby
Avatar for Arthur Nogueira Neves arthurnn
0
360

Other Decks in Technology

See All in Technology
レガシー共有バッチ基盤への挑戦 - SREドリブンなリアーキテクチャリングの取り組み
Avatar for Konishi tatsuhiro tatsukoni
0
220
GitHub Issue Templates + Coding Agentで簡単みんなでIaC/Easy IaC for Everyone with GitHub Issue Templates + Coding Agent
Avatar for AEON aeonpeople
1
240
All About Sansan – for New Global Engineers
Avatar for Sansan, Inc. sansan33
PRO
1
1.4k
Sansan Engineering Unit 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
1
3.9k
Codex 5.3 と Opus 4.6 にコーポレートサイトを作らせてみた / Codex 5.3 vs Opus 4.6
Avatar for ama-ch ama_ch
0
160
OpenShiftでllm-dを動かそう！
Avatar for jpishikawa jpishikawa
0
110
Contract One Engineering Unit 紹介資料
Avatar for Sansan, Inc. sansan33
PRO
0
13k
SREじゃなかった僕らがenablingを通じて「SRE実践者」になるまでのリアル / SRE Kaigi 2026
Avatar for AEON aeonpeople
6
2.4k
登壇駆動学習のすすめ — CfPのネタの見つけ方と書くときに意識していること
Avatar for おおいし bicstone
3
110
15 years with Rails and DDD (AI Edition)
Avatar for Andrzej Krzywda andrzejkrzywda
0
200
Greatest Disaster Hits in Web Performance
Avatar for Estela Franco guaca
0
260
プロダクト成長を支える開発基盤とスケールに伴う課題
Avatar for yuu26 yuu26
4
1.3k

Featured

See All Featured
Navigating Algorithm Shifts & AI Overviews - #SMXNext
Avatar for Aleyda Solis aleyda
0
1.1k
Measuring Dark Social's Impact On Conversion and Attribution
Avatar for Stephen Akadiri stephenakadiri
1
130
Hiding What from Whom? A Critical Review of the History of Programming languages for Music
Avatar for Tomoya Matsuura tomoyanonymous
2
420
Claude Code のすすめ
Avatar for schroneko schroneko
67
210k
The AI Search Optimization Roadmap by Aleyda Solis
Avatar for Aleyda Solis aleyda
1
5.2k
Navigating the Design Leadership Dip - Product Design Week Design Leaders+ Conference 2024
Avatar for Andy Polaine apolaine
0
180
Intergalactic Javascript Robots from Outer Space
Avatar for Vicent Martí tanoku
273
27k
技術選定の審美眼（2025年版） / Understanding the Spiral of Technologies 2025 edition
Avatar for Takuto Wada twada
PRO
117
110k
ReactJS: Keep Simple. Everything can be a component!
Avatar for Pedro Nauck pedronauck
666
130k
How to Build an AI Search Optimization Roadmap - Criteria and Steps to Take #SEOIRL
Avatar for Aleyda Solis aleyda
1
1.9k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
Avatar for mongodb mongodb
49
9.9k
How Software Deployment tools have changed in the past 20 years
Avatar for Geshan Manandhar geshan
0
32k

Transcript

  1. MongoDB Toronto User group 1 Saturday, 23 February, 13

  2. Hi. 2 Saturday, 23 February, 13

  3. Hi. @arthurnn 2 Saturday, 23 February, 13

  4. Hi. @arthurnn 2 Saturday, 23 February, 13

  5. Hi. @arthurnn 2 Saturday, 23 February, 13

  6. 3 Saturday, 23 February, 13

  7. Tech 4 Saturday, 23 February, 13

  8. Tech 4 Saturday, 23 February, 13

  9. Tech 4 Saturday, 23 February, 13

  10. Tech 4 Saturday, 23 February, 13

  11. Tech 4 Saturday, 23 February, 13

  12. RPC Tech 4 Saturday, 23 February, 13

  13. Databases 5 Saturday, 23 February, 13

  14. Databases 5 Saturday, 23 February, 13

  15. Databases 5 Saturday, 23 February, 13

  16. Databases Memcached 5 Saturday, 23 February, 13

  17. Databases Memcached Sphinx -> ElasticSearch 5 Saturday, 23 February, 13

  18. + 6 Saturday, 23 February, 13

  19. + Activity Feed 6 Saturday, 23 February, 13

  20. + Activity Feed User Settings 6 Saturday, 23 February, 13

  21. + Activity Feed User Settings Push Notification 6 Saturday, 23

    February, 13

  22. + Activity Feed User Settings Push Notification Device track 6

    Saturday, 23 February, 13

  23. + Activity Feed User Settings Push Notification Device track Stats

    6 Saturday, 23 February, 13

  24. Servers 1.0 7 Saturday, 23 February, 13

  25. Servers 1.0 mongod primary 7 Saturday, 23 February, 13

  26. Servers 1.0 mongod primary mongod secondary 7 Saturday, 23 February,

    13

  27. Servers 1.0 mongod primary mongod secondary mongod secondary 7 Saturday,

    23 February, 13

  28. Servers 1.0 mongod primary mongod secondary mongod secondary mongod hidden

    7 Saturday, 23 February, 13

  29. Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod

    mongoc mongod mongoc mongod mongoc Main 8 Saturday, 23 February, 13

  30. Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod

    mongoc mongod mongoc mongod mongoc mongod Main 8 Saturday, 23 February, 13

  31. Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod

    mongoc mongod mongoc mongod mongoc mongod mongod Main 8 Saturday, 23 February, 13

  32. Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod

    mongoc mongod mongoc mongod mongoc mongod mongod mongod Main 8 Saturday, 23 February, 13

  33. Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod

    mongoc mongod mongoc mongod mongoc mongod mongod mongod Main 8 Saturday, 23 February, 13

  34. User Settings { "_id" : 123, "stats_page_empty" : true, "statistics_page_order"

    : [ 1, 4, 2, 3 ] } 9 Saturday, 23 February, 13

  35. Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"

    : 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications 10 Saturday, 23 February, 13

  36. Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"

    : 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"user_id": 1} 10 Saturday, 23 February, 13

  37. Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"

    : 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"user_id": 1} 10 Saturday, 23 February, 13

  38. Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"

    : 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} 10 Saturday, 23 February, 13

  39. Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"

    : 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} index: {"user_id": 1} 10 Saturday, 23 February, 13

  40. Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"

    : 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} index: {"user_id": 1, "updated_at": -1} index: {"user_id": 1} 10 Saturday, 23 February, 13

  41. Activity Feed - Flow 200M+ { "_id" : ObjectId(""), "aid"

    : 1127689, # Actor id "otype" : 9, # Object type for cast "v" : 7, # Verb of the action "oid" : 25000475, # Object id "uid" : 1127689 # User id } 11 Saturday, 23 February, 13

  42. db.ensureIndex ??? { aid: 1 } { aid: 1, _id:

    1 } { _id: 1, aid: 1 } { aid: 1, v: 1, _id: 1 } 12 Saturday, 23 February, 13

  43. Data access 13 Saturday, 23 February, 13

  44. Data access db.activities.find( aid: 22, v: {'$in': [2,3,4]}) .sort(_id: -1).limit(10)

    13 Saturday, 23 February, 13

  45. { aid: 1 } 14 Saturday, 23 February, 13

  46. { aid: 1, v:1 } { aid: 1, v:1, _id:

    -1 } 15 Saturday, 23 February, 13

  47. { aid: 1, _id: -1 } { aid: 1, _id:

    -1, v: 1 } 16 Saturday, 23 February, 13

  48. 17 Saturday, 23 February, 13

  49. db.activities.find( aid: {'$in': [1,10]}, v: {'$in': [2,3,4]}) .sort(_id: -1).limit(10) 17

    Saturday, 23 February, 13

  50. Shard key ??? 18 Saturday, 23 February, 13

  51. Shard key ??? { aid: 1, v: 1, _id: 1

    } 18 Saturday, 23 February, 13

  52. Mistakes on shard 19 Saturday, 23 February, 13

  53. Mistakes on shard Automatic shard 19 Saturday, 23 February, 13

  54. Mistakes on shard Automatic shard SSDs on Primaries 19 Saturday,

    23 February, 13

  55. After sharding? 20 Saturday, 23 February, 13

  56. After sharding? Good performance on writes 20 Saturday, 23 February,

    13

  57. After sharding? Good performance on User Flow reads Good performance

    on writes 20 Saturday, 23 February, 13

  58. After sharding? Good performance on User Flow reads Bad performance

    on Friends Flow reads Good performance on writes 20 Saturday, 23 February, 13

  59. Solutions: 21 Saturday, 23 February, 13

  60. Solutions: #1: denormalize 21 Saturday, 23 February, 13

  61. { "_id" : ObjectId(""), "user" : 123, # viewer "page"

    : 1, "feed" : [{activity}, {activity}] } Flow Feed by user 22 Saturday, 23 February, 13

  62. { "_id" : ObjectId(""), "user" : 123, # viewer "page"

    : 1, "feed" : [{activity}, {activity}] } Flow Feed by user Fan out on write 22 Saturday, 23 February, 13

  63. Solutions: #1: denormalize 23 Saturday, 23 February, 13

  64. Solutions: #1: denormalize #2: cache 23 Saturday, 23 February, 13

  65. key: user_id 24 Saturday, 23 February, 13

  66. key: user_id prepend activity 24 Saturday, 23 February, 13

  67. key: user_id prepend activity activity append 24 Saturday, 23 February,

    13

  68. key: user_id prepend activity activity append activity append 24 Saturday,

    23 February, 13

  69. Future Denormaziled solution. 25 Saturday, 23 February, 13

  70. Thank you 26 Saturday, 23 February, 13