Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Toronto MongoDB User Group

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →
Avatar for Arthur Nogueira Neves Arthur Nogueira Neves
February 20, 2013
Technology
87
1

Toronto MongoDB User Group

Presetation given on Toronto MongoDB User group

Avatar for Arthur Nogueira Neves

Arthur Nogueira Neves

February 20, 2013

More Decks by Arthur Nogueira Neves

See All by Arthur Nogueira Neves
Using multiple connections in ActiveRecord
Avatar for Arthur Nogueira Neves arthurnn
3
1k
Rails talk - RubyLightningTalksTO
Avatar for Arthur Nogueira Neves arthurnn
0
110
Abril Pro Ruby
Avatar for Arthur Nogueira Neves arthurnn
0
380

Other Decks in Technology

See All in Technology
Spring Boot における​ AOT Cache 活用テクニックと​ 起動時間改善事例​
Avatar for NTTドコモソリューションズ　Java担当 ntt_dsol_java
0
200
はじめてのDatadog
Avatar for KairiM kairim0
0
260
JJUG CCC 2026 Spring　AI時代の開発こそ標準化を武器に！ ― 方式・プロセス・プラットフォームの標準化
Avatar for WatanabeShun s27watanabe
2
670
「嘘をつくテスト」の失敗例から学ぶ 良いテストコード #frontend_phpcon_do
Avatar for asumikam asumikam
0
140
エンジニアは生成AIと どのように向き合うべきか？ ことばの意味という観点から
Avatar for Hitomi Yanaka verypluming
3
320
もりもり新機能を一挙紹介！ AgentCoreに入門して、AWS上にAIエージェントを構築しよう
Avatar for みのるん minorun365
PRO
6
650
Javaで学ぶSOLID原則
Avatar for Negima negima
1
260
ルールやカスタム機能、どう使う？理想の出力を引き出すために今知りたいIBM Bob 5つの機能
Avatar for mu (Mizuho Uehara) muehara
0
190
TROCCOで始めるクラウドコストを民主化するためのFinOps
Avatar for tk3fftk tk3fftk
3
540
個人の発見を、組織の知恵に 〜生成AI活用を"探索"から"組織の仕組み"へ〜
Avatar for KintoTech_Dev kintotechdev
2
490
20260528_生成AIを専属DSに_Howの次にすべきことを考える
Avatar for NobuakiOshiro doradora09
PRO
0
280
Platform Engineering as a Product: Criteria for Improvement and Multi-Tenant Design
Avatar for Kumo Ishikawa kumorn5s
0
470

Featured

See All Featured
Getting science done with accelerated Python computing platforms
Avatar for Jacob Tomlinson jacobtomlinson
2
220
JavaScript: Past, Present, and Future - NDC Porto 2020
Avatar for David Neal reverentgeek
52
5.9k
Chasing Engaging Ingredients in Design
Avatar for Sebastian Deterding codingconduct
0
200
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
Avatar for Michelle Schulp Hunt marktimemedia
31
2.8k
Docker and Python
Avatar for Tania Allard trallard
47
3.9k
How to audit for AI Accessibility on your Front & Back End
Avatar for davetheseo davetheseo
0
400
Hiding What from Whom? A Critical Review of the History of Programming languages for Music
Avatar for Tomoya Matsuura tomoyanonymous
2
840
DevOps and Value Stream Thinking: Enabling flow, efficiency and business value
Avatar for Helen Beal helenjbeal
1
210
Between Models and Reality
Avatar for mayunak mayunak
4
320
Why Mistakes Are the Best Teachers: Turning Failure into a Pathway for Growth
Avatar for Umar Saidu Auna auna
0
150
The Success of Rails: Ensuring Growth for the Next 100 Years
Avatar for Eileen M. Uchitelle eileencodes
47
8.1k
The Psychology of Web Performance [Beyond Tellerrand 2023]
Avatar for Tammy Everts tammyeverts
49
3.5k

Transcript

  1. MongoDB Toronto User group 1 Saturday, 23 February, 13

  2. Hi. 2 Saturday, 23 February, 13

  3. Hi. @arthurnn 2 Saturday, 23 February, 13

  4. Hi. @arthurnn 2 Saturday, 23 February, 13

  5. Hi. @arthurnn 2 Saturday, 23 February, 13

  6. 3 Saturday, 23 February, 13

  7. Tech 4 Saturday, 23 February, 13

  8. Tech 4 Saturday, 23 February, 13

  9. Tech 4 Saturday, 23 February, 13

  10. Tech 4 Saturday, 23 February, 13

  11. Tech 4 Saturday, 23 February, 13

  12. RPC Tech 4 Saturday, 23 February, 13

  13. Databases 5 Saturday, 23 February, 13

  14. Databases 5 Saturday, 23 February, 13

  15. Databases 5 Saturday, 23 February, 13

  16. Databases Memcached 5 Saturday, 23 February, 13

  17. Databases Memcached Sphinx -> ElasticSearch 5 Saturday, 23 February, 13

  18. + 6 Saturday, 23 February, 13

  19. + Activity Feed 6 Saturday, 23 February, 13

  20. + Activity Feed User Settings 6 Saturday, 23 February, 13

  21. + Activity Feed User Settings Push Notification 6 Saturday, 23

    February, 13

  22. + Activity Feed User Settings Push Notification Device track 6

    Saturday, 23 February, 13

  23. + Activity Feed User Settings Push Notification Device track Stats

    6 Saturday, 23 February, 13

  24. Servers 1.0 7 Saturday, 23 February, 13

  25. Servers 1.0 mongod primary 7 Saturday, 23 February, 13

  26. Servers 1.0 mongod primary mongod secondary 7 Saturday, 23 February,

    13

  27. Servers 1.0 mongod primary mongod secondary mongod secondary 7 Saturday,

    23 February, 13

  28. Servers 1.0 mongod primary mongod secondary mongod secondary mongod hidden

    7 Saturday, 23 February, 13

  29. Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod

    mongoc mongod mongoc mongod mongoc Main 8 Saturday, 23 February, 13

  30. Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod

    mongoc mongod mongoc mongod mongoc mongod Main 8 Saturday, 23 February, 13

  31. Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod

    mongoc mongod mongoc mongod mongoc mongod mongod Main 8 Saturday, 23 February, 13

  32. Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod

    mongoc mongod mongoc mongod mongoc mongod mongod mongod Main 8 Saturday, 23 February, 13

  33. Servers 2.0 mongod mongod mongod mongod mongod mongod Primary mongod

    mongoc mongod mongoc mongod mongoc mongod mongod mongod Main 8 Saturday, 23 February, 13

  34. User Settings { "_id" : 123, "stats_page_empty" : true, "statistics_page_order"

    : [ 1, 4, 2, 3 ] } 9 Saturday, 23 February, 13

  35. Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"

    : 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications 10 Saturday, 23 February, 13

  36. Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"

    : 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"user_id": 1} 10 Saturday, 23 February, 13

  37. Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"

    : 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"user_id": 1} 10 Saturday, 23 February, 13

  38. Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"

    : 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} 10 Saturday, 23 February, 13

  39. Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"

    : 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} index: {"user_id": 1} 10 Saturday, 23 February, 13

  40. Push notifications { "_id" : ObjectId("4fe60f72a3a6a5654b000001"), "status" : 24, "user_id"

    : 1, "photo_id" : 8026403, "actors" : [ 71 ], "action_id" : 0, "updated_at" : ISODate("2012-06-23T18:48:18Z"), "created_at" : ISODate("2012-06-23T18:48:18Z") } { "_id" : ObjectId("4ffc9eb55328f538ad00002d"), "permissions" : [ 0, 1, 5, 10 ], "type" : 1, "user_id" : 357534, "token" : "xxxx xxxx xxxx" } db.mobile_devices db.mobile_notifications index: {"token": 1}, {sparse: true} index: {"registration_id": 1}, {sparse: true} index: {"user_id": 1} index: {"user_id": 1, "updated_at": -1} index: {"user_id": 1} 10 Saturday, 23 February, 13

  41. Activity Feed - Flow 200M+ { "_id" : ObjectId(""), "aid"

    : 1127689, # Actor id "otype" : 9, # Object type for cast "v" : 7, # Verb of the action "oid" : 25000475, # Object id "uid" : 1127689 # User id } 11 Saturday, 23 February, 13

  42. db.ensureIndex ??? { aid: 1 } { aid: 1, _id:

    1 } { _id: 1, aid: 1 } { aid: 1, v: 1, _id: 1 } 12 Saturday, 23 February, 13

  43. Data access 13 Saturday, 23 February, 13

  44. Data access db.activities.find( aid: 22, v: {'$in': [2,3,4]}) .sort(_id: -1).limit(10)

    13 Saturday, 23 February, 13

  45. { aid: 1 } 14 Saturday, 23 February, 13

  46. { aid: 1, v:1 } { aid: 1, v:1, _id:

    -1 } 15 Saturday, 23 February, 13

  47. { aid: 1, _id: -1 } { aid: 1, _id:

    -1, v: 1 } 16 Saturday, 23 February, 13

  48. 17 Saturday, 23 February, 13

  49. db.activities.find( aid: {'$in': [1,10]}, v: {'$in': [2,3,4]}) .sort(_id: -1).limit(10) 17

    Saturday, 23 February, 13

  50. Shard key ??? 18 Saturday, 23 February, 13

  51. Shard key ??? { aid: 1, v: 1, _id: 1

    } 18 Saturday, 23 February, 13

  52. Mistakes on shard 19 Saturday, 23 February, 13

  53. Mistakes on shard Automatic shard 19 Saturday, 23 February, 13

  54. Mistakes on shard Automatic shard SSDs on Primaries 19 Saturday,

    23 February, 13

  55. After sharding? 20 Saturday, 23 February, 13

  56. After sharding? Good performance on writes 20 Saturday, 23 February,

    13

  57. After sharding? Good performance on User Flow reads Good performance

    on writes 20 Saturday, 23 February, 13

  58. After sharding? Good performance on User Flow reads Bad performance

    on Friends Flow reads Good performance on writes 20 Saturday, 23 February, 13

  59. Solutions: 21 Saturday, 23 February, 13

  60. Solutions: #1: denormalize 21 Saturday, 23 February, 13

  61. { "_id" : ObjectId(""), "user" : 123, # viewer "page"

    : 1, "feed" : [{activity}, {activity}] } Flow Feed by user 22 Saturday, 23 February, 13

  62. { "_id" : ObjectId(""), "user" : 123, # viewer "page"

    : 1, "feed" : [{activity}, {activity}] } Flow Feed by user Fan out on write 22 Saturday, 23 February, 13

  63. Solutions: #1: denormalize 23 Saturday, 23 February, 13

  64. Solutions: #1: denormalize #2: cache 23 Saturday, 23 February, 13

  65. key: user_id 24 Saturday, 23 February, 13

  66. key: user_id prepend activity 24 Saturday, 23 February, 13

  67. key: user_id prepend activity activity append 24 Saturday, 23 February,

    13

  68. key: user_id prepend activity activity append activity append 24 Saturday,

    23 February, 13

  69. Future Denormaziled solution. 25 Saturday, 23 February, 13

  70. Thank you 26 Saturday, 23 February, 13