Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
mongoDB at Intercom
Search
ciaranlee
May 23, 2012
Technology
3
380
mongoDB at Intercom
These slides are from a talk I gave at the first meeting of the Dublin mongoDB users group.
ciaranlee
May 23, 2012
Tweet
Share
More Decks by ciaranlee
See All by ciaranlee
Query Fingerprinting
ciaranlee
0
41
Other Decks in Technology
See All in Technology
Autonomous Database - Dedicated 技術詳細 / adb-d_technical_detail_jp
oracle4engineer
PRO
4
10k
「どこから読む?」コードとカルチャーに最速で馴染むための実践ガイド
zozotech
PRO
0
450
[ JAWS-UG 東京 CommunityBuilders Night #2 ]SlackとAmazon Q Developerで 運用効率化を模索する
sh_fk2
3
430
COVESA VSSによる車両データモデルの標準化とAWS IoT FleetWiseの活用
osawa
1
290
Rustから学ぶ 非同期処理の仕組み
skanehira
1
140
Automating Web Accessibility Testing with AI Agents
maminami373
0
1.3k
DDD集約とサービスコンテキスト境界との関係性
pandayumi
3
280
まずはマネコンでちゃちゃっと作ってから、それをCDKにしてみよか。
yamada_r
2
110
「Linux」という言葉が指すもの
sat
PRO
4
130
いま注目のAIエージェントを作ってみよう
supermarimobros
0
290
MCPで変わる Amebaデザインシステム「Spindle」の開発
spindle
PRO
3
3.3k
AIエージェント開発用SDKとローカルLLMをLINE Botと組み合わせてみた / LINEを使ったLT大会 #14
you
PRO
0
120
Featured
See All Featured
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
8
520
The Cult of Friendly URLs
andyhume
79
6.6k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
139
34k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
32
1.6k
Optimising Largest Contentful Paint
csswizardry
37
3.4k
Rebuilding a faster, lazier Slack
samanthasiow
83
9.2k
YesSQL, Process and Tooling at Scale
rocio
173
14k
Building a Modern Day E-commerce SEO Strategy
aleyda
43
7.6k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
285
13k
Java REST API Framework Comparison - PWX 2021
mraible
33
8.8k
Done Done
chrislema
185
16k
Transcript
Wednesday 23 May 12
Ciaran Lee @ciaran_lee CTO of @intercom Wednesday 23 May 12
MongoDB at Intercom Wednesday 23 May 12
• What we do • Why we started using mongoDB
• Learnings • Problems • Hosting Overview Wednesday 23 May 12
CRM & Messaging Tool for Web Apps Wednesday 23 May
12
<script> var intercomSettings = { app_id:"tx2p130c", email:"
[email protected]
" }; </script> <script
src="https://api.intercom.io/api/ js/library.js"</script> Wednesday 23 May 12
Wednesday 23 May 12
Wednesday 23 May 12
• Rails • MySQL • MongoDB • Redis • Memcached
• Solr Technology Stack Wednesday 23 May 12
• Open beta • Installed on thousands of apps •
Tracking millions of end users Current status Wednesday 23 May 12
MySQL • Initially used for all data • We had
a built high traffic apps with it before (@exceptional) Wednesday 23 May 12
Wednesday 23 May 12
var intercomSettings = { app_id:"tx2p130c", email:"
[email protected]
", custom_data: { plan: "Small",
teammates: 4, revenue: 312, } }; Wednesday 23 May 12
http://en.wikipedia.org/wiki/Entity-attribute-value_model Entity–attribute–value pattern Wednesday 23 May 12
Wednesday 23 May 12
class User < ActiveRecord::Base serialize :custom_data end user.custom_data = {
:plan => "Small", :team_mates => 4, :revenue => 312, } Wednesday 23 May 12
• Reading and writing worked fine • Sorting or filtering
was a disaster Results Wednesday 23 May 12
Why MongoDB? • Handles arbitrary document structure • Flexible querying
• Can use an ActiveRecord compatible ORM (mongomapper) • Available as a heroku addon Wednesday 23 May 12
Users -> mongoDB Wednesday 23 May 12
Wednesday 23 May 12
The Mongo way • Different way of thinking to relational
DB • Often denormalised • Desired read scenarios influence design Wednesday 23 May 12
Tagging Wednesday 23 May 12
Tagging • Relational way is users + tags + join
table • Mongo way is a tag_ids array in the user record Wednesday 23 May 12
db.users.update( { app_id: 6, 'custom_data.apps': { $gte: 5 } },
{ $addToSet: { tag_ids:1234 } } ); Wednesday 23 May 12
db.users.update( { app_id: 6, 'custom_data.apps': { $gte: 5 } },
{ $addToSet: { tag_ids:1234 } } ); Wednesday 23 May 12
db.users.update( { app_id: 6, 'custom_data.apps': { $gte: 5 } },
{ $addToSet: { tag_ids:1234 } } ); Wednesday 23 May 12
Challenges Wednesday 23 May 12
Activity Stream Wednesday 23 May 12
Wednesday 23 May 12
Problem #1 • Querying users has gotten slower • Users
collection is 1/4 the size of activities • Activity stream growing too fast • Knocks the users collection out of RAM Wednesday 23 May 12
Solution • Move the activity stream over to a separate
database • We can use the more powerful database for complex user queries • sorting + filtering should be quicker Wednesday 23 May 12
Problem #2 • Can’t index every custom data attribute •
Intend to experiment with app specific user collections (+ index all custom data attrs on that app’s collection) Wednesday 23 May 12
Slowness • Always seems to involve disk access • Slow
performance of simple queries after big addToSet operations (> 200k users) Wednesday 23 May 12
Hosting • MongoHQ • Replica set - heroku addon •
ec2 cluster • Dedicated hardware with SSDs • Amazing service Wednesday 23 May 12
Thanks for listening! Wednesday 23 May 12
Wednesday 23 May 12