Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
mongoDB at Intercom
Search
ciaranlee
May 23, 2012
Technology
3
380
mongoDB at Intercom
These slides are from a talk I gave at the first meeting of the Dublin mongoDB users group.
ciaranlee
May 23, 2012
Tweet
Share
More Decks by ciaranlee
See All by ciaranlee
Query Fingerprinting
ciaranlee
0
41
Other Decks in Technology
See All in Technology
サンドボックス技術でAI利活用を促進する
koh_naga
0
190
AI エージェントとはそもそも何か? - 技術背景から Amazon Bedrock AgentCore での実装まで- / AI Agent Unicorn Day 2025
hariby
4
1.2k
開発者を支える Internal Developer Portal のイマとコレカラ / To-day and To-morrow of Internal Developer Portals: Supporting Developers
aoto
PRO
1
400
バッチ処理で悩むバックエンドエンジニアに捧げるAWS Glue入門
diggymo
3
130
Skrub: machine-learning with dataframes
gaelvaroquaux
0
120
Snowflakeの生成AI機能を活用したデータ分析アプリの作成 〜Cortex AnalystとCortex Searchの活用とStreamlitアプリでの利用〜
nayuts
1
390
Bye-Bye Query Spaghetti: Write Queries You'll Actually Understand Using Pipelined SQL Syntax
tobiaslampertlotum
0
150
エラーとアクセシビリティ
schktjm
0
1.1k
Kubernetes における cgroup v2 でのOut-Of-Memory 問題の解決
pfn
PRO
0
460
自作JSエンジンに推しプロポーザルを実装したい!
sajikix
1
160
おやつは300円まで!の最適化を模索してみた
techtekt
PRO
0
280
ガチな登山用デバイスからこんにちは
halka
1
220
Featured
See All Featured
Become a Pro
speakerdeck
PRO
29
5.5k
The Art of Programming - Codeland 2020
erikaheidi
55
13k
Rebuilding a faster, lazier Slack
samanthasiow
83
9.2k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
How to Ace a Technical Interview
jacobian
279
23k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.8k
Automating Front-end Workflow
addyosmani
1370
200k
The Language of Interfaces
destraynor
161
25k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
111
20k
The Pragmatic Product Professional
lauravandoore
36
6.9k
Six Lessons from altMBA
skipperchong
28
4k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
Transcript
Wednesday 23 May 12
Ciaran Lee @ciaran_lee CTO of @intercom Wednesday 23 May 12
MongoDB at Intercom Wednesday 23 May 12
• What we do • Why we started using mongoDB
• Learnings • Problems • Hosting Overview Wednesday 23 May 12
CRM & Messaging Tool for Web Apps Wednesday 23 May
12
<script> var intercomSettings = { app_id:"tx2p130c", email:"
[email protected]
" }; </script> <script
src="https://api.intercom.io/api/ js/library.js"</script> Wednesday 23 May 12
Wednesday 23 May 12
Wednesday 23 May 12
• Rails • MySQL • MongoDB • Redis • Memcached
• Solr Technology Stack Wednesday 23 May 12
• Open beta • Installed on thousands of apps •
Tracking millions of end users Current status Wednesday 23 May 12
MySQL • Initially used for all data • We had
a built high traffic apps with it before (@exceptional) Wednesday 23 May 12
Wednesday 23 May 12
var intercomSettings = { app_id:"tx2p130c", email:"
[email protected]
", custom_data: { plan: "Small",
teammates: 4, revenue: 312, } }; Wednesday 23 May 12
http://en.wikipedia.org/wiki/Entity-attribute-value_model Entity–attribute–value pattern Wednesday 23 May 12
Wednesday 23 May 12
class User < ActiveRecord::Base serialize :custom_data end user.custom_data = {
:plan => "Small", :team_mates => 4, :revenue => 312, } Wednesday 23 May 12
• Reading and writing worked fine • Sorting or filtering
was a disaster Results Wednesday 23 May 12
Why MongoDB? • Handles arbitrary document structure • Flexible querying
• Can use an ActiveRecord compatible ORM (mongomapper) • Available as a heroku addon Wednesday 23 May 12
Users -> mongoDB Wednesday 23 May 12
Wednesday 23 May 12
The Mongo way • Different way of thinking to relational
DB • Often denormalised • Desired read scenarios influence design Wednesday 23 May 12
Tagging Wednesday 23 May 12
Tagging • Relational way is users + tags + join
table • Mongo way is a tag_ids array in the user record Wednesday 23 May 12
db.users.update( { app_id: 6, 'custom_data.apps': { $gte: 5 } },
{ $addToSet: { tag_ids:1234 } } ); Wednesday 23 May 12
db.users.update( { app_id: 6, 'custom_data.apps': { $gte: 5 } },
{ $addToSet: { tag_ids:1234 } } ); Wednesday 23 May 12
db.users.update( { app_id: 6, 'custom_data.apps': { $gte: 5 } },
{ $addToSet: { tag_ids:1234 } } ); Wednesday 23 May 12
Challenges Wednesday 23 May 12
Activity Stream Wednesday 23 May 12
Wednesday 23 May 12
Problem #1 • Querying users has gotten slower • Users
collection is 1/4 the size of activities • Activity stream growing too fast • Knocks the users collection out of RAM Wednesday 23 May 12
Solution • Move the activity stream over to a separate
database • We can use the more powerful database for complex user queries • sorting + filtering should be quicker Wednesday 23 May 12
Problem #2 • Can’t index every custom data attribute •
Intend to experiment with app specific user collections (+ index all custom data attrs on that app’s collection) Wednesday 23 May 12
Slowness • Always seems to involve disk access • Slow
performance of simple queries after big addToSet operations (> 200k users) Wednesday 23 May 12
Hosting • MongoHQ • Replica set - heroku addon •
ec2 cluster • Dedicated hardware with SSDs • Amazing service Wednesday 23 May 12
Thanks for listening! Wednesday 23 May 12
Wednesday 23 May 12