Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
MongoDB for Analytics - John Nunemaker, Ordered List
Search
mongodb
November 01, 2011
Technology
9
7.6k
MongoDB for Analytics - John Nunemaker, Ordered List
MongoChicago 2011
mongodb
November 01, 2011
Tweet
Share
More Decks by mongodb
See All by mongodb
NoSQL Now! 2012
mongodb
17
3.1k
MongoDB 2.2 At the Silicon Valley MongoDB User Group
mongodb
7
1.2k
Turning off the LAMP Hunter Loftis, Skookum Digital Works
mongodb
1
1.4k
Mobilize Your MongoDB! Developing iPhone and Android Apps in the Cloud Grant Shipley, Red Hat
mongodb
0
440
Beanstalk Data - MongoDB In Production Chris Siefken, CTO Beanstalk Data
mongodb
0
410
New LINQ support in C#/.NET driver Robert Stam, 10gen
mongodb
9
40k
Welcome and Keynote Aaron Heckman, 10gen
mongodb
0
390
Webinar Introduction to MongoDB's Java Driver
mongodb
0
1.1k
Webinar Intro to Schema Design
mongodb
4
1.7k
Other Decks in Technology
See All in Technology
Além do else! Categorizando Pokemóns com Pattern Matching no JavaScript
wmsbill
0
640
ゼロから始めるVue.jsコミュニティ貢献 / first-vuejs-community-contribution-link-and-motivation
lmi
1
130
Grafana x PagerDuty Better Together
jacopen
0
110
Google Cloud の AI を支える裏側のインフラを垣間見る!
maroon1st
0
360
GraphQL 成熟度モデルの紹介と、プロダクトに当てはめた事例 / GraphQL maturity model
mh4gf
7
1.4k
Gitlab本から学んだこと - そーだいなるプレイバック / gitlab-book
soudai
4
440
エンジニアのキャリアをちょっと楽しくする3本の軸/Three Pillars to Make an Engineer's Career More Enjoyable
kwappa
0
2.7k
Cloud Native Java with Spring Boot (CNCF Aarhus, April 2024)
thomasvitale
1
180
プロトタイピングによる不確実性の低減 / Reducing Uncertainty through Prototyping
ohbarye
5
390
地理空間データ可視化・解析・活用ソリューション Pacific Spatial Solutions (PSS)
pacificspatialsolutions
0
300
検証を通して見えてきたTiDBの性能特性
lycorptech_jp
PRO
7
3.8k
LLM開発・活用の舞台裏@2024.04.25
yushin_n
2
550
Featured
See All Featured
Code Review Best Practice
trishagee
55
15k
Web Components: a chance to create the future
zenorocha
305
41k
Optimising Largest Contentful Paint
csswizardry
8
2.4k
Into the Great Unknown - MozCon
thekraken
10
1k
Art, The Web, and Tiny UX
lynnandtonic
289
19k
Typedesign – Prime Four
hannesfritz
36
2.1k
The Invisible Side of Design
smashingmag
294
49k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
9
8.3k
Mobile First: as difficult as doing things right
swwweet
216
8.6k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
30
6k
5 minutes of I Can Smell Your CMS
philhawksworth
199
19k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
121
39k
Transcript
Ordered List John Nunemaker MongoChi 2011 October 18, 2011 MongoDB
for Analytics A loving conversation with @jnunemaker
Background As presented through interpretive dance
None
None
None
~1 month Of evenings and weekends
~4 dog years Since public launch
~6 tiny servers 2 web, 2 app, 2 db
~1-2 Million Page views per day
None
None
Implementation Imma show you how we do what we do
baby
Doing It Live No aggregate querying
get('/track.gif') do Hit.record(...) TrackGif end
class Hit def record site.atomic_update(site_updates) Resolution.record(self) Technology.record(self) Location.record(self) Referrer.record(self) Content.record(self)
Search.record(self) Notification.record(self) View.record(self) end end
class Resolution def record(hit) query = {'_id' => "..."} update
= {'$inc' => {}} update['$inc']["sx.#{hit.screenx}"] = 1 update['$inc']["bx.#{hit.browserx}"] = 1 update['$inc']["by.#{hit.browsery}"] = 1 collection(hit.created_on) .update(query, update, :upsert => true) end end end
Pros
Pros Space
Pros Space RAM
Pros Space RAM Reads
Pros Space RAM Reads Live
Cons
Cons Writes
Cons Writes Constraints
Cons Writes Constraints More Forethought
Cons Writes Constraints More Forethought No raw data
Time Frame Minute, hour, month, day, year, forever?
# of Variations One document vs many
Single Document Per Time Frame
None
{ "t" => 336381, "u" => 158951, "2011" => {
"02" => { "18" => { "t" => 9, "u" => 6 } } } }
{ '$inc' => { 't' => 1, 'u' => 1,
'2011.02.18.t' => 1, '2011.02.18.u' => 1, } }
Single Document For all ranges in time frame
None
{ "_id" =>"...:10", "bx" => { "320" => 85, "480"
=> 318, "800" => 1938, "1024" => 5033, "1280" => 6288, "1440" => 2323, "1600" => 3817, "2000" => 137 }, "by" => { "480" => 2205, "600" => 7359,
"600" => 7359, "768" => 4515, "900" => 3833, "1024"
=> 2026 }, "sx" => { "320" => 191, "480" => 179, "800" => 195, "1024" => 1059, "1280" => 5861, "1440" => 3533, "1600" => 7675, "2000" => 1279 } }
{ '$inc' => { 'sx.1440' => 1, 'bx.1280' => 1,
'by.768' => 1, } }
Many Documents Search terms, content, referrers...
None
[ { "_id" => "<oid>:<hash>", "t" => "ruby class variables",
"sid" => BSON::ObjectId('<oid>'), "v" => 352 }, { "_id" => "<oid>:<hash>", "t" => "ruby unless", "sid" => BSON::ObjectId('<oid>'), "v" => 347 }, ]
Writes {'_id' => "#{site_id}:#{hash}"}
Reads [['sid', 1], ['v', -1]]
Growth The best laid plans of mice and men
Partition Hot Data Currently using collections for time frames
Bigger, Faster Server More CPU, RAM, Disk Space
Users Sites Content Referrers Terms Engines Resolutions Locations Users Sites
Content Referrers Terms Engines Resolutions Locations
Partition by Function Spread writes across a few servers
Users Sites Content Referrers Terms Engines Resolutions Locations
Partition by Server Spread writes across a ton of servers,
way down the road, not worried yet
Ordered List Thank you!
[email protected]
John Nunemaker MongoChi 2011 October
18, 2011 @jnunemaker