MongoDB Analytics for Online Advertising at Magnetic

The Magnetic data marketplace empowers advertisers and publishers to use search data as the key indicator of intent and retarget campaigns to the most relevant audience online. With more than 270 million search profiles, Magnetic significantly lifts the value of media and improves campaign performance.

This webinar will examine how Magnetic uses MongoDB for their online advertising platform. Mark Weiss, the Senior Director of Engineering at Magnetic, will discuss how the company drastically simplified its architecture by taking advantage of MongoDB's rich combination of features. Topics will include:

How Magnetic takes advantage of MongoDB's flexible data model
Leveraging atomic counters for real-time analytics on impression counts and frequency caps
Using map/reduce to offer a new tier of near-real-time reporting

Transcript

1. MongoDB for Online Advertising Mark Weiss- Sr. Dir. of Engineering
2. OVERVIEW • About our Business and Systems • Magnetic and Typical Ad Tech Challenges • Benefits of MongoDB • Future Evolution: Mongo on the Roadmap
3. ABOUT MAGNETIC Magnetic is Search Retargeting
4. 4 !"#$%&'#()* !"#$%&'()*+ +',)-#.*/0'$-1*/(.231*/(0.-* ,-.'%"-/ 0-1-2')+ •  3-.4567-+&.78.9:1+7.1.:-7-1'+ •  ;&.4.<4-+91=".)'"%&'%"-+ •  >-$9.+#8679?.6#1+ •  >%4684-+91@-1'#"A+)#%"&-)+(&#7;&#"-+BCDE+ -F&G.1:-+91@-1'#"AE+-'&H*+ •  ,%445)-"@9&-+I+)-4=5)-"@9&-+)#4%6#1)+ •  ;-"@-)+<9449#1)+#=+978"-))9#1)+7#1'G4A+ 425%-).-*6-,$-)7'#()* !"#$%&'()*+ 6--58(053* ,-.'%"-/ 0-1-2')+ •  J-AK#"$+:-1-".6#1+'##4+ •  L&&-))+'#+MCD+79449#1+%19N%-+%)-"+ )-."&G-)+ •  L<949'A+'#+7.'&G+;O>+P-AK#"$+49)'+ •  Q.":-'+&#78-6'#"+P-AK#"$)+ 9'7'*+')',-$-)7*:;'<(0$* !"#$%&'()*+ +',)-#.*/%-;5* ,-.'%"-/ 0-1-2')+ •  O.)A+'#+%)-+RS+ •  ;9784-+91'-:".6#1+ •  !#K-"=%4+-1#%:G+'#+G.1$4-+T+<9449#1+ 7#1'G4A+)-."&G-)++ •  !."'1-")+K9'G+4-.$91:+$.'.+8"#@9$-")+ •  O1.<4-)+8."'1-")+'#+7#1-6?-+$.'.+
5. ABOUT SEARCH RETARGETING • Time 0 • User searches • Targeting based on the search term • Times 1 .. N • Show ads over time based on targeting
6. ABOUT SEARCH RETARGETING 6 !"#$%&'()%*+%" ( ,-&#+.$&/% ( 0--1+'$&/% ( 231145"67+'" ( !"1845"67+'"( ( 09(!"67"65 ( :$6*"&%*()%*+%" ( 0%$1;&'5 ( !"$6'<(=$>$ ( • !"#$%&'!#()*(#+! ! • !,-./$%*+-(!+'-//*()!#()*(#+! ! • !0-.$*(!.$($)#%+ ! • !"*1#+!2*1'!3+#%!+#$%&'#+ ! 4567589:;!"9<:,7"! 0=8=!>:96407:"! '/#!'/6"(>/-(?@A(-6/-"6&"5(
7. MAGNETIC TECHNICAL CHALLENGES • High throughput, low latency • Business logic per request -- richer logic means better targeting • Distributed servers • Scaling for growth • Keeping data in sync and using and delivering real-time data • Continuous improvement
8. ONLINE AD TECHNICAL CHALLENGES • High throughput, low latency • Business logic per request • Distributed servers • Scaling for growth • Keeping data in sync and using and delivering real-time data • Continuous improvement
9. OVERVIEW • About our Business and Systems • Magnetic and Typical Ad Tech Challenges • Benefits of MongoDB • Future Evolution: Mongo on the Roadmap
10. OVERVIEW • About our Business and Systems • Magnetic and Typical Ad Tech Challenges • Benefits of MongoDB • Future Evolution: Mongo on the Roadmap
11. OVERVIEW • About our Business and Systems • Magnetic and Typical Ad Tech Challenges • Benefits of MongoDB • Future Evolution: Mongo on the Roadmap
12. OVERVIEW • About our Business and Systems • Magnetic and Typical Ad Tech Challenges • Benefits of MongoDB • Future Evolution: Mongo on the Roadmap
13. • Simplified architecture • Faster performance • Richer data model • Real time updates and all data in synch • Near real-time reporting 13
14. BENEFIT: SIMPLIFIED ARCHITECTURE User modifies data Generate flat file of current state MySQL Push to file server Download file to each ad server File server Load into APC on each server Targeting & ad servers
15. BENEFIT: SIMPLIFIED ARCHITECTURE MySQL Mongo Mongo Mongo Mongo Mongo Mongo Targeting & ad servers User modifies data
16. BENEFIT: FASTER PERFORMANCE AND RICHER DATA MODEL • APC is a key-value store • Mongo is document-based • Mongo is 40% faster than APC for us
17. BENEFIT: FASTER PERFORMANCE db.ads.findOne({“ad_group_id” : ObjectId (“47cc67093475061e3d95369d”), “state”:”active”, “size”:”banner”, “geo”:”US”})
18. BENEFIT: REAL TIME UPDATES User modifies data Generate flat file of current state MySQL Push to file server Download file to each ad server File server Load into APC on each server Targeting & ad servers
19. BENEFIT: REAL TIME UPDATES 19
20. BENEFIT: ALL DATA IN SYNC Local APC Local APC Local APC Local APC Mongo Cluster
21. db.ad_facts.update({'year' : 2011, 'month' : 6, 'day' : 12, 'advertiser_id' : ObjectId (“47cc67093474061e3d95369d”)...}, {‘$inc’ : {“impressions” : 1}}, true); db.ad_facts.update({'year' : 2011, 'month' : 6, 'day' : 12, 'advertiser_id' : ObjectId (“47cc67093474061e3d95369d”)...}, {‘$inc’ : {“clicks” : 1}}, true); db.ad_facts.update({'year' : 2011, 'month' : 6, 'day' : 12, 'advertiser_id' : ObjectId (“47cc67093474061e3d95369d”)...}, {‘$inc’ : {“view_thru_conversions” : 1}}, true); BENEFIT: NEAR REAL TIME REPORTING
22. {“month”:6, “year” : 2011,”day” : 23 “ad_id”: ObjectId(“47cc67093475061e3d95369d”), “ad_group_id”, ObjectId (“47cc67093474061e3d95369d”), “campaign_id” : ObjectId (“47cc67093475061e3d95368d”), “advertiser_id” : ObjectId (“47cc67093475061e3d95339d”), “impressions”, 1983272, “clicks” : 33498, “view_thru_conversions” : 3231, “click_thru_conversions” : 878} BENEFIT: NEAR REAL TIME REPORTING
23. map_function = function() { emit( { ad_group_id: this.ad_group_id, month: this.month, year: this.year, advertiser: this.advertiser_id }, { impressions: this.impressions, clicks: this.clicks, view_thru: this.view_thru_conversions, click_thru: this.click_thru_conversions } ); emit( { campaign_id: this.campaign_id, month: this.month, year: this.year, advertiser: this.advertiser_id }, { impressions: this.impressions, clicks: this.clicks, view_thru: this.view_thru_conversions, click_thru: this.click_thru_conversions } ); emit( { ad_id: this.ad_id, month: this.month, year: this.year, advertiser: this.advertiser_id }, { impressions: this.impressions, clicks: this.clicks, view_thru: this.view_thru_conversions, click_thru: this.click_thru_conversions } ); emit( { advertiser: this.advertiser_id, month: this.month, year: this.year }, { impressions: this.impressions, clicks: this.clicks, view_thru: this.view_thru_conversions, click_thru: this.click_thru_conversions } ) } BENEFIT: NEAR REAL TIME REPORTING
24. reduce_function = function(key, vals) { var ret = { impressions: 0, clicks: 0, conversions: 0, view_thru: 0, click_thru: 0 }; for(var i = 0; i < vals.length; i++) { ret.impressions += vals[i].impressions; ret.clicks += vals[i].clicks ; ret.view_thru += vals[i].view_thru; ret.click_thru += vals[i].click_thru; } return ret; } BENEFIT: NEAR REAL TIME REPORTING
25. final_function = function() { db.result_fact.find().forEach( function(record) { if (record._id.hasOwnProperty('ad_group_id')) db.month_ad_group_metric.update( {ad_group_id: record._id.ad_group_id, year: record._id.year, month: record._id.month, advertiser: record._id.advertiser}, {"$set" : {impressions: record.value.impressions, clicks: record.value.clicks, view_thru: record.value.view_thru, click_thru: record.value.click_thru}}, {upsert : true}) ; if (record._id.hasOwnProperty('ad_id')) db.month_ad_metric.update( {ad_id: record._id.ad_id, year: record._id.year, month: record._id.month, advertiser: record._id.advertiser }, {"$set" : {impressions: record.value.impressions,clicks: record.value.clicks, view_thru: record.value.view_thru, click_thru: record.value.click_thru}}, {upsert : true}) ; if (record._id.hasOwnProperty('campaign_id')) db.month_campaign_metric.update( {campaign_id: record._id.campaign_id, year: record._id.year, month: record._id.month ,advertiser: record._id.advertiser}, {"$set" : {impressions: record.value.impressions, clicks: record.value.clicks, view_thru: record.value.view_thru, click_thru: record.value.click_thru}}, {upsert : true}) ; if (record._id.hasOwnProperty('advertiser')) db.month_advertiser_metric.update( {advertiser: record._id.advertiser, year: record._id.year, month: record._id.month, advertiser: record._id.advertiser}, {"$set" : {impressions: record.value.impressions, clicks: record.value.clicks, view_thru: record.value.view_thru, click_thru: record.value.click_thru}}, {upsert : true}) ; }) } BENEFIT: NEAR REAL TIME REPORTING
26. OVERVIEW • About our Business and Systems • Magnetic and Typical Ad Tech Challenges • Benefits of MongoDB • Future Evolution: Mongo on the Roadmap
27. MONGODB ON THE ROADMAP • About our Business and Systems • Magnetic and Typical Ad Tech Challenges • Benefits of MongoDB • Future Evolution: Mongo on the Roadmap
28. www.magnetic.is jobs@magnetic.is Twitter: @marksweiss @nielsmeer @magneticis .is hiring