Hero Conf - Prep your Measurement for Life after Cookies

Presented at Hero Conf San Diego on September 24th 2025 by Ben Vigneron, VP of Analytics with Blackbird PPC. Enjoy!
Visit http://www.blackbirdppc.com/

Transcript

1. Prep Your Measurement for Life After Cookies Ben Vigneron Blackbird PPC @benjaminvigneron
2. Let’s ask ourselves these questions… DO YOU REMEMBER THE LAST AD YOU CLICKED BEFORE BUYING SOMETHING ONLINE? Prep Your Measurement for Life After Cookies, Ben Vigneron
3. Let’s ask ourselves these questions… WHAT % OF OVERALL REVENUE IS UNATTRIBUTED IN YOUR BACKEND CURRENTLY? Prep Your Measurement for Life After Cookies, Ben Vigneron % OF UNATTRIBUTED REVENUE Most revenue doesn't get attributed correctly
4. Let’s ask ourselves these questions… HOW MUCH CAN MY CURRENT REPORTING BE TRUSTED? Prep Your Measurement for Life After Cookies, Ben Vigneron LEVEL OF TRUST Most marketers trust their current reporting
5. Let’s ask ourselves these questions… AM I ABLE TO PREDICT REVENUE BASED ON VARIOUS MARKETING ACTIVITIES? Prep Your Measurement for Life After Cookies, Ben Vigneron ABILITY TO PREDICT REVENUE Most marketers struggle to make accurate predictions
6. …piecing together Most revenue doesn't get attributed correctly… …and are unable to make accurate predictions …however most marketers trust their current reporting… Prep Your Measurement for Life After Cookies, Ben Vigneron
7. …although attribution is not a new problem! Prep Your Measurement for Life After Cookies, Ben Vigneron John Wanamaker (1838-1922) Merchant and former US Postmaster General HALF THE MONEY I SPEND ON ADVERTISING IS WASTED; THE TROUBLE IS, I DON'T KNOW WHICH HALF
8. Agenda 1. Why is cookie-based attribution outdated? 2. How are MMM and incrementality testing replacing traditional attribution? 3. MMM deep dive 4. Incrementality testing deep dive 5. Proxy metrics 6. Recap Prep Your Measurement for Life After Cookies, Ben Vigneron
9. Why is cookie-based attribution outdated? Prep Your Measurement for Life After Cookies, Ben Vigneron
10. 10+ Years of Privacy Regulation 2012 ePrivacy Directive 2018 GDPR, CCPA 2020 Google Consent Mode v1, Apple iOS 14 blocks 3rd party cookies 2024 Google Consent Mode v2 Prep Your Measurement for Life After Cookies, Ben Vigneron
11. Impact on Users & Advertisers 2009 ePrivacy Directive • Introduction of Cookie Consent Banners • Emergence of consent management platforms (CMPs) 2020-2024 Google Consent Mode v1 then v2 • Set default consent states to “Denied”, tags are effectively blocked unless users grant permission. 2018-2020 GDPR, CCPA • Data minimization and purpose limitation • Development of privacy-enhancing technologies (PETs) Prep Your Measurement for Life After Cookies, Ben Vigneron
12. Observed vs. Modeled data More users opting out from personalized content and ads means… What if you don’t implement consent mode? • Less accurate reporting • Reduced audience insights • Reduced data modeling Prep Your Measurement for Life After Cookies, Ben Vigneron More modeled data Less observed data
13. Piping backend data More users opting out from cookies also means that… Prep Your Measurement for Life After Cookies, Ben Vigneron Piping backend data has become crucial for: • Algorithmic bidding: bid up keywords and audiences which produce high customer lifetime value, or high lead quality • Audience creation: site visitors who took action X or Y, etc…
14. How are MMM and incrementality testing replacing traditional attribution? Prep Your Measurement for Life After Cookies, Ben Vigneron
15. How to attribute all types of exposure and engagement Views TV OOH Display Print SEM Social Impressions Clicks As opposed to traditional digital reporting which relies solely on clicks, incrementality testing and MMM account for views and impressions, online and offline. Prep Your Measurement for Life After Cookies, Ben Vigneron
16. MMM and Incrementality testing help you get the full picture = Traditional reporting Prep Your Measurement for Life After Cookies, Ben Vigneron Visible Icebergs float on the ocean with only about 10% of their mass appearing above the ocean Hidden The remaining 90% is concealed beneath the waves = MMM and Incrementality testing
17. MMM deep dive Prep Your Measurement for Life After Cookies, Ben Vigneron
18. What is MMM? • Marketing Mix Modeling (MMM) is a holistic attribution method which relies on historical spend, organic initiatives, and context. • Therefore, MMM is not affected by the deprecation of third-party cookies, or other online tracking challenges. • Ultimately, MMM helps find the optimal media mix across online and offline efforts. Aligns with the ground-truth Reduces human bias Enables actionable decision making Private by design Prep Your Measurement for Life After Cookies, Ben Vigneron
19. MMM process 1. Gather the data Historical ad spend across channels by week, and overall revenue also by week 3. Reallocate media mix Further validate learnings through incrementality testing 2. Run MMM unsupervised, and supervised Customize hyperparameters if need be until model accuracy is satisfactory Prep Your Measurement for Life After Cookies, Ben Vigneron
20. MMM: model accuracy In this case, model accuracy was 65%, meaning that 65% of week-over-week revenue variations can be explained by marketing initiatives and other factors. Goal is model accuracy>80% Prep Your Measurement for Life After Cookies, Ben Vigneron
21. MMM: response decomposition In this example, the top contributor was Google Nonbrand, as opposed to Google Brand in the backend. Interestingly, Podcasts and YouTube were not getting any credit in the backend. Prep Your Measurement for Life After Cookies, Ben Vigneron
22. MMM: Robyn vs. Meridian Good news: both Robyn and Meridian attributed similar influence across most channels Bad news: Meridian attributed a lot more organic influence to channel 0. This indicates that we probably need further testing to validate our findings. Prep Your Measurement for Life After Cookies, Ben Vigneron
23. MMM: optimal budget allocation In this example, revenue could be increased from 13.2M (Before) up to 14.2M (After), that is a by +7.73% increase. Before After Prep Your Measurement for Life After Cookies, Ben Vigneron
24. MMM: Pros and Cons Pros • Allows advertisers to estimate the contribution of every single initiative (!!!) • Does not rely on cookies • Only requires channel spend, and aggregated revenue from back-end data sources • Accounts for seasonality, and latency Cons • Requires 2 years of data • Requires some fine-tuning • Requires some R or Python knowledge • Requires dense-enough data, otherwise soft metrics should be used Search Social Display YouTube Podcasts Prep Your Measurement for Life After Cookies, Ben Vigneron
25. Incrementality testing deep dive Prep Your Measurement for Life After Cookies, Ben Vigneron
26. What is incrementality testing? Incrementality focuses on causal impact — only measuring moments where your ad caused a customer to convert. That is the difference in performance between users who saw your ads and users who did not. Prep Your Measurement for Life After Cookies, Ben Vigneron
27. Incrementality testing: test design examples Geo lift: Test vs. Control states ABM lift: Test vs. Control accounts All accounts Qualified accounts Test accounts Control accounts Prep Your Measurement for Life After Cookies, Ben Vigneron
28. Incrementality testing: Causal inference Causal inference allows marketers to estimate how the response metric would have evolved if the intervention had never occurred - more specifically if your ad had not been seen. Source: https://google.github.io/CausalImpact/CausalImpact.html Prep Your Measurement for Life After Cookies, Ben Vigneron Control Test
29. Incrementality testing Pros and Cons Pros • Establish causality • Does not rely on cookies - just like MMM • Does not require years of data, typically only 90 days • Accounts for seasonality, and latency Cons • Estimates the contribution of a single initiative • Requires some R or Python knowledge • Requires dense-enough data, otherwise soft metrics should be used Prep Your Measurement for Life After Cookies, Ben Vigneron
30. Proxy metrics Prep Your Measurement for Life After Cookies, Ben Vigneron
31. Proxy metrics Proxy metrics are reliable predictors for future success that can be observed earlier or more easily. For example, more Engaged Sessions are most likely correlated with more Leads, and more MQLs are most likely correlated with more Opportunities. Prep Your Measurement for Life After Cookies, Ben Vigneron HOW MANY OF YOU FIND IT DIFFICULT TO PREDICT FUTURE HIGH-LEVEL BUSINESS PERFORMANCE ? Typical B2B Customer Journey
32. Identifying Reliable Proxy Metrics As a marketer, you need to achieve high-level business goals which are often not observable yet. Therefore you need to identify a KPI that is: • Dense enough (think algorithmic bidding) • Recent enough (within 90 days post-click or so) • Accurate enough (needs to correlate strongly with the actual outcome of interest) Prep Your Measurement for Life After Cookies, Ben Vigneron Balancing Density, Recency, and Accuracy
33. Recap Prep Your Measurement for Life After Cookies, Ben Vigneron
34. Recap Prep Your Measurement for Life After Cookies, Ben Vigneron Cookie-based attribution is no longer enough. Piping backend data is crucial for finding high-quality customers Incrementality testing and MMM provide the future of measurement Proxy metrics help guide us Key points
35. Prep Your Measurement for Life After Cookies, Ben Vigneron Moment of Reinvention for Marketers We’re not losing visibility — we’re gaining clarity. Be skeptical, and keep experimenting
36. About me Ben Vigneron • 20+ years in digital marketing • 10+ years at the intersection of AI and advertising • Founded Adobe’s Product Analytics Group in the Adobe Advertising Cloud • Blends platform knowledge with advanced analytics techniques to get more out of client budgets • Always up for a good analytics chat: Ben@BlackbirdPPC.com Feel free to reach out! Thank you