Data Exclusions - What they can tell us about how Smart Bidding really works behind the scenes | WeDiscover

More information and speaker notes available here: https://docs.google.com/presentation/d/177neb3klX7YX_RFFWNtysTSvDcoU9DZaESmznuz3YzI/edit?usp=sharing

Transcript

1. Data Exclusions What they can tell us about how Smart Bidding really works behind the scenes
2. The basics The absolute fundamentals
3. What are data exclusions? Smart Bidding uses conversions and conversion value data in Google Ads to help meet your goals. If you have any issues with conversion tracking, you can use data exclusions to help reduce the impact that these issues may have on Smart Bidding Performance. Data exclusions are an advanced tool that helps you to inform your Smart Bidding strategy of an issue with conversion data, to help reduce the impact to performance. Conversion issues that may aect Smart Bidding include anything that causes reported conversions or conversion value in Google Ads to be incorrect. For example: ➔ Tagging issues ➔ Website outages ➔ Data Import issues
4. What you need to know about data exclusions Data exclusions can only be used with conversions and conversion value based Smart Bidding strategies. ➔ Data exclusions apply to clicks. Your data exclusions must exclude clicks that could have had the aected conversions aributed to them. When these clicks are excluded, the associated conversions are also excluded ➔ Data exclusions apply to all campaign types. If you applied an exclusion for the entire account, conversions for Search, Display, Shopping Network and Performance Max campaigns will be excluded. Video campaigns aren’t available ➔ Data exclusions don’t aect conversion reporting. Data exclusions only aect the data that Smart Bidding uses. You can still find these excluded conversions in your reporting ➔ Data exclusions may take up to a week to fully process. In the meantime, adjust CPA or ROAS targets to achieve desired performance, while ensuring budgets are set to acceptable levels
5. How to add data exclusions With UI changes, specific instructions will probably be outdated by the time you finish reading this sentence. The gist is the following: 1. Navigate to Exclusions and click the plus buon. PRO TIP: Search for “Exclusions” using the search bar 2. Fill in your seings (name, description, start and end dates) 3. Select your scope (campaign types, specific campaigns or devices) and save You can find detailed instructions here.
6. Best practices Things you should do (and one thing you definitely shouldn’t do)
7. Best practices Example: If there was a data upload issue from 15 October to 18 October, and the conversion delay is five days, apply the exclusion as quickly as possible, including any days with impacted clicks. In this case, you would exclude 10 October to 18 October ➔ Apply data exclusions as quickly as possible at the time you’ve identified a conversion data issue ➔ Data exclusions are typically taken into eect by Smart Bidding within a week. After applying a data exclusion, adjust CPA/ROAS targets to achieve desired performance, while ensuring budgets are set to acceptable levels ➔ Exclusion dates selected will apply to clicks, so make sure that you consider your conversion delay and exclude any days of clicks that may have been impacted. It’s a best practice to exclude 90% of clicks associated with impacted conversion data
8. How to see conversion lag
9. Best practices (cont’d) ➔ Data exclusions aren’t intended to be used frequently or for extended periods of time. Doing so could negatively impact Smart Bidding performance ➔ Data exclusions are only intended for issues with conversion tracking, and are not recommended for excluding irregular performance periods, such as promotional periods. If you anticipate major changes in conversion rate, consider using a seasonality adjustment ➔ Data exclusions applied to future dates, such as for scheduled site maintenance, will typically take eect immediately What’s the most important best practice of all? 🥁
10. Don’t remove a data exclusion after it has been applied. You can still backfill the accurate conversion data for reporting purposes (although you should wait one week before backfilling), but it’s not recommended to remove data exclusions as it may cause undesired performance fluctuations. Use data exclusions for conversion outages Google Ads Help
11. How data exclusions work (Maybe.)
12. How do data exclusions work under the hood? Honestly, no one knows. However, Google has given us a few clues. On the following slides are a few quotes from Google about various parts of Google Ads (with emphasis added in green) which give us an indication of what it’s doing and why. After each quote, I’ll give my interpretation of what it tells us about data exclusions.
13. If you've recently had a conversion tracking issue, we suggest that you use data exclusions to exclude the period with bad data. By doing this, it prevents our systems from training on data that might be inaccurate. For example, if a tag was accidentally removed from your site, we suggest applying a data exclusion on the days that are missing conversion data. Note: Data exclusions are meant to account for outages or major issues. Using them often or for long periods could negatively impact Smart Bidding performance. How to steer AI-powered search ads Google Ads Help
14. Training data
15. Training data ➔ A machine learning model’s performance is very dependent on the dataset it is being trained on ➔ An accepted approach is to divide the data into 70% for training, 20% for testing and 10% for validation ➔ The model is optimised based on the version of the world represented in the training data ➔ Once training is done, we use the validation data to evaluate how well the model does. Based on the results, we tune the hyperparameters of the model and train again ➔ The model is being optimised to show the best performance on the validation data ➔ For time-based data sets, the split cannot be done randomly since the values of today might depend on the data point from yesterday. In those cases, the split is done sequentially
16. Don’t remove a data exclusion after it has been applied. Advertisers may wish to backfill the accurate conversion data for their reporting purposes, but we recommend not removing data exclusions as it may cause undesired performance fluctuations. Advertisers should wait one week before backfilling. Use data exclusions for conversion data outages Google Ads Help
17. Time series forecasting
18. Time series forecasting ➔ Whenever data or observations of something are recorded at regular time intervals, you’re looking a time series data ➔ Time series forecasting is looking at data over time to forecast or predict what will happen in the next time period, based on paerns and recurring trends from previous time periods. What has happened in the past is likely to happen in the future ➔ The prediction is based on a given time, looking at a sequence of observations over time ➔ It is also possible to model data with no recognisable paern or trend in your time series ➔ If there is no recognisable paern, your best bet is to rely more on what’s recently happened and less on what’s happened far in history ➔ What’s happened recently is more useful in guiding us to what will happen next
19. Smart Bidding is always learning, even from campaigns using manual bidding…When you have lile to no conversion data available, Smart Bidding can still use query-level data beyond your bid strategy to build more accurate initial conversion rate models. This helps it make more informed bidding decisions from the start. It then uses Bayesian learning to continuously improve these models as it accrues conversion rate data at more granular levels (e.g. for a search query mapped to specific ad copy or landing pages). Smart Bidding takes seasonality into account, and in most cases, it will automatically handle seasonal increases in traic without requiring any input. How our bidding algorithms learn
20. ARIMA models
21. ARIMA models ➔ Autoregressive - you’re trying to predict the value of a time series today based on a value of the time series in the past ➔ Integrated - the time series has an upwards or downwards trend so you use dierencing to get rid of that ➔ Moving average - using the error from a previous period to inform your prediction of your time series today ➔ Seasonality - there’s a repeating paern within a year which happens over and over again over time in your time series
22. What’s the connection with Smart Bidding? Based on the descriptions on Google Ads Help, Google almost certainly uses a seasonality ARIMA model to predict future traic levels for Smart Bidding. Since it’s autoregressive, we know that it predicts future values by looking at the past. The problem is that data exclusions change what the past looks like. Fortunately for us, you can actually still fit ARIMA models with missing values (for example, those missing due to a data exclusion) by estimating them using something called a Kalman filter. Using this model is how Google can infer seasonal increases without requiring any input from you.
23. Bayes’ Theorem
24. ➔ This is a theorem which helps you work out the probability of a hypothesis given a specific condition ➔ You update your probabilities based on evidence ◆ If you see dark clouds, it’s probably going to rain ◆ If you get stuck in traic, it’s more likely that you’re going to be late ➔ Here’s the formula (minus the horror films and clowns) Bayes’ Theorem
25. MATHS ALERT Consider yourself warned.
26. Bayes’ Theorem in action Suppose there are two full crates of beer. Crate #1 has 6 pale ales and 18 lagers and Crate #2 has 12 of each. Ash picks a crate at random and then picks a beer at random. We have no reason to believe that Ash treats one crate dierently to the other, likewise for the beers. The beer Ash picks turns out to be a lager. How likely is it that Ash picked a beer out of Crate #1? Intuitively, we know the answer should be more than 50% because there are more lagers in Crate #1. However, the exact answer is given by Bayes' theorem.
27. Bayes’ Theorem in action (cont’d) Let H 1 correspond to Crate #1 and H 2 to Crate #2. Since both crates were identical from Ash’s point of view, P(H 1 ) = P(H 2 ). The probabilities must add up to 1 so both are equal to 0.5. This was our belief before we saw any evidence and is called the “prior probability”. The event E is the observation of a lager. From the contents of the crates, we know that P(E | H 1 ) = 18/24 = 0.75 and P(E | H 2 ) = 12/24 = 0.5. Bayes’ formula then yields: After seeing the lager, we revise our probability of Ash choosing a beer from Crate #1 to 0.6. This is called the “posterior probability”.
28. What’s any of this got to do with Smart Bidding? Put simply, when you remove a data exclusion (even if you have backfilled the data), you change every single one of the probabilities that the bidding strategy uses. Now it’s not 24 beers in a crate but 35. Now there aren’t 12 pale ales in Crate 1 but 27. Everything is completely dierent. Since Google uses Bayesian learning and updates probabilities based o of evidence, you change all of the probabilities that is uses to bid by removing the data exclusion. Sure, if you remove it, the data the strategy uses may now be correct but it causes volatility in the short term as it has to relearn everything. Performance will likely be more stable if you allow it to learn from future correct data (which is new evidence) rather than altering the evidence of the past.
30. Our algorithms apply adaptive historical weighting to rely more heavily on recent data when adjusting bids…recent performance is likely more predictive of future performance, but this should weigh less heavily against clicks that aren’t yet seeing conversions due to conversion delays. For example, if you’re an advertiser…with lengthier conversion cycles, your recent data may not be as useful because those ad clicks require a longer period of time to yield conversions. As a result, we’ll weigh that recent data less heavily compared to advertisers with shorter conversion cycles…This helps prevent overreactions to recent clicks that are experiencing conversion delays, which could lead to unnecessary bid reductions. We also automate this process… How our bidding algorithms learn Google Ads Help
31. Data exclusions created for past dates are typically taken into eect by Smart Bidding within a week…Smart Bidding may be using the incorrect conversion data during the week following a data exclusion event… Use data exclusions for conversion data outages Google Ads Help
32. The three main takeaways: 1. Data exclusions are recommended for a conversion data outage. Conversion issues that may aect Smart Bidding include anything that causes reported conversions or conversion value in Google Ads to be incorrect 2. Data exclusions apply to clicks NOT conversions. Your data exclusions must exclude clicks that could have had the aected conversions aributed to them. When these clicks are excluded, the associated conversions are also excluded 3. Don’t remove a data exclusion after it has been applied. You can still backfill the accurate conversion data for reporting purposes (although you should wait one week before backfilling), but it’s not recommended to remove data exclusions as it may cause undesired performance fluctuations Summary
33. Thank You