Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Ad Copy Testing with AI Models - Gokce Yesilbas

Ad Copy Testing with AI Models - Gokce Yesilbas

Gokce Yesilbas

February 07, 2025
Tweet

More Decks by Gokce Yesilbas

Other Decks in Marketing & SEO

Transcript

  1. Gokce Yesilbas 󰠁 Senior Paid Media Consultant at Vervaunt 🎓

    MSc Business Intelligence & Digital Marketing Brunel University of London ⭐ BrightonSEO & Hero Conf Speaker 🤍 “All About Digital Marketing & Data Analytics” newsletter on Linkedin Cat Mother 🐈
  2. Agenda: 1) Types of AI Models (LLMs) 2) Ad Copy

    Testing • Hypothesis • Methodology • Test Design 3) Case Study
  3. Relying only on ChatGPT is like painting with one color

    – you need a full palette of AI models. #ppclive in/gokceyesilbas/
  4. Text-focused (GPT, Claude’s models) Image Generation (DALL-E, Midjourney, Flux) Video

    & Audio Generation (Luma, Runway, Sora, ElevenLabs, AudioCraft) Multi Model #ppclive in/gokceyesilbas/
  5. Ad copy generated by [AI Model X] will achieve a

    higher CTR than human-written ad copy. #ppclive in/gokceyesilbas/
  6. Ad copy generated by [AI Model A] will outperform ad

    copy generated by [AI Model B] in terms of performance metrics. #ppclive in/gokceyesilbas/
  7. Multi-lingual AI-generated ad copy tailored to each region’s language will

    yield higher engagement metrics than a English-only one. #ppclive in/gokceyesilbas/
  8. EXPERIMENTAL SETUP AI MODEL SELECTION AD COPY GENERATION KPIs &

    ANALYSIS A/B Testing Multivariate Testing Metrics: CTR, Ad Strength, CPC, conversion Statistical Analysis ANOVA Analysis Standardised prompt Prompt Techniques: Few Shot, Cross Lingual, Expert Simulator, CoT Brand Guidelines USPs & CTAs Campaign Specific Needs (number of copies, length, search keywords) ChatGPT - 4o ChatGPT - o1, o3 mini Claude 3.5 Sonnet Gemini 2.0 Llama 3.3 Deepseek R1 AI Model’s Native Platforms Langtail Thinkbuddy Search Ads PMAX Meta Ads Consistency (time, targeting, budget,bid strategy)
  9. Why Langtail? 01 Direct access to various AI model APIs

    Greater flexibility Avoidance of predefined limitations (e.g., word count restrictions) Customisation to suit specific AI model characteristics Simple to use for marketers All-in-one access to all AI models 02 03 04 05 06 Playground
  10. min 4 weeks Testing Timeframe: Budget Level: Medium-level Num. of

    AI Models: 2 Experiment Tool: Experiments A/B Test
  11. Experiment Method Create a trial campaign & test different ad

    copies generated by 2 AI Models Independent Variables: AI model types (ChatGPT-4o and Claude Sonnet 3.5 etc.) Dependent Variables: CTR , ad strength, key performance indicators for ad copy Search Campaign #1 AI Model X generated ad copy !! start your campaigns on the same day Campaign Structure Search Campaign #2 AI Model Y generated ad copy
  12. min 4-6 weeks Testing Timeframe: Budget Level: Medium or High-level

    Num. of AI Models: more than 2 Experiment Tool: Manuel Multivariate
  13. Experiment Method Use optimise or "do not optimise: Rotate ads

    indefinitely" setting Ensure equal impression distribution Independent Variables: *AI model types (ChatGPT-4o and Claude Sonnet 3.5, human-generated) * language of the ad copy (English, Turkish, French and German) Dependent Variables: CTR , ad strength, key performance indicators for ad copy Search Campaign Campaign Structure English Ad Groıup German Ad Groıup Turkish Ad Groıup Ads: ChatGPT-4o Claude 3.5 Sonnet Gemini or human-generated
  14. EXPERIMENTAL SETUP AI MODEL SELECTION PROMPT ENGINEERING TOOLS & ANALYSIS

    A/B Testing Multivariate Testing Metrics: CTR, Ad Strength, CPC Descriptive Statistics Two-way ANOVA Analysis standardised prompt Prompt Techniques: Cross Lingual, Expert Simulator, CoT Brand Guidelines USPs & CTAs Campaign Specific Needs (number of copies, length, search keywords) ChatGPT - 4o Claude 3.5 Sonnet +human-generated In Various Languages: English, Turkish and German Langtail 1 4 9 2 3 5 6 7 8 10 11 Search Ads PMAX Meta Ads Consistency (time, targeting, budget,bid strategy)
  15. While human-generated ads received the majority of traffic, AI-generated ads

    were more efficient in converting impressions to clicks. #ppclive in/gokceyesilbas/
  16. ChatGPT-4o, with its larger number of parameters, performed well in

    the low-source languages like German and Turkish. #ppclive in/gokceyesilbas/
  17. Claude models are trained to act as more friendly assistants

    in English. #ppclive in/gokceyesilbas/
  18. ChatGPT-4o produced similar output across multiple ad copies in English,

    potentially limiting the diversity of the generated content. #ppclive in/gokceyesilbas/
  19. TEST WISELY WITH Test Share ChatGPT - o1 ChatGPT -

    o3 mini high Gemini 2.0 llama 3.3 Deepseek V3 with the PPC Live community! the latest AI models… Continuously test and refine AI-generated content Learn & Scale #ppclive in/gokceyesilbas/