Ad Copy Testing with AI Models - Gokce Yesilbas

Transcript

1. Ad Copy Testing with AI Models Gokce Yesilbas #ppclive linkedin.com/in/gokceyesilbas/
2. Gokce Yesilbas 󰠁 Senior Paid Media Consultant at Vervaunt 🎓 MSc Business Intelligence & Digital Marketing Brunel University of London ⭐ BrightonSEO & Hero Conf Speaker 🤍 “All About Digital Marketing & Data Analytics” newsletter on Linkedin Cat Mother 🐈
3. BUFTEK SUFLE
4. Agenda: 1) Types of AI Models (LLMs) 2) Ad Copy Testing • Hypothesis • Methodology • Test Design 3) Case Study
5. Is ChatGPT enough for our tasks?
6. Relying only on ChatGPT is like painting with one color – you need a full palette of AI models. #ppclive in/gokceyesilbas/
7. Text-focused (GPT, Claude’s models) Image Generation (DALL-E, Midjourney, Flux) Video & Audio Generation (Luma, Runway, Sora, ElevenLabs, AudioCraft) Multi Model #ppclive in/gokceyesilbas/
8. Text-focused (GPT, Claude’s models) #ppclive in/gokceyesilbas/
9. Text-focused (GPT, Claude’s models) Gemini llama ChatGPT Claude Deepseek #ppclive in/gokceyesilbas/
10. Ad Copy Testing
12. Ad copy generated by [AI Model X] will achieve a higher CTR than human-written ad copy. #ppclive in/gokceyesilbas/
13. Ad copy generated by [AI Model A] will outperform ad copy generated by [AI Model B] in terms of performance metrics. #ppclive in/gokceyesilbas/
14. Multi-lingual AI-generated ad copy tailored to each region’s language will yield higher engagement metrics than a English-only one. #ppclive in/gokceyesilbas/
17. EXPERIMENTAL SETUP AI MODEL SELECTION AD COPY GENERATION KPIs & ANALYSIS A/B Testing Multivariate Testing Metrics: CTR, Ad Strength, CPC, conversion Statistical Analysis ANOVA Analysis Standardised prompt Prompt Techniques: Few Shot, Cross Lingual, Expert Simulator, CoT Brand Guidelines USPs & CTAs Campaign Specific Needs (number of copies, length, search keywords) ChatGPT - 4o ChatGPT - o1, o3 mini Claude 3.5 Sonnet Gemini 2.0 Llama 3.3 Deepseek R1 AI Model’s Native Platforms Langtail Thinkbuddy Search Ads PMAX Meta Ads Consistency (time, targeting, budget,bid strategy)
19. Why Langtail? 01 Direct access to various AI model APIs Greater flexibility Avoidance of predefined limitations (e.g., word count restrictions) Customisation to suit specific AI model characteristics Simple to use for marketers All-in-one access to all AI models 02 03 04 05 06 Playground
20. Test Design
21. min 4 weeks Testing Timeframe: Budget Level: Medium-level Num. of AI Models: 2 Experiment Tool: Experiments A/B Test
22. Experiment Method Create a trial campaign & test different ad copies generated by 2 AI Models Independent Variables: AI model types (ChatGPT-4o and Claude Sonnet 3.5 etc.) Dependent Variables: CTR , ad strength, key performance indicators for ad copy Search Campaign #1 AI Model X generated ad copy !! start your campaigns on the same day Campaign Structure Search Campaign #2 AI Model Y generated ad copy
23. min 4-6 weeks Testing Timeframe: Budget Level: Medium or High-level Num. of AI Models: more than 2 Experiment Tool: Manuel Multivariate
24. Experiment Method Use optimise or "do not optimise: Rotate ads indefinitely" setting Ensure equal impression distribution Independent Variables: *AI model types (ChatGPT-4o and Claude Sonnet 3.5, human-generated) * language of the ad copy (English, Turkish, French and German) Dependent Variables: CTR , ad strength, key performance indicators for ad copy Search Campaign Campaign Structure English Ad Groıup German Ad Groıup Turkish Ad Groıup Ads: ChatGPT-4o Claude 3.5 Sonnet Gemini or human-generated
25. Prompt Engineering instead of this… #ppclive in/gokceyesilbas/
26. Prompt Engineering write like this #ppclive in/gokceyesilbas/
27. Prompt Engineering Use CoT prompting with expert simulator Split your prompt as steps
28. Prompt Engineering Feed your prompt with examples
29. Prompt Engineering Specify your search keywords & negative keywords
30. Prompt Engineering Include your USPs & CTAs
31. Prompt Engineering Mention campaign specific ad copy length & number
32. Prompt Engineering Include: do not show final review at the end of the reply
34. What happened in my case study?
35. EXPERIMENTAL SETUP AI MODEL SELECTION PROMPT ENGINEERING TOOLS & ANALYSIS A/B Testing Multivariate Testing Metrics: CTR, Ad Strength, CPC Descriptive Statistics Two-way ANOVA Analysis standardised prompt Prompt Techniques: Cross Lingual, Expert Simulator, CoT Brand Guidelines USPs & CTAs Campaign Specific Needs (number of copies, length, search keywords) ChatGPT - 4o Claude 3.5 Sonnet +human-generated In Various Languages: English, Turkish and German Langtail 1 4 9 2 3 5 6 7 8 10 11 Search Ads PMAX Meta Ads Consistency (time, targeting, budget,bid strategy)
36. Ad Strength GERMAN AD GROUP TURKISH AD GROUP ENGLISH AD GROUP #ppclive in/gokceyesilbas/
37. CTR TURKISH AD GROUP GERMAN AD GROUP #ppclive in/gokceyesilbas/
38. While human-generated ads received the majority of traffic, AI-generated ads were more efficient in converting impressions to clicks. #ppclive in/gokceyesilbas/
39. ChatGPT-4o, with its larger number of parameters, performed well in the low-source languages like German and Turkish. #ppclive in/gokceyesilbas/
40. CTR ENGLISH AD GROUP #ppclive in/gokceyesilbas/
41. Claude models are trained to act as more friendly assistants in English. #ppclive in/gokceyesilbas/
42. ChatGPT-4o produced similar output across multiple ad copies in English, potentially limiting the diversity of the generated content. #ppclive in/gokceyesilbas/
43. What’s Next?
44. TEST WISELY WITH Test Share ChatGPT - o1 ChatGPT - o3 mini high Gemini 2.0 llama 3.3 Deepseek V3 with the PPC Live community! the latest AI models… Continuously test and refine AI-generated content Learn & Scale #ppclive in/gokceyesilbas/
45. Let’s connect for more PPC! linkedin.com/in/gokceyesilbas/ 01 02 #PPCLive