[NAACL2022 Industry Track] Aspect-based Analysis of Advertising Appeals for Search Engine Advertising

Soichiro Murakami, Peinan Zhang, Sho Hoshino, Hidetaka Kamigaito, Hiroya Takamura, and Manabu Okumura. 2022. Aspect-based Analysis of Advertising Appeals for Search Engine Advertising. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track, pages 69–78, Hybrid: Seattle, Washington + Online. Association for Computational Linguistics.

https://aclanthology.org/2022.naacl-industry.9/

Transcript

1. Aspect-based Analysis of Advertising Appeals for Search Engine Advertising Soichiro Murakami, Peinan Zhang, Sho Hoshino, Hidetaka Kamigaito, Hiroya Takamura, Manabu Okumura CyberAgent, Inc., Tokyo Institute of Technology 1

2. Motivation Ad creator TokyoHotels.com - Best Price Guarantee Find your Hotel in Tokyo. Register Now. Members Get an Extra 20% OFF. Ad text Ad・https://www.tokyohotels.com/ ▼ Advertising appeals help to increase the attractiveness Special deals Discount price, Limited offer 2
 • Writing an attractive ad text is essential for the success of online advertising ◦ Can persuade people to click an ad and buy a product.
3. Motivation • Writing an attractive ad text is essential for the success of online advertising ◦ Can persuade people to click an ad and buy a product. Ad creator TokyoHotels.com - Best Price Guarantee Find your Hotel in Tokyo. Register Now. Members Get an Extra 20% OFF. Ad text Ad・https://www.tokyohotels.com/ ▼ Advertising appeals help to increase the attractiveness Special deals Discount price, Limited offer 3
 We explore effective aspects of advertising appeals in advertising texts.
4. Motivation e-commerce (EC) industry automobile industry Limited offers Quality of product 4
 • Effective aspects might be different for each industry
5. Motivation • Effective aspects might be different for each industry e-commerce (EC) industry automobile industry Limited offers Quality of product What are the effective aspects for each industry? 5

6. Main Idea large-scale but unlabeled… Advertising performance metric: CTR = clicks ÷ impression • We investigate the relationship between CTR and aspects of advertising appeals 6
 (Ad text, IndustryType, CTR) (“Members Get an Extra 20% OFF”, Travel, 0.7) Ad text IndustryType CTR
7. Main Idea Aspect detection model labeled dataset large-scale but unlabeled… • We investigate the relationship between CTR and aspects of advertising appeals 7
 (Ad text, IndustryType, CTR) (Ad text, IndustryType, CTR, Aspect)
8. Main Idea Aspect detection model Annotated dataset Train labeled dataset (Ad text, IndustryType, CTR) (Ad text, IndustryType, CTR, Aspect) large-scale but unlabeled… • We investigate the relationship between CTR and aspects of advertising appeals 8
 (Ad text, Aspect)
9. Goal • Create a dataset of ad texts annotated with the advertising appeals • Develop an aspect detection model to identify the aspects • Explore the effective aspects through correlation analysis in different industires ◦ between advertising appeals and performance metric (click-through-rate; CTR) Dataset construction Aspect detection model Correlation analysis 9

10. Dataset construction • Data Resource ◦ 2,738 ads from Mar. 1, 2020 to Feb. 28, 2021 through Google Ads (in Japanese) • Two steps to define our aspect labels ◦ Step 1: Found eight aspects through our preliminary analysis ▪ special deals, quality, problem solving, speed, user-friendliness, limited offers, product lineup, trend ◦ Step 2: Refined the aspect labels based on the opinion of experienced ad creator （1）Special deals （2）　Discount price （3）　Reward points （4）　Free （5）　Special gift （6）Features （7）　Quality （8）　Problem sovling （9）　Speed （10） User-friendliness （11） Transportation （12）Limited offers （13）　Limited time （14）　Limited target （15）　First-time limited （16）Performance （17）　Largest/no.1 （18）　Product lineup （19）　Trend （20）Others （21）　Story 10

11. Annotation Example • Define an advertising expression as a span text to be annotated • Allow annotators to provide multiple labels for each span No.1 Insurance For Children. Enroll Now While You Can. Get a Free Health Insurance Quote in 3 Easy Steps. ✅ Free ✅ User-friendliness ✅ Largest/no.1 11

12. Statistics of annotated dataset “#spans” represents the number of span texts annotated with each label. • 2,738 ad texts in total ◦ include 4,219 spans 12

13. Aspect Detection Watch Docs, Movies & More - Get the First Month Free 
 Input: Ad text
 Output: Aspect labels
 Free First-time Limited Offer 13

14. Aspect Detection Model • Span-based model [Zhang+, 2019] ◦ Extracts spans and predicts aspect labels for each span • Doc-based model [Devlin+, 2019] ◦ Predicts labels for an entire ad text ◦ A document classification model BERT & CRF [CLS] Watch Docs, Movies & More - Get the First Month Free Free, First-time limited offer MLP O O O O O O O … O O O O O O … B I I I I … I I I I E
 Span Detection Label Prediction BERT [CLS] Watch Docs, Movies & More - Get the First Month Free MLP Label Prediction Free, First-time limited offer 14

15. Experiments • Aspect detection ◦ Evaluate two models: the span-based and doc-based models • Correlation analysis ◦ Analyze which aspects labels are effective in each industry • CTR prediction ◦ Investigate whether the aspect labels contribute to CTR prediction Aspect detection CTR prediction Train 1,857 136,352 Valid 465 16,084 Test 410 15,976 Also used for 
 correlation analysis 
 15

16. Results - Aspect Detection • The doc-based model outperformed the span-based model. 16

17. Results - Aspect Detection • The doc-based model outperformed the span-based model. • “Free”, “Speed”, “Transportation”, “Largest/no.1” are easy to detect. 17
 “Free shipping”
18. Results - Aspect Detection • The doc-based model outperformed the span-based model. • “Free”, “Speed”, “Transportation”, “Largest/no.1” are easy to detect. • The limited number of annotation causes low performance 18

19. Correlation analysis • Observed a weak correlation between the aspect labels and CTR 19

20. Correlation analysis • Observed a weak correlation between the aspect labels and CTR “One of the largest websites in Japan” (国内最大級サイト) “Get [N] points for new membership” (新規入会&利用で[N]ポイント) 20

21. CTR Prediction • Investigate the effectiveness of the aspect labels in CTR prediction ◦ Can be a potential application for ad creation support • Model ◦ Input: ad text, “predicted” aspect labels ◦ Output: CTR score [0-1] “Watch Docs, Movies & More - Get the First Month Free” 
 Free，First-time Limited Offer CTR score [0-1] BERT 0.74 21

22. Results - CTR Prediction • Identification of aspect labels contributes to CTR prediction • Doc-based model improved the performance than span-based model 22

23. Summary • Explored the effective aspects of advertising appeals in different industries ◦ Dataset construction ◦ Aspect detection ◦ Correlation analysis between aspects and CTR • Demonstrated that each industry exhibits unique effective aspects and that identification of aspect labels contributes to CTR prediction Future direction ➔ Investigate whether presenting the effective aspects can help ad creators write effective ad texts in a real-world scenario ➔ Aspect-based ad text geneartion 23

24. Acknowledgement • Icons in the slides are from flaticon ◦ https://www.flaticon.com/ 24