[NAACL2022 Industry Track] Aspect-based Analysis of Advertising Appeals for Search Engine Advertising

Soichiro Murakami, Peinan Zhang, Sho Hoshino, Hidetaka Kamigaito, Hiroya Takamura, and Manabu Okumura. 2022. Aspect-based Analysis of Advertising Appeals for Search Engine Advertising. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track, pages 69–78, Hybrid: Seattle, Washington + Online. Association for Computational Linguistics.

https://aclanthology.org/2022.naacl-industry.9/

Transcript

1. Aspect-based Analysis of Advertising Appeals
   for Search Engine Advertising
   Soichiro Murakami, Peinan Zhang, Sho Hoshino,
   Hidetaka Kamigaito, Hiroya Takamura, Manabu Okumura
   CyberAgent, Inc., Tokyo Institute of Technology
   1

   

   View Slide

2. Motivation
   Ad creator
   TokyoHotels.com - Best Price Guarantee
   Find your Hotel in Tokyo. Register Now.
   Members Get an Extra 20% OFF.
   Ad text
   Ad・https://www.tokyohotels.com/ ▼
   Advertising appeals help to increase the attractiveness
   Special deals
   Discount price, Limited offer
   2

   ● Writing an attractive ad text is essential for the success of online advertising
   ○ Can persuade people to click an ad and buy a product.
   

   View Slide

3. Motivation
   ● Writing an attractive ad text is essential for the success of online advertising
   ○ Can persuade people to click an ad and buy a product.
   Ad creator
   TokyoHotels.com - Best Price Guarantee
   Find your Hotel in Tokyo. Register Now.
   Members Get an Extra 20% OFF.
   Ad text
   Ad・https://www.tokyohotels.com/ ▼
   Advertising appeals help to increase the attractiveness
   Special deals
   Discount price, Limited offer
   3

   We explore effective aspects of advertising appeals in advertising texts.
   

   View Slide

4. Motivation
   e-commerce (EC) industry automobile industry
   Limited offers Quality of product
   4

   ● Effective aspects might be different for each industry
   

   View Slide

5. Motivation
   ● Effective aspects might be different for each industry
   e-commerce (EC) industry automobile industry
   Limited offers Quality of product
   What are the effective aspects for each industry? 5

   

   View Slide

6. Main Idea
   large-scale but unlabeled…
   Advertising performance metric:
   CTR = clicks ÷ impression
   ● We investigate the relationship between CTR and aspects of advertising appeals
   6

   (Ad text, IndustryType, CTR)
   (“Members Get an Extra 20% OFF”, Travel, 0.7)
   Ad text IndustryType CTR
   

   View Slide

7. Main Idea
   Aspect detection model labeled dataset
   large-scale but unlabeled…
   ● We investigate the relationship between CTR and aspects of advertising appeals
   7

   (Ad text, IndustryType, CTR) (Ad text, IndustryType, CTR, Aspect)
   

   View Slide

8. Main Idea
   Aspect detection model
   Annotated dataset
   Train
   labeled dataset
   (Ad text, IndustryType, CTR) (Ad text, IndustryType, CTR, Aspect)
   large-scale but unlabeled…
   ● We investigate the relationship between CTR and aspects of advertising appeals
   8

   (Ad text, Aspect)
   

   View Slide

9. Goal
   ● Create a dataset of ad texts annotated with the advertising appeals
   ● Develop an aspect detection model to identify the aspects
   ● Explore the effective aspects through correlation analysis in different industires
   ○ between advertising appeals and performance metric (click-through-rate; CTR)
   Dataset construction Aspect detection model Correlation analysis
   9

   

   View Slide

10. Dataset construction
    ● Data Resource
    ○ 2,738 ads from Mar. 1, 2020 to Feb. 28, 2021 through Google Ads (in Japanese)
    ● Two steps to define our aspect labels
    ○ Step 1: Found eight aspects through our preliminary analysis
    ■ special deals, quality, problem solving, speed, user-friendliness, limited offers, product lineup, trend
    ○ Step 2: Refined the aspect labels based on the opinion of experienced ad creator
    （1）Special deals
    （2）　Discount price
    （3）　Reward points
    （4）　Free
    （5）　Special gift
    （6）Features
    （7）　Quality
    （8）　Problem sovling
    （9）　Speed
    （10） User-friendliness
    （11） Transportation
    （12）Limited offers
    （13）　Limited time
    （14）　Limited target
    （15）　First-time limited
    （16）Performance
    （17）　Largest/no.1
    （18）　Product lineup
    （19）　Trend
    （20）Others
    （21）　Story
    10

    

    View Slide

11. Annotation Example
    ● Define an advertising expression as a span text to be annotated
    ● Allow annotators to provide multiple labels for each span
    No.1 Insurance For Children. Enroll Now While You Can.
    Get a Free Health Insurance Quote in 3 Easy Steps.
    ✅ Free
    ✅ User-friendliness
    ✅ Largest/no.1
    11

    

    View Slide

12. Statistics of annotated dataset
    “#spans” represents the number of span texts annotated with each label.
    ● 2,738 ad texts in total
    ○ include 4,219 spans
    12

    

    View Slide

13. Aspect Detection
    Watch Docs, Movies & More - Get
    the First Month Free 

    Input: Ad text
 Output: Aspect labels

    Free
    First-time Limited Offer
    13

    

    View Slide

14. Aspect Detection Model
    ● Span-based model [Zhang+, 2019]
    ○ Extracts spans and predicts aspect
    labels for each span
    ● Doc-based model [Devlin+, 2019]
    ○ Predicts labels for an entire ad text
    ○ A document classification model
    BERT & CRF
    [CLS] Watch Docs, Movies & More - Get the
    First Month Free
    Free, First-time limited offer
    MLP
    O O O O O O O … O O O O O O … B I I I I … I I I I
    E

    Span Detection
    Label Prediction
    BERT
    [CLS] Watch Docs, Movies & More - Get the
    First Month Free
    MLP
    Label Prediction
    Free, First-time limited offer
    14

    

    View Slide

15. Experiments
    ● Aspect detection
    ○ Evaluate two models: the span-based and doc-based models
    ● Correlation analysis
    ○ Analyze which aspects labels are effective in each industry
    ● CTR prediction
    ○ Investigate whether the aspect labels contribute to CTR prediction
    Aspect detection CTR prediction
    Train 1,857 136,352
    Valid 465 16,084
    Test 410 15,976
    Also used for 

    correlation analysis 

    15

    

    View Slide

16. Results - Aspect Detection
    ● The doc-based model outperformed the span-based model.
    16

    

    View Slide

17. Results - Aspect Detection
    ● The doc-based model outperformed the span-based model.
    ● “Free”, “Speed”, “Transportation”, “Largest/no.1” are easy to detect.
    17

    “Free shipping”
    

    View Slide

18. Results - Aspect Detection
    ● The doc-based model outperformed the span-based model.
    ● “Free”, “Speed”, “Transportation”, “Largest/no.1” are easy to detect.
    ● The limited number of annotation causes low performance
    18

    

    View Slide

19. Correlation analysis
    ● Observed a weak correlation between the aspect labels and CTR
    19

    

    View Slide

20. Correlation analysis
    ● Observed a weak correlation between the aspect labels and CTR
    “One of the largest websites in
    Japan” (国内最大級サイト)
    “Get [N] points for new membership”
    (新規入会&利用で[N]ポイント)
    20

    

    View Slide

21. CTR Prediction
    ● Investigate the effectiveness of the aspect labels in CTR prediction
    ○ Can be a potential application for ad creation support
    ● Model
    ○ Input: ad text, “predicted” aspect labels
    ○ Output: CTR score [0-1]
    “Watch Docs, Movies & More - Get
    the First Month Free” 

    Free，First-time Limited Offer
    CTR score [0-1]
    BERT 0.74
    21

    

    View Slide

22. Results - CTR Prediction
    ● Identification of aspect labels contributes to CTR prediction
    ● Doc-based model improved the performance than span-based model
    22

    

    View Slide

23. Summary
    ● Explored the effective aspects of advertising appeals in different industries
    ○ Dataset construction
    ○ Aspect detection
    ○ Correlation analysis between aspects and CTR
    ● Demonstrated that each industry exhibits unique effective aspects and that identification
    of aspect labels contributes to CTR prediction
    Future direction
    ➔ Investigate whether presenting the effective aspects can help ad creators write
    effective ad texts in a real-world scenario
    ➔ Aspect-based ad text geneartion
    23

    

    View Slide

24. Acknowledgement
    ● Icons in the slides are from flaticon
    ○ https://www.flaticon.com/
    24

    

    View Slide