Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What Web Search Behaviors Lead to Online Purchase Satisfaction? (EN)

What Web Search Behaviors Lead to Online Purchase Satisfaction? (EN)

Yuki Yanagida, Makoto P. Kato, Yuka Kawada, Takehiro Yamamoto, Hiroaki Ohshima and Sumio Fujita. What Web Search Behaviors Lead to Online Purchase Satisfaction?. In Proceedings of the 15th ACM Web Science Conference (WebSci 2023). Online & Austin, Texas, USA., Apr. 2023.

YANAGIDA Yuki

May 01, 2023
Tweet

More Decks by YANAGIDA Yuki

Other Decks in Research

Transcript

  1. What Web Search Behaviors Lead to
    Online Purchase Satisfaction?
    Yuki Yanagida, Makoto P. Kato (University of Tsukuba)
    Yuka Kawada, Takehiro Yamamoto, Hiroaki Ohshima
    (University of Hyogo)
    Sumio Fujita (Yahoo Japan Corporation)
    15th ACM Web Science Conference (WebSci 2023)

    View full-size slide

  2. Satisfying people purchasing products is an important issue in EC
    • “Post-purchase” satisfaction improves to be continuously use of EC sites [1]
    • Customer satisfaction can have a big impact since 41% of people shop
    online at least once a week [2]
    [1] Gustafsson et al. The effects of customer satisfaction, relationship commitment dimensions, and triggers on customer retention. Journal of
    marketing, 69(4), 2005.
    [2] PwC. December 2021 global consumer insights pulse survey. https://www.pwc.com/gx/en/industries/consumer-markets/consumer-
    insights-survey/archive/consumer-insights-survey-december-2021.html, (accessed 2022-11-28).
    Background 2

    View full-size slide

  3. We investigate the relationship between
    information-seeking behavior on the web and post-purchase satisfaction
    • The findings of this study enables us to:
    ‒ Predict post-purchase satisfaction from web search behavior for non-reviewing users
    • Also addressed in this study
    Purpose 3
    Examine the relationship between web search behavior and review ratings on EC site


    Def
    Subset
    Data Satisfaction
    Studied
    search behavior
    Existing post-purchase
    satisfaction study
    Product or user
    characteristics
    “Post-purchase”
    satisfaction

    Existing search behavior
    study
    Logs, subject experiment
    “Search”
    satisfaction
    “EC” search
    This study
    Logs, product or user
    characteristics
    “Post-purchase”
    satisfaction
    “Web” search
    Comparison to existing studies

    View full-size slide

  4. • RQ1: Are web search behaviors different for SAT (satisfied) and DSAT
    (dissatisfied) users?
    A Within a week prior to the purchase, SAT users more frequently searched for a wider
    range of product-related information
    • RQ2: Are web search behaviors different depending on post-purchase
    satisfaction and product/user characteristics?
    A SAT users searched with more specific queries prior to purchase when:
    • They are relatively familiar with web search
    • They are purchasing a relatively expensive product
    • RQ3: What is the relationship between query terms and satisfaction?
    A Users looking for opinions of others are more likely to be satisfied with their purchase
    RQs and Their Answers 4

    View full-size slide

  5. We estimate the search intent of each query
    to characterize web search behaviors
    Analysis Flow 5
    Query
    EOS R5 price
    popular camera
    night photography
    weather in Tokyo
    Query log
    Query Intent
    EOS R5 price TF (narrow)
    popular camera DM (broad)
    night photography Other
    weather in Tokyo Other
    Query Intent
    EOS R5 price TF (narrow)
    popular camera DM (broad)
    night photography DM (broad)
    weather in Tokyo Other
    Rule-based estimation
    Estimation by
    weak supervision
    Classify queries
    =
    Query frequency calculation
    for each time segment
    Time
    Frequency
    Web queries are enormous
    → Identify each intent systematically based on an existing taxonomy
    SAT
    DSAT
    We sampled logs from a major commercial search engine and EC sites
    (The same account was used in these services)

    View full-size slide

  6. We want to estimate the search intent of each query
    → Create product dictionary and classify queries based on rules
    • TF: considering purchase and
    searching narrowly
    ‒ Corresponds to Target Finding in [3]
    ‒ It contains a product name, a brand name,
    or model number in camera
    • DM: searching broadly than TF
    ‒ Corresponds to Decision Making in [3]
    ‒ Condition TF is not met but contains
    “camera”
    • Other
    Rule-based Estimation 6
    Query Intent
    EOS R5 price TF (narrow)
    popular camera DM (broad)
    night photography Other
    weather in Tokyo Other
    Related to camera but classified as Other
    [3] Su et al. User intent, behaviour, and perceived satisfaction in product search. WSDM 2018.
    e.g. : Camera category
    Search intent was defined
    based on previous research [3]
    Analyze differences in search behavior by intent
    We use a weak supervision for words that cannot
    be classified by the rule-based approach
    Measure

    View full-size slide

  7. • RQ1: Are web search behaviors different for SAT (satisfied) and DSAT
    (dissatisfied) users?
    A Within a week prior to the purchase, SAT users more frequently searched for a wider
    range of product-related information
    A
    RQs and Their Answers 7

    View full-size slide

  8. Temporal distribution of TF (narrow) queries
    RQ1: Are Web Search Behaviors Different for SAT and DSAT Users? 8
    0
    0.0001
    0.0002
    0.0003
    0.0004
    0.0005
    (-4w
    ,-3w
    ]
    (-3w
    ,-2w
    ]
    (-2w
    ,-1w
    ]
    (-1w
    ,0]
    [0,+
    1w
    )
    [+
    1w
    ,+
    2w
    )
    [+
    2w
    ,+
    3w
    )
    [+
    3w
    ,+
    4w
    )
    Search frequency
    Time segment
    SAT
    DSAT
    DM (broad) was similar
    Within a week prior to the
    purchase, SAT users more
    searched with TF (narrow)
    SAT users looked for a broader
    range of information
    Hypothesize
    Normalized frequency by
    total number of searches
    e.g. [0,+1w): the time segment
    within a week of purchase
    Purchase
    6/Mar 10/Mar 13/Mar 17/Mar
    Purchase
    Time
    Align and to count

    View full-size slide

  9. Comparison of Query Term Entropy 9
    We compared query term entropy prior to the purchase
    to examine whether SAT users looked for a broader range of information
    SAT users used more diverse query terms than
    DSAT users for both TF and DM queries
    0
    0.2
    0.4
    0.6
    0.8
    1
    1.2
    TF (narrow) DM (broad)
    Average Entropy
    5.8
    6
    6.2
    6.4
    6.6
    6.8
    7
    Other
    SAT
    DSAT
    Intent

    View full-size slide

  10. A
    • RQ2: Are web search behaviors different depending on post-purchase
    satisfaction and product/user characteristics?
    A SAT users searched with more specific queries prior to purchase when:
    • They are relatively familiar with web search
    • They are purchasing a relatively expensive product
    RQs and Their Answers 10
    • Search intent
    ‒ TF (narrow): Narrower search
    ‒ DM (broad): Broader search than TF

    View full-size slide

  11. RQ2: Are Web Search Behaviors Different Depending on Post-purchase Satisfaction and Product/user Characteristics? 11
    Search Frequency
    Temporal distribution of TF (narrow) queries for each
    combination of product price and search frequency levels
    SAT
    DSAT
    Search Frequency
    Purchase
    Purchase
    Low-price
    Low-search-
    frequency
    High-price
    Low-search-
    frequency
    High-price
    High-search-
    frequency
    Low-price
    High-search-
    frequency
    Purchase
    Purchase

    View full-size slide

  12. Temporal distribution of TF (narrow) queries for each
    combination of product price and search frequency levels
    RQ2: Are Web Search Behaviors Different Depending on Post-purchase Satisfaction and Product/user Characteristics? 12
    SAT
    DSAT
    High-price
    High-search-
    frequency Purchase
    If they are familiar with web search, they might turn to conduct
    web search about a specific product to lower the purchasing risk
    Only when the product price and search frequency
    were high, SAT users searched more frequently with
    TF queries prior to the purchase

    View full-size slide

  13. A
    A
    • RQ3: What is the relationship between query terms and satisfaction?
    A Users looking for opinions of others are more likely to be satisfied with their purchase
    RQs and Their Answers 13

    View full-size slide












  14. • Clustering query
    ‒ Averaged the embeddings of words in TF
    and DM queries used by each user and
    obtained a vector representing each user
    ‒ Normalized each cluster by the number
    of searches for purchase-related terms
    • Compare satisfaction with each cluster
    ‒ C3 has the highest percentage of satisfaction
    ‒ C2 has the lowest percentage of satisfaction
    RQ3: What Is The Relationship between Query Terms and Satisfaction? 14
    purchase,
    store, etc.
    coupon,
    sale, etc.
    warranty,
    repair, etc.
    Cluster (C)
    recommen-
    dation, etc.
    reputation,
    etc.
    latest,
    2017, etc.
    searched
    with years
    looked for
    reviews
    Purchase-related
    word group
    looked for
    suggestion
    visited
    EC sites
    average
    cluster
    C1 C2 C3 C4 C5
    Users looking for opinions of others are more
    likely to be satisfied with their purchase
    Suggest

    View full-size slide

  15. • Prediction by temporal distribution of search intents and query words
    ‒ Frequency of search intents
    ‒ Bag of Words created from the queries
    • No characteristic weights were seen
    ‒ Averaged word embeddings from the queries
    ‒ Additional pre-training BERT by the queries
    • Ordered TF or DM queries chronologically
    and split by [SEP]
    Satisfaction Prediction 15
    0.531
    0.559
    0.530 0.529
    0.559
    0.548
    0.533
    0.48
    0.5
    0.52
    0.54
    0.56
    0.58
    Accuracy
    Frequency of search intents
    Frequency of search intents + product and user features
    Bag of Words
    Word embeddings
    Word embeddings + product and user features
    BERT
    BERT + product and user features
    Accuracy of each model
    ① BERT has the highest accuracy
    • Time series of queries may be useful for predicting
    ② The product and user features
    improve accuracy
    • BERT does not learn enough word/the features
    combinations due to lack of training data


    0

    View full-size slide

  16. • Implication
    ‒ SAT users searched for products more frequently and diversely before the purchase
    ‒ In the marketing research literature, an increase in the amount of search leads to
    customer satisfaction indirectly [4]
    Implication 16
    Suggest
    It should encourage a user to search
    more frequently and diversely before purchasing
    [4] Punj and Staelin. A model of consumer information search behavior for new automobiles. Journal of consumer research, 9(4), 1983.
    popular camera
    EOS R5 reputation
    Nikon camera
    Typical
    SAT user
    Typical
    DSAT user
    EOS R5 sale
    camera sale
    Purchase
    Search timeline
    Canon camera popular
    Recommend queries
    e.g.

    View full-size slide

  17. • Within a week prior to the purchase, SAT users more frequently
    searched for a wider range of product-related information
    • SAT users searched with more specific queries prior to purchase when:
    ‒ They are relatively familiar with web search
    ‒ They are purchasing a relatively expensive product
    • Users looking for opinions of others are more likely to be satisfied with
    their purchase
    Summary 17
    Main findings
    We investigate the relationship between
    information-seeking behavior on the web and post-purchase satisfaction
    Examine the relationship between Web search behavior and review ratings on EC site

    Subset

    Def

    View full-size slide

  18. Related Work: Customer Satisfaction 19
    • Relationship between the level of interest in the product and
    “post-purchase” satisfaction [5]
    • Relationship between perceived product quality and “post-purchase”
    satisfaction [6]
    • User interest and product quality affect “post-purchase” satisfaction
    • Whereas the effect of web information-seeking behaviors on the post-
    purchase satisfaction has not been widely explored yet
    Satisfaction
    Positive
    correlation
    Product
    quality
    Satisfaction
    Positive
    correlation
    Level of
    interest
    [5] Richins et al. Post-purchase product satisfaction: Incorporating the effects of involvement and time. Journal of Business Research, 23(2), 1991.
    [6] Tsiotsou. The role of perceived product quality and overall satisfaction on purchase intentions. International journal of consumer studies, 30(2), 2006.

    View full-size slide

  19. • Relationship between search behaviors in EC sites and satisfaction
    with search results [3]
    ‒ They also showed that searches directly related to purchasing can be classified into
    two types of intent
    • The relationship between search intent and "search result"
    satisfaction was examined, but not "post-purchase" satisfaction
    [3] Su et al. User intent, behaviour, and perceived satisfaction in product search. WSDM 2018.
    Related Work: Search Behaviors Related to Purchasing 20
    Search sessions on EC sites
    Display
    search results
    Satisfaction with search results
    Satisfied
    e.g.
    Query
    popular camera
    night photography
    Query
    EOS spec
    EOS R5 price
    EOS R5 used
    TF
    (narrow)
    DM
    (broad)
    Dissatisfied
    Intent to consider purchasing
    and search narrowly
    Intent to search products broadly
    EOS: Camera Brand

    View full-size slide

  20. • Yahoo! JAPAN Search: logs of users routinely using the search engine
    (≡ who conducted web search at least 10 days in each month)
    ‒ Sampled 13,882 users and 48,758,880 queries
    • Yahoo! JAPAN Shopping: Purchase log and Reviews
    ‒ Out of 5-point scale, reviews with a rating of three or lower tend to include negative
    comments so purchase with a 4 or 5 rating was defined as satisfactory purchase
    ‒ Num. of purchases: 15,277
    • Since the same account was used in these services, we could identify
    the same user's web search behavior and purchases
    Dataset 21

    View full-size slide

  21. • 32 categories were grouped into 5 meta categories:
    Appliance, Audio, Beauty, Gadgets, and Outdoor
    • We grouped 32 categories into 5 meta categories and analyzed about
    fifteen thousand purchases
    Data Statistics 22
    Search log Purchase log Reviews
    Service Yahoo! JAPAN Search Yahoo! JAPAN Shopping
    Period 2016 - 2017 2016 - 2018
    Num. of users 13,882
    Num. of purchases N/A SAT: 12,620; DSAT: 2,657
    Num. of queries 48,758,880 N/A
    Ave. query length 1.70 N/A
    Ave. session length 3.03 N/A

    View full-size slide

  22. We use a weak supervision approach for
    words that cannot be classified by the rule-based approach
    • We want to identify query related to camera
    • Identify words related to camera from queries classified as “Other”
    ‒ Weak supervision classifies them as DM (broad) because the word is not included in
    the product dictionary
    Intent Estimation by Weak Supervision 23
    Query Intent
    EOS R5 price TF (narrow)
    popular camera DM (broad)
    night photography Other
    weather in Tokyo Other
    Query Intent
    EOS R5 price TF (narrow)
    popular camera DM (broad)
    night photography DM (broad)
    weather in Tokyo Other
    Classify
    Words related to camera but classified as Other
    =

    View full-size slide

  23. Pseudo positive: query between model
    number, product name, or “camera”
    Detail of Weak Supervision 24
    Search session Session probably not
    related to camera
    Tokyo sightseeing
    Mt. Fuji history
    camera for beginners
    how to choose a lens
    EOS R5 spec
    Intent estimated
    by rule-base
    DM (broad)
    Other
    TF (narrow)
    Other
    Other
    Search timeline
    Probably related
    to camera
    Pseudo negative: queries for sessions where
    TF or DM never appear
    • We used logistic regression to predict
    ‒ Prediction results with support vector machines, etc. were almost the same
    • Sampled 100 queries for each meta category and take a majority vote with three annotators
    ‒ Accuracy: 0.91, F1 score: 0.53
    • Predict camera accessories, etc. as positive

    View full-size slide

  24. How to Normalize Search Frequency 25
    Intent ・・・ (-1w, 0] [0, +1w) ・・・
    TF (narrow) 6 3
    DM (broad) 4 2
    Other 10 5
    Num. of searches user u
    Normalized by
    the total search frequency
    Intent ・・・ (-1w, 0] [0, +1w) ・・・
    TF (narrow) 6/30 3/30
    DM (broad) 4/30 2/30
    Other 10/30 5/30
    Normalize
    : 30
    Average the value of each cell across
    all users and graph the resulting values
    𝑢!,#
    𝑢∗,∗
    Normalized by time
    segment t and intent i
    Sum of

    View full-size slide

  25. RQ2: Are Web Search Behaviors Different Depending on Post-purchase Satisfaction and Product/user Characteristics? (DM) 26
    Search Frequency
    Temporal distribution of DM (broad) queries for each
    combination of product price and search frequency levels
    SAT
    DSAT
    Search Frequency
    Purchase
    Purchase
    Low-price
    Low-search-
    frequency
    High-price
    Low-search-
    frequency
    High-price
    High-search-
    frequency
    Low-price
    High-search-
    frequency
    Purchase
    Purchase

    View full-size slide

  26. Temporal distribution of DM (broad) queries for each
    combination of product price and search frequency levels
    RQ2: Are Web Search Behaviors Different Depending on Post-purchase Satisfaction and Product/user Characteristics? (DM) 27
    Purchase
    Only when the product price and search frequency
    were low, DSAT users searched more frequently
    with DM queries posterior to the purchase
    When customers are dissatisfied with their purchase
    and the product price is low, they might turn to find
    alternatives to the purchased product

    View full-size slide










  27. Compare satisfaction with each cluster
    Characteristics of Each Cluster 28
    Users looking for opinions of others are
    more likely to be satisfied with their purchase
    C3 has the highest percentage of satisfaction
    C2 has the lowest percentage of satisfaction
    Ave. Num. of
    Search TF (narrow)
    Ave. Num. of
    Search DM (broad)
    Ratio of SAT
    Ave. Price
    Cluster (C)
    C1 C2 C3 C4 C5











    Cluster (C)
    searched
    with years
    looked for
    reviews
    looked for
    suggestion
    visited
    EC sites
    average
    cluster
    C1 C2 C3 C4 C5
    purchase,
    store, etc.
    coupon,
    sale, etc.
    warranty,
    repair, etc.
    recommen-
    dation, etc.
    reputation,
    etc.
    latest,
    2017, etc.
    Purchase-related
    word group

    View full-size slide

  28. SAT DSAT
    S 2(1f c2(-,
    1: 0e.l ceme,2, .0-blem, (,q3(07
    0e. (0, b37(,g, 5 00 ,27
    2: (eC-m A), (eC-m B), m (l -0de0
    3c2(-,, 1ell(,g, 12-0e, (eC-m C)
    1h-..(,g, (eC-m D), .30ch 1e
    (0e2 (le0 X), (eC-m E), b--)(,g
    3: c-m. 0(1-,, d(ffe0e,ce, beg(,,e0
    27.e, h-5 2- ch--1e, m )e0
    4: .0(ce, (,e6.e,1(4e, f-0 1-,g, c-12
    5: l(gh2, 1(8e, 1.ec, 5e(gh2
    6: 1(l4e0, 5h(2e, 0ed, c-l-0, bl c)
    7: 1 le, .-(,2, c m. (g,, c-3.-,
    8: W-M, e4 l3 2(-,, 0e4(e5, 0e.32 2(-,
    9: 2015, 2016, 2017, ,e5-27.e, l 2e12
    10: h-5 2- 31e, 1e23.
    11: 0ec-mme,d, .-.3l 0, 0 ,)(,g
    Te0m g0-3.
    0.033 0.042
    0.173 0.180
    0.072 0.061
    0.099 0.130
    0.025 0.019
    0.043 0.042
    0.024 0.028
    0.115 0.123
    0.070 0.070
    0.021 0.017
    0.325 0.287
    DIFF
    -0.009
    -0.007
    0.012
    -0.031
    0.006
    0.001
    -0.004
    -0.009
    -0.000
    0.004
    0.038
    The Difference of Query Terms 29
    SAT users searched for
    suggestions
    DSAT users
    searched for price

    View full-size slide

  29. -0.2
    -0.1
    0
    0.1
    0.2
    0.3
    0.4
    (-4w
    ,-3w
    ]
    (-3w
    ,-2w
    ]
    (-2w
    ,-1w
    ]
    (-1w
    ,0]
    [0, +
    1w
    )
    [+
    1w
    , +
    2w
    )
    [+
    2w
    , +
    3w
    )
    [+
    3w
    , +
    4w
    )
    Price
    Search
    Frequency
    Weight
    Product and user features and time segment for each intent
    TF (narrow)
    DM (broad)
    Product and user features *
    *
    *
    *
    • Prediction by temporal distribution of search intents
    ‒ We used Random Forest and Logistic
    Regression to indicate feature weights
    ‒ Accuracy using
    product and user features: 0.56
    • = The level of price and search frequency
    Prediction by Temporal Distribution of Search Intents 30
    Weights in Logistic Regression
    (* indicates that the factor is statistically significant)
    We try to predict (explain) satisfaction based on differences
    in the temporal distribution of search intent and query words
    Accuracy is not high, but the effects of some factors are statistically significant

    View full-size slide

  30. Limitation 31
    • The limitations of our study
    ‒ it does not take into account the information actually collected by users
    • More detailed relationship between user behavior and purchase satisfaction will be the
    subject of future research

    View full-size slide