Mining Alternative Actions from Community Q&A Corpus for Task- Oriented Web Search Suppanut Pothirattanachaikul Kyoto University Takehiro Yamamoto Kyoto University Sumio Fujita Yahoo Japan Corporation Akira Tajima Yahoo Japan Corporation Katsumi Tanaka Kyoto University

Introduction (1/2) Many people use Web search engine to achieve their goal • 57.5% users answered that one main motivation for using Web search engines is to find ways or means to solve their goals[1] [1] http://www.dl.kuis.kyoto-u.ac.jp/i-explosion/report/index.html User Take sleeping pills 2 I want to sleep well

Introduction (2/2) 3 Many people prefer to modify their query based on the suggestion from the search engines User sleeping pills sleeping pills online sleeping pills name sleeping pills overdose Query Suggestions • 29% of the modified queries are submitted in a search session[2] [2] Kajta Hoffman et al. An Eye-tracking Study of User Interactions with Query Auto Completion

Problem People may not be aware of another solutions for achieving their goal Take sleeping pills ？ Have a cup of hot milk Stroll before bedtime 4 I want to sleep well

Research Goal Suggest another solutions for achieving their goal so that they may find more appropriate solutions System suggest Take sleeping pills Have a cup of hot milk Stroll before bedtime 5 I want to sleep well

Definition (1/3) • Action • an activity which the user wants to achieve • represented as a verbal phrase • e.g. “take sleeping pills” , “sleep well” take sleeping pills sleep well Action 6

Definition (2/3) • Goal Relationship • a relationship between two actions If achieving an action ai also helps to achieve an action a g we call ag is a goal of ai sleep well is a goal of ag ai take sleeping pills 7

Definition (3/3) • Alternative-Action • a relationship between two actions • If action ai and action aj share the same goal as ag , ai and aj are alternative-actions sleep well take sleeping pills have a cup of hot milk is a goal of ag ai aj Alternative- Actions 8

Relation to existing works • Yamamoto et al and Yang et al focus on finding sub-tasks of a query • Given the query “sleeping problem”, returning “take sleeping pills” • However, our study focuses on finding alternative solutions behind a query 9

Ambiguity of a Goal • There may exist multiple goals • We cannot determine which goal a user has in his mind Take sleeping pills ？ 10 I want to sleep well I want to relieve anxiety

Problem Statement Given a query representing an action, generate diversified ranked list of alternative-actions which satisfy as many goals as possible Take sleeping pills Query System Have a cup of hot milk See a therapist Eat Healthier sleep well cure anxiety relive stress 11

Challenges (1/2) • Data Resource for Mining • Query log cannot be used with our research • Few users reformulate their query to an alternative- action (i.e. “take sleeping pills” to “go for a jogging”) Query Input Time ambien 2006-03-19 16:15:04 ambien side effects 2006-03-21 16:36:30 klonopin side effects 2006-03-21 16:38:34 12

Challenges (2/2) • Textual Similarity cannot be used to find alternative-actions • Even “take sleeping pills” and “have a cup of hot milk” are textually dissimilar, they can be regarded as alternative-actions sleep well take sleeping pills have a cup of hot milk 13

Approach: Mining Alternative-Action from cQA Corpus • Question-and-Answer Service • Community website where questioner asks the solutions from various respondents • e.g. Reddit, Yahoo Chiebukuro Q: How can I sleep better? A: You should take sleeping pills Q: How to solve sleeping problems A: Please ask a doctor first Alternative-Actions 14

Approach: Mining Alternative-Action from cQA Corpus query q = take sleeping pills 1) Retrieve set of question-answer pairs cQA Corpus … Q1 A1 Q2 Qn 2) Extract candidate actions from answer A1 An 15 A2 An a11 a12 a13 … an1 an2 … anm a1m an3

Approach: Mining Alternative-Action from cQA Corpus 3) Compute an alternativeness through question-answer bipartite-graph 4) Generate a diversified ranked list of alternative actions have a cup of hot milk see a doctor eat healthier 16

Step 2: Extract candidate actions from answers What should I do to sleep well ? Questions Answers How about go for a run ? CONDITIONAL RANDOM FIELD (CRF) go for a run, have a cup of hot milk, … Candidate Actions 17

Step 3: Measure alternativeness between actions • Alternativeness • The value that measures how two actions share the same goal • If action has the high alternativeness to a query, it should be ranked high go to Stinson beach take sleeping pills have a cup of hot milk High Alternativeness Low Alternativeness How to compute alternativeness? 18

Step 3: Measure alternativeness between actions • Hypothesis (1): If two questions are likely to represent the same goal, actions in their answers are likely to be alternative-actions QUESTION ACTION How can I sleep better ? Have a cup of hot milk Take a hot shower How to solve sleeping problems ? 19

Step 3: Measure alternativeness between actions • Hypothesis (2): If two actions are likely to be alternative-actions, questions of the answers containing these actions are likely to represent the same goal QUESTION ACTION Have a cup of hot milk Take a hot shower 20 How can I sleep better ? How to solve sleeping problems ?

Step 3: Measure alternativeness between actions • Since two hypotheses are recursive to each other, SimRank[3] is applicable to compute similarity between nodes in a question-answer bipartite graph G [3] Glen et al. SimRank: a measure of structural-context similarity QUESTION ACTION How can I sleep well ? Have a cup of hot milk Take a hot shower To relieve stress, what should I do ? See a therapist How should I handle my tiredness? 21

Step 3: Measure alternativeness between actions • Based on the SimRank, alternativeness between two actions can be denoted as Where constant value C is a confident value and I(ai ) is a set of in-neighbors of ai which are questions 22 QUESTION ACTION Qi Qj ai aj

Step 3: Measure alternativeness between actions • While goal similarity between two questions can be denoted as QUESTION ACTION Qi Qj ai aj Where constant value C is a confident value and O(Qi ) is a set of out-neighbors of Qi which are actions 23

Step 4: Generate diversified ranked list of alternative-actions • MMR[4] is applied to diversify the alternative-actions by make it takes both relevance and diversity under its consideration • MMR selects ar, an action ranked at the r-th position using [4] Jaime et al. The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries. Action with high alternativeness to a query should be ranked high Action with high alternativeness to an already ranked action should be ranked low (goal diversification) 24

Evaluation - Datasets • Japanese Collection • Corpus archived in Yahoo! Chiebukuro • English Collection • Corpus archived in Reddit 25 Question Answer

Evaluation - Queries 26 • Query • 50 queries from three domains (health, recreation, and education) • Example: sleeping pills, airbnb, acupuncture, etc. Query Goal 1 Goal 2 Goal 3 sleeping pills be sleep well relieve anxiety airbnb reserve a room become a host acupucture relieve pain relieve headache solve a digestive problem

Evaluation - Method • Baselines: Query suggestions from two commercial search engines (QS1 and QS2). • Proposed Method: Alternative-Action Mining from Q&A Corpus 27

Evaluation - Results • D#nDCG: Measure that evaluates a ranked list in terms of both diversity and relevance • Using D#-nDCG@8, proposed method outperforms baselines(QS1 and QS2) D#-nDCG@8 (English) QS1 0.11 QS2 0.12 Proposed Method 0.39 D#-nDCG@8 (Japanese) QS1 0.02 QS2 0.01 Proposed Method 0.41 28

Evaluation - Examples Query = “chamomile tea” Proposed Method QS1 QS2 1st drink a cup of hot milk chamomile tea effect chamomile tea effect 2nd put it on your eyes how to make chamomile tea chamomile tea cough 29

Discussion (1) • We found that D#-nDCG obtained by our method is relatively low compare to other diversification problems. • This problem is caused by the list of candidate actions still contains irrelevant actions. • We aim to improve the step of candidate action extraction to accurately extract alternative-actions instead of irrelevant actions. 30

Discussion (2) • People usually use web search engines to confirm their belief[5]. • Thus, users may not take alternative-actions under their consideration. • Providing credibility or suggesting dispute sentence may raise an awareness of the searcher [5] Ryen et al. Beliefs and biases in web search. 31

Conclusion • Objective • Suggest alternative actions of the goal represented by a query • Approach : Mining Alternative-Action from cQA Corpus • Retrieve questions and answers from Q&A corpus • Extract candidate alternative-actions • Measure an alternativeness between actions • Result Diversification • Evaluation and Results • Discussions 32