Slide 1

Slide 1 text

Mining Alternative Actions from Community Q&A Corpus for Task- Oriented Web Search Suppanut Pothirattanachaikul Kyoto University Takehiro Yamamoto Kyoto University Sumio Fujita Yahoo Japan Corporation Akira Tajima Yahoo Japan Corporation Katsumi Tanaka Kyoto University

Slide 2

Slide 2 text

Introduction (1/2) Many people use Web search engine to achieve their goal • 57.5% users answered that one main motivation for using Web search engines is to find ways or means to solve their goals[1] [1] http://www.dl.kuis.kyoto-u.ac.jp/i-explosion/report/index.html User Take sleeping pills 2 I want to sleep well

Slide 3

Slide 3 text

Introduction (2/2) 3 Many people prefer to modify their query based on the suggestion from the search engines User sleeping pills sleeping pills online sleeping pills name sleeping pills overdose Query Suggestions • 29% of the modified queries are submitted in a search session[2] [2] Kajta Hoffman et al. An Eye-tracking Study of User Interactions with Query Auto Completion

Slide 4

Slide 4 text

Problem People may not be aware of another solutions for achieving their goal Take sleeping pills ? Have a cup of hot milk Stroll before bedtime 4 I want to sleep well

Slide 5

Slide 5 text

Research Goal Suggest another solutions for achieving their goal so that they may find more appropriate solutions System suggest Take sleeping pills Have a cup of hot milk Stroll before bedtime 5 I want to sleep well

Slide 6

Slide 6 text

Definition (1/3) • Action • an activity which the user wants to achieve • represented as a verbal phrase • e.g. “take sleeping pills” , “sleep well” take sleeping pills sleep well Action 6

Slide 7

Slide 7 text

Definition (2/3) • Goal Relationship • a relationship between two actions If achieving an action ai also helps to achieve an action a g we call ag is a goal of ai sleep well is a goal of ag ai take sleeping pills 7

Slide 8

Slide 8 text

Definition (3/3) • Alternative-Action • a relationship between two actions • If action ai and action aj share the same goal as ag , ai and aj are alternative-actions sleep well take sleeping pills have a cup of hot milk is a goal of ag ai aj Alternative- Actions 8

Slide 9

Slide 9 text

Relation to existing works • Yamamoto et al and Yang et al focus on finding sub-tasks of a query • Given the query “sleeping problem”, returning “take sleeping pills” • However, our study focuses on finding alternative solutions behind a query 9

Slide 10

Slide 10 text

Ambiguity of a Goal • There may exist multiple goals • We cannot determine which goal a user has in his mind Take sleeping pills ? 10 I want to sleep well I want to relieve anxiety

Slide 11

Slide 11 text

Problem Statement Given a query representing an action, generate diversified ranked list of alternative-actions which satisfy as many goals as possible Take sleeping pills Query System Have a cup of hot milk See a therapist Eat Healthier sleep well cure anxiety relive stress 11

Slide 12

Slide 12 text

Challenges (1/2) • Data Resource for Mining • Query log cannot be used with our research • Few users reformulate their query to an alternative- action (i.e. “take sleeping pills” to “go for a jogging”) Query Input Time ambien 2006-03-19 16:15:04 ambien side effects 2006-03-21 16:36:30 klonopin side effects 2006-03-21 16:38:34 12

Slide 13

Slide 13 text

Challenges (2/2) • Textual Similarity cannot be used to find alternative-actions • Even “take sleeping pills” and “have a cup of hot milk” are textually dissimilar, they can be regarded as alternative-actions sleep well take sleeping pills have a cup of hot milk 13

Slide 14

Slide 14 text

Approach: Mining Alternative-Action from cQA Corpus • Question-and-Answer Service • Community website where questioner asks the solutions from various respondents • e.g. Reddit, Yahoo Chiebukuro Q: How can I sleep better? A: You should take sleeping pills Q: How to solve sleeping problems A: Please ask a doctor first Alternative-Actions 14

Slide 15

Slide 15 text

Approach: Mining Alternative-Action from cQA Corpus query q = take sleeping pills 1) Retrieve set of question-answer pairs cQA Corpus … Q1 A1 Q2 Qn 2) Extract candidate actions from answer A1 An 15 A2 An a11 a12 a13 … an1 an2 … anm a1m an3

Slide 16

Slide 16 text

Approach: Mining Alternative-Action from cQA Corpus 3) Compute an alternativeness through question-answer bipartite-graph 4) Generate a diversified ranked list of alternative actions have a cup of hot milk see a doctor eat healthier 16

Slide 17

Slide 17 text

Step 2: Extract candidate actions from answers What should I do to sleep well ? Questions Answers How about go for a run ? CONDITIONAL RANDOM FIELD (CRF) go for a run, have a cup of hot milk, … Candidate Actions 17

Slide 18

Slide 18 text

Step 3: Measure alternativeness between actions • Alternativeness • The value that measures how two actions share the same goal • If action has the high alternativeness to a query, it should be ranked high go to Stinson beach take sleeping pills have a cup of hot milk High Alternativeness Low Alternativeness How to compute alternativeness? 18

Slide 19

Slide 19 text

Step 3: Measure alternativeness between actions • Hypothesis (1): If two questions are likely to represent the same goal, actions in their answers are likely to be alternative-actions QUESTION ACTION How can I sleep better ? Have a cup of hot milk Take a hot shower How to solve sleeping problems ? 19

Slide 20

Slide 20 text

Step 3: Measure alternativeness between actions • Hypothesis (2): If two actions are likely to be alternative-actions, questions of the answers containing these actions are likely to represent the same goal QUESTION ACTION Have a cup of hot milk Take a hot shower 20 How can I sleep better ? How to solve sleeping problems ?

Slide 21

Slide 21 text

Step 3: Measure alternativeness between actions • Since two hypotheses are recursive to each other, SimRank[3] is applicable to compute similarity between nodes in a question-answer bipartite graph G [3] Glen et al. SimRank: a measure of structural-context similarity QUESTION ACTION How can I sleep well ? Have a cup of hot milk Take a hot shower To relieve stress, what should I do ? See a therapist How should I handle my tiredness? 21

Slide 22

Slide 22 text

Step 3: Measure alternativeness between actions • Based on the SimRank, alternativeness between two actions can be denoted as Where constant value C is a confident value and I(ai ) is a set of in-neighbors of ai which are questions 22 QUESTION ACTION Qi Qj ai aj

Slide 23

Slide 23 text

Step 3: Measure alternativeness between actions • While goal similarity between two questions can be denoted as QUESTION ACTION Qi Qj ai aj Where constant value C is a confident value and O(Qi ) is a set of out-neighbors of Qi which are actions 23

Slide 24

Slide 24 text

Step 4: Generate diversified ranked list of alternative-actions • MMR[4] is applied to diversify the alternative-actions by make it takes both relevance and diversity under its consideration • MMR selects ar, an action ranked at the r-th position using [4] Jaime et al. The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries. Action with high alternativeness to a query should be ranked high Action with high alternativeness to an already ranked action should be ranked low (goal diversification) 24

Slide 25

Slide 25 text

Evaluation - Datasets • Japanese Collection • Corpus archived in Yahoo! Chiebukuro • English Collection • Corpus archived in Reddit 25 Question Answer

Slide 26

Slide 26 text

Evaluation - Queries 26 • Query • 50 queries from three domains (health, recreation, and education) • Example: sleeping pills, airbnb, acupuncture, etc. Query Goal 1 Goal 2 Goal 3 sleeping pills be sleep well relieve anxiety airbnb reserve a room become a host acupucture relieve pain relieve headache solve a digestive problem

Slide 27

Slide 27 text

Evaluation - Method • Baselines: Query suggestions from two commercial search engines (QS1 and QS2). • Proposed Method: Alternative-Action Mining from Q&A Corpus 27

Slide 28

Slide 28 text

Evaluation - Results • D#nDCG: Measure that evaluates a ranked list in terms of both diversity and relevance • Using D#-nDCG@8, proposed method outperforms baselines(QS1 and QS2) D#-nDCG@8 (English) QS1 0.11 QS2 0.12 Proposed Method 0.39 D#-nDCG@8 (Japanese) QS1 0.02 QS2 0.01 Proposed Method 0.41 28

Slide 29

Slide 29 text

Evaluation - Examples Query = “chamomile tea” Proposed Method QS1 QS2 1st drink a cup of hot milk chamomile tea effect chamomile tea effect 2nd put it on your eyes how to make chamomile tea chamomile tea cough 29

Slide 30

Slide 30 text

Discussion (1) • We found that D#-nDCG obtained by our method is relatively low compare to other diversification problems. • This problem is caused by the list of candidate actions still contains irrelevant actions. • We aim to improve the step of candidate action extraction to accurately extract alternative-actions instead of irrelevant actions. 30

Slide 31

Slide 31 text

Discussion (2) • People usually use web search engines to confirm their belief[5]. • Thus, users may not take alternative-actions under their consideration. • Providing credibility or suggesting dispute sentence may raise an awareness of the searcher [5] Ryen et al. Beliefs and biases in web search. 31

Slide 32

Slide 32 text

Conclusion • Objective • Suggest alternative actions of the goal represented by a query • Approach : Mining Alternative-Action from cQA Corpus • Retrieve questions and answers from Q&A corpus • Extract candidate alternative-actions • Measure an alternativeness between actions • Result Diversification • Evaluation and Results • Discussions 32