Mining Alternative Actions from Community Q&A Corpus for Task-Oriented Web Search

Suppanut Pothirattanachaikul, Takehiro Yamamoto, Sumio Fujita, Akira Tajima and Katsumi Tanaka
Mining Alternative Actions from Community Q&A Corpus for Task-Oriented Web Search
In Proceeding of the 2017 IEEE/WIC/ACM International Conference on Web Intelligence (WI 2017), pp.607-614, August 2017.

Transcript

1. Mining Alternative Actions from Community Q&A Corpus for Task- Oriented Web Search Suppanut Pothirattanachaikul Kyoto University Takehiro Yamamoto Kyoto University Sumio Fujita Yahoo Japan Corporation Akira Tajima Yahoo Japan Corporation Katsumi Tanaka Kyoto University
2. Introduction (1/2) Many people use Web search engine to achieve their goal • 57.5% users answered that one main motivation for using Web search engines is to find ways or means to solve their goals[1] [1] http://www.dl.kuis.kyoto-u.ac.jp/i-explosion/report/index.html User Take sleeping pills 2 I want to sleep well
3. Introduction (2/2) 3 Many people prefer to modify their query based on the suggestion from the search engines User sleeping pills sleeping pills online sleeping pills name sleeping pills overdose Query Suggestions • 29% of the modified queries are submitted in a search session[2] [2] Kajta Hoffman et al. An Eye-tracking Study of User Interactions with Query Auto Completion
4. Problem People may not be aware of another solutions for achieving their goal Take sleeping pills ？ Have a cup of hot milk Stroll before bedtime 4 I want to sleep well
5. Research Goal Suggest another solutions for achieving their goal so that they may find more appropriate solutions System suggest Take sleeping pills Have a cup of hot milk Stroll before bedtime 5 I want to sleep well
6. Definition (1/3) • Action • an activity which the user wants to achieve • represented as a verbal phrase • e.g. “take sleeping pills” , “sleep well” take sleeping pills sleep well Action 6
7. Definition (2/3) • Goal Relationship • a relationship between two actions If achieving an action ai also helps to achieve an action a g we call ag is a goal of ai sleep well is a goal of ag ai take sleeping pills 7
8. Definition (3/3) • Alternative-Action • a relationship between two actions • If action ai and action aj share the same goal as ag , ai and aj are alternative-actions sleep well take sleeping pills have a cup of hot milk is a goal of ag ai aj Alternative- Actions 8
9. Relation to existing works • Yamamoto et al and Yang et al focus on finding sub-tasks of a query • Given the query “sleeping problem”, returning “take sleeping pills” • However, our study focuses on finding alternative solutions behind a query 9
10. Ambiguity of a Goal • There may exist multiple goals • We cannot determine which goal a user has in his mind Take sleeping pills ？ 10 I want to sleep well I want to relieve anxiety
11. Problem Statement Given a query representing an action, generate diversified ranked list of alternative-actions which satisfy as many goals as possible Take sleeping pills Query System Have a cup of hot milk See a therapist Eat Healthier sleep well cure anxiety relive stress 11
12. Challenges (1/2) • Data Resource for Mining • Query log cannot be used with our research • Few users reformulate their query to an alternative- action (i.e. “take sleeping pills” to “go for a jogging”) Query Input Time ambien 2006-03-19 16:15:04 ambien side effects 2006-03-21 16:36:30 klonopin side effects 2006-03-21 16:38:34 12
13. Challenges (2/2) • Textual Similarity cannot be used to find alternative-actions • Even “take sleeping pills” and “have a cup of hot milk” are textually dissimilar, they can be regarded as alternative-actions sleep well take sleeping pills have a cup of hot milk 13
14. Approach: Mining Alternative-Action from cQA Corpus • Question-and-Answer Service • Community website where questioner asks the solutions from various respondents • e.g. Reddit, Yahoo Chiebukuro Q: How can I sleep better? A: You should take sleeping pills Q: How to solve sleeping problems A: Please ask a doctor first Alternative-Actions 14
15. Approach: Mining Alternative-Action from cQA Corpus query q = take sleeping pills 1) Retrieve set of question-answer pairs cQA Corpus … Q1 A1 Q2 Qn 2) Extract candidate actions from answer A1 An 15 A2 An a11 a12 a13 … an1 an2 … anm a1m an3
16. Approach: Mining Alternative-Action from cQA Corpus 3) Compute an alternativeness through question-answer bipartite-graph 4) Generate a diversified ranked list of alternative actions have a cup of hot milk see a doctor eat healthier 16
17. Step 2: Extract candidate actions from answers What should I do to sleep well ? Questions Answers How about go for a run ? CONDITIONAL RANDOM FIELD (CRF) go for a run, have a cup of hot milk, … Candidate Actions 17
18. Step 3: Measure alternativeness between actions • Alternativeness • The value that measures how two actions share the same goal • If action has the high alternativeness to a query, it should be ranked high go to Stinson beach take sleeping pills have a cup of hot milk High Alternativeness Low Alternativeness How to compute alternativeness? 18
19. Step 3: Measure alternativeness between actions • Hypothesis (1): If two questions are likely to represent the same goal, actions in their answers are likely to be alternative-actions QUESTION ACTION How can I sleep better ? Have a cup of hot milk Take a hot shower How to solve sleeping problems ? 19
20. Step 3: Measure alternativeness between actions • Hypothesis (2): If two actions are likely to be alternative-actions, questions of the answers containing these actions are likely to represent the same goal QUESTION ACTION Have a cup of hot milk Take a hot shower 20 How can I sleep better ? How to solve sleeping problems ?
21. Step 3: Measure alternativeness between actions • Since two hypotheses are recursive to each other, SimRank[3] is applicable to compute similarity between nodes in a question-answer bipartite graph G [3] Glen et al. SimRank: a measure of structural-context similarity QUESTION ACTION How can I sleep well ? Have a cup of hot milk Take a hot shower To relieve stress, what should I do ? See a therapist How should I handle my tiredness? 21
22. Step 3: Measure alternativeness between actions • Based on the SimRank, alternativeness between two actions can be denoted as Where constant value C is a confident value and I(ai ) is a set of in-neighbors of ai which are questions 22 QUESTION ACTION Qi Qj ai aj
23. Step 3: Measure alternativeness between actions • While goal similarity between two questions can be denoted as QUESTION ACTION Qi Qj ai aj Where constant value C is a confident value and O(Qi ) is a set of out-neighbors of Qi which are actions 23
24. Step 4: Generate diversified ranked list of alternative-actions • MMR[4] is applied to diversify the alternative-actions by make it takes both relevance and diversity under its consideration • MMR selects ar, an action ranked at the r-th position using [4] Jaime et al. The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries. Action with high alternativeness to a query should be ranked high Action with high alternativeness to an already ranked action should be ranked low (goal diversification) 24
25. Evaluation - Datasets • Japanese Collection • Corpus archived in Yahoo! Chiebukuro • English Collection • Corpus archived in Reddit 25 Question Answer
26. Evaluation - Queries 26 • Query • 50 queries from three domains (health, recreation, and education) • Example: sleeping pills, airbnb, acupuncture, etc. Query Goal 1 Goal 2 Goal 3 sleeping pills be sleep well relieve anxiety airbnb reserve a room become a host acupucture relieve pain relieve headache solve a digestive problem
27. Evaluation - Method • Baselines: Query suggestions from two commercial search engines (QS1 and QS2). • Proposed Method: Alternative-Action Mining from Q&A Corpus 27
28. Evaluation - Results • D#nDCG: Measure that evaluates a ranked list in terms of both diversity and relevance • Using D#-nDCG@8, proposed method outperforms baselines(QS1 and QS2) D#-nDCG@8 (English) QS1 0.11 QS2 0.12 Proposed Method 0.39 D#-nDCG@8 (Japanese) QS1 0.02 QS2 0.01 Proposed Method 0.41 28
29. Evaluation - Examples Query = “chamomile tea” Proposed Method QS1 QS2 1st drink a cup of hot milk chamomile tea effect chamomile tea effect 2nd put it on your eyes how to make chamomile tea chamomile tea cough 29
30. Discussion (1) • We found that D#-nDCG obtained by our method is relatively low compare to other diversification problems. • This problem is caused by the list of candidate actions still contains irrelevant actions. • We aim to improve the step of candidate action extraction to accurately extract alternative-actions instead of irrelevant actions. 30
31. Discussion (2) • People usually use web search engines to confirm their belief[5]. • Thus, users may not take alternative-actions under their consideration. • Providing credibility or suggesting dispute sentence may raise an awareness of the searcher [5] Ryen et al. Beliefs and biases in web search. 31
32. Conclusion • Objective • Suggest alternative actions of the goal represented by a query • Approach : Mining Alternative-Action from cQA Corpus • Retrieve questions and answers from Q&A corpus • Extract candidate alternative-actions • Measure an alternativeness between actions • Result Diversification • Evaluation and Results • Discussions 32