of medical Web sites authorized by medical experts: < 50%* * E. Sillence et al., “Trust and Mistrust of Online Health Sites”, ACM CHI, pp.663-670, 2004
Tanaka. Enhancing Credibility Judgment of Web Search Results. In Proceedings of the 29th ACM SIGCHI Conference on Human Factors in Computing Systems (CHI 2011), pages 1235–1244, 2011. *1 Yin, X., Han, J., & Philip, S. Y. (2008). Truth discovery with multiple conflicting information providers on the web. IEEE Transactions on Knowledge and Data Engineering, 20(6), 796-808. TruthFinder*1 Scores the consistency of fact describing objects CowSearch*2 Provides supporting information for credibility judgment The analysis does not guarantee the correctness of information Limitation 4
82% # of young Japanese people who trust Web information *1 *1 Adobe Inc., “The State of Content : Rules of Engagement”, 2015 *2 S.Nakamura et al., “Trustworthiness analysis of Web search results (ECDL 2007) Never mind, never mind. # of people who trust information on SERP *2 I trust Google 6
lTrust in search engine’s ranking1 lWrong metrics for quality judgment (e.g. appearance of websites2) lCognitive bias3 7 1:Kahneman, D.: Thinking, fast and slow, Macmillan (2011) 2:Pan, B., Hembrooke, H., Joachims, et al In Google We Trust: Users’ Decisions on Rank, Position, and Relevance 3:Fogg, B. J, Soohoo, Cathy ,Danielson, David R, et al How Do Users Evaluate the Credibility of Web Sites? A Study with over 2,500 Participants ♥
... ” l“Research has shown...” 14 l“ It is often said ... ” l“ It is widely thought ... ” Who said that? What is the truth? Weasel Expressions create an impression that something specific and meaningful has been said, although their claim is ambiguous and lacks in evidence.
Wikipedia editting rules. Some weasel expressions are annotated with the special tags in Wikipedia. Ex: Who said？ 16 It is said that Company A has said this hoax. [By whom?]
been company A, a media and advertising company that wanted to show off their influence.[by whom?] ” l“ ~ is an established theory, but there are some objections. [who?]” 17 Image reference https://en.wikipedia.org/wiki/Wikipedia_logo
senteneces) Sentences annotated with [who?][by whom?] on Wikipedia 18 Negative examples(2236 senteneces) Sentences without [who?][by whom?] tags in Wikipedia articles where weasel expressions appear.
more data from editing histories on Wikipedia Frequency of weasel annotation varies depending on topics on Wikipedia Topic categories of positive examples may be biased Improvements in dataset
web information seeking. lH2 The proposed system increases the number of visited webpages. lH3 The proposed system improves the users’ confidence in their decisions made through web information seeking. lH4 The above-mentioned effects vary with users’ familiarity with the search topics. 25
to answer medical questions. • Fixed search results: Participants sought 100 fixed search results • 4 search tasks: We prepared 4 search tasks about medical topics. “Is cinnamon effective for diabetes? Report your answer by Web search ”
to two groups by UI condition lProposed group（105 participants ） – The system highlighted the weasel sentences while viewing the webpages – Authors manually decided which sentence should be highlighted as weasel sentences – Participants learnt what highlighted sentences meant before the task starts lControlled group（83 participants ） –Highlighting function was disabled 27
familiarity affected confidence change. If participants were not familiar with topics, our prototype changed prior confidence more significantly. If participants were familiar with topics, our prototype changed prior confidence less significantly.
enhance user engagement in careful information seeking on the web – The system increased pageview and session time. lWe need to investigate how user feel and use highlighted sentences – we didn’t understand why participants with our prototype viewed more webpages and spent longer time in search session 36
classification performance of weasel sentences were generally good performance but needs more improvement lUser study to examine the system effect – The proposed system can enhance user engagement in critical information seeking on the web. lFuture works – Improvement of weasel classifier – Investigating weasel sentences on the Web 37
reverentgeek
rmw
skipperchong
carmenhchung
keathley
productmarketing
clairelew
thoeni
jcasabona
cherdarchuk
lauravandoore
danielanewman
