APWeb2012発表資料

 APWeb2012発表資料

発表タイトル:Disputed sentence suggestion towards credibility-oriented Web search

059fb717431a8cd2b509ffebc57d905a?s=128

Y. Yamamoto

April 12, 2012
Tweet

Transcript

  1. %JTQVUFETFOUFODFTVHHFTUJPOUPXBSET DSFEJCJMJUZPSJFOUFE8FCTFBSDI :VTVLF:BNBNPUP ,ZPUP6OJWFSTJUZ yamamoto@kura.kyoto-u.ac.jp

  2. Q. *TUIF8FCDSFEJCMF

  3. 4FSJPVTTJUVBUJPOPG8FCDSFEJCJMJUZ ※2 S. Nakamura et al., Trustworthiness analysis of Web

    search results (ECDL 2007) -BSHFBNPVOUPG TVTQJDJPVT8FCJOGP .BOZQFPQMFUSVTU 8FCTFBSDISFTVMUT 57 43 % % Credible Non-credible Questionnaire on Web search engine More than 50% of medical HP are unauthorized(*1) I can trust Google! ※1 E. Sillence et al., Trust and mistrust of online health sites (CHI 2004)
  4. $PX4FBSDIDSFEJCJMJUZPSJFOUFE8FCTFBSDIFOHJOF 7JTVBMJ[FT TDPSFTPG8FCQBHFT POJNQPSUBOU DSFEJCJMJUZDSJUFSJB 1. 2. +VTJDF%JFU(PPEPS#BE 3. 4.

    F⒎FDUJWFEJFUNFUIPE 4LJQCSFBLGBTUGPSEJFU )$(EJFUJTUIFCFTUXBZ %JFUMBCPSBUPSZ 3DJH5DQN &RYHUDJH 7\SLFDOLW\ 5HSXWDWLRQ )UHVKQHVV 3DJH5DQN &RYHUDJH 7\SLFDOLW\ 5HSXWDWLRQ )UHVKQHVV 3DJH5DQN &RYHUDJH 7\SLFDOLW\ 5HSXWDWLRQ )UHVKQHVV 3DJH5DQN &RYHUDJH 7\SLFDOLW\ 5HSXWDWLRQ )UHVKQHVV 3DJH5DQN &RYHUDJH 7\SLFDOLW\ 5HSXWDWLRQ )UHVKQHVV TFBSDI Coverage and reputation are important! * Enhancing credibility judgment of Web search results, ACM CHI 2011. 2$BO$PX4FBSDI IFMQVTFST "/PUQFSGFDU
  5. &MBCPSBUJPOMFWFMUPDSFEJCJMJUZKVEHNFOU 1FPQMF XJUIPVUFYQFSUJTFPSJOUFSFTUDBOOPU OPUJDFTVTQJDJPVTJOGPSNBUJPOTPXFMM )JHI GBNJMJBSJUZ .VDI JOUFSFTU 'PSRVFSZ 4FMFDUTDSFEJCMF

    8FCQBHFT 4IBMMPX DSFEJCJMJUZ KVEHNFOU -PX GBNJMJBSJUZ -JUUMFJOUFSFTU 'PSRVFSZ User 4USJDUDSFEJCJMJUZ KVEHNFOU 5 (FUTTVTQJDJPVT QBHFT XJUIPVUOPUJDF
  6. &MBCPSBUJPOMFWFMUPDSFEJCJMJUZKVEHNFOU 1FPQMF XJUIPVUFYQFSUJTFPSJOUFSFTUDBOOPU OPUJDFTVTQJDJPVTJOGPSNBUJPOTPXFMM )JHI GBNJMJBSJUZ .VDI JOUFSFTU 'PSRVFSZ (FUTDSFEJCMF

    8FCQBHFT 4IBMMPX DSFEJCJMJUZ KVEHNFOU -PX GBNJMJBSJUZ -JUUMFJOUFSFTU 'PSRVFSZ User 4USJDUDSFEJCJMJUZ KVEHNFOU 6 (FUTTVTQJDJPVT QBHFT XJUIPVUOPUJDF 'PSVTJOH$PX4FBSDI NPSFFGGFDUJWFMZ *U`TOFDFTTBSZUPNBLFVTFSTNPSFDBSFGVM JOTFBSDIQSPDFTT
  7. 3FTFBSDIHPBM %JTQVUFETFOUFODFTVHHFTUJPO GPSTFBSDIRVFSZ .BLFVTFSTNPSFDBSFGVMBCPVUDSFEJCJMJUZKVEHNFOU

  8. effective diet method %JTQVUFETFOUFODFTVHHFTUJPOJOTFBSDIQSPDFTT Disputed sentences about “effective diet method”

    5IFBULJOT EJFUJTNPSFFGGFDUJWFUIBOPUIFSEJFUNFUIPET )$(EJFUNFUIPEJTRVJUFFGGFDUJWF )JHIQSPUFJOEJFUTBSFUIFNPTUFGGFDUJWF I didn’t know that HCG diet is suspicious!
  9. 4ZTUFNGMPX *OQVUBRVFSZ &YUSBDU EJTQVUFETFOUFODFDBOEJEBUFTGSPN8FC 3BOL UIFDBOEJEBUFTCZDPOTJEFSJOH UIFJSUZQJDBMJUZRVFSZMJLFMJIPPE %JTQVUFETFOUFODFTVHHFTUJPO

  10. &YUSBDUJOHEJTQVUFETFOUFODFTGSPNUIF8FC Core idea 6TFMJOHVJTUJDQBUUFSOTPGEJTQVUFTJHOBM There is no evidence that more

    effective than other diets available the Raw Food Diet is
  11. &YUSBDUJOHEJTQVUFETFOUFODFTGSPNUIF8FC Core idea 6TFMJOHVJTUJDQBUUFSOTPGEJTQVUFTJHOBM There is no evidence that more

    effective than other diets available the Raw Food Diet is 4JHOBMQBUUFSO %JTQVUFETFOUFODF
  12. &YBNQMFTPGMJOHVJTUJDQBUUFSO lno proof that ~ lmistakenly believe that ~ lno

    evidence that ~ lit is not true that ~ lit is no clear that ~ lmisconception that ~ lmyth that ~ lit is denied that ~ lit is speculated that ~ * Rob Ennals et. al, What is disputed on the Web (WICOW2010) 6TFlUIBUDMBVTFTzUPEFUFDUEJTQVUFT
  13. TU TUFQFYUSBDUJOHEJTQVUFETFOUFODFTGSPNUIF8FC Query: effective diet + “no evidence that” is

    no evidence that Alone causes weight loss. Some weight loss … the use of diet supplements 0CUBJO8FCTFBSDISFTVMUTGSPNTFBSDIFOHJOF &YUSBDUEJTQVUFETFOUFODFTVTJOHEFMJNJUFST “the use of diet supplements alone causes weight loss” (“.”, “,”, “and”, “but”, “where”, etc.)
  14. 4ZTUFNGMPX *OQVUBRVFSZ &YUSBDU EJTQVUFETFOUFODFDBOEJEBUFTGSPN8FC 3BOL UIFDBOEJEBUFTCZDPOTJEFSJOH UIFJSUZQJDBMJUZRVFSZMJLFMJIPPE %JTQVUFETFOUFODFTVHHFTUJPO

  15. OE TUFQSBOLJOHPGEJTQVUFETFOUFODFT rank(s | q) = Pr(s | q) ∝

    Pr(s) ((1− λ)Pr(t | M c t∈q ∏ )+ λ Pr(t | M s )) 2VFSZ MJLFMJIPPE MBOHVBHF NPEFM ᶄ )PX GSFRVFOUMZ EPFT B EJTQVUFE TFOUFODF BQQFBS ᶃ )PX NBOZ RVFSZ UFSNT BSF DPOUBJOFE JO UIF TFOUFODF 4NPPUIJOHGBDUPS 2VFSZMJLFMJIPPE -FOHUI T GSFR U Given sentence s & query q , )PXUPDBMDVMBUF
  16. 5ZQJDBMJUZPGEJTQVUFETFOUFODFT 1S TFOUFODF ≈ TFOUFODF UZQJDBMJUZ 6TF5FYU3BOL GPSDBMDVMBUJPO Typ(s t

    ) = Sim text (s i ,s j )⋅Typ(s i ) ∑ Rada Mihalcea et. al., TextRank: Brinking Order into Texts (EMNLP2004) )$(EJFUJTFGGFDUJWF 4LJQQJOHCSFBLGBTU XPSLTXFMM &GGFDUJWFEJFUJT)$( )$(EJFUJTUIF NPTUFGGFDUJWFXBZ 1FPQMFMJLF)$( BTEJFUXFMM *GBUFYUIBTNPSFTJNJMBSUFYUT JUDPVMECFNPSFUZQJDBM
  17. 0VUQVUFYBNQMFT

  18. &WBMVBUJPO Relevance )PXSFMFWBOUBSFTVHHFTUFEEJTQVUFETFOUFODFTUPBRVFSZ Alertingness )PXDSJUJDBM XJMMVTFSTCFDPNFBCPVU8FCTFBSDISFTVMUTGPSBRVFSZ XIFOMPPLJOHBUUIFTVHHFTUFETFOUFODFT Usefulness )PXVTFGVM BSFUIFTVHHFTUFETFOUFODFTUPTFBSDIGPS8FCQBHFT

    BCPVUDSFEJCJMJUZFWJEFODFT Metrics These are effective for enhancing my credibility judgment?
  19. &WBMVBUJPONFUIPE  5IFTZTUFNHFOFSBUFE TFOUFODFTGPSBRVFSZVTJOHBNFUIPE  5IF TFOUFODFTXFSFTVHHFTUFE BTTVTQJDJPVTUP FWBMVBUPS 

    5IFFWBMVBUPSTKVEHFEUIFNXJUIZFTPSOPPOUIFDSJUFSJB  4UFQT  XFSFEPOFGPS RVFSJFTPOUIFNFUIPET $PNQBSFENFUIPET CZVTJOHFWBMVBUPST 0VSNFUIPE UZQJDBMJUZ MJLFMJIPPE :BIPP,FZXPSETVHHFTUJPO 5ZQJDBMJUZ 5%4 vs. Some people doubt the following sentences about effective diets Some people doubt the following sentences about effective diets Some people doubt the following sentences about effective diets
  20. 2VFSZTFU Category Query Controversy dinosaurs extinction, global warming, earthquake cause,

    plastic recycling, renewable energy Health effective diet, cancer treatment, mobile phone health, vaccine side effects, caffeine overdose Misunderstanding light bulb inventor, telephone inventor, civil war reason, caesar salad name Food coffee health, grapefruit seed extract benefit, potato poison Finance forex risk, mortgage refinancing, personal debt reduction
  21. 3FMFWBODF PGTVHHFTUFETFOUFODFTGPSJOQVURVFSZ 0.80 0.70 0.69 0.60 0.67 0.64 1.00 0.93

    0.94 0 0.2 0.4 0.6 0.8 1 r@1 r@3 r@5 Ours TYP Yahoo ˔ 2VFSZMPH CBTFE Yahoo IBT UIF IJHIFTU SFMFWBODF ˔ Ours  TYP ˰ RVFSZMJLFMJIPPE NPEFM XPSLT XFMM 33% improved
  22. "MFSUJOHOFTT PGTVHHFTUFETFOUFODFTGPSJOQVURVFSZ 0.70 0.63 0.65 0.50 0.57 0.51 0.05 0.07

    0.09 0 0.2 0.4 0.6 0.8 1 a@1 a@3 a@5 Ours TYP Yahoo 8JUIOurs & TYP BUMFBTU50% PGTVHHFTUFETFOUFODFT DBMMFEVTFST`BUUFOUJPOUPDSFEJCJMJUZKVEHNFOU
  23. 6TFGVMOFTT PGTVHHFTUFETFOUFODFTGPSJOQVURVFSZ 0.35 0.45 0.51 0.45 0.52 0.51 0.20 0.23

    0.27 0 0.2 0.4 0.6 0.8 1 u@1 u@3 u@5 Ours TYP Yahoo ˔ Ours  TYP BSF CFUUFS UIBO Yahoo ˔ Ours is less useful to search for evidences than TYP?
  24. 8IZXBTPVSNFUIPEKVEHFEBTMFTTVTFGVM Evaluator’s comment 5IFTVHHFTUFEEJTQVUFEQISBTFTUIFNTFMWFT XFSFPGUFOVTFGVMBTFWJEFODFPSDMVFTUPKVEHF UIFDSFEJCJMJUZPG8FCQBHFTBCPVUHJWFO RVFSJFT4P*KVEHFEUIBU*EJEO`UOFFEUPTFBSDI GPS8FCQBHFTVTJOHUIFN A. .PTUTVHHFTUFEEJTQVUFEQISBTFTXFSF

    VTFGVMDMVFTJOUIFNTFMWFT 0VSNFUIPEJTFGGFDUJWFFOPVHIUPNBLF VTFSTNPSFDSJUJDBMGPSDSFEJCJMJUZKVEHNFOU Conclusion
  25. &SSPSBOBMZTJT Error type Query Suggested disputed sentence Non-relevant potato poison

    potatoes are fattening (5) Ambiguous global warming global warming was the cause (1) Grammatical error plastic recycling shipping plastics for recycling (3) Difficult to understand telephone inventor agb was not the inventor (2) A. 'BJMVSFUPFTUJNBUFSFMFWBODF B. 'BJMVSFUPTQMJUTFOUFODFT C. 5IFFWBMVBUPSTIBEOPJEFBTBCPVUUFSNT Why? A B C B
  26. $PODMVTJPO 5PEBZ`TUBML %JTQVUFETFOUFODFTVHHFTUJPO - More sophisticated disputed sentence extraction -

    Consideration of sentence comprehensibility - Evidence search to judge credibility of disputed sentences  5PNBLFVTFSTNPSFDSJUJDBMGPSDSFEJCJMJUZKVEHNFOU UIFTZTUFNTVHHFTUTTVTQJDJPVTTFOUFODFT $SFEJCJMJUZPSJFOUFE8FCTFBSDI 'VUVSFXPSL with Thank you so much for your listening, 謝謝!
  27. None
  28. ޮՌతͳμΠΤοτ $POWFOUJPOBMRVFSZTVHHFTUJPO Suggest disputed sentences about search keyword 4VHHFTUSFMFWBOULFZXPSETUPRVFSZ 8PSLTXFMMJGVTFSTDBOOPUTFBSDIXFMM

    ʢ/PUSFMBUFEUPDSFEJCJMJUZKVEHNFOUʣ &GGFDUJWFEJFU &GGFDUJWFEJFUGPPE &GGFDUJWFEJFUNFUIPE &GGFDUJWFEJFUFYFSDJTF .PTU&GGFDUJWFEJFU &GGFDUJWFEJFUXBMLJOH &GGFDUJWFEJFUNFEJDJOF Related keywords to “Effective diet”
  29. 2VFTUJPOOBJSFJOFYQFSJNFOUBMFWBMVBUJPO lUsefulness