The Web Conference 2020 Report -多目的最適化における確率的ラベル集約-

Transcript

1. ©2020 Wantedly, Inc. ଟ໨త࠷దԽʹ͓͚Δ֬཰తϥϕϧू໿ The Web Conference Taipei 2020 Report

   Apr 30, 2020 - Yuko Ishizaki

2. ©2020 Wantedly, Inc. 5IF
   8FC
   $POGFSFODF
   
   ࿦จɾجௐߨԋ
    .BUDIJOH
   $SPTT
   /FUXPSL
   GPS
   -FBSOJOH
   UP
   3BOL
   JO
   1FSTPOBM
   4FBSDI
   ɹGoogle Drive ͷݕࡧʹ͓͍ͯγϯϓϧͰޮՌతͳϞσϧΛಋೖͨ͠࿩ 

   .VMUJ0CKFDUJWF
   3BOLJOH
   0QUJNJ[BUJPO
   GPS
   1SPEVDU
   4FBSDI
   6TJOH
   4UPDIBTUJD
   -BCFM
   "HHSFHBUJPO
   ɹAmazon Echo ͷ঎඼ݕࡧʹ͓͍ͯଟ໨త࠷దԽ໰୊ΛऔΓѻͬͨ࿦จ  #JBT
   PO
   UIF
   8FC
   BOE
   #FZPOE
   ɹWeb4ALLͰͷجௐߨԋ ࣮຿ͰͷΞΠσΟΞʹͳΓͦ͏ͳ΋ͷɺ࣮૷͠΍ͦ͢͏ͳ΋ͷɺݕূσʔλྔ͕े෼ͳ΋ͷΛϐοΫΞοϓ
   

3. ©2020 Wantedly, Inc. Matching Cross Network for Learning to Rank

   in Personal Search Zhen Qin, Zhongliang Li, Michael Bendersky, Donald Metzler Google LLC, Mountain View, CA, USA

4. ©2020 Wantedly, Inc. ঺հ࿦จ̍ Matching Cross Network for Learning to

   Rank in Personal Search Motivation γϯϓϧͰޮ཰తͳυΩϡϝϯτݕࡧ ࠷ۙͷυΩϡϝϯτݕࡧ͸ɺݕࡧΫΤϦʔͱυΩϡϝϯ τͷςΩετ಺༰ͱͷҙຯత߹கʹॏ఺Λஔ͍͍ͯΔɻ ͔͠͠ɺͦΕҎ֎ͷαΠυ৘ใ΋ॏཁͰɺྫ͑͹ۭߓͰ ʮྖऩॻʯͱυϥΠϒΛݕࡧͨ͠৔߹ɺ୳͍ͯ͠Δ΋ͷ ͸ߤۭ݊ͷྖऩॻͰ͋ΔՄೳੑ͕ߴ͍ɻ·ͨɺେ͖ͳϞ σϧ(BERTͳͲ)Λ࢖༻ͨ͠৔߹ɺϨΠςϯγʔ͕໰୊ʹ ͳΔɻ γϯϓϧͰޮՌతͳϞσϧΛ࡞੒͠ɺGoogle Driveͷݕ ࡧʹ࢖༻͍ͨ͠ɻ

5. ©2020 Wantedly, Inc. ݕࡧΫΤϦʔʹର͠ɺυϥΠϒ಺ͷυΩϡϝϯτΛϥϯΫ෇͚͢ΔϞσϧ 
   .BUDI
   'JSTU
   2VFSZͱ%PDVNFOU
   ͷ
   &NCFEEJOH
   ͷཁૉੵ • ηϚϯςΟοΫϚονϯά • &NCFEEJOHʹ͸ಉࣙ͡ॻͱ-PPLVQ
   5BCMFΛ࢖༻

   
   4JEF
   *OGPSNBUJPO
   ࡞੒͞Ε͔ͯΒͷܦա࣌ؒ
   ࣌ࠁ
   ϢʔβΫϦοΫ৘ใ
   ౳ 
   -BUFOU
   $SPTT
   • "UUFOUJPOͷҰछ • w͕&NCFEEJOHཁૉੵͷग़ྗ • ಛ௃ྔؒͷ૬ޓ࡞༻Λޮ཰తʹֶश ঺հ࿦จ̍ Matching Cross Network for Learning to Rank in Personal Search

6. ©2020 Wantedly, Inc. σʔληοτ ɹ(NBJMͱ(PPHMF
   %SJWF
   ͷ
   ਺ԯ
   TBNQMF
   Λ࢖༻
   ධՁࢦඪ ɹw͸DMJDL
   QPTJUJPO
   QPTJUJPO
   όΠΞεิਖ਼ ൺֱϞσϧ w

   #BTF
   MJOF
   ಛ௃ྔΛ͢΂ͯDPODBUFOBUF
   w -BUFOU
   $SPTT
   2VFSZ
   2VFSZ͚ͩ-BUFOU
   $SPTT
   w .VMUJQMJDBUJPO
   'JSTU
   
   -BUFOU
   $SPTT
   ΛDPODBUFOBUFʹ
   w ,FSOFM
   .$/
   .$/ΛҰ෦มߋ
   ɹ Offline Experimental Results ঺հ࿦จ̍ Matching Cross Network for Learning to Rank in Personal Search &NCFEEJOHཁૉੵͱ-BUFOU
   $SPTTΛ
   ૊Έ߹ΘͤΔ͜ͱ͕ޮՌతͰ͋Δ͜ͱ͕Θ͔Δ 1͸୯Ґߦྻ

7. ©2020 Wantedly, Inc. σʔλ ɹ(PPHMF
   %SJWF
   ͷ೥݄ͷສճͷݕࡧͰ
   ɹطଘͷϞσϧͱ"#ςετΛ࣮ࢪ
   ධՁࢦඪ w $53
   ΫϦοΫ཰
   w

   "$1
   
   ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ .͸ΫϦοΫ͞Εͨηογϣϯ w "CBOEPO
   3BUF
   ཭୤཰
   w -POH
   $MJDL
   QFS
   4FTTJPO
   υΩϡϝϯτΛඵҎ্Ӿཡ
   w 5JNF
   UP
   $MJDL
   ΫϦοΫʹ͔͔Δ࣌ؒɹ ঺հ࿦จ̍ Matching Cross Network for Learning to Rank in Personal Search Online Experimental Results ͢΂ͯͷࢦඪ͕༗ҙʹվળ͞ΕͨͨΊ
   (PPHMF
   %SJWFͷݕࡧͷϞσϧΛϦϓϨʔε ˣ͸਺஋͕௿͍ํ͕ྑ͍ࢦඪ

8. ©2020 Wantedly, Inc. ·ͱΊ w ݕࡧΫΤϦʔͱυΩϡϝϯτͱαΠυ৘ใΛޮՌతʹֶशͰ͖ΔγϯϓϧͳϥϯΫ෇͚ϞσϧΛ࣮ݱ
   w "#
   ςετͰ༗ҙʹվળ͕ݟΒΕɺੈք࠷େͷύʔιφϧݕࡧΤϯδϯͱͯ͠(PPHMF
   %SJWFʹಋೖ
   ײ૝ w

   (PPHMF
   %SJWFͷݕࡧʹ͍ͭͯ஌Εͯ໘ന͔ͬͨ
   ୯ʹpMUFSJOHͩͱࢥͬͯͨ
   w ηϚϯςΟοΫϚονϯάͷ෦෼Λ΋ͬͱෳࡶͳ΋ͷʹ͢ΔͳͲɺԠ༻͕Ͱ͖ͦ͏
   ঺հ࿦จ̍ Matching Cross Network for Learning to Rank in Personal Search

9. ©2020 Wantedly, Inc. Multi-Objective Ranking Optimization for Product Search Using

   Stochastic Label Aggregation David Carmel, Elad Haramaty, Arnon Lazerson, Liane Lewin-Eytan

10. ©2020 Wantedly, Inc. Ի੠ϕʔεͷݕࡧʹ͓͚Δଟ໨త࠷దԽ໰୊ ঎඼ݕࡧ͸ɺݕࡧΫΤϦʔͱ঎඼ͱͷؔ࿈ੑ΍ɺߪೖͷ Մೳੑͱ͍ͬͨɺෳ਺ͷ໨తΛ࠷దԽͯ͠ϥϯΫ෇͚͢ Δඞཁ͕͋ΔɻeίϚʔεͰͷ঎඼ݕࡧ͸ଟ͘ͷઌߦݚڀ ͕͋Δ͕ɺԻ੠ʹΑΔ঎඼ݕࡧͷଟ໨త࠷దԽ໰୊ʹؔ ͢Δ΋ͷ͸·ͩͳ͍ɻ Ի੠ʹΑΔ঎඼ݕࡧͷଟ໨త࠷దԽ໰୊ʹରͯ͠ɺ֬཰

    తϥϕϧू໿ͱ͍͏ΞϓϩʔνΛࢼΈͨɻ Amazon, Echo, Alexa and all related logos are trademarks of Amazon.com, Inc. or its affiliates. Motivation ঺հ࿦จ̎ Multi-Objective Ranking Optimization for Product Search Using Stochastic Label Aggregation

11. ©2020 Wantedly, Inc. ݕࡧΫΤϦʔʹର͠ɺߪೖՄೳੑ1ͱؔ࿈ੑ3ͱͷ྆ํΛ࠷దԽͯ͠ɺ঎඼ΛϥϯΫ෇͚͢ΔϞσϧ 
    ֬཰తϥϕϧू໿
    Ћͷ֬཰ͰϥϕϧΛ1ʹЋͷ֬཰Ͱ3ʹ͢Δ
    
    5XP1IBTF
    .PEFM
    
    ϥϕϧ1Ͱֶशͤͨ͞Ϟσϧͱ
    ϥϕϧ3Ͱֶशͤͨ͞ϞσϧΛ࡞Δ
    

    ͦͷग़ྗΛ֬཰తϥϕϧू໿ͷϞσϧͷಛ௃ྔʹֶश
    ঺հ࿦จ̎ Multi-Objective Ranking Optimization for Product Search Using Stochastic Label Aggregation 1IBTF
    .PEFM
    4UPDIBTUJD
    -BCFM
    "HHSFHBUJPO
    1IBTF
    .PEFM
    4UPDIBTUJD
    -BCFM
    "HHSFHBUJPO
    

12. ©2020 Wantedly, Inc. ঺հ࿦จ̎ Multi-Objective Ranking Optimization for Product Search

    Using Stochastic Label Aggregation Experiments σʔληοτ "MFYBͷϩά
    ສΫΤϦʔ
    w ू໿Ի੠ݕࡧσʔλ
    w ະՃ޻Ի੠ݕࡧσʔλ
    w 8FCݕࡧσʔλ
    Ϟσϧ 9(#PPTU MBNCEB
    NBSU
    ධՁࢦඪ /%$(!
    ಛ௃ྔ
    w +BDDBSE܎਺
    w ϨʔϕϯγϡλΠϯڑ཭
    w 'BTU5FYU
    ίαΠϯྨࣅ౓  
    1IBTF
    
    ֬཰తϥϕϧू໿
     
    1IBTF
    
    ֬཰తϥϕϧू໿
     
    1IBTF
    
    ઢܗϥϕϧू໿
     
    1IBTF
    
    ઢܗϥϕϧू໿
     'VTJPO
    
    είΞͷ
    λΠτϧɺΧςΰϦʔɺϒϥϯυ౳ 1IBTF 4UPDL͕࠷΋ྑ͍

13. ©2020 Wantedly, Inc. ֬཰తϥϕϧू໿ͷίετؔ਺͸ɺҎԼͷΑ͏ʹ
    1ͷίετؔ਺ͱ3ͷίετؔ਺ͷઢܗ࿨ʹ౳͍͠
    ূ໌
    ֬཰తϥϕϧू໿͸྆ํͷϥϕϧ͕ͳͯ͘΋ֶश͕Մೳ
    Ξϊςʔγϣϯ͸Ұൠతʹίετ͕ߴ͍ ঺հ࿦จ̎ Multi-Objective

    Ranking Optimization for Product Search Using Stochastic Label Aggregation Theoretical Analysis Pareto frontiers ֬཰తϥϕϧू໿Ͱֶशͨ͠Ϟσϧ͸ύϨʔτ࠷దղͷ ू߹Λग़ྗ͢Δɻ
    ࠨͷࣜʹ͓͚ΔЋ͕ͷ࣌ͱͷ࣌ɻЋͷ஋ʹΑͬͯύϨʔτ࠷దղͷू߹Λग़ྗͰ͖Δͱ͍͏ ௚ײతͳΠϝʔδɻ

14. ©2020 Wantedly, Inc. ঺հ࿦จ̎ Multi-Objective Ranking Optimization for Product Search

    Using Stochastic Label Aggregation ·ͱΊ w Ի੠ʹΑΔ঎඼ݕࡧͰଟ໨తϥϯΩϯά࠷దԽΛߦͬͨݚڀ
    w ֬཰తϥϕϧू໿ͱ
    1IBTF
    NPEFM͸ޮՌతͰ͋ͬͨ
    ײ૝ w 8BOUFEMZ
    7JTJUͷਪન΋ଟ໨త࠷దԽͱ͍͏ଆ໘΋͋ΔͷͰɺ࣮຿ʹΞΠσΟΞΛར༻Ͱ͖ͦ͏
    w ࿦ཧత෼ੳͱղઆ͕ৄ͘͠ॻ͔Ε͍ͯͯྑ͍ͱࢥͬͨ

15. ©2020 Wantedly, Inc. Bias on the Web and Beyond Ricardo

    Baeza-Yates

16. ©2020 Wantedly, Inc. Web্ͷόΠΞεʹ͍ͭͯ ΄ͱΜͲͷWebγεςϜ͸ϢʔβͷϑΟʔυόοΫʹ Αͬͯ࠷దԽΛߦ͍ͬͯΔɻ͔͠͠ɺϢʔβͷϑΟʔυ όοΫ͸༷ʑͳόΠΞε͕͔͔͍ͬͯΔ৔߹͕͋Δɻಛ ʹɺݕࡧɾਪનγεςϜͳͲ͸௨ৗɺػցֶशΛ࢖༻͠ ͍ͯΔͨΊɺόΠΞεΛڧԽͤͯ͞͠·͏͜ͱ͕͋Δɻ Web্ͷόΠΞεΛཧղ͠ɺͦͷೝࣝϨϕϧΛ্͛Α

    ͏ɻ 8FC"MMجௐߨԋ Bias on the Web and Beyond Bias on the web. Ricardo Baeza-Yates https://doi.org/10.1145/3209581 Motivation όΠΞεʹؾ͚ͮͣʹγεςϜ։ൃΛ͢Δͱ
    ϢʔβɾαʔϏεʹѱӨڹΛٴ΅͢Մೳੑ΋

17. ©2020 Wantedly, Inc. Data Bias https://www.languageconnect.net/blog/market-research/the- language-of-web-content-infographic/ ݴޠόΠΞε WebαΠτͷ52%͕ӳޠͰهࡌ͞Ε͍ͯΔ͕ɺӳޠ͕ωΠςΟϒɾୈ2ݴޠͰ͋Δਓ͸ੈքͷ16%. (ԁάϥϑ͸ͪΐͬͱݹ͍)

    ݱࡏɺੈքͰ࢖ΘΕ͍ͯΔݴޠ͸7100छྨ΋͋ΔͷʹɺWikipediaͰ࢖ΘΕ͍ͯΔͷ͸299ݴޠ͚ͩɻ 8FC"MMجௐߨԋ Bias on the Web and Beyond ڭҭɾܦࡁɾٕज़తόΠΞεʹΑͬͯ
    ΠϯλʔωοτʹΞΫηε͍ͯ͠Δਓ͸ภ͍ͬͯΔ https://upload.wikimedia.org/wikipedia/commons/1/1d/ Www.wikipedia.org_screenshot_2018.png

18. ©2020 Wantedly, Inc. Wikipediaͷঁੑͷฤूऀ͸11ˋ͔͍͠ͳ͍ɻ ҎԼͷਤ͸Wikipediaͷঁੑͷ఻هͷྦྷੵׂ߹ɻ εϖΠϯͷ༌ग़ֹͱWeb্ϦϯΫ਺ͷάϥϑɻઌਐࠃͰ ͋Δ΄Ͳ૬ؔ܎਺͕େ͖͍ɻ Data Bias ܦࡁόΠΞε

    8FC"MMجௐߨԋ Bias on the Web and Beyond ͜ΕҎ֎ʹ΋ਓछ ೥ྸ फڭ ੓࣏ ஍ཧʜ
    ༷ʑͳཁҼͰσʔλʹόΠΞε͕͔͔Δ ੑผόΠΞε Bias on the web. Ricardo Baeza-Yates https://doi.org/10.1145/3209581

19. ©2020 Wantedly, Inc. Web্ͷ΄ͱΜͲͷίϯςϯπ͸গ਺ͷΞΫςΟϒϢʔ βʹΑͬͯ࡞ΒΕ͍ͯΔɻ֤αʔϏεͷ50%ͷίϯςϯ πΛ࡞੒͢ΔΞΫςΟϒϢʔβͷׂ߹ΛάϥϑԽͨ͠΋ ͷɻ Activity Bias গ਺ͷΞΫςΟϒϢʔβ

    8FC"MMجௐߨԋ Bias on the Web and Beyond ωοτ্ͰΈΜͳͷҙݟͩͱࢥΘΕ͍ͯΔ͜ͱ͸
    ͦ͏Ͱ͸ͳ͍ͱ͍͏͜ͱ΋͋Δ Amazon͸༗ྉͷِϨϏϡʔ๷ࢭΩϟϯϖʔϯͰ1000ਓ ͷϑΣΠΫϨϏϡΞʔΛఠൃͨ͜͠ͱ΋͋Δ ϑΣΠΫϨϏϡʔ https://www.theguardian.com/international

20. ©2020 Wantedly, Inc. Bias in the Interaction Ϣʔβૢ࡞ͷόΠΞε w ϓϨθϯςʔγϣϯόΠΞε
    Ϣʔβ͸ݟͨ΋ͷ͔͠ΫϦοΫ͠ͳ͍
    

    w ϙδγϣϯόΠΞε
    ࠨ্͸ΫϦοΫ͞Ε΍͍͢
    w ϥϯΩϯάόΠΞε
    ϥϯΩϯά্͕Ґͷ΋ͷ΄ͲΫϦοΫ͞Ε΍͍͢
    Bias on the web. Ricardo Baeza-Yates ɹhttps://dl.acm.org/doi/pdf/10.1145/3209581 8FC"MMجௐߨԋ Bias on the Web and Beyond ͜ΕΒ͸૬ޓʹ࡞༻͢ΔͨΊ
    όΠΞεͷ෼཭͸ඇৗʹࠔ೉ The Evolution of Google's Search Results Pages & Effects on User Behaviour

21. ©2020 Wantedly, Inc. Vicious Cycle of Bias 2࣍όΠΞε όΠΞε͸όΠΞεΛੜΉɻ ৽͍͠WebίϯςϯπΛ࡞੒͢Δͱ͖ɺผͷWebίϯς

    ϯπΛࢀߟʹͯ͠ϦϯΫΛ͸Δέʔε΋͋Δɻෳ੡͞Ε ͨ৘ใͱϦϯΫʹΑͬͯɺݕࡧΤϯδϯ͕͞ΒʹόΠΞ εΛ͔͚ͯ͠·͏৔߹͕͋Δɻ ·ͨɺա౓ͳύʔιφϥΠζΞϧΰϦζϜ͸ดͨ͡ੈք Λ࡞Γग़ͯ͠͠·͏ɻ Degenerate Feedback Loops in Recommender Systems Ray Jiang, Silvia Chiappa, Tor Lattimore, Andras Agyorgy, and Pushmeet Kohli. 8FC"MMجௐߨԋ Bias on the Web and Beyond ଟ༷ੑɾηϨϯσΟϐςΟͷେ੾͞

22. ©2020 Wantedly, Inc. 8FC"MMجௐߨԋ Bias on the Web and Beyond

    ·ͱΊ w όΠΞεͷରॲʹ͸ɺ·ͣόΠΞεΛೝ஌͢Δ͜ͱ͔Β࢝·Δ
    w όΠΞεΛߟྀ͠ɺ༏ΕͨγεςϜઃܭΛ͢Δඞཁ͕͋Δ
    ײ૝ w ͜ͷεϥΠυʹ͸هࡌग़དྷͳ͔͕ͬͨɺଞʹ΋୔ࢁͷࣄྫ͕͋ͬͯͱͯ΋ڵຯਂ͔ͬͨ
    w ଟ͘͸ฉ͍ͨ͜ͱ͕͋Δ͜ͱɾ૝૾͕༰қͳ͜ͱ͕ͩͬͨɺؾΛ͚͍ͭͯͳ͍ͱݟಀͯ͠͠·͏͜ͱͰ ΋͋Δͱࢥ͍ɺͱͯ΋ؾ͕࣋ͪҾ͖క·ͬͨ