The Web Conference 2020 Report -多目的最適化における確率的ラベル集約-

The Web Conference 2020 Report -多目的最適化における確率的ラベル集約-

C108d59e6e6cd9a58e4d73ab9f922d81?s=128

Yuko Ishizaki

April 30, 2020
Tweet

Transcript

  1. ©2020 Wantedly, Inc. ଟ໨త࠷దԽʹ͓͚Δ֬཰తϥϕϧू໿ The Web Conference Taipei 2020 Report

    Apr 30, 2020 - Yuko Ishizaki
  2. ©2020 Wantedly, Inc. 5IF8FC$POGFSFODF࿦จɾجௐߨԋ  .BUDIJOH$SPTT/FUXPSLGPS-FBSOJOHUP3BOLJO1FSTPOBM4FBSDI ɹGoogle Drive ͷݕࡧʹ͓͍ͯγϯϓϧͰޮՌతͳϞσϧΛಋೖͨ͠࿩ 

    .VMUJ0CKFDUJWF3BOLJOH0QUJNJ[BUJPOGPS1SPEVDU4FBSDI6TJOH4UPDIBTUJD -BCFM"HHSFHBUJPO ɹAmazon Echo ͷ঎඼ݕࡧʹ͓͍ͯଟ໨త࠷దԽ໰୊ΛऔΓѻͬͨ࿦จ  #JBTPOUIF8FCBOE#FZPOE ɹWeb4ALLͰͷجௐߨԋ ࣮຿ͰͷΞΠσΟΞʹͳΓͦ͏ͳ΋ͷɺ࣮૷͠΍ͦ͢͏ͳ΋ͷɺݕূσʔλྔ͕े෼ͳ΋ͷΛϐοΫΞοϓ
  3. ©2020 Wantedly, Inc. Matching Cross Network for Learning to Rank

    in Personal Search Zhen Qin, Zhongliang Li, Michael Bendersky, Donald Metzler Google LLC, Mountain View, CA, USA
  4. ©2020 Wantedly, Inc. ঺հ࿦จ̍ Matching Cross Network for Learning to

    Rank in Personal Search Motivation γϯϓϧͰޮ཰తͳυΩϡϝϯτݕࡧ ࠷ۙͷυΩϡϝϯτݕࡧ͸ɺݕࡧΫΤϦʔͱυΩϡϝϯ τͷςΩετ಺༰ͱͷҙຯత߹கʹॏ఺Λஔ͍͍ͯΔɻ ͔͠͠ɺͦΕҎ֎ͷαΠυ৘ใ΋ॏཁͰɺྫ͑͹ۭߓͰ ʮྖऩॻʯͱυϥΠϒΛݕࡧͨ͠৔߹ɺ୳͍ͯ͠Δ΋ͷ ͸ߤۭ݊ͷྖऩॻͰ͋ΔՄೳੑ͕ߴ͍ɻ·ͨɺେ͖ͳϞ σϧ(BERTͳͲ)Λ࢖༻ͨ͠৔߹ɺϨΠςϯγʔ͕໰୊ʹ ͳΔɻ γϯϓϧͰޮՌతͳϞσϧΛ࡞੒͠ɺGoogle Driveͷݕ ࡧʹ࢖༻͍ͨ͠ɻ
  5. ©2020 Wantedly, Inc. ݕࡧΫΤϦʔʹର͠ɺυϥΠϒ಺ͷυΩϡϝϯτΛϥϯΫ෇͚͢ΔϞσϧ .BUDI'JSTU 2VFSZͱ%PDVNFOUͷ&NCFEEJOHͷཁૉੵ • ηϚϯςΟοΫϚονϯά • &NCFEEJOHʹ͸ಉࣙ͡ॻͱ-PPLVQ5BCMFΛ࢖༻

     4JEF*OGPSNBUJPO ࡞੒͞Ε͔ͯΒͷܦա࣌ؒ ࣌ࠁ ϢʔβΫϦοΫ৘ใ౳  -BUFOU$SPTT • "UUFOUJPOͷҰछ • w͕&NCFEEJOHཁૉੵͷग़ྗ • ಛ௃ྔؒͷ૬ޓ࡞༻Λޮ཰తʹֶश ঺հ࿦จ̍ Matching Cross Network for Learning to Rank in Personal Search
  6. ©2020 Wantedly, Inc. σʔληοτ ɹ(NBJMͱ(PPHMF%SJWFͷ਺ԯTBNQMFΛ࢖༻ ධՁࢦඪ ɹw͸DMJDLQPTJUJPO QPTJUJPOόΠΞεิਖ਼ ൺֱϞσϧ w

    #BTFMJOFಛ௃ྔΛ͢΂ͯDPODBUFOBUF w -BUFOU$SPTT2VFSZ2VFSZ͚ͩ-BUFOU$SPTT w .VMUJQMJDBUJPO'JSTU-BUFOU$SPTTΛDPODBUFOBUFʹ w ,FSOFM.$/.$/ΛҰ෦มߋɹ Offline Experimental Results ঺հ࿦จ̍ Matching Cross Network for Learning to Rank in Personal Search &NCFEEJOHཁૉੵͱ-BUFOU$SPTTΛ ૊Έ߹ΘͤΔ͜ͱ͕ޮՌతͰ͋Δ͜ͱ͕Θ͔Δ 1͸୯Ґߦྻ
  7. ©2020 Wantedly, Inc. σʔλ ɹ(PPHMF%SJWFͷ೥݄ͷສճͷݕࡧͰ ɹطଘͷϞσϧͱ"#ςετΛ࣮ࢪ ධՁࢦඪ w $53ΫϦοΫ཰ w

    "$1 ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ .͸ΫϦοΫ͞Εͨηογϣϯ w "CBOEPO3BUF཭୤཰ w -POH$MJDLQFS4FTTJPOυΩϡϝϯτΛඵҎ্Ӿཡ w 5JNFUP$MJDLΫϦοΫʹ͔͔Δ࣌ؒɹ ঺հ࿦จ̍ Matching Cross Network for Learning to Rank in Personal Search Online Experimental Results ͢΂ͯͷࢦඪ͕༗ҙʹվળ͞ΕͨͨΊ (PPHMF%SJWFͷݕࡧͷϞσϧΛϦϓϨʔε ˣ͸਺஋͕௿͍ํ͕ྑ͍ࢦඪ
  8. ©2020 Wantedly, Inc. ·ͱΊ w ݕࡧΫΤϦʔͱυΩϡϝϯτͱαΠυ৘ใΛޮՌతʹֶशͰ͖ΔγϯϓϧͳϥϯΫ෇͚ϞσϧΛ࣮ݱ w "#ςετͰ༗ҙʹվળ͕ݟΒΕɺੈք࠷େͷύʔιφϧݕࡧΤϯδϯͱͯ͠(PPHMF%SJWFʹಋೖ ײ૝ w

    (PPHMF%SJWFͷݕࡧʹ͍ͭͯ஌Εͯ໘ന͔ͬͨ ୯ʹpMUFSJOHͩͱࢥͬͯͨ  w ηϚϯςΟοΫϚονϯάͷ෦෼Λ΋ͬͱෳࡶͳ΋ͷʹ͢ΔͳͲɺԠ༻͕Ͱ͖ͦ͏ ঺հ࿦จ̍ Matching Cross Network for Learning to Rank in Personal Search
  9. ©2020 Wantedly, Inc. Multi-Objective Ranking Optimization for Product Search Using

    Stochastic Label Aggregation David Carmel, Elad Haramaty, Arnon Lazerson, Liane Lewin-Eytan
  10. ©2020 Wantedly, Inc. Ի੠ϕʔεͷݕࡧʹ͓͚Δଟ໨త࠷దԽ໰୊ ঎඼ݕࡧ͸ɺݕࡧΫΤϦʔͱ঎඼ͱͷؔ࿈ੑ΍ɺߪೖͷ Մೳੑͱ͍ͬͨɺෳ਺ͷ໨తΛ࠷దԽͯ͠ϥϯΫ෇͚͢ Δඞཁ͕͋ΔɻeίϚʔεͰͷ঎඼ݕࡧ͸ଟ͘ͷઌߦݚڀ ͕͋Δ͕ɺԻ੠ʹΑΔ঎඼ݕࡧͷଟ໨త࠷దԽ໰୊ʹؔ ͢Δ΋ͷ͸·ͩͳ͍ɻ Ի੠ʹΑΔ঎඼ݕࡧͷଟ໨త࠷దԽ໰୊ʹରͯ͠ɺ֬཰

    తϥϕϧू໿ͱ͍͏ΞϓϩʔνΛࢼΈͨɻ Amazon, Echo, Alexa and all related logos are trademarks of Amazon.com, Inc. or its affiliates. Motivation ঺հ࿦จ̎ Multi-Objective Ranking Optimization for Product Search Using Stochastic Label Aggregation
  11. ©2020 Wantedly, Inc. ݕࡧΫΤϦʔʹର͠ɺߪೖՄೳੑ1ͱؔ࿈ੑ3ͱͷ྆ํΛ࠷దԽͯ͠ɺ঎඼ΛϥϯΫ෇͚͢ΔϞσϧ ֬཰తϥϕϧू໿ Ћͷ֬཰ͰϥϕϧΛ1ʹЋͷ֬཰Ͱ3ʹ͢Δ  5XP1IBTF.PEFM ϥϕϧ1Ͱֶशͤͨ͞Ϟσϧͱ ϥϕϧ3Ͱֶशͤͨ͞ϞσϧΛ࡞Δ

    ͦͷग़ྗΛ֬཰తϥϕϧू໿ͷϞσϧͷಛ௃ྔʹֶश ঺հ࿦จ̎ Multi-Objective Ranking Optimization for Product Search Using Stochastic Label Aggregation 1IBTF.PEFM4UPDIBTUJD-BCFM"HHSFHBUJPO 1IBTF.PEFM4UPDIBTUJD-BCFM"HHSFHBUJPO
  12. ©2020 Wantedly, Inc. ঺հ࿦จ̎ Multi-Objective Ranking Optimization for Product Search

    Using Stochastic Label Aggregation Experiments σʔληοτ "MFYBͷϩάສΫΤϦʔ w ू໿Ի੠ݕࡧσʔλ w ະՃ޻Ի੠ݕࡧσʔλ w 8FCݕࡧσʔλ Ϟσϧ 9(#PPTU MBNCEBNBSU  ධՁࢦඪ /%$(! ಛ௃ྔ w +BDDBSE܎਺ w ϨʔϕϯγϡλΠϯڑ཭ w 'BTU5FYUίαΠϯྨࣅ౓  1IBTF ֬཰తϥϕϧू໿  1IBTF ֬཰తϥϕϧू໿  1IBTF ઢܗϥϕϧू໿  1IBTF ઢܗϥϕϧू໿  'VTJPOείΞͷ λΠτϧɺΧςΰϦʔɺϒϥϯυ౳ 1IBTF 4UPDL͕࠷΋ྑ͍
  13. ©2020 Wantedly, Inc. ֬཰తϥϕϧू໿ͷίετؔ਺͸ɺҎԼͷΑ͏ʹ 1ͷίετؔ਺ͱ3ͷίετؔ਺ͷઢܗ࿨ʹ౳͍͠ ূ໌ ֬཰తϥϕϧू໿͸྆ํͷϥϕϧ͕ͳͯ͘΋ֶश͕Մೳ Ξϊςʔγϣϯ͸Ұൠతʹίετ͕ߴ͍ ঺հ࿦จ̎ Multi-Objective

    Ranking Optimization for Product Search Using Stochastic Label Aggregation Theoretical Analysis Pareto frontiers ֬཰తϥϕϧू໿Ͱֶशͨ͠Ϟσϧ͸ύϨʔτ࠷దղͷ ू߹Λग़ྗ͢Δɻ ࠨͷࣜʹ͓͚ΔЋ͕ͷ࣌ͱͷ࣌ɻЋͷ஋ʹΑͬͯύϨʔτ࠷దղͷू߹Λग़ྗͰ͖Δͱ͍͏ ௚ײతͳΠϝʔδɻ
  14. ©2020 Wantedly, Inc. ঺հ࿦จ̎ Multi-Objective Ranking Optimization for Product Search

    Using Stochastic Label Aggregation ·ͱΊ w Ի੠ʹΑΔ঎඼ݕࡧͰଟ໨తϥϯΩϯά࠷దԽΛߦͬͨݚڀ w ֬཰తϥϕϧू໿ͱ1IBTFNPEFM͸ޮՌతͰ͋ͬͨ ײ૝ w 8BOUFEMZ7JTJUͷਪન΋ଟ໨త࠷దԽͱ͍͏ଆ໘΋͋ΔͷͰɺ࣮຿ʹΞΠσΟΞΛར༻Ͱ͖ͦ͏ w ࿦ཧత෼ੳͱղઆ͕ৄ͘͠ॻ͔Ε͍ͯͯྑ͍ͱࢥͬͨ
  15. ©2020 Wantedly, Inc. Bias on the Web and Beyond Ricardo

    Baeza-Yates
  16. ©2020 Wantedly, Inc. Web্ͷόΠΞεʹ͍ͭͯ ΄ͱΜͲͷWebγεςϜ͸ϢʔβͷϑΟʔυόοΫʹ Αͬͯ࠷దԽΛߦ͍ͬͯΔɻ͔͠͠ɺϢʔβͷϑΟʔυ όοΫ͸༷ʑͳόΠΞε͕͔͔͍ͬͯΔ৔߹͕͋Δɻಛ ʹɺݕࡧɾਪનγεςϜͳͲ͸௨ৗɺػցֶशΛ࢖༻͠ ͍ͯΔͨΊɺόΠΞεΛڧԽͤͯ͞͠·͏͜ͱ͕͋Δɻ Web্ͷόΠΞεΛཧղ͠ɺͦͷೝࣝϨϕϧΛ্͛Α

    ͏ɻ 8FC"MMجௐߨԋ Bias on the Web and Beyond Bias on the web. Ricardo Baeza-Yates https://doi.org/10.1145/3209581 Motivation όΠΞεʹؾ͚ͮͣʹγεςϜ։ൃΛ͢Δͱ ϢʔβɾαʔϏεʹѱӨڹΛٴ΅͢Մೳੑ΋
  17. ©2020 Wantedly, Inc. Data Bias https://www.languageconnect.net/blog/market-research/the- language-of-web-content-infographic/ ݴޠόΠΞε WebαΠτͷ52%͕ӳޠͰهࡌ͞Ε͍ͯΔ͕ɺӳޠ͕ωΠςΟϒɾୈ2ݴޠͰ͋Δਓ͸ੈքͷ16%. (ԁάϥϑ͸ͪΐͬͱݹ͍)

    ݱࡏɺੈքͰ࢖ΘΕ͍ͯΔݴޠ͸7100छྨ΋͋ΔͷʹɺWikipediaͰ࢖ΘΕ͍ͯΔͷ͸299ݴޠ͚ͩɻ 8FC"MMجௐߨԋ Bias on the Web and Beyond ڭҭɾܦࡁɾٕज़తόΠΞεʹΑͬͯ ΠϯλʔωοτʹΞΫηε͍ͯ͠Δਓ͸ภ͍ͬͯΔ https://upload.wikimedia.org/wikipedia/commons/1/1d/ Www.wikipedia.org_screenshot_2018.png
  18. ©2020 Wantedly, Inc. Wikipediaͷঁੑͷฤूऀ͸11ˋ͔͍͠ͳ͍ɻ ҎԼͷਤ͸Wikipediaͷঁੑͷ఻هͷྦྷੵׂ߹ɻ εϖΠϯͷ༌ग़ֹͱWeb্ϦϯΫ਺ͷάϥϑɻઌਐࠃͰ ͋Δ΄Ͳ૬ؔ܎਺͕େ͖͍ɻ Data Bias ܦࡁόΠΞε

    8FC"MMجௐߨԋ Bias on the Web and Beyond ͜ΕҎ֎ʹ΋ਓछ ೥ྸ फڭ ੓࣏ ஍ཧʜ ༷ʑͳཁҼͰσʔλʹόΠΞε͕͔͔Δ ੑผόΠΞε Bias on the web. Ricardo Baeza-Yates https://doi.org/10.1145/3209581
  19. ©2020 Wantedly, Inc. Web্ͷ΄ͱΜͲͷίϯςϯπ͸গ਺ͷΞΫςΟϒϢʔ βʹΑͬͯ࡞ΒΕ͍ͯΔɻ֤αʔϏεͷ50%ͷίϯςϯ πΛ࡞੒͢ΔΞΫςΟϒϢʔβͷׂ߹ΛάϥϑԽͨ͠΋ ͷɻ Activity Bias গ਺ͷΞΫςΟϒϢʔβ

    8FC"MMجௐߨԋ Bias on the Web and Beyond ωοτ্ͰΈΜͳͷҙݟͩͱࢥΘΕ͍ͯΔ͜ͱ͸ ͦ͏Ͱ͸ͳ͍ͱ͍͏͜ͱ΋͋Δ Amazon͸༗ྉͷِϨϏϡʔ๷ࢭΩϟϯϖʔϯͰ1000ਓ ͷϑΣΠΫϨϏϡΞʔΛఠൃͨ͜͠ͱ΋͋Δ ϑΣΠΫϨϏϡʔ https://www.theguardian.com/international
  20. ©2020 Wantedly, Inc. Bias in the Interaction Ϣʔβૢ࡞ͷόΠΞε w ϓϨθϯςʔγϣϯόΠΞεϢʔβ͸ݟͨ΋ͷ͔͠ΫϦοΫ͠ͳ͍

    w ϙδγϣϯόΠΞεࠨ্͸ΫϦοΫ͞Ε΍͍͢ w ϥϯΩϯάόΠΞεϥϯΩϯά্͕Ґͷ΋ͷ΄ͲΫϦοΫ͞Ε΍͍͢ Bias on the web. Ricardo Baeza-Yates ɹhttps://dl.acm.org/doi/pdf/10.1145/3209581 8FC"MMجௐߨԋ Bias on the Web and Beyond ͜ΕΒ͸૬ޓʹ࡞༻͢ΔͨΊ όΠΞεͷ෼཭͸ඇৗʹࠔ೉ The Evolution of Google's Search Results Pages & Effects on User Behaviour
  21. ©2020 Wantedly, Inc. Vicious Cycle of Bias 2࣍όΠΞε όΠΞε͸όΠΞεΛੜΉɻ ৽͍͠WebίϯςϯπΛ࡞੒͢Δͱ͖ɺผͷWebίϯς

    ϯπΛࢀߟʹͯ͠ϦϯΫΛ͸Δέʔε΋͋Δɻෳ੡͞Ε ͨ৘ใͱϦϯΫʹΑͬͯɺݕࡧΤϯδϯ͕͞ΒʹόΠΞ εΛ͔͚ͯ͠·͏৔߹͕͋Δɻ ·ͨɺա౓ͳύʔιφϥΠζΞϧΰϦζϜ͸ดͨ͡ੈք Λ࡞Γग़ͯ͠͠·͏ɻ Degenerate Feedback Loops in Recommender Systems Ray Jiang, Silvia Chiappa, Tor Lattimore, Andras Agyorgy, and Pushmeet Kohli. 8FC"MMجௐߨԋ Bias on the Web and Beyond ଟ༷ੑɾηϨϯσΟϐςΟͷେ੾͞
  22. ©2020 Wantedly, Inc. 8FC"MMجௐߨԋ Bias on the Web and Beyond

    ·ͱΊ w όΠΞεͷରॲʹ͸ɺ·ͣόΠΞεΛೝ஌͢Δ͜ͱ͔Β࢝·Δ w όΠΞεΛߟྀ͠ɺ༏ΕͨγεςϜઃܭΛ͢Δඞཁ͕͋Δ ײ૝ w ͜ͷεϥΠυʹ͸هࡌग़དྷͳ͔͕ͬͨɺଞʹ΋୔ࢁͷࣄྫ͕͋ͬͯͱͯ΋ڵຯਂ͔ͬͨ w ଟ͘͸ฉ͍ͨ͜ͱ͕͋Δ͜ͱɾ૝૾͕༰қͳ͜ͱ͕ͩͬͨɺؾΛ͚͍ͭͯͳ͍ͱݟಀͯ͠͠·͏͜ͱͰ ΋͋Δͱࢥ͍ɺͱͯ΋ؾ͕࣋ͪҾ͖క·ͬͨ