ヒューマンコンピュテーション（2019年度「人工知能」第3回）/ Human Computation (GB20301)

Transcript

1. 
   ஜ೾େֶ৘ใՊֶྨ
 ೥౓ʮਓ޻஌ೳʯୈճ
 അ৔
   ઇ೫ ώϡʔϚϯίϯϐϡςʔγϣϯ 1
2. / 30 ώϡʔϚϯίϯϐϡςʔγϣϯ ਓؒͱਓ޻஌ೳͷ૊߹ͤͰ೉͍͠໰୊ͷղܾΛ໨ࢦ͢ ਓ޻஌ೳγεςϜ ώϡʔϚϯίϯϐϡςʔγϣϯ ਓؒͱਓ޻஌ೳΛ૊Έ߹Θͤͯ
 ͲͪΒ͔Ұํ͚ͩͰ͸ղܾ͕೉͍͠໰୊Λղܾ͢ΔͨΊͷํ๏࿦ ώϡʔϚϯίϯϐϡςʔγϣϯ ਓ޻஌ೳγεςϜͷ಺෦ʹਓؒΛ૊ࠐΈਓؒΛܭࢉࢿݯͱͯ͠ѻ͏ 2
3. / 30 ώϡʔϚϯίϯϐϡςʔγϣϯͷྫᶃɿSF$"15$)" จࣈೝࣝγεςϜʹਓؒΛ૊ࠐΉ େྔͷॻ੶ͷจࣈೝࣝΛߦ͍͍͕ͨਓ޻஌ೳʹ͸೉͍͠จࣈ͕͋Δ

   ೉͍͠จࣈ͸ਓؒʹೝࣝͤ͞Δ morning 4UFQ
   
   ݁Ռ͕ෆҰகͷͱ͖ʹਓؒʹ໰͍߹ΘͤΔ moroiog morpipg 4UFQ
   
   ॻ੶தͷจࣈΛ̎ͭͷਓ޻஌ೳγεςϜʹೝࣝͤ͞Δ 3
4. / 30 ώϡʔϚϯίϯϐϡςʔγϣϯͷྫᶃɿSF$"15$)" w ΢Σϒϖʔδ΁ͷΞΫηεݖ΍ΞΧ΢ϯτ࡞੒ͷݖརΛใुʹ͢Δ
   w SF$"15$)"͸ද޲͖ʹ͸ʮਓؒͰ͋Δ͜ͱΛূ໌͢Δʯ࢓૊Έ
   w ࢀՃऀ͸ɼࣗ෼͕ਓؒͩͱূ໌͠ΞΫηεݖΛಘΔͨΊʹจࣈೝࣝΛߦ͏ ਓؒΛจࣈೝࣝʹࢀՃͤ͞ΔͨΊΞΫηεݖΛใुͱ͢Δ ΞΫηεΛ
 ڐՄ͍ͯͩ͘͠͞ʂ จࣈೝࣝΛͯ͠
 ͋ͳ͕ͨਓؒͩͱূ໌͠ͳ͍͞ 4
5. / 30 ώϡʔϚϯίϯϐϡςʔγϣϯͷྫᶃɿSF$"15$)" ਖ਼ղط஌ͷ໰୊ΛࠞͥΔ͜ͱͰෆਅ໘໨ͳਓΛআ֎͢Δ ᶃ
   ਓ͔ؒϘοτ͔ͷ൑ఆɹᶄ
   ॻ੶ͷจࣈೝࣝ SF$"15$)"ͷ໨త Type the word Type the word reCAPTCHA ೋͭͷจࣈΛೝࣝ͢ΔΑ͏ʹࢦࣔ͢Δ ˞
   ͲͪΒ͕ਖ਼ղط஌͔͸఻͑ͳ͍ ᶃɹਖ਼ղͳΒˠਓؒ
   
 ɹෆਖ਼ղͳΒˠϘοτ
   ᶄʮਓؒ
   ʯͷ݁Ռ͚ͩΛ
 ɹ
   ೝࣝ݁Ռͱͯ͠࠾༻ ਖ਼ղط஌ ਖ਼ղະ஌ 5
6. / 30 ώϡʔϚϯίϯϐϡςʔγϣϯͷྫᶃɿSF$"15$)" w ʮਓؒʯ͕ͨ·ͨ·ؒҧ͑ΔՄೳੑ͕͋Δ
   w ಉ͡จࣈྻΛෳ਺ਓʹೝࣝͤͯ͞Ұఆਓ਺ͷ౴͕͑Ұகͨ͠ͱ͖͚ͩ
 ࠷ऴతͳೝࣝ݁Ռͱͯ͠࠾༻͢Δ ෳ਺ਓ͕߹ҙͨ͠ճ౴͚ͩΛ࠾༻͢Δ morning
 Λೝࣝ݁Ռͱͯ͠࠾༻ morning morning morning morping 6
7. / 30 ώϡʔϚϯίϯϐϡςʔγϣϯͷྫᶄɿ7J[8J[ ࢹ֮ো͕͍ऀͷ࣭໰Ԡ౴γεςϜʹਓؒΛ૊ΈࠐΉ  -tap to take a photo.  -tap to begin recording your question and again to stop. 
   
    
   
   
    
   side,
   
    
    
   
    User  
   
   
   
   ? Database - Local Client Remote Services and Worker Interface 4UFQ
   
   Ϣʔβ͕࣭໰Λ౤ߘ 4UFQ
   
   γεςϜ಺෦ͷਓ͕ؒճ౴ ίʔϯͷ؈͸ͲΕʁ Ұ൪ӈͷ؈Ͱ͢ 7 J. Bigham et al.: VizWiz: Nearly real-time answers to visual questions, In UIST, 2010.
8. / 30 Ϋϥ΢υιʔγϯά w ۚમใु͸൚༻తͳखஈ
   w ΠϯλʔωοτΛ௨ͯ͡ෆಛఆଟ਺ͷਓʑʹ࡞ۀΛൃ஫͢Δ࢓૊Έ
 ʮΫϥ΢υιʔγϯάʯʹΑΓ͓ۚΛࢧ෷͍ࢀՃऀΛूΊΔ͜ͱ͕Ͱ͖Δ
   ⿞Ϋϥ΢υιʔγϯάͷྫɿ"NB[PO
   .FDIBOJDBM
   5VSL
   ϥϯαʔζ
   w 7J[8J[͸Ϋϥ΢υιʔγϯάΛ༻͍ͯࢀՃऀΛ֬อ ෆಛఆଟ਺ͷਓʹใुΛ෷͍࡞ۀΛґཔ Ϋϥ΢υιʔγϯά ࡞ۀΛൃ஫ අ༻Λࢧ෷͏ ࡞ۀΛड஫ ใुΛ੥ٻ 8
9. / 30 ώϡʔϚϯίϯϐϡςʔγϣϯͷ՝୊ w ώϡʔϚϯίϯϐϡςʔγϣϯ͸ෆಛఆଟ਺ͷਓؒʹཔΒ͟ΔΛಘͳ͍
   w ࢀՃऀͷೳྗɾಈػ͸༷ʑͰ
 શһ͕༩͑ΒΕͨ࢓ࣄΛे෼ͳ඼࣭ͰॲཧͰ͖Δอূ͸ͳ͍ʢଟ༷ੑʣ
   w ܭࢉػͱൺ΂ͯਓؒͷੑೳ͸҆ఆ͠ͳ͍ʢෆ࣮֬ੑʣ
   w ਓ͔ؒΒਖ਼͍͠౴͑ΛಘΔͨΊͷٕज़͕ඞཁ ਓؒ͸͍ͭͰ΋ɾ୭Ͱ΋ਖ਼͍͠౴͑Λฦ͢ͱ͸ݶΒͳ͍ 9 ࣸਅʹೣ͕ࣸͬͯ·͔͢ʁ :&4 /0 NO
10. / 30 ຊ೔ͷςʔϚɿूஂΛ׆͔ͨ͠Ξϓϩʔν w ूஂͰ໰୊Λղ͔ͤΔɾղ͚ΔਓΛݟ͚ͭΔ͜ͱͰ
 ਓ͔ؒΒਖ਼͍͠౴͑ΛҾ͖ग़͢ ूஂͷྗͰਖ਼͍͠౴͑ΛҾ͖ग़͢ ਓ޻஌ೳʹΑΔ໰୊ղܾΛ
 ਓ͕ؒࢧԉ
    ਓؒʹΑΔ໰୊ղܾΛ
 ਓ޻஌ೳ͕ࢧԉ 10 ᶃ
    ฒྻ໰߹ͤ ᶄ
    ௚ྻ໰߹ͤ ᶅ
    ઐ໳Ոൃݟ ᶆ
    ڠௐࢧԉ ؆୯ɾ୯७ͳ໰୊ ࠔ೉ɾෳࡶͳ໰୊
11. ᶃ
    ฒྻ໰߹ͤ
12. / 30 ฒྻ໰߹ͤ ผʑʹղ͔ͤͯ౴͑Λ·ͱΊΔ 12 ࣸਅʹௗ͕ࣸͬͯ·͔͢ʁ :&4 /0 YES NO YES ଟ਺ܾͷ݁Ռ͸
    YES ࣸਅʹௗ͕ࣸͬͯ·͔͢ʁ :&4 /0 YES YES NO ଟ਺ܾͷ݁Ռ͸
    YES w ৴པੑ޲্ͷͨΊ
 ৴པͰ͖Δਓʹߴ͍ॏΈΛ༩͍͑ͨ w ୯७ͳํ๏͸ଟ਺ܾɿ
 ಉ࣭͡໰Λෳ਺ਓʹ໰͍߹Θͤ
 ճ౴Λ౷߹͢Δ
13. / 30 ౷ܭతճ౴౷߹ɿճ౴ऀͷ৴པੑΛߟྀ w ճ౴͔Β֤ࣗͷ৴པੑΛਪఆ্ͨ͠Ͱ֤໰୊ͷਖ਼ղΛ༧ଌ͢Δ
    w ੜెͷճ౴͚͔ͩΒࢼݧͷਖ਼ղΛ༧ଌ͢ΔΑ͏ͳ΋ͷ ճ౴͔Β֤ࣗͷ৴པੑΛ౷ܭతʹਪఆ͢Δ 13 YES YES YES YES NO YES NO YES YES YES YES NO ? ? ? ਖ਼ղ ໰୊ A. P. Dawid and A. M. Skene: Maximum likelihood estimation of observer error-rates using the EM algorithm, Journal of the Royal Statistical Society. Series C (Applied Statistics), 1979.
14. / 30 ౷ܭతճ౴౷߹ɿճ౴ऀͷ৴པੑΛߟྀ ճ౴ͷੜ੒ϞσϧΛ༻͍ճ౴ऀͷ৴པੑͱਖ਼ղΛಉ࣌ʹਪఆ 14 ɿճ౴ऀ
    ͕ਖ਼ղ͕
    YES ͷ໰୊ʹ
    YES ͱ౴͑Δ֬཰ θj j ɿճ౴ऀ
    ͕ਖ਼ղ͕
    NO ͷ໰୊ʹ
    NO ͱ౴͑Δ֬཰ λj j ti ਖ਼ղ
 :&4
    PS
    /0 YES ti = NO ti = yij θj λj ճ౴ऀͷ৴པੑύϥϝʔλ ճ౴ͷੜ੒Ϟσϧʢ໰୊Jʹର͢Δճ౴ऀKͷճ౴ʣ ੜ੒Ϟσϧʹج͖ͮ
 ճ౴σʔλ͔Β֤ࣗͷ৴པੑ ͱ֤໰୊ͷਖ਼ղΛਪఆ͢Δ ճ౴
 :&4
    PS
    /0
15. / 30 ౷ܭతճ౴౷߹ɿճ౴ऀͷ֬৴౓Λߟྀ w ճ౴ऀʹ֬৴౓΋ฉ͘
    w ֬৴౓ͷߴ͍ਓ΄Ͳೳྗ͕ߴͦ͏͕ͩ֬৴౓Λਖ਼͘͠౴͑Δอূ͸ͳ͍
    ⿞ޡ౴ͳͷʹʮࣗ৴͕͋Δʯͱ౴͑Δਓ͕͍Δʢࣗ৴ա৒ʣ
    ⿞ਖ਼౴ͳͷʹʮࣗ৴͕ͳ͍ʯͱ౴͑Δਓ͕͍Δʢࣗ৴աখʣ
    w ֬৴౓ͷճ౴܏޲΋ߟྀͯ͠ճ౴ऀͷ৴པੑͱਖ਼ղΛ༧ଌ͢Δ ճ౴ऀʹ֬৴౓΋ฉ͍ͯਖ਼ղ༧ଌʹ༻͍Δ 15 
    ΦʔετϥϦΞͷट౎͸γυχʔͰ͔͢ʁ
 
    ճ౴ʹࣗ৴͕͋Γ·͔͢ʁ
    
    
    ͸͍ ͍͍͑ ͸͍ ͍͍͑ S. Oyama, Y. Baba, Y. Sakurai, and H, Kashima: Accurate integration of crowdsourced labels using workers' self-reported confidence scores, In IJCAI, 2013.
16. ᶄ
    ௚ྻ໰߹ͤ
17. / 30 ௚ྻ໰߹ͤ w ଞਓͷ౴͑Λ࢖͍ղ͔ͤΔ͜ͱͰਖ਼͍͠౴͑ΛҾ͖ग़͢ͷ͕௚ྻ໰߹ͤ
    w 'JOE'JY7FSJGZͷ̏ஈ֊Ͱจॻதͷ໰୊ͷ͋ΔՕॴΛݕग़͠ߍਖ਼Λߦ͏
    ⿞'JOE
    ໰୊ͷ͋ΔՕॴͷݕग़
    ⿞'JY
    ߍਖ਼ͷ࣮ࢪ
    ⿞7FSJGZ
    ߍਖ਼͕ਖ਼͍͔֬͠ೝ ଞਓͷ౴͑Λ࢖ͬͯղ͔ͤΔ 17 Figure 2. Crowdproof is a human-augmented proofreader. The drop-down explains the problem (blue title) and suggests Find Fix Verify J. Bigham et al.: Soylent: A word processor with a crowd inside, In UIST, 2010.
18. / 30 ࣗݾగਖ਼ w 4UFQ
    
    Կ΋ݟͤͣʹճ౴ͤ͞Δ
 
 
 
 w 4UFQ
    
    ଞͷճ౴ऀͷଟ਺ܾ݁ՌΛఏࣔ͠࠶౓ճ౴ͤ͞Δ ଞਓͷ౴͑Λఏࣔ͠గਖ਼Λଅ͢ 18 αϯϑϥϯγείͷࣸਅΛબΜͰ͍ͩ͘͞ ͋ͳͨͷճ౴ ଟ਺ܾ݁Ռ ͋ͳͨͷճ౴͕ؒҧ͍ͬͯΔͱࢥ͏৔߹͸గਖ਼͍ͯͩ͘͠͞ N. B. Shah and D. Zhou: No oops, you won’t do it again: Mechanisms for self-correction in crowdsourcing, In ICML, 2016.
19. / 30 मਖ਼ͱධՁͷஞ࣮࣍ߦ
    w 5VS,POUSPM
    ੒Ռ෺ͷमਖ਼ͱධՁΛ܁Γฦ͢ΞϧΰϦζϜ
    ⿞मਖ਼ɾධՁͷཁෆཁΛɺ
 ඼࣭Λঢ়ଶͱ͢Δ෦෼؍ଌϚϧίϑܾఆաఔͰܾఆ
    ⿞े෼ʹमਖ਼Ͱ͖ͨͱ͜ΖͰऴྃ ΞϧΰϦζϜ੍ޚʹΑΓमਖ਼ͱධՁΛ܁Γฦ͢ 19 ੒Ռ෺ मਖ਼͕ ඞཁʁ मਖ਼Λ ࣮ࢪ ධՁ͕ ඞཁʁ मਖ਼લޙͷ ൺֱ ऴྃ YES NO NO YES मਖ਼લޙͷͲͪΒ͔Λ࠾༻ P. Dai et al.: Decision-theoretic control of crowd-sourced workflows, In AAAI, 2010.
20. ᶅ
    ઐ໳Ոൃݟ
21. / 30 ઐ໳Ոൃݟ w ઐ໳ੑ͕ඞཁͳ໰୊Ͱ͸ղ͚ΔਓΛݟ͚ͭͳ͍ͱਖ਼͍͠౴͕͑ಘΒΕͳ͍
    w ؆୯ͳํ๏ɿਖ਼ղط஌໰୊Ͱͷਖ਼౴཰Ͱઐ໳ՈΛݟ͚ͭΔ
    w ςετΛશһʹड͚ͤ͞Δͷ͸൥ࡶͳͷͰଞͷख͕͔ΓΛར༻͍ͨ͠ ղ͚ΔਓΛूஂͷத͔Βݟ͚ͭΔ 21 2
    ݂தೱ౓Λ֬ೝ͢Δඞཁੑ͕࠷΋ߴ͍ҩༀ඼͸ͲΕ͔ʁ
    "
    ΞεϐϦϯɹɹɹɹɹ #
    ϑϩηϛυ
    $
    ςΦϑΟϦϯɹɹɹɹ %
    Πϯυϝλγϯ
    ઐ໳ੑ͕ඞཁͳ໰୊ྫ
22. / 30 ݕࡧΫΤϦʹΑΔઐ໳Ոൃݟ w ݕࡧ࿈ಈܕ޿ࠂ͔Β໰୊΁༠ಋ͠ઐ໳Ո͕Α͘࢖͏ݕࡧΫΤϦΛൃݟ ઐ໳Ո͕Α͘࢖͏ݕࡧΫΤϦΛख͕͔Γͱ͢Δ 22 Figure 3: Example ad to attract users time a user clicks on the ad and we record a conversion event, a the advertising system. This w the system to optimize the adv mizing the number of conversio contribution yield, instead of t the number of clicks. Although optimizing for conv Figure 1: Screenshot of the Quizz system. Intern Intern Fi healthline P. Ipeirotis and E. Gabrilovich: Quizz: targeted crowdsourcing with a billion (potential) users, In WWW, 2014.
23. / 30 ਪનʹΑΔઐ໳Ոൃݟ w શถՕॴʹ੺͍෩ધΛઃஔ
    w ࠷ॳʹશͯͷ੺͍෩ધΛݟ͚ͭͨνʔϜʹ৆͕ۚࢧ෷ΘΕΔίϯςετ
    w ༏উνʔϜ͸ʮ෩ધΛݟ͚ͭͦ͏ͳਓʯΛଞऀ͔ΒͷਪનͰൃݟ %"31"
    3FE
    #BMMPPO
    $IBMMFOHF
    ੺͍෩ધΛ୳͢ίϯςετ 23 contributed articles platform for viral collaboration that used recursive incentives to align the public’s interest with the goal of win- ning the Challenge. This approach was inspired by the work of Peter S. Dodds et al.5 that found that success in us- ing social networks to tackle widely distributed search problems depends on individual incentives. The work of Mason and Watts7 also informed the use of financial incentives to motivate crowdsourcing productivity. The MIT team’s winning strategy was to use the prize money as a finan- cial incentive structure rewarding not only the people who correctly located balloons but also those connecting the finder to the MIT team. Should the team win, they would allocate $4,000 in prize money to each balloon. They Figure 1. Locations in the DARPA Red Balloon Challenge. Figure 2. Example recursive incentive-structure process for the MIT team. J. Tang et al.: Reflecting on the DARPA red balloon challenge, In Communications of the ACM, 2011.
24. / 30 ਪનʹΑΔઐ໳Ոൃݟ w ඃਪનऀ͕෩ધΛൃݟͨ͠Βਪનऀʹ΋ใु͕෼഑͞ΕΔΑ͏ʹઃܭ ༗ӹͳਪન͕ಘΒΕΔΑ͏ʹใुΛઃܭ 24 ਪન ਪન ਪન ෩ધΛൃݟʂ  ͷใु  ͷใु ෩ધΛൃݟʂ  ͷใु ͷใु  ͷใु
25. ᶆ
    ڠௐࢧԉ
26. / 30 ڠௐࢧԉ w ࠔ೉ɾෳࡶͳ໰୊ͷղܾʹ͸ਓؒಉ࢜ͷڠௐ͕ෆՄܽ
    ⿞໰୊Λ෼ׂ͠ΈΜͳͰղ͘
    ⿞ྑ͍ղΛΈΜͳͰݟ͚ͭΔ
    w ݴޠʹΑΔٞ࿦͸εέʔϧ͠ͳ͍ͨΊ
 γεςϚςΟοΫͳڠௐࢧԉ͕ٻΊΒΕΔ ूஂʹΑΔ໰୊ղܾΛࢧԉ͢Δ 26
27. / 30 ڠௐ໰୊෼ׂ w 5VSLPNBUJD
    ໰୊෼ׂɾղܾΛूஂͰߦ͏࢓૊Έ
    ⿞4UFQ
    
    ໰୊Λ෼ׂ͢Δ
    ‣ े෼୯७ͳ෦෼໰୊ʹͳΔ·Ͱ෼ׂΛ܁Γฦ͢
    ⿞4UFQ
    
    ֤෦෼໰୊Λਓ͕ؒղ͘
    ⿞4UFQ
    
    ෦෼໰୊ͷղΛू໿͠࠷ऴతͳղΛग़ྗ ໰୊෼ׂΛूஂͰߦ͍େن໛ͳ໰୊Λղܾ͢Δ 27 ໰୊ྫɿɹ
    ʮʰӕ΋ํศʱ͸ਖ਼͍͔͠ʁʯ͕ςʔϚͷখ࿦จࣥච
    ෦෼໰୊
    ʮӕ͸Α͘ͳ͍ʯͱ͍͏ஈམΛࣥච
    ෦෼໰୊
    ʮඞཁͳӕ͕͋Δʯͱ͍͏ஈམΛࣥච
 ʜ A. Kulkarni et al.: Collaboratively crowdsourcing workflows with Turkomatic, In CSCW, 2012.
28. / 30 ΞΠσΞͷऩू
    w *OOP$FOUJWF
    ಛఆͷ໰୊ͷղʢΞΠσΞʣΛืू͢ΔϓϥοτϑΥʔϜ
    w ໰୊ྫɿʮϑοૉύ΢μʔΛۭؾதʹ֦ࢄͤͣ͞ʹࣃຏ͖νϡʔϒʹೖΕ Δํ๏͸ʁʯ
    w ΞΠσΞ͕࠾༻͞ΕΔͱ৆͕ۚࢧ෷ΘΕΔ ूஂ͔ΒΞΠσΞΛืू͢ΔϓϥοτϑΥʔϜ 28 InnoCentive
29. / 30 ΞΠσΞͷ୳ࡧ w ूஂͰΞΠσΞͷબൈɾަࠥΛ܁Γฦ͢ Ҩ఻తΞϧΰϦζϜʹ฿ͬͯΞΠσΞΛ୳ࡧ͢Δ 29 4UFQ
    
    
 ୈੈ୅ͷݸମΛ֤͕ࣗੜ੒
    4UFQ
    
    
 ౤ථͰ༏ྑݸମΛબൈ͠
 ਓ͕ؒͦΕΛަࠥͯ࣍͠ੈ୅ͷݸମ Λੜ੒
    
    4UFQ
    
    બൈɾަࠥΛ܁Γฦ͢
    
    ྫɿࢠڙ޲͚ͷҜࢠͷσβΠϯ L. Yu and J. V. Nickerson: Cooks or cobblers? crowd creativity through combination, In CHI, 2011.
30. / 30 ·ͱΊ ਓؒͷूஂͱਓ޻஌ೳʹΑΔ໰୊ղܾͷख๏Λ঺հ ਓ޻஌ೳʹΑΔ໰୊ղܾΛ
 ਓ͕ؒࢧԉ
    ਓؒʹΑΔ໰୊ղܾΛ
 ਓ޻஌ೳ͕ࢧԉ 30 ᶃ
    ฒྻ໰߹ͤ ᶄ
    ௚ྻ໰߹ͤ ᶅ
    ઐ໳Ոൃݟ ᶆ
    ڠௐࢧԉ ؆୯ɾ୯७ͳ໰୊ ࠔ೉ɾෳࡶͳ໰୊