○○2vec 再考

builderscon 2019 「○○2vec 再考」の発表資料です

Transcript

1. ©2019 Wantedly, Inc. ʓʓ2vec ࠶ߟ Builderscon 2019 Aug 31, 2019 - Naomichi Agata Photo by Mika Baumeister on Unsplash
2. ©2019 Wantedly, Inc. Naomichi Agata @agatan_ ࣗݾ঺հ - 2018/04~ Wantedly, Inc. - Wantedly People ͷػցֶशΤϯδχΞ - Data Driven Developer Meetup ӡӦϝϯόʔ
   - https://d3m.connpass.com
3. ©2019 Wantedly, Inc. ػցֶश΍ͬͯ·͢
   ! Ξϯέʔτ
4. ©2019 Wantedly, Inc. word2vecͱ͍͏ݴ༿Λ
 ฉ͍ͨ͜ͱ͕͋Δ
   ! Ξϯέʔτ “ύϦ
   
   ϑϥϯε
   
   ೔ຊ
   
   ౦ژz
   Ͱ༗໊ͳ͋ΕͰ͢
5. ©2019 Wantedly, Inc.  ◦◦2vec
   ֓ཁ
    άϥϑߏ଄Λ࢖ͬͨ
   ◦◦2vec  ޮՌతʹ͔ͭ͏ͨΊʹ
    ·ͱΊ ΞδΣϯμ
6. ©2019 Wantedly, Inc.  ◦◦2vec
   ֓ཁ
    άϥϑߏ଄Λ࢖ͬͨ
   ◦◦2vec  ޮՌతʹ͔ͭ͏ͨΊʹ
    ·ͱΊ ΞδΣϯμ
7. ©2019 Wantedly, Inc.  ◦◦2vec ֓ཁ
   
   w ◦◦2vec
   ͱ͸
   w ͳʹ͕͏Ε͍͠ͷ͔
   w ݁ہͲΜͳΞϓϦέʔγϣϯʹ͔ͭ͑Δͷ͔ ΞδΣϯμ
8. ©2019 Wantedly, Inc.  ◦◦2vec
   ֓ཁ
   w ◦◦2vec
   ͱ͸
   w ͳʹ͕͏Ε͍͠ͷ͔
   w ݁ہͲΜͳΞϓϦέʔγϣϯʹ͔ͭ͑Δͷ͔ ΞδΣϯμ
9. ©2019 Wantedly, Inc. ◦◦2vec ͓͍͍͠ ͏·͍ ΤϯδχΞ Word2vec y x word2vec: ୯ޠΛ௿࣍ݩͷϕΫτϧʹม׵͢Δ ͓͍͍͠
 
    ͏·͍
   
    ΤϯδχΞ
   
   
10. ©2019 Wantedly, Inc. ◦◦2vec ͱ͸ word2vec
    ͸
 ʮ୯ޠʯ
    Λ
    ʮҙຯ͕ࣅͨ୯ޠ͸ࣅͨϕΫτϧʹͳΔʯ
    Λຬͨ͢ϕΫτϧʹม׵͢Δ Paris Tokyo France word2vec ( 0.1, 0.3, -0.2, …) (-0.7, -0.6, 0.9, …) ( 0.8, 0.4, -0.7, …)
11. ©2019 Wantedly, Inc. ◦◦2vec ͱ͸ Entity 1 Entity 2 Entity 3 … ◦◦2vec ( 0.1, 0.3, -0.2, …) (-0.7, -0.6, 0.9, …) ( 0.8, 0.4, -0.7, …) … ◦◦2vec
    ͸
 ʮ◦◦ʯ
    Λ
    ʮ͋Δੑ࣭ʯ
    Λຬͨ͢ϕΫτϧʹม׵͢Δ
12. ©2019 Wantedly, Inc.  ◦◦2vec
    ֓ཁ
    w ◦◦2vec
    ͷఆٛʹ͍ͭͯ
    w ͳʹ͕͏Ε͍͠ͷ͔
    w ݁ہͲΜͳΞϓϦέʔγϣϯʹ͔ͭ͑Δͷ͔ ΞδΣϯμ
13. ©2019 Wantedly, Inc. ◦◦2vec Λ࢖͏͜ͱͰɺෳࡶͰந৅తͳ৘ใΛ
 ػցతʹѻ͍΍͍͢ϕΫτϧʹม׵Ͱ͖Δʂ ◦◦2vec
    ֓ཁ ͳʹ͕͏Ε͍͠ͷ͔
14. ©2019 Wantedly, Inc. w ෳࡶͳੑ࣭΍ؔ܎ੑ͕ѻ͍΍͘͢ͳΔ
    w l୯ޠͷҙຯz
    ͷΑ͏ͳෳࡶͰந৅తͳ΋ͷ͸ѻ͍ͮΒ͍
    w lϕΫτϧz
    ͸ͨͩͷ਺ࣈͳͷͰ؆୯ʂ
    w Entity
    ͷੑ࣭ɺEntity
    ؒͷؔ܎ੑ͕਺ࣈͰදݱͰ͖Δ
    w ϕΫτϧʹ͸
    Entity
    ͷੑ࣭͕ຒΊࠐ·Ε͍ͯΔʢ͸ͣʣ
    w ϕΫτϧಉ࢜ͷԋࢉͰ
    Entity
    ؒͷؔ܎ੑ͕දͤΔ
    w ࣅ͍ͯΔॱʹ୯ޠΛฒͼସ͑Δͱ͍ͬͨૢ࡞͕ՄೳʹͳΔ ◦◦2vec
    ֓ཁ ͳʹ͕͏Ε͍͠ͷ͔
15. ©2019 Wantedly, Inc.  ◦◦2vec
    ֓ཁ
    w ◦◦2vec
    ͷఆٛʹ͍ͭͯ
    w ͳʹ͕͏Ε͍͠ͷ͔
    w ݁ہͲΜͳΞϓϦέʔγϣϯʹ͔ͭ͑Δͷ͔ ΞδΣϯμ
16. ©2019 Wantedly, Inc. w ಉ͡ਓ෺͕͍ࣸͬͯΔը૾͕ࣅͨϕΫτϧʹͳΔΑ͏ʹ إը૾ݕࡧ CNN ( 0.1, 0.3, -0.2, …) (-0.7, -0.6, 0.9, …) (-0.7, -0.5, 0.7, …)
17. ©2019 Wantedly, Inc. w ϕΫτϧಉ࢜ͷൺֱͰʮඃࣸମͷಉҰੑʯΛ൑ఆͰ͖Δ
    w ը૾Λ௚઀ൺֱ͢Δͷͱൺ΂ͯʜ
    w ܭࢉ͕ߴ଎ʢ಺ੵऔΔ͚ͩʣ
    w ࡱӨ؀ڥͷҧ͍ͳͲΛٵऩͰ͖Δ إը૾ݕࡧ CNN ⃗ ⃗ ⃗ ⃗ ⃗ ⃗ ⃗ ⃗ ⃗ Index ྨࣅϕΫτϧ Λݕࡧ
18. ©2019 Wantedly, Inc. w ◦◦2vec ͱผͷΞϧΰϦζϜΛ૊Έ߹Θͤͯ࢖͏͜ͱ΋Α͋͘Δ
    w ྫ
    Doc2vec + NN
    ͰจॻͷΧςΰϦ෼ྨ จॻ෼ྨ Doc2vec ( 0.1, 0.3, -0.2, …) “builderscon͸
 ʮ஌Βͳ͔ͬͨɺΛฉ͘ʯΛ
 ςʔϚͱٕͨ͠ज़ΛѪ͢Δ
 શͯͷΪʔΫୡͷ͓ࡇΓͰ͢ɻ” Neural Network ΧςΰϦ༧ଌ
19. ©2019 Wantedly, Inc. ◦◦2vec Λ࢖͏͜ͱͰɺෳࡶͰந৅తͳ৘ใΛ
 ػցతʹѻ͍΍͍͢ϕΫτϧʹม׵Ͱ͖Δʂ ◦◦2vec
    ֓ཁ ͳʹ͕͏Ε͍͠ͷ͔ʢ࠶ܝʣ
20. ©2019 Wantedly, Inc.  ◦◦2vec
    ֓ཁ
     άϥϑߏ଄Λ࢖ͬͨ
    ◦◦2vec  ޮՌతʹ͔ͭ͏ͨΊʹ
     ·ͱΊ ΞδΣϯμ
21. ©2019 Wantedly, Inc. άϥϑߏ଄ ϊʔυͱϊʔυ͕ΤοδͰͭͳ͕͍ͬͯΔߏ଄ ൚༻ੑ͕ߴ͘ɺ͍ΖΜͳ৘ใΛάϥϑͰදͤΔ
22. ©2019 Wantedly, Inc. άϥϑߏ଄ ͨͱ͑͹ʜ
23. ©2019 Wantedly, Inc. άϥϑߏ଄ Ϣʔβ͕
    follow
    ͍ͯ͠ΔϢʔβ
24. ©2019 Wantedly, Inc. άϥϑߏ଄ Ϣʔβ͕ॴଐ͍ͯ͠Δاۀ
25. ©2019 Wantedly, Inc. άϥϑߏ଄ Ϣʔβ͕
    like
    ͨ͠هࣄ
26. ©2019 Wantedly, Inc. άϥϑߏ଄ χϡʔεʹొ৔͢Δاۀ
27. ©2019 Wantedly, Inc. άϥϑߏ଄ දݱྗ͸ߴ͍ʜ
    ͕ɺѻ͍ʹ͍͘
28. ©2019 Wantedly, Inc. w άϥϑߏ଄
    
    ੈͷதͷ͞·͟·ͳෳࡶͳσʔλΛදݱͰ͖Δ
    w ◦◦2vec 
    ෳࡶͳσʔλΛѻ͍΍͘͢Ͱ͖Δ άϥϑߏ଄Λ͔ͭͬͨ ◦◦2vec ૊Έ߹ΘͤͨΒศརͦ͏
29. ©2019 Wantedly, Inc. άϥϑߏ଄ ͨͱ͑͹ʜ άϥϑߏ଄Λ࢖ͬͨ
    ◦◦2vec
30. ©2019 Wantedly, Inc. άϥϑߏ଄Λ͔ͭͬͨ ◦◦2vec ◦◦2vec ( 0.1, 0.3, -0.2, …) (-0.7, -0.6, 0.9, …) ( 0.8, 0.4, -0.7, …) … 
 ʮάϥϑதͷ֤ϊʔυʯ
    Λ
    ʮྡΓ߹ͬͨϊʔυ͕ࣅͨϕΫτϧʹͳΔʯ
    Λຬͨ͢ϕΫτϧʹม׵͢Δ
31. ©2019 Wantedly, Inc. Ͳ͏΍ֶͬͯश͢Δͷ͔ ྡΓ߹͍ͬͯΔ ཭Ε͍ͯΔ ͨ͘͞Μͷख๏͕ݚڀ͞Ε͍ͯͯ೔ਐ݄าɻ ͜͜Ͱ͸γϯϓϧͳ࣮૷Πϝʔδ͚ͩ࿩͠·͢ɻ
32. ©2019 Wantedly, Inc. Ͳ͏΍ֶͬͯश͢Δͷ͔ ͱ ͸ྡΓ߹͍ͬͯΔ ͸཭Ε͍ͯΔ y x ࠷ॳ͸ద౰ͳϕΫτϧ͔Βελʔτ
33. ©2019 Wantedly, Inc. Ͳ͏΍ֶͬͯश͢Δͷ͔ ͱ ͸ྡΓ߹͍ͬͯΔ ͸཭Ε͍ͯΔ y x ྡΓ߹ͬͨϊʔυͷϕΫτϧΛ͚ۙͮΔ
34. ©2019 Wantedly, Inc. Ͳ͏΍ֶͬͯश͢Δͷ͔ ͱ ͸ྡΓ߹͍ͬͯΔ ͸཭Ε͍ͯΔ y x ཭Ε͍ͯΔϊʔυͷϕΫτϧΛԕ͚͟Δ
35. ©2019 Wantedly, Inc. Ͳ͏΍ֶͬͯश͢Δͷ͔ ͱ ͸ྡΓ߹͍ͬͯΔ ͸཭Ε͍ͯΔ y x ͜ΕΛऩଋ͢Δ·Ͱ܁Γฦ͢
36. ©2019 Wantedly, Inc. Ͳ͏΍ֶͬͯश͢Δͷ͔ ͱ ͸ྡΓ߹͍ͬͯΔ ͸཭Ε͍ͯΔ y x ͜ΕΛऩଋ͢Δ·Ͱ܁Γฦ͢
37. ©2019 Wantedly, Inc. Ͳ͏΍ֶͬͯश͢Δͷ͔ ͱ ͸ྡΓ߹͍ͬͯΔ ͸཭Ε͍ͯΔ y x ͜ΕΛऩଋ͢Δ·Ͱ܁Γฦ͢
38. ©2019 Wantedly, Inc. Ͳ͏΍ֶͬͯश͢Δͷ͔ ͋ΔϊʔυͷϕΫτϧ͸ɺपลϊʔυͱͷؔ܎ੑʹΑܾͬͯఆ͢Δ
39. ©2019 Wantedly, Inc. Ͳ͏΍ֶͬͯश͢Δͷ͔ ࣅͨϊʔυͱͭͳ͕͍ͬͯΔϊʔυಉ࢜͸ࣅͨϕΫτϧʹͳΔ
40. ©2019 Wantedly, Inc. ͲΜͳ͜ͱ͕Ͱ͖Δ͔ άϥϑߏ଄Λ͔ͭͬͨ
    ʓʓ2vec
    ͷΞϓϦέʔγϣϯ
41. ©2019 Wantedly, Inc. άϥϑߏ଄Λ͔ͭͬͨ ʓʓ2vec ͷΞϓϦέʔγϣϯ Ϣʔβʹ
    follow
    ͷਪનΛ͍ͨ͠ʂ
42. ©2019 Wantedly, Inc. άϥϑߏ଄Λ͔ͭͬͨ ʓʓ2vec ͷΞϓϦέʔγϣϯ Ϣʔβؒͷ
    follow
    ؔ܎ user2vec
43. ©2019 Wantedly, Inc. άϥϑߏ଄Λ͔ͭͬͨ ʓʓ2vec ͷΞϓϦέʔγϣϯ w ڞ௨ͷͭͳ͕Γ͕ଟ͍ਓ͸ࣅͨϕΫτϧʹͳΔ
    w ϕΫτϧ͕ࣅ͍ͯΔϢʔβΛਪન͢Δ
44. ©2019 Wantedly, Inc. άϥϑߏ଄Λ͔ͭͬͨ ʓʓ2vec ͷΞϓϦέʔγϣϯ ϢʔβʹχϡʔεͷਪનΛ͍ͨ͠ʂ
45. ©2019 Wantedly, Inc. άϥϑߏ଄Λ͔ͭͬͨ ʓʓ2vec ͷΞϓϦέʔγϣϯ Ϣʔβͱχϡʔεͷ
    like
    ؔ܎ user2vec & news2vec
46. ©2019 Wantedly, Inc. άϥϑߏ଄Λ͔ͭͬͨ ʓʓ2vec ͷΞϓϦέʔγϣϯ w ҟͳΔछྨͷϊʔυͰ΋௚઀ϕΫτϧΛൺֱͰ͖Δʂ
    w ϢʔβʹʮϕΫτϧ͕ࣅ͍ͯΔهࣄʯΛਪન͢Δ
47. ©2019 Wantedly, Inc. άϥϑߏ଄Λ͔ͭͬͨ ʓʓ2vec ͷΞϓϦέʔγϣϯ Follower
    ͕ಡΜͩχϡʔε͸
 ڵຯ͕͋ΔͷͰ͸ʁ
48. ©2019 Wantedly, Inc. άϥϑߏ଄Λ͔ͭͬͨ ʓʓ2vec ͷΞϓϦέʔγϣϯ Ϣʔβͱχϡʔεͷ
    like
    ؔ܎ user2vec & news2vec Ϣʔβؒͷ
    follow
    ؔ܎
49. ©2019 Wantedly, Inc. άϥϑߏ଄Λ͔ͭͬͨ ʓʓ2vec ͷΞϓϦέʔγϣϯ Ϣʔβ͸
    follower
    ͷձࣾͷχϡʔεʹ
 ڵຯ͕͋Δͷ͔΋ʁ
50. ©2019 Wantedly, Inc. άϥϑߏ଄Λ͔ͭͬͨ ʓʓ2vec ͷΞϓϦέʔγϣϯ Ϣʔβͱχϡʔεͷ
    like
    ؔ܎ user2vec & news2vec
 & company2vec Ϣʔβؒͷ
    follow
    ؔ܎ اۀͱؔ࿈χϡʔεͷؔ܎ Ϣʔβͱاۀͷॴଐؔ܎
51. ©2019 Wantedly, Inc.  ◦◦2vec
    ֓ཁ
     άϥϑߏ଄Λ࢖ͬͨ
    ◦◦2vec  ޮՌతʹ͔ͭ͏ͨΊʹ
     ·ͱΊ ΞδΣϯμ
52. ©2019 Wantedly, Inc. w ͱʹ͔͘൚༻ੑ͕ߴ͍ɻ͍͍ͩͨͷ͜ͱ͸දݱ͸Ͱ͖Δɻ
    w ༩͑Δάϥϑ͸มΘͬͯ΋ΞϧΰϦζϜ͸ҰॹͰྑ͍
    w ͞·͟·ͳཁҼΛಉ࣌ʹߟֶྀͨ͠श͕Ͱ͖Δ
    w ͭͳ͕Γɺॴଐاۀɺهࣄͱاۀͷؔ܎ੑɺʜ
    
    w ͍ΖΜͳαϒλεΫʹస༻Ͱ͖Δ
    ◦◦2vec
    ͕Ͱ͖Δ άϥϑߏ଄Λ͔ͭͬͨ
    ʓʓ2vec ͷྑ͍ͱ͜Ζ
53. ©2019 Wantedly, Inc. • ͭΒ͍͜ͱ΋݁ߏ͋Δ w άϥϑͷͲͷ෦෼͕Ͳ͏ޮ͍͍ͯΔͷ͔෼͔ΓͮΒ͍
    w ߋ৽ɺ࠶ֶश͕େม
    
    όʔδϣχϯά
    w ʢαϒλεΫͰϦʔΫ͢Δࣄނ͕ൃੜ͠΍͍͢ʣ ਖ਼͘͠࢖͏ͨΊʹ
54. ©2019 Wantedly, Inc. • શϚγͷڊେάϥϑΛೖྗʹֶश͢Δͱɺ
 ༷ʑͳϊʔυؒͷӨڹΛߟྀͨ͠ਪનΛ࡞Ε͍ͯΔؾ෼ʹͳΕΔ • ຊ౰ʹͦͷؔ܎ੑ͸ʮࠓղ͖͍ͨ՝୊ʯʹͱͬͯ༗༻ͳͷ͔ʁ • υϝΠϯ஌ࣝΛ׆༻͢Δ + ࣮ݧͯ͠ධՁ͢Δ͔͠ͳ͍ άϥϑͷͲͷ෦෼͕Ͳ͏ޮ͍͍ͯΔͷ͔෼͔ΓͮΒ͍
55. ©2019 Wantedly, Inc. • ৽͍͠ϊʔυɺΤοδ͕௥Ճ͞ΕΔͨͼʹશମΛ࠶ֶश͢Δ • ࠩ෼ߋ৽తͳ͜ͱ΋ݚڀ͸͞Ε͍ͯΔ͕େม • ࠶ֶशલͱޙͰ͸ಘΒΕΔϕΫτϧ͸׬શʹผ෺ʹͳΔ • ಉ͡ Entity Ͱ΋·ͬͨ͘ผ෺ʹͳΔ • ؒҧͬͯݹ͍ϕΫτϧͱ৽͍͠ϕΫτϧͰԋࢉ͢ΔͱյΕΔ • ʓʓ2vec ͷޙΖʹผͷϞσϧΛͭͳ͍͛ͯΔ৔߹͸
 ͦͷϞσϧ΋ಉ࣌ʹ࠶ֶश͢Δඞཁ͕͋Δ ߋ৽ɺ࠶ֶश͕ͭΒ͍
    
    όʔδϣχϯά
56. ©2019 Wantedly, Inc. ΑΓޮՌతʹ࢖͏ͨΊʹ • ߴ଎ʹ࡞Εͯͦͦ͜͜༏लͳϕʔεϥΠϯͱͯ͠࢖͏ • ݶΒΕͨϦιʔεͰ੒ՌΛ࠷େԽ͢Δ • ͻͱཻͰ N ౓͓͍͍͠ঢ়ଶΛ࡞Δ • σʔλΛάϥϑతʹѻ͍΍͍͢Α͏ʹͯ͠ੵΈॏͶΔ • ಉ͡ Entity ͕ʮಉ͡ʯͰ͋Δͱѻ͑ΔΑ͏ʹ͓ͯ͘͠ • ͦͷυϝΠϯʹ͓͚Δ Knowledge Base Λ࡞Δ • λεΫʹΑͬͯద੾ͳαϒάϥϑΛఆٛ͢Δ • ҋӢͳϞσϧڞ༗ / άϥϑڞ༗͸͠ͳ͍
57. ©2019 Wantedly, Inc. w ൚༻ੑ͕ͱͯ΋ߴ͍
    w ͞·͟·ͳαϒλεΫʹస༻͠΍͍͢
    w ߴ଎ʹ༏लͳϕʔεϥΠϯϞσϧΛ࡞ΕΔ ·ͱΊ άϥϑߏ଄Λ࢖ͬͨ ◦◦2vec ͸ͱͯ΋ศར ◦◦2vec Ͱෳࡶͳ΋ͷΛѻ͍΍͘͢͠Α͏ w ୯ޠͷҙຯ
    ඃࣸମͷ৘ใ
    ʜ
    w ϕΫτϧʹ͢Ε͹ػցతʹѻ͍΍͍͢ʂ
58. ©2019 Wantedly, Inc. ࢀߟจݙ - Efficient Estimation of Word Representations in Vector Space - Tomas Mikolov, et al., 2013, ICLR - https://arxiv.org/abs/1301.3781 - FaceNet: A Unified Embedding for Face Recognition and Clustering - Florian Schroff, et al., 2015, CVPR - https://arxiv.org/abs/1503.03832 - PyTorch-BigGraph: A Large-scale Graph Embedding Framework - Adam Lerer, et al., 2019, SysML - https://arxiv.org/abs/1903.12287 - StarSpace: Embed All The Things! - Ledell Wu, et al. - https://arxiv.org/abs/1709.03856