Speaker Deck
Speaker Deck Pro
Sign in
Sign up
for free
ELMoで文脈に応じた類似キーワード検索システムを作った話 / ELMo for Searching Similar Keywords
tagucci
March 24, 2020
Technology
2
1.2k
ELMoで文脈に応じた類似キーワード検索システムを作った話 / ELMo for Searching Similar Keywords
ML@Loft #11. 類似画像/テキスト検索
tagucci
March 24, 2020
Tweet
Share
Other Decks in Technology
See All in Technology
humank
0
210
cygames
1
330
kurochan
0
530
ocise
1
1.3k
line_developers
PRO
3
2.6k
satoryu
0
2k
ocise
0
110
kentaro
1
320
osonoi
0
160
clustervr
0
210
hecateball
1
12k
yuyaabo
0
250
Featured
See All Featured
maggiecrowley
8
390
colly
186
14k
aarron
258
36k
roundedbygravity
241
21k
bermonpainter
342
26k
morganepeng
18
1.1k
maltzj
500
36k
lynnandtonic
270
16k
philnash
8
480
smashingmag
229
18k
jmmastey
8
530
swwweet
206
6.8k
Transcript
&-.PͰจ຺ʹԠͨ͡ ྨࣅΩʔϫʔυݕࡧγεςϜ Λ࡞ͬͨ ©The Asahi Shimbun Company 2020 .-!-PGU גࣜձࣾே৽ฉࣾాޱ༤࠸
݄
ே৽ฉͷݚڀ։ൃ!ϝσΟΞϥϘ 2 ࣗಈཁɺࣗಈߍਖ਼ͳͲࣗવݴޠॲཧͷجૅݚڀΛߦ͍ͬͯ ΔɻΞτϓοτͷܗֶࣜձൃද͕ϝΠϯ ݚڀدΓ ઌͷ/-1ͰݴޠࢿݯΛ͍͖ͨͩ·ͨ͠ ݚڀՌཁ"1*ͱͯ͠ެ։தͳͷͰͬͯΈ͍ͯͩ͘͞ʂ .-!-PGUओʹ"84্ͰػցֶशϫʔΫϩʔυΛ ӡ༻͍ͯ͠ΔσϕϩούʔɾσʔλαΠΤϯςΟετ
ͷͨΊͷʮ͓Έ૬ஊձʯͰ͢ɻ݄ʹճ"84-PGU 5PLZPͰ։࠵͠ɺຖճ׆ൃͳ͕ٞߦΘΕ͍ͯ·͢ɻ ୈճྨࣅը૾ݕࡧΛςʔϚʹɺಛྔʹམͱ͠ࠐ ΉωοτϫʔΫͷɺಛྔݕࡧͷɾՄ༻ੑ ্ͷɺྨࣅը૾ݕࡧͷγεςϜઃܭશମͷͳͲʹ ͍ͭͯٞ͠·͢ɻ͡Ίʹొஃऀͷํʑ͔Βͣ ͭͷࣗݾհ-5ܗࣜͰྨࣅը૾ݕࡧʹ͓͚Δ՝Λ ఏى͍͍ͨͩͨޙɺࢀՃऀͷօ͞Μ͔Β͍࣭ͨ ΛݩʹύωϧσΟεΧογϣϯΛ࣮ࢪ͠·͢ɻ ϞόΠϧ͚χϡʔεΞϓϦ༻ จࣈ Ͱੜ "84Ͱʮ͓Έ૬ஊձʯ -PGU͕ʮ͓Έ૬ஊձʯ "84-PGUͰ͓Έ૬ஊ -4-PGUͰ͓Έ૬ஊձ .-!-PGU͕૬ஊձ ީิΛͭग़ྗ χϡʔεهࣄͰͳͯ͋͘ΔఔͦΕΒ͍͠ͷ͕ग़ͤΔ https://cl.asahi.com/ "1*ΛͬͯΈ͍ͨਓͥͻͪ͜Β ©The Asahi Shimbun Company 2020
ே৽ฉͷݚڀ։ൃ!*$53"% 3 ࣾͷ՝ղܾΛ͢ΔνʔϜɻϓϩτλΠϓ࡞αʔϏ εԽ͕ϝΠϯ ։ൃدΓ ߴߍٿͷࣗಈઓධ࡞ͷ͓ʔͱΓ͌ɺ"*ܦࡁهऀΈͭ ͚ʔͨɺ࢛࣌ࣄΫΠζαʔϏεͷ2SJDIΛ։ൃ ©The Asahi
Shimbun Company 2020
ࠓ͢͜ͱ 4 2SJDIʹ͓͚Δ࢛ΫΠζͷ࡞ϑϩʔ XPSEWFDʹΑΔྨࣅΩʔϫʔυݕࡧͱͦͷ &-.PʹΑΔྨࣅΩʔϫʔυݕࡧ ©The Asahi Shimbun Company 2020
https://qrich.asahi.com/
5 ©The Asahi Shimbun Company 2020 2SJDIʹ͓͚Δ࢛ΫΠζͷ࡞ϑϩʔ هࣄσʔλͷऔಘ هࣄຊจ͔Β݀ൈ͖͢ΔΩʔϫʔυΛબ ॏཁޠநग़
બͨ͠Ωʔϫʔυʹྨࣅ͢Δ୯ޠΛݕࡧ͢Δ ͱΛ߹Θ࢛ͤͯͱ͢Δ
6 ©The Asahi Shimbun Company 2020 2SJDIʹ͓͚Δ࢛ΫΠζͷ࡞ϑϩʔ هࣄσʔλͷऔಘ هࣄຊจ͔Β݀ൈ͖͢ΔΩʔϫʔυΛબ ॏཁޠநग़
બͨ͠Ωʔϫʔυʹྨࣅ͢Δ୯ޠΛݕࡧ͢Δ ͱΛ߹Θ࢛ͤͯͱ͢Δ ࠓ͢ͷ͜ͷ෦
7 ©The Asahi Shimbun Company 2020 2SJDIʹ͓͚Δ࢛ΫΠζͷ࡞ϑϩʔ هࣄσʔλͷऔಘ هࣄຊจ͔Β݀ൈ͖͢ΔΩʔϫʔυΛબ ॏཁޠநग़
બͨ͠Ωʔϫʔυʹྨࣅ͢Δ୯ޠΛݕࡧ͢Δ ͱΛ߹Θ࢛ͤͯͱ͢Δ ࠓ͢ͷ͜ͷ෦ ˞ʮΩʔϫʔυʯͷཻ/&PMPHE ࣙॻͰׂ͞Ε໊ͨࢺΛࢦ͠·͢
8 ©The Asahi Shimbun Company 2020 XPSEWFDʹΑΔྨࣅΩʔϫʔυݕࡧ ே৽ฉͷهࣄσʔλΛ༻͍ͯXPSEWFD TLJQHSBN ͷ
୯ޠϕΫτϧΛߏங ୯ޠϕΫτϧHFOTJNͰಡΈࠐΊΔͷͰ؆୯ʹۙͷΩʔ ϫʔυ͕ݕࡧՄೳ
9 ©The Asahi Shimbun Company 2020 هࣄͷ༰ʹؔΘΒͣۙͷ୯ޠ͕ৗʹಉ͡ XPSEWFDʹΑΔྨࣅΩʔϫʔυݕࡧ ே৽ฉͷهࣄσʔλΛ༻͍ͯXPSEWFD TLJQHSBN
ͷ ୯ޠϕΫτϧΛߏங ୯ޠϕΫτϧHFOTJNͰಡΈࠐΊΔͷͰ؆୯ʹۙͷΩʔ ϫʔυ͕ݕࡧՄೳ
10 ©The Asahi Shimbun Company 2020 XPSEWFDͩͱࠔΔϙΠϯτ ͦͦྑ͍දݱ͕֫ಘͰ͖͍ͯͳ͍ Ωʔϫʔυͷ୯ޠϕΫτϧͷۙL୯ޠৗʹಉ͡
11 ©The Asahi Shimbun Company 2020 XPSEWFDͩͱࠔΔϙΠϯτ ͦͦྑ͍දݱ͕֫ಘͰ͖͍ͯͳ͍ Ωʔϫʔυͷ୯ޠϕΫτϧͷۙL୯ޠৗʹಉ͡ ถΧϦϑΥϧχΞभͷશҬʹ̍̕ɺ࣮࣭తͳ֎ग़ېࢭ
໋ྩ͕ग़͞ΕͨɻϩαϯθϧεαϯϑϥϯγείͳͲͷ େࢢͷ΄͔ɺγϦίϯόϨʔΛ༴͠ɺ׆ؾ͋Δ๛͔ͳ Ҭͱͯ͠ΒΕΔಉभ͕ͩɺܦࡁ׆ಈΑΓ৽ܕίϩφΠ ϧεରࡦΛ༏ઌͨ͠ܗͩɻഎܠʹઌखΛଧͭ͜ͱͰײછ ֦େΛ৯͍ࢭΊΔૂ͍͕͋Δɻ هࣄຊจ ૂͬͨ෦ͷҨࢠΛࣗ༝ʹվมͰ͖ΔήϊϜฤूٕज़ɻ ಛʹɺʹొͨ͠ʮ$3*413$BTʢΫϦεύʔ ΩϟεφΠϯʣʯɺϊʔϕϧީิͱݴΘΕΔɻҩྍ ৯ͳͲ͞·͟·ͳͰԠ༻͕ظ͞ΕΔҰํɺͦͷ ಛڐΛ८ͬͯੈքதͰ૪͍͕܁Γ͛ΒΕ͍ͯΔɻڊֹͷ རӹʹͭͳ͕ΔՄೳੑ͕͋Δಛڐͷߦํʕʕɻ
12 ©The Asahi Shimbun Company 2020 XPSEWFDͩͱࠔΔϙΠϯτ ͦͦྑ͍දݱ͕֫ಘͰ͖͍ͯͳ͍ Ωʔϫʔυͷ୯ޠϕΫτϧͷۙL୯ޠৗʹಉ͡ ถΧϦϑΥϧχΞभͷશҬʹ̍̕ɺ࣮࣭తͳ֎ग़ېࢭ
໋ྩ͕ग़͞ΕͨɻϩαϯθϧεαϯϑϥϯγείͳͲͷ େࢢͷ΄͔ɺγϦίϯόϨʔΛ༴͠ɺ׆ؾ͋Δ๛͔ͳ Ҭͱͯ͠ΒΕΔಉभ͕ͩɺܦࡁ׆ಈΑΓ৽ܕίϩφΠ ϧεରࡦΛ༏ઌͨ͠ܗͩɻഎܠʹઌखΛଧͭ͜ͱͰײછ ֦େΛ৯͍ࢭΊΔૂ͍͕͋Δɻ هࣄຊจ ૂͬͨ෦ͷҨࢠΛࣗ༝ʹվมͰ͖ΔήϊϜฤूٕज़ɻ ಛʹɺʹొͨ͠ʮ$3*413$BTʢΫϦεύʔ ΩϟεφΠϯʣʯɺϊʔϕϧީิͱݴΘΕΔɻҩྍ ৯ͳͲ͞·͟·ͳͰԠ༻͕ظ͞ΕΔҰํɺͦͷ ಛڐΛ८ͬͯੈքதͰ૪͍͕܁Γ͛ΒΕ͍ͯΔɻڊֹͷ རӹʹͭͳ͕ΔՄೳੑ͕͋Δಛڐͷߦํʕʕɻ ʮγϦίϯόϨʔʯͷۙΛXPSEWFDͰऔಘ͢Δͱɺ *5اۀ αϯϊθ όΠΦϕϯνϟʔ ͱ͍͏݁Ռʹ ʮήϊϜฤूʯͷۙΛXPSEWFDͰऔಘ͢Δͱɺ &4ࡉ๔ ώτ ສೳࡉ๔ ͱ͍͏݁Ռʹ
13 ©The Asahi Shimbun Company 2020 ιϑτόϯΫ̑ɺ࣍ੈͷߴҠಈ௨৴ํࣜʮ̜̑ʯͷαʔ ϏεΛࠓ݄̎̓ʹ࢝ΊΔͱൃදͨ͠ɻ͍·ͷ̜̐ͷྉۚʹɺ݄ ֹ̍ઍԁʢ੫ൈ͖ʣΛՃ͢Ε͑ΔΑ͏ʹ͢Δɻܞଳిେ ख̏ࣾͰ̜̑ͷ։࢝࣌ظͱྉۚΛൃදͨ͠ͷॳΊͯɻ
ιϑτόϯΫͷ౦ڊख͕ࠓقͷ։ນखΛΊΔ͜ ͱ͕̒·Ͱʹܾ·ͬͨɻϓϩ̔Ͱॳͷେͱͳ Δɻ̎̌ͷ։ນઓຊڌͷԬϖΠϖΠυʔϜͰ ϩοςͱରઓ͢Δɻ ถάʔάϧͷձࣾʮΞϧϑΝϕοτʯͷ࣌Ձ૯ֹ͕̍ ̒ɺ̍ஹυϧʢ̍̍̌ஹԁʣΛಥഁͨ͠ɻถاۀͰ ɺΞοϓϧɺΞϚκϯɺϚΠΫϩιϑτʹଓ͖̐ࣾ ʹͳΔɻຊͷ࣌Ձ૯ֹτοϓͷτϤλࣗಈंʢ̎̑ஹ ԁʣͳͲɺຊͱͷ͕ࠩ·͢·֦͢େ͍ͯ͠Δɻ ͔Βʹ͔͚ͯɺੈք֤Ͱʮաڈ࠷ѱʯͱݺΕ ΔྛՐࡂ͕૬࣍͗·ͨ͠ɻٿԹஆԽ͕ҰҼͱࢦఠ͞Ε͍ͯ ·͕͢ɺݱͰऔࡐͯ͠ΈΔͱผͷଆ໘ݟ͖͑ͯ·ͨ͠ɻΦʔ ετϥϦΞɺϒϥδϧͷΞϚκϯɺถΧϦϑΥϧχΞͰԿ͕ى͖ ͍ͯΔͷ͔ɺಛһ͕ใࠂ͠·͢ɻ XPSEWFDͩͱࠔΔϙΠϯτ εϙʔπɺܦࡁهࣄʹ͓͚ΔʮιϑτόϯΫʯɻۙ୯ޠͲͪΒʮָఱʯɺʮ%F/"ʯɺʮΦϦοΫεʯ اۀͷΞϚκϯͱଳӍྛͷΞϚκϯɻۙ୯ޠͲͪΒʮωοτ௨ൢʯɺʮάʔάϧʯɺʮΩϯυϧʯ ͦͦྑ͍දݱ͕֫ಘͰ͖͍ͯͳ͍ Ωʔϫʔυͷ୯ޠϕΫτϧͷۙL୯ޠৗʹಉ͡
ΑΓྑ͍୯ޠͷϕΫτϧදݱΛ ٻΊͯ&-.PʹͨͲΓண͘ ©The Asahi Shimbun Company 2020 14
15 ©The Asahi Shimbun Company 2020 ͦͦ&-.Pͱʁ ݴޠϞσϧʹΑΔϕΫτϧදݱ l&NCFEEJOHGSPN-BOHVBHF.PEFMzͷུশ &-.P<>ࣗମCJ-45.Λ༻͍ͯݴޠϞσϧɻXPSEWFDʹ
ΑΔ୯ޠϕΫτϧͱҟͳΓɺจΛCJ-45.ͰಡΈࠐΉͷͰ จ຺ʹԠͯ͡୯ޠͷϕΫτϧදݱ͕มԽ͢Δ
16 ©The Asahi Shimbun Company 2020 จ຺ʹԠͨ͡୯ޠϕΫτϧ &-.PϕΫτϧ .- !
-PGU ओ ʹ "84 ʜ DIBS$//Λͬͯ ୯ޠͷϕΫτϧΛදݱ CJ-45.ͷ CJ-45.ͷ ࠓճCJ-45.ͷͱͷӅΕঢ়ଶΛ݁߹ͯ͠&-.P ϕΫτϧ ࣍ݩ ͱ͢Δ &-.PϕΫτϧ
17 ©The Asahi Shimbun Company 2020 ͳͥ&-.Pͳͷ͔ʁ XPSEWFD TLJQHSBNDCPX Ҏ֎ͷ୯ޠϕΫτϧͷϞσϧ
(MP7F GBTU5FYU Ͱ୯ޠʹͭͷϕΫτϧ͕͋Δ͚ͩͰ ҙຯͷଟ༷ੑʹରॲͰ͖ͳ͍ #&35<>܇࿅ʹ͕͔͔࣌ؒΔ্ʹɺ#1&ʹΑΔ୯ޠׂ͕ લఏͳͷͰࠓճͷλεΫʹ߹Θͳ͍ /&PMPHEࣙॻʹΑΔׂͰ#&35ͭ͘ΕΔ͕ίετେ &-.PDIBS$//ʹΑͬͯ୯ޠͷϕΫτϧදݱΛͭ͘ΔͷͰ ޠኮαΠζ͕େ͖͘ͳͬͯରԠ͕Ͱ͖Δ
18 ©The Asahi Shimbun Company 2020 ɹ&-.PʹಡΈࠐ·ͤΔهࣄΛબͿ &-.PʹಡΈࠐ·ͤΔهࣄΛ֤Ωʔϫʔυ͝ͱʹͭબͿ ෳͷҙຯʹରॲ͢ΔͨΊɺࠓճશͯͷΩʔϫʔυ͕ ͭͷҙຯΛ࣋ͭͱԾఆ
֤୯ޠͰͭͷϕΫτϧΛͭ͘Δ ͳΔ͘ҙຯ͕ҟͳΔهࣄΛͭݟ͚͍ͭͨ ʮιϑτόϯΫʯͷεϙʔπهࣄΛͭΛ&-.PʹಡΈࠐ ·ͤͯࣅͨϕΫτϧ͕Ͱ͖ͯ͠·͏ 5'*%'ͱ48&.<>Λ݁߹ͯ͠จॻΛϕΫτϧԽ͠ɺͭͷ هࣄΛબͿ த৺ʹ࠷͍ۙͷΛީิهࣄ த৺͔Β࠷ԕ͍ͷΛީิهࣄ
19 ©The Asahi Shimbun Company 2020 ࣮ࡍʹͲͷΑ͏ͳهࣄ͕બΕ͔ͨ த৺ʹ͋Δͷ͕ٿஂͷʮιϑτόϯΫʯɺத৺͔Β࠷ԕ ͍ͱ͜Ζʹ͋Δͷ͕اۀͷʮιϑτόϯΫʯ εϓϦϯτΛΊ͙Δ࠶ฤڠٞʹͭ
͍ͯιϑτόϯΫͷଙਖ਼ٛձ݉ ݄ࣾɺถχϡʔϤʔΫͰه ऀஂʹʮண࣮ʹਐΜͰ͍Δʯͱ ޠͬͨɻ ٿஂɺલౡͷ܀ݪ݈ଠ ख ɺલϩοςͷຊྑ ฏัख ɺલιϑτόϯΫͷ ۚແӳख ͷબख͕ ͔ΒԬࢁݝෑࢢͰͷळقΩϟ ϯϓʹࢀՃ͠ɺೖஂςετΛड ͚Δͱൃදͨ͠ɻ ީิهࣄ ީิهࣄ̎
20 ©The Asahi Shimbun Company 2020 ۙݕࡧ༻ͷΠϯσοΫεΛ࡞ ֤ΩʔϫʔυΛͭͷهࣄΛͬͯ&-.PϕΫτϧΛऔಘ͠ɺ "OOPZΛͬͯΠϯσοΫεΛ࡞ 2SJDIͷΫΠζ࡞࣌ɺهࣄຊจͱ݀ൈ͖ରͷΩʔϫʔ
υΛಡΈࠐ·ͤͯ&-.PϕΫτϧΛऔಘ͠ɺͦͷϕΫτϧͷ ۙʹ͋Δ୯ޠΛݕࡧ ৽ޠ͕ొͨ͠߹ਓखͰΩʔϫʔυͱهࣄΛબΜͰ &-.PϕΫτϧΛऔಘ͠ɺ"OOPZͷΠϯσοΫεʹՃɾߋ ৽͢Δӡ༻ʹ͍ͯ͠Δ
21 ©The Asahi Shimbun Company 2020 &-.PͰۙͷ୯ޠͲ͏มΘ͔ͬͨʁ &-.Pͷۙ୯ޠΦϦοΫεɺڊਓɺϨουιοΫε &-.Pͷۙ୯ޠϥΠϒυΞɺιχʔɺΩϟϊϯ ถάʔάϧͷձࣾʮΞϧϑΝϕοτʯͷ࣌Ձ૯ֹ͕̍
̒ɺ̍ஹυϧʢ̍̍̌ஹԁʣΛಥഁͨ͠ɻถاۀͰ ɺΞοϓϧɺΞϚκϯɺϚΠΫϩιϑτʹଓ͖̐ࣾ ʹͳΔɻຊͷ࣌Ձ૯ֹτοϓͷτϤλࣗಈंʢ̎̑ஹ ԁʣͳͲɺຊͱͷ͕ࠩ·͢·֦͢େ͍ͯ͠Δɻ ͔Βʹ͔͚ͯɺੈք֤Ͱʮաڈ࠷ѱʯͱݺΕ ΔྛՐࡂ͕૬࣍͗·ͨ͠ɻٿԹஆԽ͕ҰҼͱࢦఠ͞Ε͍ͯ ·͕͢ɺݱͰऔࡐͯ͠ΈΔͱผͷଆ໘ݟ͖͑ͯ·ͨ͠ɻΦʔ ετϥϦΞɺϒϥδϧͷΞϚκϯɺถΧϦϑΥϧχΞͰԿ͕ى͖ ͍ͯΔͷ͔ɺಛһ͕ใࠂ͠·͢ɻ XPSEWFDͰۙ୯ޠͲͪΒʮωοτ௨ൢʯɺʮάʔάϧʯɺʮΩϯυϧʯ &-.Pͷۙ୯ޠϨϊϘɺμΠϜϥʔɺωοτϑϦοΫε &-.Pͷۙ୯ޠϝϯϑΟεɺϘεχΞɺΨϘϯ ιϑτόϯΫ̑ɺ࣍ੈͷߴҠಈ௨৴ํࣜʮ̜̑ʯͷαʔ ϏεΛࠓ݄̎̓ʹ࢝ΊΔͱൃදͨ͠ɻ͍·ͷ̜̐ͷྉۚʹɺ݄ ֹ̍ઍԁʢ੫ൈ͖ʣΛՃ͢Ε͑ΔΑ͏ʹ͢Δɻܞଳిେ ख̏ࣾͰ̜̑ͷ։࢝࣌ظͱྉۚΛൃදͨ͠ͷॳΊͯɻ ιϑτόϯΫͷ౦ڊख͕ࠓقͷ։ນखΛΊΔ͜ ͱ͕̒·Ͱʹܾ·ͬͨɻϓϩ̔Ͱॳͷେͱͳ Δɻ̎̌ͷ։ນઓຊڌͷԬϖΠϖΠυʔϜͰ ϩοςͱରઓ͢Δɻ XPSEWFDͰۙ୯ޠͲͪΒʮָఱʯɺʮ%F/"ʯɺʮΦϦοΫεʯ
·ͱΊ 22 2SJDIʹ͓͚ΔΫΠζ࡞ϑϩʔͷհ XPSEWFDͷۙͷΩʔϫʔυ͕ৗʹಉ͡ &-.PͰจ຺ʹԠͨ͡ྨࣅΩʔϫʔυγεςϜΛͭͬͨ͘ ޠኮαΠζ͕େ͖ͯ͘܇࿅͕ՄೳͳͷͰ&-.PΛ࠾༻ &-.PʹಡΈࠐ·ͤΔهࣄબ͕ॏཁ จ຺ʹԠͯۙ͡ͷ୯ޠมԽ͠ɺXPSEWFDͱൺֱ͠ ͯྨࣅΩʔϫʔυΛ͋Δఔ͏·͘औΕ͍ͯΔ ©The
Asahi Shimbun Company 2020
ࢀߟจݙ 23 [1] Peters+18, Deep contextualized word representations, NAACL2018 [2]
Devlin+19, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, NAACL2019 [3] Shen+18, Baseline Needs More Love: On Simple Word- Embedding-Based Models and Associated Pooling Mechanisms, ACL2018 ©The Asahi Shimbun Company 2020
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ ©The Asahi Shimbun Company 2020