Natural Language APIとその裏側を覗く〜完全版〜

5分に収まるように削らなかったバージョンの資料です。
削った方は08/28 potatotips #43で使用します。

Transcript

1. Natural Language APIͱ ͦͷཪଆΛ೷͘ ʙ׬શ൛ʙ @akatsuki174
2. ࣗݾ঺հ • ਢ౻ຘ • iOSΤϯδχΞ • גࣜձࣾZaimॴଐ • Twitter, Qiita, SlideShare, Github, SpeakerDeck →akatsuki174
4. એ఻̍ 10/13(ۚ)ɺ10/20(ۚ)ͷ2days 19:20ʙ
5. એ఻̎ 9/1(ۚ) 19:30ʙ
6. ࢲͱࣗવݴޠॲཧ ɾେֶͷଔۀݚڀͰΏΔΏΔ1೥ؒ ʢॏཁޠ൑ఆʣ ɾ࠷ॳͷձࣾͰ2,3ϲ݄ ʢτϐοΫ൑ఆʣ ɾ2ࣾ໨ͰΏΔΏΔ൒೥ ʢSolrͰݕࡧνϡʔχϯάʣ
7. ࠓճͷࢿྉ͸ɺ͜ͷొஃࢿྉΛ࡞ΔաఔͰ୤ઢͯ͠ ௐ΂ͨ͜ͱΛ·ͱΊͨࢿྉʹͳΓ·͢ɻ 9/17(೔) Track C 14:40ʙΑΖ͓͘͠ئ͍͠·͢ʂ
8. ɾNatural Language Processing and your ɹApps[WWDC 2017]ͷͬ͘͟Γͱͨ͠಺༰ ɾཪଆͰ࢖ΘΕ͍ͯΔٕज़ʹ͍ͭͯ ࠓ೔࿩͢͜ͱ
9. ɾϑΝʔετύʔςΟΞϓϦͰͲͷΑ͏ͳ ɹ׆༻͕͞ΕΔ͔ ɾNLP APIͱ͸ ɾ͋ͳͨͷΞϓϦͰͲͷΑ͏ʹ࢖͏ͱ ɹྑ͍͔ʢͨͿΜɻ·ͩશ෦؍ͯͳ͍ʣ Natural Language Processing and your Apps
10. ɾϑΝʔετύʔςΟΞϓϦͰͲͷΑ͏ͳ ɹ׆༻͕͞ΕΔ͔ ɾNLP APIͱ͸ ɾ͋ͳͨͷΞϓϦͰͲͷΑ͏ʹ࢖͏ͱ ɹྑ͍͔ʢͨͿΜɻ·ͩશ෦؍ͯͳ͍ʣ Natural Language Processing and your Apps
11. Natural Language Processing and your Apps https://developer.apple.com/videos/play/wwdc2017/208/ Go to iMessage app
12. Natural Language Processing and your Apps https://developer.apple.com/videos/play/wwdc2017/208/ Go to News app
13. Natural Language Processing and your Apps https://developer.apple.com/videos/play/wwdc2017/208/ Back to iMessage app
14. ݻ༗දݱநग़ͷٕज़Λ࢖͍ͬͯΔ ͷͰɺͦ͜Λ۷ΓԼ͛ͯΈ·͢ ʢҎԼɺiOS͸શؔ͘܎ͳ͍࿩ʹͳΓ·͢ʣ ͓ͦΒ͘ɺ
15. ݻ༗໊ࢺʢਓ໊ɺ஍໊౳ʣɺ࣌ؒදݱ ʢ೔෇ɺ࣌ࠁʣͳͲΛςΩετ͔Βநग़͢Δٕज़ ͜ͷٕज़͕ొ৔ͨ͠എܠ ɹɾࣙॻʹ୯ޠ͕ొ࿥͞Ε͍ͯͳ͍ͱղੳ ɹɹޡΓΛҾ͖ى͜͢ͷͰࣗಈͰநग़͍ͨ͠ ɹɾจষʹ͓͍ͯॏཁͳ৘ใ͸্هͷ ɹɹΑ͏ͳݻ༗දݱͰ͋Δ ݻ༗දݱநग़ͱ͸
16. ଠ࿠͸5݄18೔ͷே9࣌ʹՖࢠʹձ͍ʹߦͬͨɻ ↓ɹநग़ʂ ଠ࿠ʢpersonʣ͸5݄18೔ʢdateʣͷே9࣌ ʢtimeʣʹՖࢠʢpersonʣʹձ͍ʹߦͬͨɻ ݻ༗දݱநग़ͷྫ
17. ɾ࣭໰Ԡ౴γεςϜɺର࿩γεςϜɹ ɹʢWatsonɺνϟοτϘοτ౳ʣ ɾ৘ใநग़ ݻ༗දݱநग़ͷ׆༻υίϩ
18. ɾ࣭໰Ԡ౴γεςϜɺର࿩γεςϜɹ ɹʢWatsonɺνϟοτϘοτ౳ʣ ɾ৘ใநग़ ݻ༗දݱநग़ͷ׆༻υίϩ ࣗવݴޠ͔Βݻ༗දݱΛநग़͠ɺͦͷݻ༗දݱΛ ΫΤϦͱͯ͠౤͛ͯจॻݕࡧΛߦ͏ɻಘΒΕͨճ౴ ͱݻ༗දݱΛ΋ͱʹճ౴ީิΛ࡞੒͠ɺͦͷதͰ είΞ͕ߴ͍΋ͷΛճ౴ͱ͢Δɻ
19. ࣭໰Ԡ౴γεςϜɺର࿩γεςϜ ौ୩Ͱ໌೔19:30͔Β༧໿ Ͱ͖ΔళΛ୳ͯ͠ʂ q=ौ୩, ໌೔19:30 ճ౴ީิ࡞੒ ※࣮ࡍʹ͸ଞͷޠ΋౤͍͛ͯΔ͸ͣ ͳΔ΄Ͳ͒ʂ είΞܭࢉ
20. ɾ࣭໰Ԡ౴γεςϜɺର࿩γεςϜɹ ɹʢWatsonɺνϟοτϘοτ౳ʣ ɾ৘ใநग़ ݻ༗දݱநग़ͷ׆༻υίϩ ༧ΊܾΊͨʮ஌Γ͍ͨ͜ͱ߲໨ʯΛຒΊΔܗͰ৘ใ Λநग़͢Δ͜ͱɻߏ଄Խ͞Ε͍ͯͳ͍ςΩετ৘ใ Λߏ଄Խ͞Εͨσʔλϕʔεܗࣜʹม׵͢Δ͜ͱͱ ΋ଊ͑Δ͜ͱ͕Ͱ͖Δɻ
21. ݻ༗දݱநग़ͷ׆༻υίϩ ࡢ೔͋ͬͨ஍਒ͷ࠷େ਒౓ɺ Ϛάχνϡʔυɺ਒ݯ஍ʹ ͍ͭͯ஌Γ͍ͨ ਒౓ ʔʔʔ Ϛάχνϡʔυ Mʔ ਒ݯ஍ ʙݝʙࢢ ※ͪΐͬͱΠϝʔδਤҧ͏͔΋ ςΩετ͔Βݻ༗දݱΛநग़
22. ɾ஌ࣝϕʔεख๏ ɾ౷ܭతख๏ ݻ༗දݱநग़Λ͢Δʹ͸
23. ɾنଇʢύλʔϯʣΛਓखͰ࡞੒͠ɺ ɹͦΕʹ౰ͯ͸·Δ΋ͷΛநग़͢Δ ɹྫʣ໊ࢺʴʮ͞Μʯˠਓ໊ ɹɹɹ໊ࢺʴʮࢯʯˠ஍໊ ɾنଇΛ໢ཏతʹߏங͠Α͏ͱ͢Δͱ ɹലେͳίετ͕͔͔Δ ஌ࣝϕʔεख๏
24. ɾڭࢣ͋ΓֶशΛͤ͞ɺநग़نଇΛࣗಈ ɹֶश͢ΔΑ͏ʹ͢Δ ɾSVMʢSupport Vector Machine / ࣝผ ɹؔ਺ʣ΍CRFʢConditional Random ɹFields / ࣝผϞσϧʣ͕༗໊ ౷ܭతख๏
25. ͍Ζ͍Ζ೉͍͠࿩͕ग़͖ͯͨ͠ɺ೔ຊޠͷ ղੳ͸ͱͯ΋ෳࡶ ͳͷͰݻ༗දݱநग़Λָʹ͢ΔπʔϧΛ ঺հͯ͠Έ·͢ɻ
26. ɾMeCab ͱڞʹ࢖͏୯ޠ෼͔ͪॻ͖ࣙॻ ɾ࠷௿݄2ճΞοϓσʔτ ɹɾ৽ޠɺݻ༗දݱʹڧ͍ ɾApache License 2.0 ɾࠃ࢈ mecab-ipadic-NEologd
27. ɾಈ࡞ʹඞཁͳϥΠϒϥϦͷΠϯετʔϧ $ brew install mecab mecab-ipadic git curl xz ɾgit clone $ git clone --depth 1 https://github.com/ neologd/mecab-ipadic-neologd.git ɾcloneͨ͠σΟϨΫτϦͰΠϯετʔϧ $ ./bin/install-mecab-ipadic-neologd -n Πϯετʔϧ
28. ɾશ෦ೖΓͳmecab-ipadic-NEologdΛ ɹΠϯετʔϧ $ ./bin/install-mecab-ipadic-neologd -n -a ɾΠϯετʔϧઌΛ֬ೝ $ echo `mecab-config --dicdir`"/mecab- ipadic-neologd" Πϯετʔϧ
29. ɾී௨ͷIPAࣙॻΛ࢖͏৔߹ $ mecab ɾNEologdΛ࢖͏৔߹ $ mecab -d [ΠϯετʔϧઌΛࢦఆ] ࢖ͬͯΈΔ
30. ɾී௨ͷIPAࣙॻ ྫจʣϝϩϯύϯΛ৯΂·ͨ͠ ϝϩϯ ໊ࢺ,Ұൠ,*,*,*,*,ϝϩϯ,ϝϩϯ,ϝϩϯ ύϯ ໊ࢺ,Ұൠ,*,*,*,*,ύϯ,ύϯ,ύϯ Λ ॿࢺ,֨ॿࢺ,Ұൠ,*,*,*,Λ,Ϯ,Ϯ ৯΂ ಈࢺ,ཱࣗ,*,*,Ұஈ,࿈༻ܗ,৯΂Δ,λϕ,λϕ ·͠ ॿಈࢺ,*,*,*,ಛघɾϚε,࿈༻ܗ,·͢,Ϛγ,Ϛγ ͨ ॿಈࢺ,*,*,*,ಛघɾλ,جຊܗ,ͨ,λ,λ ൺֱͯ͠ΈΔ̍
31. ɾNEologd ྫจʣϝϩϯύϯΛ৯΂·ͨ͠ ϝϩϯύϯ ໊ࢺ,ݻ༗໊ࢺ,Ұൠ,*,*,*,ϝϩϯύϯ,ϝ ϩϯύϯ,ϝϩϯύϯ Λ ॿࢺ,֨ॿࢺ,Ұൠ,*,*,*,Λ,Ϯ,Ϯ ৯΂ ಈࢺ,ཱࣗ,*,*,Ұஈ,࿈༻ܗ,৯΂Δ,λϕ,λϕ ·͠ ॿಈࢺ,*,*,*,ಛघɾϚε,࿈༻ܗ,·͢,Ϛγ,Ϛγ ͨ ॿಈࢺ,*,*,*,ಛघɾλ,جຊܗ,ͨ,λ,λ ൺֱͯ͠ΈΔ̍
32. ɾී௨ͷIPAࣙॻ ྫจʣ3݄ͷϥΠΦϯΛಡΜͩ 3 ໊ࢺ,਺,*,*,*,*,* ݄ ໊ࢺ,Ұൠ,*,*,*,*,݄,πΩ,πΩ ͷ ॿࢺ,࿈ମԽ,*,*,*,*,ͷ,ϊ,ϊ ϥΠΦϯ ໊ࢺ,Ұൠ,*,*,*,*,ϥΠΦϯ,ϥΠΦϯ,ϥΠΦϯ Λ ॿࢺ,֨ॿࢺ,Ұൠ,*,*,*,Λ,Ϯ,Ϯ ಡΜ ಈࢺ,ཱࣗ,*,*,ޒஈɾϚߦ,࿈༻λ઀ଓ,ಡΉ,Ϥϯ,Ϥϯ ͩ ॿಈࢺ,*,*,*,ಛघɾλ,جຊܗ,ͩ,μ,μ ൺֱͯ͠ΈΔ̎
33. ɾNEologd ྫจʣ3݄ͷϥΠΦϯΛಡΜͩ 3݄ͷϥΠΦϯ ໊ࢺ,ݻ༗໊ࢺ,Ұൠ,*,*,*,3݄ͷϥΠ Φϯ,αϯΨπϊϥΠΦϯ,αϯΨπϊϥΠΦϯ Λ ॿࢺ,֨ॿࢺ,Ұൠ,*,*,*,Λ,Ϯ,Ϯ ಡΜ ಈࢺ,ཱࣗ,*,*,ޒஈɾϚߦ,࿈༻λ઀ଓ,ಡΉ,Ϥ ϯ,Ϥϯ ͩ ॿಈࢺ,*,*,*,ಛघɾλ,جຊܗ,ͩ,μ,μ ൺֱͯ͠ΈΔ̎ ΞχϝλΠτϧ ͳͲ΋OK
34. ɾී௨ͷIPAࣙॻ ྫจʣApple WatchΛങͬͨɻ Apple ໊ࢺ,ݻ༗໊ࢺ,૊৫,*,*,*,* Watch ໊ࢺ,Ұൠ,*,*,*,*,* Λ ॿࢺ,֨ॿࢺ,Ұൠ,*,*,*,Λ,Ϯ,Ϯ ങͬ ಈࢺ,ཱࣗ,*,*,ޒஈɾϫߦଅԻศ,࿈༻λ઀ଓ,ങ͏, Χο,Χο ͨ ॿಈࢺ,*,*,*,ಛघɾλ,جຊܗ,ͨ,λ,λ ɻ ه߸,۟఺,*,*,*,*,ɻ,ɻ,ɻ ൺֱͯ͠ΈΔ̏
35. ɾNEologd ྫจʣApple WatchΛങͬͨɻ Apple Watch ໊ࢺ,ݻ༗໊ࢺ,Ұൠ,*,*,*,Apple Watch,Ξο ϓϧ΢Υον,Ξοϓϧ΢Υον Λ ॿࢺ,֨ॿࢺ,Ұൠ,*,*,*,Λ,Ϯ,Ϯ ങͬ ಈࢺ,ཱࣗ,*,*,ޒஈɾϫߦଅԻศ,࿈༻λ઀ଓ,ങ͏, Χο,Χο ͨ ॿಈࢺ,*,*,*,ಛघɾλ,جຊܗ,ͨ,λ,λ ɻ ه߸,۟఺,*,*,*,*,ɻ,ɻ,ɻ ൺֱͯ͠ΈΔ̏ ۭനεϖʔε͕ ͋ͬͯ΋OK
36. ͍͢͝ ʢখฒײʣ
37. Կ͕ݴ͍͍͔ͨΘ͔Βͳ͘ ͳ͖ͬͯͨͷͰऴΘΓʹ͠·͢
38. ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ