Suggestions and techniques for building neural networks with Go

Goで単語をベクトル化するためのニューラルネットワーク「Word2Vec」 をスクラッチで構築した経験から、Goでニューラルネットワークを組む際のエッセンスについてお話しします。「Go+ニューラルネットワーク」と言う組み合わせはあまりイメージが無いかもしれませんが、仕組みと行列計算のクセを理解すれば難しくはありません。Goによる行列計算の現状と今後についてのお話はもちろん、Goでニューラルネットワークを組む際のアーキテクチャの考え方や、数値計算のテスト方法やモデルの保存方法などを紹介します。このトークで「Goでもデータサイエンスできそう！」と思ってもらうことが今回のトークのゴールです。

Transcript

1. Suggestions and techniques for building neural networks with Go Go Conference Tokyo 2019 Autumn / October 28, 2019 @ po3rin
2. pon (@po3rin) Software Enginner @ Shiroyagi Corporation Golang / Python / Rust / Docker / AWS / Elasticsearch
3. QBHF
    നϠΪίʔϙϨʔγϣϯ͸ɺػցֶशɾࣗવ ݴޠॲཧ෼໺ͷϓϩμΫτʹྗΛೖΕ͍ͯΔձ ࣾͰ͢ɻ։ൃʹ͸ओʹ
   (P
   ΍
   1ZUIPO
   ͕࠾༻ ͞Ε͍ͯ·͢ɻ നϠΪίʔϙϨʔγϣϯ
4. Package for Machine Lerning in Go Ὂ (P
   
   (POVN
   Ͱχϡʔϥϧ ωοτϫʔΫΛ૊ΉͨΊͷί ϯϙʔωϯτूʂ ./*45σʔληοτΛ(Pͷ ߦྻ࣮૷ͰಡΈࠐΉͨΊͷ ύοέʔδ
5. 1: Why do data-science with Go?
6. Why Go ? Ὂ (PͷڧΈ͕ͦͷ··σʔλαΠΤϯεΛ(PͰ΍Δཧ༝
   
   
   ίʔυ͕ಡΈ΍͘͢ϝϯς͠΍͍͢
   
   ύϑΥʔϚϯε͕ྑ͍
   
   ฒߦԽ͠΍͍͢
   ҰํͰ΍͸Γ·ͩ·ͩࢿ࢈తʹ1ZUIPOʹ܉ഋ্͕͕Δͷ͸͔֬ɻɻ
7. Marchine Larning Packages Ὂ ͔͠͠ɺϞσϧΛಈ͔͢؀ڥ͸͔ͳΓॆ࣮͖͍ͯͯ͠ΔͷͰɺਪ࿦ΛϚΠΫϩαʔϏ εͱཱͯͯ͠Ε͹
   (P
   
   ϚΠΫϩαʔϏεͷخ͠Έ͕ಘΒΕΔɻ ֶशࡁΈϞσϧΛ
   5FOTPS'MPX
   ͕ ఏڙ͢Δ
   $
   "1*
   Λ࢖༻ͯ͠ಈ͔͢ɻ ΋ͪΖΜ(P΋αϙʔτ͍ͯ͠Δɻ 1VSF
   (P
   Ͱॻ͔Ε͍ͯΔɻ0//9 ϑΥʔϚοτͰॻ͔Εͨχϡʔϥϧ ωοτΛΠϯϙʔτͯ͠ಈ͔ͤΔ
8. Why Go for Biginners ? Ὂ ॳ৺ऀͰɺجૅతͳͱ͜Ζ͔Βษڧ͔ͨͬͨ͠Β(P͕ ྑ͍ͱײͨ͡ɻ1ZUIPO͸ศརͳύοέʔδ͕ଟ͍ͷ Ͱɺ(PͰ࣮૷͍ͯ͘͠ͱࠜఈͷཧղ͕ͳ͍ͱ࣮૷Ͱ͖ ͳ͔ͬͨΓ͢Δɻ
9. Why Go ? Ὂ ྫ͑͹ɺ֬཰෼෍ΛݩʹཁૉΛϥϯμϜʹબ୒͢Δ࣮૷͸1ZUIPOͰ͸ߦͩ ͕ɺ(PͰ͸ࣗ෼ͰΞϧΰϦζϜΛษڧ࣮ͯ͠૷͢Δඞཁ͕͋ͬͨɻ͜͏͍ͬͨ ࡉ͔͍ॴͷ࢓૊Έ΋ษڧʹͳΔɻ https://qiita.com/po3rin/items/f28a8ce322cb1bd95175 ʮ༩͑ΒΕͨॏΈʹैͬͯϥϯμϜʹ஋Λฦ͢ʮ8FJHIUFE
   3BOEPN
   4FMFDUJPOʯΛ(PͰ࣮૷͢Δʂʯ
10. 2: Basic of matrix calculation with Go
11. Matrix operation package choices Ὂ gonum.org/v1/gonum/mat (PͰߦྻΛѻ͏ͨΊͷ൚༻తͳ
    ύοέʔδɻ׆ൃʹϝϯς͞Εͯ Δɻૄߦྻ΍ࡾ֯ߦྻͳͲʹ΋ରԠ gorgonia.org/tensor (Pͷػցֶश༻ϥΠϒϥϦ ʮHPSHPOJBʯͰ࢖͏͜ͱΛ໨తͱ ͨ͠ߦྻɻଟ࣍ݩʹ΋ରԠ
12. Matrix operation (1) Ὂ (Pʹ͸1ZUIPOʹ͋ΔΑ͏ͳԋࢉࢠΦʔόʔϩʔυ͕ͳ͍ͷͰؔ਺ͰԋࢉΛߦ͏ɻ
13. Matrix operation (2) Ὂ ߦྻΛදݱ͢Δߏ଄ମͰ͋Δ NBU%FOTF
    ͸಺෦Ͱશཁૉͷ εϥΠε
    <>qPBU
    ͱߦ਺ྻ਺ ͳͲͷ৘ใΛอ͍࣋ͯ͠Δ͚ͩɻ
14. Matrix operation (3) Ὂ NBU%FOTF
    ͸
    .BUSJY
    ΠϯλʔϑΣʔεΛ࣮૷͍ͯ͠Δɻ͜ΕʹΑΓσʔλߏ଄ͷҧ͍ ΛҙࣝͤͣʹߦྻΛར༻Ͱ͖Δɻଞʹ΋
    7FD%FOTFʢϕΫτϧΛදݱʣ΍
    5SJ%FOTF ʢࡾ֯ߦྻΛදݱʣͳͲ
    .BUSJY
    Λ࣮૷͢Δ༷ʑͳߏ଄ମ͕͋Δɻ
15. Matrix operation (4) Ὂ ࣍ݩͷߦྻ͸
    HPOVNNBU
    Ͱ͸ߦྻͷεϥΠεͰ͔͠දݱͰ͖ͳ͍ɻଟ࣍ݩߦྻ ʹରԠ͢ΔύοέʔδΛ࡞Δ৔߹͸
    UFOTPSUFOTPS
    ʹ܉ഋ্͕͕Δɻ
16. ࠷ॳʹݟͨͱ͓Γɺ.BUSJY
    ΠϯλʔϑΣʔεͷ࣮ଶͱͯ͠
    %FOTF
    ͕͋ΔɻNBU
    ϑΟʔϧυͷܕͰ͋Δ
    CMBT(FOFSBM
    ͱ ͸ͳΜͰ͠ΐ͏͔ɻ Use BLAS (1) Ὂ
17. Use BLAS (2) Ὂ #-"4ͱ͸ɺ#BTJD
    -JOFBS
    "MHFCSB
    4VCQSPHSBNTͷུͰɺجຊతͳઢ ܗԋࢉͷϥΠϒϥϦɻ#-"4ʹ͸͞·͟·ͳछྨ͕͋Γ
    0QFO#-"4
    ΍
    "5-"4
    ͳͲ͕ଘࡏ͢Δɻ
18. Use BLAS (3) Ὂ (POVNͰ͸
    6TF
    ͱ͍͏ؔ਺Ͱ࢖͏#-"4Λ͙͢ʹ੾Γସ͑ΕΔɻ
19. Use BLAS (4) Ὂ ͨͱ͑͹
    %PU
    ཁૉ͝ͱͷੵͷ߹ܭΛܭࢉ͢Δ
    Ͱ͸಺෦Ͱ͸
    CMBT%EPU
    Λݺͼग़͍ͯ͠Δɻ౰વCMBT%EPU
    ͸ΠϯλʔϑΣʔεͰఆٛ͞Εͨϝιο υͳͷͰCMBTͷ࣮ମ͝ͱʹҧ͏ॲཧΛݺ΂Δɻ
20. Use BLAS (5) Ὂ ͨͱ͑͹ཁૉͷϕΫτϧಉ࢜Λ
    %PU
    ؔ਺ʹ͔͚Δ࣌ͷϕϯνϚʔΫͰ ͸Լهͷ؀ڥͰ͸
    
    ഒ΄Ͳ0QFO#-"4ͷํ͕ૣ͘ͳ͍ͬͯ·͢ɻ
21. 2JJUBʹ
    (P
    
    (POVN
    ʹΑΔߦྻܭࢉΛ ମܥతʹ·ͱ·͍ͬͯΔɻߦྻܭࢉͷΤ ϥʔϋϯυϦϯά΍ςετͳͲ΋Χόʔ͞ Ε͍ͯ·͢ɻ
    https://qiita.com/po3rin/items/82c9c0195f9e3e38e2f1 Study Gonum Ὂ
22. IUUQTXXXDIFBUPHSBQIZDPNDIFXYZDIFBUTIFFUTEBUBTDJFODFJOHPB Study matrix operation using Go Ὂ (PSHPOJB࡞ऀʹΑΔ ʮOVNQZʯʮHPOVNNBUʯ ʮHPSHPOJBUFOTPSʯʹ͓͚Δ ߦྻԋࢉͷνʔτγʔτ͕ૉ੖ Β͍͠ɻOVNQZͷ͜Εͬͯ(P ͩͱͲ͏΍ΔΜͩʁ͕΄΅ࡌͬ ͯΔɻ
23. 3: Architecture for building neural networks with Go
24. Example of Machine Lerning in Go Ὂ (P
    
    (POVN
    ͰχϡʔϥϧωοτϫʔΫΛ૊ΉͨΊͷίϯϙʔωϯτूͰ͋ ΔHPOOQͷ࣮૷ܦݧ͔Β࣮૷ΞʔΩςΫνϟΛߟ͑Δɻ
25. Architecture (2) Ὂ ܇࿅༻ͷΦϒδΣΫτΛ
    .PEFM
    ͱ
    0QUJNJ[FS
    Λ౉ͯ͠ॳظԽ ͠·͢ɻॳظԽؔ਺͸ύϥϝʔλ͕ඇৗʹଟ͘ͳΔͷͰ
    'VODUJPOBM
    0QUJPO
    1BUUFSOT
    Ͱ࣮૷͓ͯ͘͠ͱྑ͍ɻ 5SBJOFS .PEFM 0QUJNJ[FS
26. Architecture (3) Ὂ 5SBJOFS 0QUJNJ[FS "⒏OF
    -BZFS 3FMV
    -BZFS "⒏OF
    -BZFS "⒏OF
    -BZFS ϨΠϠʔͷ࣮૷͸*OUFSGBDFͱͯ͠ఆ͓ٛͯ͠ ͖ɺύοέʔδΛར༻͢ΔϢʔβʔ͕ಠࣗͷϨ ΠϠʔΛ࣮૷ͯ͠૊ΈࠐΊΔΑ͏ʹ͓ͯ͘͠
27. Architecture (4) Ὂ 8FJHIU
    ΍
    #JBT
    ͳͲͷύϥϝʔλΛߋ৽͢ Δ0QUJNJ[FSɻ΋ͪΖΜ͜͜΋Ϣʔβʔ͕೚ ҙͷΞϧΰϦζϜΛར༻Ͱ͖ΔΑ͏ʹ *OUFSGBDF
    Ͱఆٛ͢Δɻ 5SBJOFS .PEFM (SBEJFOU
    %FTDFOU
28. Architecture (7) Ὂ .BUSJYΠϯλʔϑΣʔεʹ͸ "EE΍4VCͳͲͷجຊతͳԋࢉ ͕࣮૷͞Ε͍ͯͳ͍ͨΊ
    (POVN
    Ͱ
    χϡʔϥϧωοτ ϫʔΫΛߏங͍ͯ͘͠ͱҾ਺΍ ฦΓ஋Λɺߏ଄ମͰѻ͏͔Πϯ λʔϑΣʔεͰѻ͏͔೰ΈͲ͜ɻ
29. Architecture (9) Ὂ (POVN
    ͚ͩͰ͸े෼ͳߦྻԋࢉ ͕ఏڙ͞Ε͍ͯͳ͍ͷͰɺ HPOOQ
    Ͱ͸
    NBUVUJM
    ύοέʔδ Λ࣮૷͍ͯ͠Δɻͦͷ৔߹͸
    HPOVNNBU
    ͷΑ͏ʹΠϯλʔ ϑΣʔεͰड͚ɺߏ଄ମͰฦ͢
    
    ಺෦ͰܕεΠον͓ͯ͘͠ͱ࢖ ͍΍͍͢ɻ
30. Architecture (8) Ὂ ϨΠϠʔʹؔͯ͠͸༷ʑͳϨΠ Ϡʔͷ࣮૷Λίϯϙʔωϯτͱ͠ ͯ࢖͍ճͤΔΑ͏ʹ*OUFSGBDFͰ ड͚ͯ*OUFSGBDFͰฦ࣮͢૷ʹ͠ ͍ͯΔɻ ϨΠϠʔ͸ߦྻΛड͚ Δ͔ϕΫτϧΛड͚Δ͔෼͔Βͳ ͍
31. Architecture (9) Ὂ 5FOTPS'MPX
    ͷ࣮૷ͷΑ͏ʹσʔλϑϩʔάϥϑͬΆ͘ઃܭ͢Δͱ͍͏ߟ͑΋͋Δɻ
32. Architecture (9) Ὂ ࣮ࡍʹ
    HPSHPOJBHPSHPOJB
    Ͱ͸
    νϟωϧʹΑΔσʔλϑ ϩʔͱͯ͠άϥϑΛߏங͠ɺ࣮ ߦ͢Δ࣮૷Λ͍࣋ͬͯΔɻ
    
33. Architecture (10) Ὂ ͜͏͢Δͱฒߦॲཧͳάϥϑॲ ཧΛ࣮૷Ͱ͖Δɻ HPOOQͰ͸ ·ͩશͯγʔέϯεͳͷͰฒߦ ॲཧΛαϙʔτ͍͖͍ͯͨ͠ ࢀߟ
    ʮ5IJOL
    MJLF
    B
    WFSUFY
    VTJOH
    (PT
    DPODVSSFODZ
    GPS
    HSBQI
    DPNQVUBUJPOʯ
    https://blog.owulveryck.info/2019/10/14/think-like-a-vertex-using-gos-concurrency-for-graph- computation.html
34. 4: Store trained models
35. Store trained models (1) Ὂ (PͰֶशͨ͠Ϟσϧͷอଘ΍෼ࢄදݱͷอଘํ๏Λߟ͑Δɻ(PͰσʔλߏ଄ΛγϦΞϥ Πζ͢ΔͨΊʹ͍͔ͭ͘ͷύοέʔδ͕༻ҙ͞Ε͍ͯΔɻ
    ɾFODPSEJOHDTW
    ɾFODPSEJOHKTPO
    ɾFODPSEJOHYNM
36. Store trained models (2) Ὂ (Pͷ؀ڥ಺Ͱ׬݁͢ΔͷͰ͋Ε͹ɺFODPSEJOHHPC
    ͕͓͢͢Ίɻσʔλߏ଄͑͞ ෼͔Ε͹ͦͷ··γϦΞϥΠζͰ͖Δɻ·ͨόΠφϦΤϯίʔσΟϯάͳͷͰޮ཰ੑ ΋ߴ͍ɻ ࢀߟʮ(PCT
    PG
    EBUBʯ
    5IF
    (P
    #MPH
    https://blog.golang.org/gobs-of-data
37. Store trained models (3) Ὂ ྫ͑͹
    8PSE7FD
    Ͱ֫ಘ͠ ͨ୯ޠͷ෼ࢄදݱΛ(PͷΞϓ ϦέʔγϣϯͰར༻͍ͨ͠৔ ߹͸ӈͷΑ͏ͳγϯϓϧͳ ίʔυ͚ͩͰ࠶ར༻Մೳͳܗ ͰอଘͰ͖Δɻ
38. Store trained models (4) Ὂ ಠࣗͷΠϯλʔϑΣʔεܕΛHPCͰอଘ͍ͨ͠৔߹͸HPC3FHJTUFSΛ࢖࣮ͬͯଶͱ ͳΔσʔλߏ଄Λڭ͑Δඞཁ͕͋Δɻ࣮ࡍʹHPOOQʹ͓͍ͯ͸NBU.BUSJYΛJOJUؔ਺ ಺Ͱొ࿥͍ͯ͠Δ
39. Store trained models (5) Ὂ ҰํͰόΠφϦͰσʔλΛอଘ͢ΔϑΥʔϚο τͰ͋Δ
    .FTTBHF1BDL
    ͷ(P࣮૷ͳͲ΋બ୒ ࢶʹೖΓ·͢ɻฐࣾͷ(PͰֶशͨ͠ϞσϧͰ͸ ͪ͜ΒΛ࢖ͬͯอଘ͍ͯ͠Δɻ
40. 4: Use existing machine learning packages
41. Marchine Larning Packages Ὂ ಠ࣮ࣗ૷ͤͣͱ΋(PͰػցֶशΛ͢Δ؀ڥ͕੔͍ͭͭ͋Δɻ ίϯηϓτͱͯ͠͸
    5FOTPS'MPX
    ʹ͍ۙɺάϥϑΛߏங࣮ͯ͠ߦ͍ͯ͘͠ɺ಺෦ͷߦ ྻԋࢉͰ͸
    HPSHPOJBUFOTPS
    ͕࢖ΘΕ͍ͯΔɻHPOVNNBU
    ͱͷޓ׵ੑ΋͋Δɻ
42. Marchine Larning Packages Ὂ (PͷطଘύοέʔδͰطଘϞσϧΛ࢖͏؀ڥ΋੔͍ͭͭ͋Δɻ ֶशࡁΈϞσϧΛ
    5FOTPS'MPX
    ͕ఏڙ ͢Δ
    $
    "1*
    Λ࢖༻ͯ͠ಈ͔͢ɻ΋ͪΖΜ (P΋αϙʔτ͍ͯ͠Δɻ
43. Marchine Larning Packages Ὂ ֶशࡁΈϞσϧ
    1SPUPDPM
    #V⒎FST 5FOTPS'MPX
    
    1ZUIPO
    Ͱֶश 5FOTPS'MPX
    
    (P
    Ͱਪ࿦ 1SPUPDPM
    #V⒎FST
    ܗࣜͷֶशࡁΈϞσϧͰ ͋Ε͹
    5FOTPS'MPX
    
    (PͰಈ͔ͤΔɻ
44. data-science Channel in Gophers Slack Ὂ (PͰσʔλαΠΤϯεΛ΍Δ ਓ͕ؒू·Δ4MBDLνϟωϧ͕ ͋Δɻ(PपΓͷσʔλαΠΤ ϯε৘ใ͸͜͜ʹूத͢Δͷ ͰΦεεϝ
45. 6: Conclusion
46. Conclusion Ὂ ͜͜·Ͱ(PͰͷػցֶशͷ࣮૷Λߦྻͷ࢓૊Έ͔Β঺հ͍͖ͯ͠·ͨ͠ɻ
    ࣮͸·ͩ·ͩ(PͰͷ਺஋ܭࢉ͸ൃల్্Ͱ͢ɻ΋͜͠ͷൃදͰগ͠Ͱ΋ڵຯΛ ࣋ͬͨํ͸ੋඇҰॹʹ(PʹΑΔσʔλαΠΤϯεΛ΍͍͖ͬͯ·͠ΐ͏ɻ
47. Suggestions and techniques for building neural networks with Go Go Conference Tokyo 2019 Autumn / October 28, 2019 @ po3rin