Suggestions and techniques for building neural networks with Go

4d78b749c315e21a956053dcbf0508a9?s=47 po3rin
October 28, 2019

Suggestions and techniques for building neural networks with Go

Goで単語をベクトル化するためのニューラルネットワーク「Word2Vec」 をスクラッチで構築した経験から、Goでニューラルネットワークを組む際のエッセンスについてお話しします。「Go+ニューラルネットワーク」と言う組み合わせはあまりイメージが無いかもしれませんが、仕組みと行列計算のクセを理解すれば難しくはありません。Goによる行列計算の現状と今後についてのお話はもちろん、Goでニューラルネットワークを組む際のアーキテクチャの考え方や、数値計算のテスト方法やモデルの保存方法などを紹介します。このトークで「Goでもデータサイエンスできそう!」と思ってもらうことが今回のトークのゴールです。

4d78b749c315e21a956053dcbf0508a9?s=128

po3rin

October 28, 2019
Tweet

Transcript

  1. Suggestions and techniques for building neural networks with Go Go

    Conference Tokyo 2019 Autumn / October 28, 2019 @ po3rin
  2. pon (@po3rin) Software Enginner @ Shiroyagi Corporation Golang / Python

    / Rust / Docker / AWS / Elasticsearch
  3. QBHF നϠΪίʔϙϨʔγϣϯ͸ɺػցֶशɾࣗવ ݴޠॲཧ෼໺ͷϓϩμΫτʹྗΛೖΕ͍ͯΔձ ࣾͰ͢ɻ։ൃʹ͸ओʹ (P ΍ 1ZUIPO ͕࠾༻ ͞Ε͍ͯ·͢ɻ നϠΪίʔϙϨʔγϣϯ

  4. Package for Machine Lerning in Go Ὂ (P (POVNͰχϡʔϥϧ ωοτϫʔΫΛ૊ΉͨΊͷί

    ϯϙʔωϯτूʂ ./*45σʔληοτΛ(Pͷ ߦྻ࣮૷ͰಡΈࠐΉͨΊͷ ύοέʔδ
  5. 1: Why do data-science with Go?

  6. Why Go ? Ὂ (PͷڧΈ͕ͦͷ··σʔλαΠΤϯεΛ(PͰ΍Δཧ༝ ίʔυ͕ಡΈ΍͘͢ϝϯς͠΍͍͢ ύϑΥʔϚϯε͕ྑ͍ ฒߦԽ͠΍͍͢ ҰํͰ΍͸Γ·ͩ·ͩࢿ࢈తʹ1ZUIPOʹ܉ഋ্͕͕Δͷ͸͔֬ɻɻ

  7. Marchine Larning Packages Ὂ ͔͠͠ɺϞσϧΛಈ͔͢؀ڥ͸͔ͳΓॆ࣮͖͍ͯͯ͠ΔͷͰɺਪ࿦ΛϚΠΫϩαʔϏ εͱཱͯͯ͠Ε͹(P ϚΠΫϩαʔϏεͷخ͠Έ͕ಘΒΕΔɻ ֶशࡁΈϞσϧΛ5FOTPS'MPX͕ ఏڙ͢Δ$"1*Λ࢖༻ͯ͠ಈ͔͢ɻ ΋ͪΖΜ(P΋αϙʔτ͍ͯ͠Δɻ

    1VSF(PͰॻ͔Ε͍ͯΔɻ0//9 ϑΥʔϚοτͰॻ͔Εͨχϡʔϥϧ ωοτΛΠϯϙʔτͯ͠ಈ͔ͤΔ
  8. Why Go for Biginners ? Ὂ ॳ৺ऀͰɺجૅతͳͱ͜Ζ͔Βษڧ͔ͨͬͨ͠Β(P͕ ྑ͍ͱײͨ͡ɻ1ZUIPO͸ศརͳύοέʔδ͕ଟ͍ͷ Ͱɺ(PͰ࣮૷͍ͯ͘͠ͱࠜఈͷཧղ͕ͳ͍ͱ࣮૷Ͱ͖ ͳ͔ͬͨΓ͢Δɻ

  9. Why Go ? Ὂ ྫ͑͹ɺ֬཰෼෍ΛݩʹཁૉΛϥϯμϜʹબ୒͢Δ࣮૷͸1ZUIPOͰ͸ߦͩ ͕ɺ(PͰ͸ࣗ෼ͰΞϧΰϦζϜΛษڧ࣮ͯ͠૷͢Δඞཁ͕͋ͬͨɻ͜͏͍ͬͨ ࡉ͔͍ॴͷ࢓૊Έ΋ษڧʹͳΔɻ https://qiita.com/po3rin/items/f28a8ce322cb1bd95175 ʮ༩͑ΒΕͨॏΈʹैͬͯϥϯμϜʹ஋Λฦ͢ʮ8FJHIUFE3BOEPN4FMFDUJPOʯΛ(PͰ࣮૷͢Δʂʯ

  10. 2: Basic of matrix calculation with Go

  11. Matrix operation package choices Ὂ gonum.org/v1/gonum/mat (PͰߦྻΛѻ͏ͨΊͷ൚༻తͳ ύοέʔδɻ׆ൃʹϝϯς͞Εͯ Δɻૄߦྻ΍ࡾ֯ߦྻͳͲʹ΋ରԠ gorgonia.org/tensor

    (Pͷػցֶश༻ϥΠϒϥϦ ʮHPSHPOJBʯͰ࢖͏͜ͱΛ໨తͱ ͨ͠ߦྻɻଟ࣍ݩʹ΋ରԠ
  12. Matrix operation (1) Ὂ (Pʹ͸1ZUIPOʹ͋ΔΑ͏ͳԋࢉࢠΦʔόʔϩʔυ͕ͳ͍ͷͰؔ਺ͰԋࢉΛߦ͏ɻ

  13. Matrix operation (2) Ὂ ߦྻΛදݱ͢Δߏ଄ମͰ͋Δ NBU%FOTF͸಺෦Ͱશཁૉͷ εϥΠε<>qPBUͱߦ਺ྻ਺ ͳͲͷ৘ใΛอ͍࣋ͯ͠Δ͚ͩɻ

  14. Matrix operation (3) Ὂ NBU%FOTF͸.BUSJYΠϯλʔϑΣʔεΛ࣮૷͍ͯ͠Δɻ͜ΕʹΑΓσʔλߏ଄ͷҧ͍ ΛҙࣝͤͣʹߦྻΛར༻Ͱ͖Δɻଞʹ΋7FD%FOTFʢϕΫτϧΛදݱʣ΍5SJ%FOTF ʢࡾ֯ߦྻΛදݱʣͳͲ.BUSJYΛ࣮૷͢Δ༷ʑͳߏ଄ମ͕͋Δɻ

  15. Matrix operation (4) Ὂ ࣍ݩͷߦྻ͸HPOVNNBUͰ͸ߦྻͷεϥΠεͰ͔͠දݱͰ͖ͳ͍ɻଟ࣍ݩߦྻ ʹରԠ͢ΔύοέʔδΛ࡞Δ৔߹͸UFOTPSUFOTPSʹ܉ഋ্͕͕Δɻ

  16. ࠷ॳʹݟͨͱ͓Γɺ.BUSJYΠϯλʔϑΣʔεͷ࣮ଶͱͯ͠ %FOTF͕͋ΔɻNBUϑΟʔϧυͷܕͰ͋ΔCMBT(FOFSBMͱ ͸ͳΜͰ͠ΐ͏͔ɻ Use BLAS (1) Ὂ

  17. Use BLAS (2) Ὂ #-"4ͱ͸ɺ#BTJD-JOFBS"MHFCSB4VCQSPHSBNTͷུͰɺجຊతͳઢ ܗԋࢉͷϥΠϒϥϦɻ#-"4ʹ͸͞·͟·ͳछྨ͕͋Γ0QFO#-"4 ΍"5-"4ͳͲ͕ଘࡏ͢Δɻ

  18. Use BLAS (3) Ὂ (POVNͰ͸6TFͱ͍͏ؔ਺Ͱ࢖͏#-"4Λ͙͢ʹ੾Γସ͑ΕΔɻ

  19. Use BLAS (4) Ὂ ͨͱ͑͹%PU ཁૉ͝ͱͷੵͷ߹ܭΛܭࢉ͢Δ Ͱ͸಺෦Ͱ͸CMBT%EPU Λݺͼग़͍ͯ͠Δɻ౰વCMBT%EPU͸ΠϯλʔϑΣʔεͰఆٛ͞Εͨϝιο υͳͷͰCMBTͷ࣮ମ͝ͱʹҧ͏ॲཧΛݺ΂Δɻ

  20. Use BLAS (5) Ὂ ͨͱ͑͹ཁૉͷϕΫτϧಉ࢜Λ%PUؔ਺ʹ͔͚Δ࣌ͷϕϯνϚʔΫͰ ͸Լهͷ؀ڥͰ͸ഒ΄Ͳ0QFO#-"4ͷํ͕ૣ͘ͳ͍ͬͯ·͢ɻ

  21. 2JJUBʹ(P (POVNʹΑΔߦྻܭࢉΛ ମܥతʹ·ͱ·͍ͬͯΔɻߦྻܭࢉͷΤ ϥʔϋϯυϦϯά΍ςετͳͲ΋Χόʔ͞ Ε͍ͯ·͢ɻ https://qiita.com/po3rin/items/82c9c0195f9e3e38e2f1 Study Gonum Ὂ

  22. IUUQTXXXDIFBUPHSBQIZDPNDIFXYZDIFBUTIFFUTEBUBTDJFODFJOHPB Study matrix operation using Go Ὂ (PSHPOJB࡞ऀʹΑΔ ʮOVNQZʯʮHPOVNNBUʯ ʮHPSHPOJBUFOTPSʯʹ͓͚Δ

    ߦྻԋࢉͷνʔτγʔτ͕ૉ੖ Β͍͠ɻOVNQZͷ͜Εͬͯ(P ͩͱͲ͏΍ΔΜͩʁ͕΄΅ࡌͬ ͯΔɻ
  23. 3: Architecture for building neural networks with Go

  24. Example of Machine Lerning in Go Ὂ (P (POVNͰχϡʔϥϧωοτϫʔΫΛ૊ΉͨΊͷίϯϙʔωϯτूͰ͋ ΔHPOOQͷ࣮૷ܦݧ͔Β࣮૷ΞʔΩςΫνϟΛߟ͑Δɻ

  25. Architecture (2) Ὂ ܇࿅༻ͷΦϒδΣΫτΛ.PEFMͱ0QUJNJ[FSΛ౉ͯ͠ॳظԽ ͠·͢ɻॳظԽؔ਺͸ύϥϝʔλ͕ඇৗʹଟ͘ͳΔͷͰ 'VODUJPOBM0QUJPO1BUUFSOTͰ࣮૷͓ͯ͘͠ͱྑ͍ɻ 5SBJOFS .PEFM 0QUJNJ[FS

  26. Architecture (3) Ὂ 5SBJOFS 0QUJNJ[FS "⒏OF-BZFS 3FMV-BZFS "⒏OF-BZFS "⒏OF-BZFS ϨΠϠʔͷ࣮૷͸*OUFSGBDFͱͯ͠ఆ͓ٛͯ͠

    ͖ɺύοέʔδΛར༻͢ΔϢʔβʔ͕ಠࣗͷϨ ΠϠʔΛ࣮૷ͯ͠૊ΈࠐΊΔΑ͏ʹ͓ͯ͘͠
  27. Architecture (4) Ὂ 8FJHIU΍#JBTͳͲͷύϥϝʔλΛߋ৽͢ Δ0QUJNJ[FSɻ΋ͪΖΜ͜͜΋Ϣʔβʔ͕೚ ҙͷΞϧΰϦζϜΛར༻Ͱ͖ΔΑ͏ʹ *OUFSGBDFͰఆٛ͢Δɻ 5SBJOFS .PEFM (SBEJFOU%FTDFOU

  28. Architecture (7) Ὂ .BUSJYΠϯλʔϑΣʔεʹ͸ "EE΍4VCͳͲͷجຊతͳԋࢉ ͕࣮૷͞Ε͍ͯͳ͍ͨΊ (POVNͰχϡʔϥϧωοτ ϫʔΫΛߏங͍ͯ͘͠ͱҾ਺΍ ฦΓ஋Λɺߏ଄ମͰѻ͏͔Πϯ λʔϑΣʔεͰѻ͏͔೰ΈͲ͜ɻ

  29. Architecture (9) Ὂ (POVN͚ͩͰ͸े෼ͳߦྻԋࢉ ͕ఏڙ͞Ε͍ͯͳ͍ͷͰɺ HPOOQͰ͸NBUVUJMύοέʔδ Λ࣮૷͍ͯ͠Δɻͦͷ৔߹͸ HPOVNNBUͷΑ͏ʹΠϯλʔ ϑΣʔεͰड͚ɺߏ଄ମͰฦ͢ ಺෦ͰܕεΠον͓ͯ͘͠ͱ࢖

    ͍΍͍͢ɻ
  30. Architecture (8) Ὂ ϨΠϠʔʹؔͯ͠͸༷ʑͳϨΠ Ϡʔͷ࣮૷Λίϯϙʔωϯτͱ͠ ͯ࢖͍ճͤΔΑ͏ʹ*OUFSGBDFͰ ड͚ͯ*OUFSGBDFͰฦ࣮͢૷ʹ͠ ͍ͯΔɻ ϨΠϠʔ͸ߦྻΛड͚ Δ͔ϕΫτϧΛड͚Δ͔෼͔Βͳ

    ͍
  31. Architecture (9) Ὂ 5FOTPS'MPXͷ࣮૷ͷΑ͏ʹσʔλϑϩʔάϥϑͬΆ͘ઃܭ͢Δͱ͍͏ߟ͑΋͋Δɻ

  32. Architecture (9) Ὂ ࣮ࡍʹHPSHPOJBHPSHPOJB Ͱ͸νϟωϧʹΑΔσʔλϑ ϩʔͱͯ͠άϥϑΛߏங͠ɺ࣮ ߦ͢Δ࣮૷Λ͍࣋ͬͯΔɻ

  33. Architecture (10) Ὂ ͜͏͢Δͱฒߦॲཧͳάϥϑॲ ཧΛ࣮૷Ͱ͖Δɻ HPOOQͰ͸ ·ͩશͯγʔέϯεͳͷͰฒߦ ॲཧΛαϙʔτ͍͖͍ͯͨ͠ ࢀߟʮ5IJOLMJLFBWFSUFYVTJOH(PTDPODVSSFODZGPSHSBQIDPNQVUBUJPOʯ https://blog.owulveryck.info/2019/10/14/think-like-a-vertex-using-gos-concurrency-for-graph-

    computation.html
  34. 4: Store trained models

  35. Store trained models (1) Ὂ (PͰֶशͨ͠Ϟσϧͷอଘ΍෼ࢄදݱͷอଘํ๏Λߟ͑Δɻ(PͰσʔλߏ଄ΛγϦΞϥ Πζ͢ΔͨΊʹ͍͔ͭ͘ͷύοέʔδ͕༻ҙ͞Ε͍ͯΔɻ ɾFODPSEJOHDTW ɾFODPSEJOHKTPO ɾFODPSEJOHYNM

  36. Store trained models (2) Ὂ (Pͷ؀ڥ಺Ͱ׬݁͢ΔͷͰ͋Ε͹ɺFODPSEJOHHPC͕͓͢͢Ίɻσʔλߏ଄͑͞ ෼͔Ε͹ͦͷ··γϦΞϥΠζͰ͖Δɻ·ͨόΠφϦΤϯίʔσΟϯάͳͷͰޮ཰ੑ ΋ߴ͍ɻ ࢀߟʮ(PCTPGEBUBʯ5IF(P#MPH https://blog.golang.org/gobs-of-data

  37. Store trained models (3) Ὂ ྫ͑͹8PSE7FDͰ֫ಘ͠ ͨ୯ޠͷ෼ࢄදݱΛ(PͷΞϓ ϦέʔγϣϯͰར༻͍ͨ͠৔ ߹͸ӈͷΑ͏ͳγϯϓϧͳ ίʔυ͚ͩͰ࠶ར༻Մೳͳܗ

    ͰอଘͰ͖Δɻ
  38. Store trained models (4) Ὂ ಠࣗͷΠϯλʔϑΣʔεܕΛHPCͰอଘ͍ͨ͠৔߹͸HPC3FHJTUFSΛ࢖࣮ͬͯଶͱ ͳΔσʔλߏ଄Λڭ͑Δඞཁ͕͋Δɻ࣮ࡍʹHPOOQʹ͓͍ͯ͸NBU.BUSJYΛJOJUؔ਺ ಺Ͱొ࿥͍ͯ͠Δ

  39. Store trained models (5) Ὂ ҰํͰόΠφϦͰσʔλΛอଘ͢ΔϑΥʔϚο τͰ͋Δ.FTTBHF1BDLͷ(P࣮૷ͳͲ΋બ୒ ࢶʹೖΓ·͢ɻฐࣾͷ(PͰֶशͨ͠ϞσϧͰ͸ ͪ͜ΒΛ࢖ͬͯอଘ͍ͯ͠Δɻ

  40. 4: Use existing machine learning packages

  41. Marchine Larning Packages Ὂ ಠ࣮ࣗ૷ͤͣͱ΋(PͰػցֶशΛ͢Δ؀ڥ͕੔͍ͭͭ͋Δɻ ίϯηϓτͱͯ͠͸5FOTPS'MPXʹ͍ۙɺάϥϑΛߏங࣮ͯ͠ߦ͍ͯ͘͠ɺ಺෦ͷߦ ྻԋࢉͰ͸HPSHPOJBUFOTPS͕࢖ΘΕ͍ͯΔɻHPOVNNBUͱͷޓ׵ੑ΋͋Δɻ

  42. Marchine Larning Packages Ὂ (PͷطଘύοέʔδͰطଘϞσϧΛ࢖͏؀ڥ΋੔͍ͭͭ͋Δɻ ֶशࡁΈϞσϧΛ5FOTPS'MPX͕ఏڙ ͢Δ$"1*Λ࢖༻ͯ͠ಈ͔͢ɻ΋ͪΖΜ (P΋αϙʔτ͍ͯ͠Δɻ

  43. Marchine Larning Packages Ὂ ֶशࡁΈϞσϧ 1SPUPDPM#V⒎FST 5FOTPS'MPX 1ZUIPOͰֶश 5FOTPS'MPX (PͰਪ࿦

    1SPUPDPM#V⒎FSTܗࣜͷֶशࡁΈϞσϧͰ ͋Ε͹5FOTPS'MPX (PͰಈ͔ͤΔɻ
  44. data-science Channel in Gophers Slack Ὂ (PͰσʔλαΠΤϯεΛ΍Δ ਓ͕ؒू·Δ4MBDLνϟωϧ͕ ͋Δɻ(PपΓͷσʔλαΠΤ ϯε৘ใ͸͜͜ʹूத͢Δͷ

    ͰΦεεϝ
  45. 6: Conclusion

  46. Conclusion Ὂ ͜͜·Ͱ(PͰͷػցֶशͷ࣮૷Λߦྻͷ࢓૊Έ͔Β঺հ͍͖ͯ͠·ͨ͠ɻ ࣮͸·ͩ·ͩ(PͰͷ਺஋ܭࢉ͸ൃల్্Ͱ͢ɻ΋͜͠ͷൃදͰগ͠Ͱ΋ڵຯΛ ࣋ͬͨํ͸ੋඇҰॹʹ(PʹΑΔσʔλαΠΤϯεΛ΍͍͖ͬͯ·͠ΐ͏ɻ

  47. Suggestions and techniques for building neural networks with Go Go

    Conference Tokyo 2019 Autumn / October 28, 2019 @ po3rin