Suggestions and techniques for building neural networks with Go

Slide 1

Slide 1 text

Suggestions and techniques for building neural networks with Go Go Conference Tokyo 2019 Autumn / October 28, 2019 @ po3rin

Slide 2

Slide 2 text

pon (@po3rin) Software Enginner @ Shiroyagi Corporation Golang / Python / Rust / Docker / AWS / Elasticsearch

Slide 3

Slide 3 text

QBHF നϠΪίʔϙϨʔγϣϯ͸ɺػցֶशɾࣗવ ݴޠॲཧ෼໺ͷϓϩμΫτʹྗΛೖΕ͍ͯΔձ ࣾͰ͢ɻ։ൃʹ͸ओʹ (P ΍ 1ZUIPO ͕࠾༻ ͞Ε͍ͯ·͢ɻ നϠΪίʔϙϨʔγϣϯ

Slide 4

Slide 4 text

Package for Machine Lerning in Go Ὂ (P(POVNͰχϡʔϥϧ ωοτϫʔΫΛ૊ΉͨΊͷί ϯϙʔωϯτूʂ ./*45σʔληοτΛ(Pͷ ߦྻ࣮૷ͰಡΈࠐΉͨΊͷ ύοέʔδ

Slide 5

Slide 5 text

1: Why do data-science with Go?

Slide 6

Slide 6 text

Why Go ? Ὂ (PͷڧΈ͕ͦͷ··σʔλαΠΤϯεΛ(PͰ΍Δཧ༝ ίʔυ͕ಡΈ΍͘͢ϝϯς͠΍͍͢ ύϑΥʔϚϯε͕ྑ͍ ฒߦԽ͠΍͍͢ ҰํͰ΍͸Γ·ͩ·ͩࢿ࢈తʹ1ZUIPOʹ܉ഋ্͕͕Δͷ͸͔֬ɻɻ

Slide 7

Slide 7 text

Marchine Larning Packages Ὂ ͔͠͠ɺϞσϧΛಈ͔͢؀ڥ͸͔ͳΓॆ࣮͖͍ͯͯ͠ΔͷͰɺਪ࿦ΛϚΠΫϩαʔϏ εͱཱͯͯ͠Ε͹(PϚΠΫϩαʔϏεͷخ͠Έ͕ಘΒΕΔɻ ֶशࡁΈϞσϧΛ5FOTPS'MPX͕ ఏڙ͢Δ$"1*Λ࢖༻ͯ͠ಈ͔͢ɻ ΋ͪΖΜ(P΋αϙʔτ͍ͯ͠Δɻ 1VSF(PͰॻ͔Ε͍ͯΔɻ0//9 ϑΥʔϚοτͰॻ͔Εͨχϡʔϥϧ ωοτΛΠϯϙʔτͯ͠ಈ͔ͤΔ

Slide 8

Slide 8 text

Why Go for Biginners ? Ὂ ॳ৺ऀͰɺجૅతͳͱ͜Ζ͔Βษڧ͔ͨͬͨ͠Β(P͕ ྑ͍ͱײͨ͡ɻ1ZUIPO͸ศརͳύοέʔδ͕ଟ͍ͷ Ͱɺ(PͰ࣮૷͍ͯ͘͠ͱࠜఈͷཧղ͕ͳ͍ͱ࣮૷Ͱ͖ ͳ͔ͬͨΓ͢Δɻ

Slide 9

Slide 9 text

Why Go ? Ὂ ྫ͑͹ɺ֬཰෼෍ΛݩʹཁૉΛϥϯμϜʹબ୒͢Δ࣮૷͸1ZUIPOͰ͸ߦͩ ͕ɺ(PͰ͸ࣗ෼ͰΞϧΰϦζϜΛษڧ࣮ͯ͠૷͢Δඞཁ͕͋ͬͨɻ͜͏͍ͬͨ ࡉ͔͍ॴͷ࢓૊Έ΋ษڧʹͳΔɻ https://qiita.com/po3rin/items/f28a8ce322cb1bd95175 ʮ༩͑ΒΕͨॏΈʹैͬͯϥϯμϜʹ஋Λฦ͢ʮ8FJHIUFE3BOEPN4FMFDUJPOʯΛ(PͰ࣮૷͢Δʂʯ

Slide 10

Slide 10 text

2: Basic of matrix calculation with Go

Slide 11

Slide 11 text

Matrix operation package choices Ὂ gonum.org/v1/gonum/mat (PͰߦྻΛѻ͏ͨΊͷ൚༻తͳ ύοέʔδɻ׆ൃʹϝϯς͞Εͯ Δɻૄߦྻ΍ࡾ֯ߦྻͳͲʹ΋ରԠ gorgonia.org/tensor (Pͷػցֶश༻ϥΠϒϥϦ ʮHPSHPOJBʯͰ࢖͏͜ͱΛ໨తͱ ͨ͠ߦྻɻଟ࣍ݩʹ΋ରԠ

Slide 12

Slide 12 text

Matrix operation (1) Ὂ (Pʹ͸1ZUIPOʹ͋ΔΑ͏ͳԋࢉࢠΦʔόʔϩʔυ͕ͳ͍ͷͰؔ਺ͰԋࢉΛߦ͏ɻ

Slide 13

Slide 13 text

Matrix operation (2) Ὂ ߦྻΛදݱ͢Δߏ଄ମͰ͋Δ NBU%FOTF͸಺෦Ͱશཁૉͷ εϥΠε<>qPBUͱߦ਺ྻ਺ ͳͲͷ৘ใΛอ͍࣋ͯ͠Δ͚ͩɻ

Slide 14

Slide 14 text

Matrix operation (3) Ὂ NBU%FOTF͸.BUSJYΠϯλʔϑΣʔεΛ࣮૷͍ͯ͠Δɻ͜ΕʹΑΓσʔλߏ଄ͷҧ͍ ΛҙࣝͤͣʹߦྻΛར༻Ͱ͖Δɻଞʹ΋7FD%FOTFʢϕΫτϧΛදݱʣ΍5SJ%FOTF ʢࡾ֯ߦྻΛදݱʣͳͲ.BUSJYΛ࣮૷͢Δ༷ʑͳߏ଄ମ͕͋Δɻ

Slide 15

Slide 15 text

Matrix operation (4) Ὂ ࣍ݩͷߦྻ͸HPOVNNBUͰ͸ߦྻͷεϥΠεͰ͔͠දݱͰ͖ͳ͍ɻଟ࣍ݩߦྻ ʹରԠ͢ΔύοέʔδΛ࡞Δ৔߹͸UFOTPSUFOTPSʹ܉ഋ্͕͕Δɻ

Slide 16

Slide 16 text

࠷ॳʹݟͨͱ͓Γɺ.BUSJYΠϯλʔϑΣʔεͷ࣮ଶͱͯ͠ %FOTF͕͋ΔɻNBUϑΟʔϧυͷܕͰ͋ΔCMBT(FOFSBMͱ ͸ͳΜͰ͠ΐ͏͔ɻ Use BLAS (1) Ὂ

Slide 17

Slide 17 text

Use BLAS (2) Ὂ #-"4ͱ͸ɺ#BTJD-JOFBS"MHFCSB4VCQSPHSBNTͷུͰɺجຊతͳઢ ܗԋࢉͷϥΠϒϥϦɻ#-"4ʹ͸͞·͟·ͳछྨ͕͋Γ0QFO#-"4 ΍"5-"4ͳͲ͕ଘࡏ͢Δɻ

Slide 18

Slide 18 text

Use BLAS (3) Ὂ (POVNͰ͸6TFͱ͍͏ؔ਺Ͱ࢖͏#-"4Λ͙͢ʹ੾Γସ͑ΕΔɻ

Slide 19

Slide 19 text

Use BLAS (4) Ὂ ͨͱ͑͹%PU ཁૉ͝ͱͷੵͷ߹ܭΛܭࢉ͢Δ Ͱ͸಺෦Ͱ͸CMBT%EPU Λݺͼग़͍ͯ͠Δɻ౰વCMBT%EPU͸ΠϯλʔϑΣʔεͰఆٛ͞Εͨϝιο υͳͷͰCMBTͷ࣮ମ͝ͱʹҧ͏ॲཧΛݺ΂Δɻ

Slide 20

Slide 20 text

Use BLAS (5) Ὂ ͨͱ͑͹ཁૉͷϕΫτϧಉ࢜Λ%PUؔ਺ʹ͔͚Δ࣌ͷϕϯνϚʔΫͰ ͸Լهͷ؀ڥͰ͸ഒ΄Ͳ0QFO#-"4ͷํ͕ૣ͘ͳ͍ͬͯ·͢ɻ

Slide 21

Slide 21 text

2JJUBʹ(P(POVNʹΑΔߦྻܭࢉΛ ମܥతʹ·ͱ·͍ͬͯΔɻߦྻܭࢉͷΤ ϥʔϋϯυϦϯά΍ςετͳͲ΋Χόʔ͞ Ε͍ͯ·͢ɻ https://qiita.com/po3rin/items/82c9c0195f9e3e38e2f1 Study Gonum Ὂ

Slide 22

Slide 22 text

IUUQTXXXDIFBUPHSBQIZDPNDIFXYZDIFBUTIFFUTEBUBTDJFODFJOHPB Study matrix operation using Go Ὂ (PSHPOJB࡞ऀʹΑΔ ʮOVNQZʯʮHPOVNNBUʯ ʮHPSHPOJBUFOTPSʯʹ͓͚Δ ߦྻԋࢉͷνʔτγʔτ͕ૉ੖ Β͍͠ɻOVNQZͷ͜Εͬͯ(P ͩͱͲ͏΍ΔΜͩʁ͕΄΅ࡌͬ ͯΔɻ

Slide 23

Slide 23 text

3: Architecture for building neural networks with Go

Slide 24

Slide 24 text

Example of Machine Lerning in Go Ὂ (P(POVNͰχϡʔϥϧωοτϫʔΫΛ૊ΉͨΊͷίϯϙʔωϯτूͰ͋ ΔHPOOQͷ࣮૷ܦݧ͔Β࣮૷ΞʔΩςΫνϟΛߟ͑Δɻ

Slide 25

Slide 25 text

Architecture (2) Ὂ ܇࿅༻ͷΦϒδΣΫτΛ.PEFMͱ0QUJNJ[FSΛ౉ͯ͠ॳظԽ ͠·͢ɻॳظԽؔ਺͸ύϥϝʔλ͕ඇৗʹଟ͘ͳΔͷͰ 'VODUJPOBM0QUJPO1BUUFSOTͰ࣮૷͓ͯ͘͠ͱྑ͍ɻ 5SBJOFS .PEFM 0QUJNJ[FS

Slide 26

Slide 26 text

Architecture (3) Ὂ 5SBJOFS 0QUJNJ[FS "⒏OF-BZFS 3FMV-BZFS "⒏OF-BZFS "⒏OF-BZFS ϨΠϠʔͷ࣮૷͸*OUFSGBDFͱͯ͠ఆ͓ٛͯ͠ ͖ɺύοέʔδΛར༻͢ΔϢʔβʔ͕ಠࣗͷϨ ΠϠʔΛ࣮૷ͯ͠૊ΈࠐΊΔΑ͏ʹ͓ͯ͘͠

Slide 27

Slide 27 text

Architecture (4) Ὂ 8FJHIU΍#JBTͳͲͷύϥϝʔλΛߋ৽͢ Δ0QUJNJ[FSɻ΋ͪΖΜ͜͜΋Ϣʔβʔ͕೚ ҙͷΞϧΰϦζϜΛར༻Ͱ͖ΔΑ͏ʹ *OUFSGBDFͰఆٛ͢Δɻ 5SBJOFS .PEFM (SBEJFOU%FTDFOU

Slide 28

Slide 28 text

Architecture (7) Ὂ .BUSJYΠϯλʔϑΣʔεʹ͸ "EE΍4VCͳͲͷجຊతͳԋࢉ ͕࣮૷͞Ε͍ͯͳ͍ͨΊ (POVNͰχϡʔϥϧωοτ ϫʔΫΛߏங͍ͯ͘͠ͱҾ਺΍ ฦΓ஋Λɺߏ଄ମͰѻ͏͔Πϯ λʔϑΣʔεͰѻ͏͔೰ΈͲ͜ɻ

Slide 29

Slide 29 text

Architecture (9) Ὂ (POVN͚ͩͰ͸े෼ͳߦྻԋࢉ ͕ఏڙ͞Ε͍ͯͳ͍ͷͰɺ HPOOQͰ͸NBUVUJMύοέʔδ Λ࣮૷͍ͯ͠Δɻͦͷ৔߹͸ HPOVNNBUͷΑ͏ʹΠϯλʔ ϑΣʔεͰड͚ɺߏ଄ମͰฦ͢ ಺෦ͰܕεΠον͓ͯ͘͠ͱ࢖ ͍΍͍͢ɻ

Slide 30

Slide 30 text

Architecture (8) Ὂ ϨΠϠʔʹؔͯ͠͸༷ʑͳϨΠ Ϡʔͷ࣮૷Λίϯϙʔωϯτͱ͠ ͯ࢖͍ճͤΔΑ͏ʹ*OUFSGBDFͰ ड͚ͯ*OUFSGBDFͰฦ࣮͢૷ʹ͠ ͍ͯΔɻ ϨΠϠʔ͸ߦྻΛड͚ Δ͔ϕΫτϧΛड͚Δ͔෼͔Βͳ ͍

Slide 31

Slide 31 text

Architecture (9) Ὂ 5FOTPS'MPXͷ࣮૷ͷΑ͏ʹσʔλϑϩʔάϥϑͬΆ͘ઃܭ͢Δͱ͍͏ߟ͑΋͋Δɻ

Slide 32

Slide 32 text

Architecture (9) Ὂ ࣮ࡍʹHPSHPOJBHPSHPOJB Ͱ͸νϟωϧʹΑΔσʔλϑ ϩʔͱͯ͠άϥϑΛߏங͠ɺ࣮ ߦ͢Δ࣮૷Λ͍࣋ͬͯΔɻ

Slide 33

Slide 33 text

Architecture (10) Ὂ ͜͏͢Δͱฒߦॲཧͳάϥϑॲ ཧΛ࣮૷Ͱ͖Δɻ HPOOQͰ͸ ·ͩશͯγʔέϯεͳͷͰฒߦ ॲཧΛαϙʔτ͍͖͍ͯͨ͠ ࢀߟʮ5IJOLMJLFBWFSUFYVTJOH(PTDPODVSSFODZGPSHSBQIDPNQVUBUJPOʯ https://blog.owulveryck.info/2019/10/14/think-like-a-vertex-using-gos-concurrency-for-graph- computation.html

Slide 34

Slide 34 text

4: Store trained models

Slide 35

Slide 35 text

Store trained models (1) Ὂ (PͰֶशͨ͠Ϟσϧͷอଘ΍෼ࢄදݱͷอଘํ๏Λߟ͑Δɻ(PͰσʔλߏ଄ΛγϦΞϥ Πζ͢ΔͨΊʹ͍͔ͭ͘ͷύοέʔδ͕༻ҙ͞Ε͍ͯΔɻ ɾFODPSEJOHDTW ɾFODPSEJOHKTPO ɾFODPSEJOHYNM

Slide 36

Slide 36 text

Store trained models (2) Ὂ (Pͷ؀ڥ಺Ͱ׬݁͢ΔͷͰ͋Ε͹ɺFODPSEJOHHPC͕͓͢͢Ίɻσʔλߏ଄͑͞ ෼͔Ε͹ͦͷ··γϦΞϥΠζͰ͖Δɻ·ͨόΠφϦΤϯίʔσΟϯάͳͷͰޮ཰ੑ ΋ߴ͍ɻ ࢀߟʮ(PCTPGEBUBʯ5IF(P#MPH https://blog.golang.org/gobs-of-data

Slide 37

Slide 37 text

Store trained models (3) Ὂ ྫ͑͹8PSE7FDͰ֫ಘ͠ ͨ୯ޠͷ෼ࢄදݱΛ(PͷΞϓ ϦέʔγϣϯͰར༻͍ͨ͠৔ ߹͸ӈͷΑ͏ͳγϯϓϧͳ ίʔυ͚ͩͰ࠶ར༻Մೳͳܗ ͰอଘͰ͖Δɻ

Slide 38

Slide 38 text

Store trained models (4) Ὂ ಠࣗͷΠϯλʔϑΣʔεܕΛHPCͰอଘ͍ͨ͠৔߹͸HPC3FHJTUFSΛ࢖࣮ͬͯଶͱ ͳΔσʔλߏ଄Λڭ͑Δඞཁ͕͋Δɻ࣮ࡍʹHPOOQʹ͓͍ͯ͸NBU.BUSJYΛJOJUؔ਺ ಺Ͱొ࿥͍ͯ͠Δ

Slide 39

Slide 39 text

Store trained models (5) Ὂ ҰํͰόΠφϦͰσʔλΛอଘ͢ΔϑΥʔϚο τͰ͋Δ.FTTBHF1BDLͷ(P࣮૷ͳͲ΋બ୒ ࢶʹೖΓ·͢ɻฐࣾͷ(PͰֶशͨ͠ϞσϧͰ͸ ͪ͜ΒΛ࢖ͬͯอଘ͍ͯ͠Δɻ

Slide 40

Slide 40 text

4: Use existing machine learning packages

Slide 41

Slide 41 text

Marchine Larning Packages Ὂ ಠ࣮ࣗ૷ͤͣͱ΋(PͰػցֶशΛ͢Δ؀ڥ͕੔͍ͭͭ͋Δɻ ίϯηϓτͱͯ͠͸5FOTPS'MPXʹ͍ۙɺάϥϑΛߏங࣮ͯ͠ߦ͍ͯ͘͠ɺ಺෦ͷߦ ྻԋࢉͰ͸HPSHPOJBUFOTPS͕࢖ΘΕ͍ͯΔɻHPOVNNBUͱͷޓ׵ੑ΋͋Δɻ

Slide 42

Slide 42 text

Marchine Larning Packages Ὂ (PͷطଘύοέʔδͰطଘϞσϧΛ࢖͏؀ڥ΋੔͍ͭͭ͋Δɻ ֶशࡁΈϞσϧΛ5FOTPS'MPX͕ఏڙ ͢Δ$"1*Λ࢖༻ͯ͠ಈ͔͢ɻ΋ͪΖΜ (P΋αϙʔτ͍ͯ͠Δɻ

Slide 43

Slide 43 text

Marchine Larning Packages Ὂ ֶशࡁΈϞσϧ 1SPUPDPM#V⒎FST 5FOTPS'MPX1ZUIPOͰֶश 5FOTPS'MPX(PͰਪ࿦ 1SPUPDPM#V⒎FSTܗࣜͷֶशࡁΈϞσϧͰ ͋Ε͹5FOTPS'MPX(PͰಈ͔ͤΔɻ

Slide 44

Slide 44 text

data-science Channel in Gophers Slack Ὂ (PͰσʔλαΠΤϯεΛ΍Δ ਓ͕ؒू·Δ4MBDLνϟωϧ͕ ͋Δɻ(PपΓͷσʔλαΠΤ ϯε৘ใ͸͜͜ʹूத͢Δͷ ͰΦεεϝ

Slide 45

Slide 45 text

6: Conclusion

Slide 46

Slide 46 text

Conclusion Ὂ ͜͜·Ͱ(PͰͷػցֶशͷ࣮૷Λߦྻͷ࢓૊Έ͔Β঺հ͍͖ͯ͠·ͨ͠ɻ ࣮͸·ͩ·ͩ(PͰͷ਺஋ܭࢉ͸ൃల్্Ͱ͢ɻ΋͜͠ͷൃදͰগ͠Ͱ΋ڵຯΛ ࣋ͬͨํ͸ੋඇҰॹʹ(PʹΑΔσʔλαΠΤϯεΛ΍͍͖ͬͯ·͠ΐ͏ɻ

Slide 47

Slide 47 text

Suggestions and techniques for building neural networks with Go Go Conference Tokyo 2019 Autumn / October 28, 2019 @ po3rin