アットホームな分析基盤の作り方 / Homemade Machine Learning Toolkits

分析基盤トーク #1
https://daft.connpass.com/event/93036/

Transcript

1. ΞοτϗʔϜͳ

   ෼ੳج൫ͷ࡞Γํ
   BZFNPT
   

   View Slide

2. ࣗݾ঺հ
   છ୩༔Ұ࿠

   View Slide

3. ΫοΫύουݚڀ։ൃ෦
   ೥݄ʹൃ଍
   ౰࣌͸໊ͷϝϯόʔ
   ೥݄ݱࡏ͸ࠃ಺ʹ໊ւ֎ʹ໊
   

   View Slide

4. ΫοΫύουͱػցֶश
   .ZϑΥϧμ ͓ؾʹೖΓػೳ
   ಺ͷϨγϐΛࣗಈΧςΰϦ෼ྨ
   ʮྉཧ͖Ζ͘ʯεϚʔτϑΥϯ಺ͷྉཧࣸਅΛΧϨϯμʔܗࣜ
   ͰৼΓฦΓ
   ࡐྉ໊ͷਖ਼نԽ
   ౳ʑ
   ৄ͘͠͸IUUQTUFDIMJGFDPPLQBEDPN
   

   View Slide

5. BHFOEB
   ΫοΫύουͷ෼ੳج൫ͷ঺հ
   ෼ੳج൫ͷ͜Ε͔Β
   

   View Slide

6. ج൫঺հͦͷ;PPFZ
   

   View Slide

7. View Slide

8. ෼ੳͱ࣮ݧΛ֤͕ࣗࣗ༝ʹߦ͏ͨΊͷ࠷௿ݶͷج൫
   

   View Slide

9. ج൫঺հͦͷ̍;PPFZ
   ܭࢉػ؀ڥߏங༻$IBU#PU
   ܭࢉػ؀ڥͷ࡞੒ʗ্ཱͪ͛ʗఀࢭʗ࡟আ
   ΞΠυϧঢ়ଶͷܭࢉػΛࣗಈఀࢭ
   

   View Slide

10. View Slide

11. $6%" DV%//
    $6%" DV%//
    %FFQ-FBSOJOH".*+BO

    FUD
    0VUHPJOH8FCIPPLT
    
    TMBDL@VJE6
    DPNNBOEDSFBUFXPSLCFODI
    MBNCEB*OWPLF
    71$ 4VCOFU /BNFλά౳

    ؀ڥಛ༗ͷઃఆ΋͜͜Ͱ΍Δ
    ΠϯελϯεʹATMBDL@VJEAΛ

    λά෇͚ͯ͠ॴ༗ऀΛ؅ཧ͢Δ
    DSFBUFXPSLCFODI
    ;PPFZ
    

    View Slide

12. ATTIBZFNPTXPSLCFODIEOTDPNA
    ;PPFZ
    *ODPNJOH8FCIPPLT
    
    TTI
    
    

    View Slide

13. ;PPFZ
    4UPQ
    
    ΞΠυϧͳΠϯελϯεΛࣗಈఀࢭ
    

    View Slide

14. ج൫঺հͦͷ̍;PPFZ
    ಺੡ͷԾ૝*NBHF%FFQ-FBSOJOH".*
    Λར༻Մೳ
    Πϯελϯε࡞੒ͷೖΓޱΛҰຊԽ
    ωοτϫʔΫɺηΩϡϦςΟपΓͷࡉ͔͍ઃఆΛٵऩ

    ར༻ଆ͸*NBHFΛ൪߸ͰબͿ͚ͩ
    ࣾ಺ΠϯϑϥͷมԽʹରͯ͠͸πʔϧͷΞοϓσʔτͰରԠ
    ւ֎ͷϝϯόʔ΋ར༻
    
    IUUQTBXTBNB[PODPNKQNBDIJOFMFBSOJOHBNJT
    

    View Slide

15. ج൫঺հͦͷ̍;PPFZ
    ֤͕ࣗSPPUΛऔΕΔΠϯελϯεΛ෷͍ग़͢
    গʑલ࣌୅తʁ
    ॊೈੑ͸ߴ͘ɺ࣮ݧஈ֊ͷ؀ڥͱͯ͠͸࢖͍৺஍͕Α͍
    ࣗಈఀࢭ΋͋ΔͷͰ֤ࣗ҆৺ͯ͠ඞཁͳ্ཱ͚ͩͪ͛ΒΕΔ
    ʮ࣮ݧʯΛؚΊαϙʔτ͢Δڞ௨ج൫Λ࣋ͨͳ͍ͷͰ

    ʮ࣮ݧʯʮຊ൪ʯͷ1PSUBCJMJUZ͸ผ్ิڧ͢Δඞཁ͕͋Δ
    

    View Slide

16. ج൫঺հͦͷσʔλج൫
    

    View Slide

17. View Slide

18. ͍ΘΏΔ%8)42-ϕʔεͷ&5-ج൫
    

    View Slide

19. σʔλج൫
    ͜Ε͸ओʹผνʔϜ σʔλج൫άϧʔϓ
    ʹΑΔ؅ཧ
    "NB[PO3FETIJGUʹΑΔ%8)ߏங
    .Z42-3FETIJGU΁ͷΠϯϙʔτδϣϒΛίʔυ؅ཧ
    ৄ͘͠͸IUUQTUFDIMJGFDPPLQBEDPNFOUSZ
    
    

    View Slide

20. σʔλج൫ͷར༻ঢ়گ
    %8)͕ਆ
    &5-
    ࣮ݧͷ౓ʹ֤͕ࣗ౎౓ߦ͍ɺεΫϦϓτԽ͢Δࣄ͕ଟ͍
    ڞ௨&5-ج൫ͷར༻ɺதؒσʔλͷڞ༗ͳͲʹର͢Δχʔζ
    ͸ͦ͜·Ͱେ͖͘ͳ͍ ݱঢ়
    
    

    View Slide

21. ج൫঺հͦͷίʔυ؅ཧ
    

    View Slide

22. ج൫঺հͦͷίʔυ؅ཧ
    ج൫ʁ
    
    DPPLJFDVUUFSEBUBTDJFODF ࣾ಺GPSL
    Λར༻
    ϓϩδΣΫτߏ੒ͷςϯϓϨʔτ
    ATSDEBUBA ATSDNPEFMAσʔλੜ੒ɺֶशεΫϦϓτ
    AEBUBA ANPEFMTAσʔλɺϞσϧ
    HJUʹ͸DIFDLJOͤͣɺ4ͱTZOD͢Δ ANBLFTZOD@EBUB@UP@TA
    
    

    View Slide

23. ج൫঺հͦͷίʔυ؅ཧ
    IUUQTHJUIVCDPNEPDLFSTDJFODFDPPLJFDVUUFSEPDLFS
    TDJFODF
    DPPLJFDVUUFSEBUBTDJFODFΛࢀߟʹͨ͠ςϯϓϨʔτ
    EPDLFSΛར༻͠ɺϓϩδΣΫτͷ࣮ݱੑΛ͞ΒʹߴΊΔ
    OPUFCPPLͷ্ཱͪ͛1PSUGPSXBSE౳ߦ͏UBSHFU΋
    

    View Slide

24. ج൫·ͱΊ
    ݱঢ়͸౷ҰԽ͞ΕͨػցֶशϓϥοτϑΥʔϜΛར༻͍ͯ͠ͳ͍
    ࣗ੡πʔϧͷ૊Έ߹Θͤ%8)Ͱ͓͓ΉͶճ͍ͬͯΔ
    ج൫ͦͷ΋ͷͷ؅ཧίετͳͲߟྀͭͭ͠ਐΊͨ݁Ռ

    ݱঢ়͸
    ൺֱతΧδϡΞϧͳӡ༻ͱͳ͍ͬͯΔ
    

    View Slide

25. ෼ੳج൫ͷࠓޙ
    ʙΞοτϗʔϜͳ෼ੳج൫͔Βେ౷Ұج൫΁ͷಓͷΓʙ
    ͔͜͜Βߟ࡯
    
    

    View Slide

26. ෼ੳج൫ͷࠜຊత՝୊
    (FOFSJDBOE&YUFOTJCMF
    

    View Slide

27. ෼ੳج൫ͷࠜຊత՝୊
    5'9"5FOTPS'MPX#BTFE1SPEVDUJPO4DBMF.BDIJOF-FBSOJOH1MBUGPSNΑΓ
    l5IFNBDIJOFMFBSOJOHQMBUGPSNNVTUCFHFOFSJD
    FOPVHIUPIBOEMFUIFNPTUDPNNPOTFUPGMFBSOJOH
    UBTLTBTXFMMBTCFFYUFOTJCMFUPTVQQPSUPOFP⒎
    BUZQJDBMVTFDBTFTz
    ҙ༁
    ػցֶशج൫͸แׅత͔֦ͭுՄೳͰ͋Δ΂͖
    

    View Slide

28. (FOFSJDBOE&YUFOTJWF
    ʰ࣮ݧ͔Βຊ൪ӡ༻·Ͱɺ౷߹తͳ؀ڥͰػցֶशʱ

    ͠Α͏ͱ͍ͯ͠Δ
    (FOFSJDͳΔ΂͘ଟ͘ͷϢʔεέʔεʹ ಁաతʹ
    ରԠ͍ͨ͠
    ྑ͍ந৅Խͱ"1*֦ॆͷؤுΓ͖ͬͱେม
    &YUFOTJWFྫ֎తͳέʔεʹରͯ͠΋ গͳ͍࿑ྗͰ
    ରԠ͍ͨ͠
    ೚ҙͷίʔυΛࠩ͠ࠐΊΔ৔ॴΛ༻ҙ͢Δɺͱ͔ʜ
    

    View Slide

29. (FOFSJDBOE&YUFOTJWF
    αϙʔτ͢Δ໰୊ͷόϦΤʔγϣϯͱͷઓ͍͸ଓ͘
    ෼ྨʗճؼ͘Β͍ͷந৅Խ͔Β࿙Εͯ͠·͏෦෼͕ແࢹͰ͖ͳ͍
    ྔʹͳ͍ͬͯΔ ཁग़య
    
    ج൫ͱ͍͏ܕʹ͸ΊΔ͜ͱͰࣦΘΕΔॊೈੑͷ૯ྔΛ༧ଌ͢Δ͜
    ͱ͸ग़དྷͳ͍ ཁग़య
    
    

    View Slide

30. ߟ࡯
    ج൫͸େมͳ͸ͣͳͷͰ
    ࡞Δਓͱ࢖͏ਓ͕૿͑ͯίϛϡχςΟ͕੒௕
    ͢Δඞཁ͕͋Δ
    ҰํɺνʔϜͷεέʔϧʹ࠷దԽ͞ΕͨπʔϧΛར༻ʗࣗ੡ͯ͠ࡁ·ͤ
    Δͱ͍͏બ୒ࢶ΋͋Δ
    ͦͷΑ͏ͳݱ৔͔Β͸౷Ұ͞Εͨ෼ੳج൫ʹΑͬͯ΋ͨΒ͞ΕΔϝϦο
    τ͕Πϝʔδ͠ʹ͍͘
    

    View Slide

31. ߟ࡯
    ͷͰɺ લड़ͨ͠Α͏ͳ՝୊Λ্๊͑ͨͰ
    ෼ੳج൫͕ଧͪग़͢ϝϦοτ͸Կ͔ɺΛߟ͑Δ
    ͷ͕େࣄͩͱࢥ͏
    ػցֶशΛج൫ͳ͠Ͱεέʔϧͤ͞Δͷ͕ ෆՄೳʹۙ͘
    ೉͍͔͠Βɺ

    ಋೖʹ൐͏ίετ΍՝୊Λड͚ೖΕͯͰ΋ج൫ʹैͬͯ։ൃ͢Δͱ͍͏બ୒Λ͢΂͖ɺ
    ͳͷ͔
    εέʔϧ͕೉͍͠ͱ͍͏࣮ײΛ ۀքશମ͕
    ࠓͷ࣌఺Ͱ
    ಘ͍ͯΔͷ͔
    ڞ௨ج൫Խ͢Δ͜ͱʹΑͬͯಘΒΕΔߴڃͳػೳ "#ςετɺ؂ࢹͳͲ
    

    ͷັྗΛԡ͠ग़͢ͷ͔
    ౳ʑ
    

    View Slide

32. ߟ࡯
    ʮσʔλͷྲྀΕ͕͋ΓɺγεςϜ͕σʔλͱڞʹ੒௕͢ΔΑ͏ͳγε
    ςϜʯ·Ͱࢹ໺Λ޿͛Δͱʜ
    ػցֶशΛ༻͍ͳ͍
    ݕࡧΤϯδϯ΋ٞ࿦ͷର৅

    ͱͳΔͷ͕ඞવ
    ݕࡧΤϯδϯࣗମ͸ಠࣗਐԽͯ͠ΔͷͰ৽ͨͳ

    ιϦϡʔγϣϯΛଧͪग़͢༨஍͸ͳ͍͔΋͠Εͳ͍͕ʜ
    ͱ͸͍͑χϡʔϥϧݕࡧ౳૊Έ߹ΘͤͨγεςϜʹର͢Δ

    χʔζ͸ࠓޙ૿͍͑ͯ͘͸ͣ
    ͜͜·Ͱ&YUFOTJCMFʹͳΕΔ͔ ͳΔ΂͖͔
    
    IUUQTXXXBNB[PODPKQ#VJMEJOH*OUFMMJHFOU4ZTUFNT-FBSOJOH&OHJOFFSJOHFCPPLEQ##82)3
    

    View Slide

33. ·ͱΊ
    ΫοΫύουݚڀ։ൃ෦͕ར༻͍ͯ͠Δ෼ੳج൫Λ঺հ
    ෦಺ʹج൫୲౰͕͍ͯɺϢʔεέʔεʹরΒ͠߹Θͤͳ͕Βۀ຿վળ
    ݱঢ়͸ࣗ੡πʔϧͷ૊Έ߹Θͤ%8)Ͱ͓͓ΉͶຬ଍
    ෼ੳج൫ͷ͜Ε͔Β
    ݱ৔ͱͯ͠͸ͨ͘͞ΜͷϢʔεέʔεΛൃ৴͍ͯ͘͠
    ػցֶशͷ๊͑Δ՝୊Λ۩ମԽ͠ɺϝοηʔδͱͯ͠఻͍͑ͯ͘ͷ͕

    ॏཁͰେมͦ͏
    

    View Slide