$30 off During Our Annual Pro Sale. View Details »

機械学習チームにおけるソフトウェアエンジニア〜役割、キャリア /devsum-2018-summer

Takahiko Ito
July 27, 2018
11k

機械学習チームにおけるソフトウェアエンジニア〜役割、キャリア /devsum-2018-summer

Takahiko Ito

July 27, 2018
Tweet

Transcript

  1. ػցֶशνʔϜʹ͓͚Δ
    ιϑτ΢ΣΞΤϯδχΞ
    ໾ׂɺΩϟϦΞ
    ҏ౻ܟ඙

    View Slide

  2. ࣗݾ঺հ
    • ΫοΫύουגࣜձࣾͰಇ͍͍ͯΔι
    ϑτ΢ΣΞΤϯδχΞ
    • ത࢜ʢ޻ֶʣ
    • TwitterΞΧ΢ϯτ: takahi_i
    • Φʔϓϯιʔεɿ RedPen
    2

    View Slide

  3. ུྺ
    3
    ݚڀऀ
    KDD
    PKDD
    ιϑτ΢ΣΞΤϯδχΞ
    ෼ࢄϑϨʔϜϫʔΫɿHadoop
    ݕࡧΤϯδϯɿSolrɺESɺSedueɺFAST ESP
    ݚڀपΓɿػցֶशɺϨίϝϯυɺNLP
    ࠷ۙ
    ػցֶश
    άϧʔϓ
    2007೥ 2017೥

    View Slide

  4. ຊ೔ͷτϐοΫ
    • ػցֶशνʔϜʹ͓͚Διϑτ΢ΣΞΤϯδχΞͷ໾ׂ
    • Webاۀʹ͓͚Δػցֶश෼໺ͷΩϟϦΞܗ੒
    4

    View Slide

  5. ຊ೔ͷτϐοΫ
    • ػցֶशνʔϜʹ͓͚Διϑτ΢ΣΞΤϯδχΞͷ໾ׂ
    • Webاۀʹ͓͚Δػցֶश෼໺ͷΩϟϦΞܗ੒
    5

    View Slide

  6. ४උɿػցֶशϓϩδΣΫτͷಛ௃
    • ௨ৗͷιϑτ΢ΣΞ։ൃͱҧ͍
    • ίʔυ͚ͩͰ׬݁͠ͳ͍
    • ίʔυΛಡΜͰ΋ৼΔ෣͍͕෼͔Βͳ͍ʢϞσϧʹৼΔ෣͍͕Ӆṭ͞Ε͍ͯΔʣ
    • ೖྗσʔλʹґଘ͢Δʢ͞Βʹೖྗͷ܏޲͕มԽ͢Δ͜ͱ͕͋Δʣ
    • ஌ݟΛڞ༗ʢ࣮૷Λཧղ͢ΔϝϯόΛෳ਺ἧ͑Δʣͯ҆͠શੑΛ֬อ͢Δͷ͕೉͍͠
    • ಛʹΞϧΰϦζϜࣗମ͕೉͍͠৔߹
    ࣮͸έΞ͠ͳ͚Ε͹ͳΒͳ͍෦෼͕ଟ͍ϓϩδΣΫτͱ͍͑Δ
    6

    View Slide

  7. ࠷ۙͷྲྀΕ
    • ػցֶशͷಋೖɺ؅ཧͷ೉͠͞ΛΤ
    ϯδχΞϦϯάͰରॲ
    • σʔλαΠΤϯςΟετΛαϙʔτ
    ͢ΔνʔϜฤ੒
    • MLΤϯδχΞɺσʔλΤϯδχΞ
    etc…
    • AIɺػցֶश෼໺Ͱ׆༂͢Διϑτ
    ΢ΣΞΤϯδχΞ͕૿Ճ
    7
    • Ͱ͸ɺػցֶशνʔϜͰιϑτ΢Σ
    ΞΤϯδχΞ͸Ͳ͏͍ͬͨߩݙΛ
    ͢Δ΂͖ͳͷ͔ʁ
    • ϓϩδΣΫτͷϥΠϑαΠΫϧʹ
    ஫໨͢Δͱ໾ׂ͕ݟ͑ͯ͘Δ

    View Slide

  8. ४උɿػցֶशϓϩδΣΫτͷαΠΫϧ
    ̏ͭͷεςʔδ͔ΒͳΔ
    1. ࣮ݧɿ Jupyter Notebook Λར༻
    ͯ͠୳ࡧతͳࢼߦࡨޡ
    2. ίʔυ੔ཧɿϦϑΝΫλϦϯάɺ
    ϥΠϒϥϦԽɺCI
    3. σϓϩΠ: αʔϏεԽɺCDɺ؂

    8

    View Slide

  9. ४උɿػցֶशϓϩδΣΫτͷαΠΫϧ
    9
    • ࣮ݧɺίʔυ੔ཧɺσϓϩΠ͕άϧά
    ϧճΔ
    • αΠΫϧΛճͭͭ͠ΑΓྑ͍γεςϜʹ
    ਫ਼౓޲্ɺγεςϜͷؤ݈Խ

    View Slide

  10. ѱ͍ϓϩδΣΫτɿαΠΫϧ͕ճΒͣ
    Ϟσϧ͕ݻఆ
    αΠΫϧ͕ճΒͳ͍ͱ
    • ਫ਼౓͕޲্͠ͳ͍
    • σϓϩΠίετ͕
    େ͖͍
    10

    View Slide

  11. ػցֶशνʔϜͷΤϯδχΞʹٻΊΒ
    ΕΔ͜ͱ
    • ϓϩδΣΫτͷαΠΫϧΛߴ଎ɺ
    ҆શʹճͤΔ؀ڥͷ੔උ
    • ֤εςʔδͷ໰୊ΛɺΤϯδχ
    ΞϦϯάͰղܾ
    11
    ҎԼɺ֤εςʔδʹ͓͚Δ໰୊
    ͱղܾํ๏ʹ͍ͭͯղઆͯ͠Ώ͘

    View Slide

  12. ࣮ݧεςʔδʹ͓͚Δ໰୊
    ೋͭଘࡏ͢Δ
    1. σʔλͷऔಘ
    2. ܭࢉػϦιʔε
    12

    View Slide

  13. ࣮ݧεςʔδʹ͓͚Δ໰୊
    ೋͭଘࡏ͢Δ
    1. σʔλͷऔಘ
    2. ܭࢉػϦιʔε

    View Slide

  14. σʔλऔಘʹؔ࿈͢ΔΞϯνύλʔϯ
    σʔλΛ؆୯ʹऔಘͰ͖Δ؀ڥΛߏங͢ΔલʹϦαʔνϟʔɺσʔλ
    αΠΤϯςΟετΛେྔʹޏ͏
    14
    ஫ҙ
    • σʔλαΠΤϯςΟετͷΞ΢τϓοτ͸σʔλͷऔಘίετʹґଘ͢Δ
    • σʔλऔಘίετ͕େ͖͘ർฐ͢Δ؀ڥͰ͸σʔλαΠΤϯςΟετ͸े෼ͳ
    ݁ՌΛ࢒ͤͳ͍

    View Slide

  15. σʔλʹΞΫηεͰ͖ͳ͍σʔλαΠ
    ΤϯςΟετͱ͸Կऀ͔ʁ
    ͍Θ͹ʮٰαʔϑΝʔʯ
    15
    αʔϑΝʔʹਫΛಧ͚Δඞཁ͕͋Δʂ

    View Slide

  16. σʔλऔಘ
    • σʔλͷछྨɿ
    1.σʔλϕʔεͷςʔϒϧ܈ʢϢʔβʣ
    • େ͖͞ɿ਺ϝΨόΠτʙ਺ΪΨόΠτ
    2. ϩά
    • େ͖͞ɿ਺ΪΨόΠτʙ਺ϖλόΠτ
    • σʔλαΠΤϯςΟετ͕྆ํͷσʔλʹࣗ෼ͰΞΫηεͰ͖Δඞཁ͕͋Δ
    16

    View Slide

  17. σʔλ෼ੳج൫
    • ڊେͳσʔλΛอ࣋͠ɺ؆୯ʹநग़Ͱ͖Δ
    ϓϥοτϑΥʔϜ
    • બ୒ࢶ͸ͨ͘͞Μɿ
    • ࣗࣾӡ༻ɿHiveʢHadoopʣɺSparkɺ
    Presto
    • ϗεςΟϯάαʔϏεɿBigQueryɺ
    RedshiftɺTreasureData
    17

    View Slide

  18. ࣾ಺ͷ͋ΒΏΔσʔλΛ෼ੳج൫ʹࡌ
    ͤΔ
    • ෼ੳج൫ͷબఆɺ࡞੒͕ऴྃ͠
    ͨΒσʔλΛ෼ੳج൫ʹࡌͤΔ
    • ϩάऩूπʔϧΛར༻ͯࣗ͠ಈ
    ͰσʔλΛ౤ೖ͢Δ࢓૊ΈΛ੔
    ͑Δ
    • ਓखʹΑΔσʔλ౤ೖ͸
    18

    View Slide

  19. ฐࣾࣄྫɿ෼ੳج൫
    • DWHνʔϜ͕୲౰ʢػցֶशνʔϜͱ͸ผʹଘࡏʣ
    • ඞཁͳσʔλΛSQLͰ؆୯ʹऔಘͰ͖Δ؀ڥΛ੔උͯ͠΋Β͍ͬͯ
    Δ
    • ৄ͘͠͸ɿ https://www.slideshare.net/mineroaoki/cookpad-
    techconf-2016-dwh
    19

    View Slide

  20. ࣮ݧεςʔδʹ͓͚Δ໰୊
    ೋͭଘࡏ͢Δ
    1. σʔλͷऔಘ
    2. ܭࢉػϦιʔε
    20

    View Slide

  21. ܭࢉػϦιʔεͷރׇ
    • ෳ਺ਓͰ୯ҰαʔόʹϩάΠϯͯ͠࡞ۀ
    • Ϧιʔε͕଍Γͳ࣮ͯ͘ݧ͕ਐ·ͳ͍ɻɻɻ
    21

    View Slide

  22. े෼ͳܭࢉػϦιʔεͷ֬อ
    • ػցֶशνʔϜͷϝϯόશһ͕
    շదʹ࣮ݧͰ͖Δ؀ڥΛ੔͑Δ
    • GPU͕଍Γͳ͍ͱ͔͸
    22

    View Slide

  23. ฐࣾࣄྫɿܭࢉػϦιʔε؅ཧ
    • ɹɹ @ayemos_y ࢯʹΑΔܭࢉػϦιʔε
    ͷ؅ཧ༻Slack΢ΣΞ
    • Slack্Ͱܭࢉثͷ֬อΛ͓ئ͍͢ΔͱEC2
    ΠϯελϯεΛ࡞ͬͯ͘ΕΔɻ
    • ܭࢉػ͕࢖ΘΕͳ͘ͳΔͱࣗಈͰམͱͯ͠
    ͘ΕΔʢ͓ۚͷઅ໿ʣ
    • ৄ͘͠͸ɿhttps://techlife.cookpad.com/
    entry/2017/10/26/174345
    23

    View Slide

  24. ίʔυ੔ཧεςʔδʹ͓͚Δ໰୊
    ̎ͭͷ໰୊͕͋Δ
    1.ίʔυ͕ཧղͰ͖ͳ͍
    2.ϙʔλϏϦςΟ͕ແ͍
    24

    View Slide

  25. ίʔυ੔ཧεςʔδʹ͓͚Δ໰୊
    ̎ͭͷ໰୊͕͋Δ
    1. ίʔυ͕ཧղͰ͖ͳ͍
    2. ϙʔλϏϦςΟ͕ແ͍
    25

    View Slide

  26. ࣮ݧεΫϦϓτ͕ཧղͰ͖ͳ͍
    • ঢ়گɿͳΜ͔ಈ࡞͍ͯ͠ΔΑ͏͕ͩɺϞσϧΛੜ੒͍ͯ͠Δίʔυ͕ཧղͰ
    ͖ͳ͍
    • ྫɿJupyter Notebook Λͦͷ··ίϐϖͨ͠εΫϦϓτ
    • ػցֶशΞϧΰϦζϜ͸೉͍͠㱺ίʔυ͕੔ཧ͞Ε͍ͳ͍ͱ΋ͬͱ೉͍͠
    • ίʔυͷ੔ཧ͕ඞཁ
    26
    ·ͣ͸NotebookΛεΫϦϓτʹͯ͠Ώ͘

    View Slide

  27. εΫϦϓτԽ
    • ࣮ݧεςʔδͰ࡞੒ͨ͠ Jupyter Notebook ʹهड़͞Ε͍ͯΔॲཧ
    Λ Python εΫϦϓτʹҠߦ
    • ࡞ۀɿ
    • ࣮ݧϑϩʔͷߏ଄Խɿ΂ͨॻ͖ͷॲཧ͔Βؔ਺ɺΫϥεͷநग़
    • ͋Θͤͯؤ݈ੑͷ֬อɿϦϑΝΫλϦϯάɺςετ௥Ճ
    27

    View Slide

  28. ϦϑΝΫλϦϯά
    • ϓϩάϥϜͷ֎෦͔Βݟͨಈ࡞
    Λม͑ͣʹιʔείʔυͷ಺෦
    ߏ଄Λ੔ཧ͢ΔʢWikipedia Α
    Γʣ
    • ॴײɿGitHub ΍ Qiita Ͱެ։͞
    Ε͍ͯΔػցֶशϓϩδΣΫτ
    ͸੔ཧ͞Ε͍ͯΔ΋ͷ͕গͳ͍
    ʢଞ෼໺ͷίʔυͱൺֱʣɻ
    28

    View Slide

  29. ϦϑΝΫλϦϯά߲໨
    ॳาతͳ੔ཧͰ΋ಡΈ΍͕͢͞޲্͢ΔʢCIɺCD΁ͷ෍ੴʣ
    • ؔ਺ͷ௕͞
    • ม਺ͷείʔϓ
    • ؔ਺͕औΔҾ਺ͷ਺
    • ϚδοΫφϯόʔͷఆ਺΁ͷஔ͖׵͑
    • ಉ͡ॲཧΛҰՕॴʹ·ͱΊΔ
    • ਂ͍ωετ෦෼Λؔ਺ͱͯ͠நग़͢Δ
    29

    View Slide

  30. ࣗಈςετ
    • ςετɿೖྗʹରͯ͠ظ଴ͨ͠Ξ
    ΢τϓοτʹͳ͍ͬͯΔ͔Λݕূ
    ͢Δίʔυ
    • ࠷௿ݶɿલॲཧɺEnd-to-Endͷς
    ετ͸ॻ͘
    30

    View Slide

  31. ςετͷԸܙ
    • ςετ=࢓༷
    • υΩϡϝϯτΛॻ͍ͯ΋࣌ؒͱͱ΋ʹᴥᴪ͕ੜ·ΕΔ
    • CIͰಈ࡞͢Δςετʹ͸ᴥᴪ͕ͳ͍
    • ॻ͍͓͍ͯͯ͋͛ΔͱɺϓϩδΣΫτΛҾ͖ܧ͙ਓͷཧղΛॿ͚Δ
    • ςετ͕ແ͍ػցֶशͷίʔυΛमਖ਼͢Δͷ͸ڪා
    31

    View Slide

  32. ͦͷଞͷίʔυ੔ཧεςʔδͰͷ࡞ۀ
    • ϫʔΫϑϩʔπʔϧͷಋೖ
    ʢmakeɺLuigi౳ʣ
    • ϩΨʔ௥Ճ
    • υΩϡϝϯτڧԽ
    • ܧଓతΠϯςάϨʔγϣϯͷ؀ڥ
    ੔උ
    32

    View Slide

  33. Ξϯνύλʔϯɿίʔυ੔ཧεςʔδʹ͓
    ͚Δ෼ۀ
    ୳ࡧతͳ࣮ݧ ίʔυ੔ཧ
    Ϟσϧͷ
    σϓϩΠ
    • ةݥɿϦαʔνϟ͕ݕূ࣮ͨ͠ݧ಺༰ΛΤϯδχΞ͕੔ཧͯ͠σϓϩΠ
    • ػցֶशͷίʔυ͸௨ৗͷϓϩάϥϜΑΓ΋Ҿ͖ܧ͗ίετ͕େ͖͍
    • ୭΋ϓϩδΣΫτΛཧղͰ͖ͳ͘ͳΔڪΕ͕͋Δ
    33
    33

    View Slide

  34. ฐࣾࣄྫɿίʔυ੔ཧΛϖΞͰऔΓ૊
    Ή
    • ίʔυ੔ཧ࣌ʹϦαʔνϟɺΤϯδχΞͷϖΞ
    Λ࡞Δ
    • ίʔυͷ஌ݟΛڞ༗ͭͭ͠࡞ۀ
    • νʔϜϝϯόͷίʔσΟϯάೳྗΛۉҰԽ
    ୳ࡧతͳ࣮ݧ ίʔυ੔ཧ
    Ϟσϧͷ
    σϓϩΠ
    34

    View Slide

  35. ίʔυ੔ཧͰͷ໰୊
    ̎ͭͷ໰୊͕͋Δ
    1. ίʔυ͕ཧղͰ͖ͳ͍
    2. ϙʔλϏϦςΟ͕ແ͍
    35

    View Slide

  36. εΫϦϓτΛ࣮ߦ͢Δ؀ڥ͕࡞Εͳ͍
    • ػցֶशΛѻ͏εΫϦϓτ͸ଟ਺ͷϥΠϒϥϦʹґଘ
    • PythonҎ֎ͷݴޠͰهड़͞Εͨπʔϧʹ΋ґଘʢMeCabͳͲʣ
    • ֤εςʔδʢ࣮ݧɺ੔ཧɺσϓϩΠʣ͝ͱʹҟͳΔ؀ڥʢܭࢉػʣ
    Ͱಈ࡞͢ΔͷͰ࣮ߦ؀ڥʹϙʔλϏϦςΟ͕ແ͍ͱਏ͍ɻɻɻ
    • ྫɿϩʔΧϧͰ͏·͘ಈ͍͍ͯͨεΫϦϓτ͕ຊ൪αʔόͰ͸ಈ
    ࡞͠ͳ͍
    36

    View Slide

  37. ղܾํ๏ɿDocker Λಋೖ
    • ܰྔͳԾ૝Խ؀ڥ
    • PythonϥΠϒϥϦҎ֎ͷɺґଘ͢Δ؀ڥ΋Dockerfileʹهड़Ͱ͖Δ
    • ϓϩδΣΫτͷϙʔλϏϦςΟ͕޲্
    37

    View Slide

  38. DockerͰ؀ڥΛԾ૝Խ
    38
    • ཧ૝ɿ࣮ݧஈ֊͔Β
    DockerͰ࡞ۀ
    • εςʔδ͕มΘͬͯ΋
    ࣮֬ʹεΫϦϓτ͕ಈ
    ࡞͢Δ؀ڥ͕खʹೖΔ
    • ݁ՌɺϓϩδΣΫτͷ
    αΠΫϧ͕ճ͠΍͘͢

    View Slide

  39. ͔͠͠ɺɺDocker͸ɺɺগʑ൥ࡶ
    ίϚϯυ͕௕͍ɻɻɻɻ(TдT)
    39
    ϓϩδΣΫτຖʹϙʔτϑΥϫʔυɺ
    Πϝʔδɺίϯςφ໊Λ֮͑Δඞཁ͋Δ(TдT)

    View Slide

  40. ྫɿDocker ίϚϯυ
    • Docker Πϝʔδͷ࡞੒
    • docker build -t ml-image -f ./docker/Dockerfile .
    • Dockerίϯςφͷ࡞੒
    • docker run -it -v `pwd`:/work -p 8888:8888 --name ml-image ml-
    container
    • ίϯςφͷ࡟আɺ࠶ੜ੒͢Δͨͼʹຖճಉ͡ίϚϯυΛଧͪࠐΉ…
    40

    View Slide

  41. ฐࣾࣄྫɿCookiecutter Docker Science
    • Docker؀ڥͰͷ࣮ݧʙσϓϩΠ·ͰΛαϙʔτ͢ΔCookiecutterςϯ
    ϓϨʔτΛ࡞੒
    • ΦʔϓϯιʔεϓϩδΣΫτʢ࢖ͬͯΈͯ΋Β͑Δͱخ͍͠Ͱ͢ʣ
    • URL: https://docker-science.github.io/
    • ιʔεɿhttps://github.com/docker-science/cookiecutter-docker-science
    • Cookiecutter: ϓϩδΣΫτͷςϯϓϨʔτੜ੒πʔϧ
    41

    View Slide

  42. ػೳɿCookicutter Docker Science
    • ΤϯδχΞϦϯάೳྗͷߴ͘ͳ͍ϝϯόͰ΋DockerΛѻ͍΍͘͢
    • DockerͷίϚϯυΛ make λʔήοτͰӅṭ
    • Πϝʔδ໊ɺϙʔτɺϑΝΠϧϚ΢ϯτઃఆɺίϯςφ࡞Γ௚͠ etc …
    • ࣮ݧ͔Β੔ཧɺσϓϩΠ·ͰΛҙࣝͨ͠σΟϨΫτϦߏ੒Λग़ྗ
    • σΟϨΫτϦߏ੒ͷڞ௨ԽʹΑΓϓϩδΣΫτͷݟ௨͠
    • Cookiecutter Data Science ͷߏ੒Λࢀߟʹͨ͠
    42

    View Slide

  43. ϑΝΠϧɺσΟϨΫτϦߏ੒ͷ౷Ұ
    43
    make init Ͱ S3͔Βσʔ
    λΛμ΢ϯϩʔυ
    ֶशεΫϦϓτ͕ग़ྗ͢Δ
    ϞσϧΛอ࣋
    ࣮ݧ༻ͷϊʔτϒο
    ΫΛอ࣋
    ίʔυ੔ཧ࣌ʹ࡞
    ΒΕΔϝιουɺΫϥε
    Λอ࣋
    ϓϩδΣΫτͷϫʔ
    ΫϑϩʔΛ௥ه

    View Slide

  44. Cookiecutter Docker Science ͷ࢖͍ํʢϓ
    ϩδΣΫτੜ੒ʣ
    $cookiecutter [email protected]:docker-science/cookiecutter-docker-science.git
    project_name [project_name]: image-classification
    project_slug [image_classification]:
    jupyter_host_port [8888]:
    description [Please Input a short description]: Classify images into several categories
    data_source [Please Input data source in S3]: s3://research-data/food-images
    44

    View Slide

  45. σϞ: Cookiecutter Docker Science
    • ϓϩδΣΫτͷੜ੒
    • https://asciinema.org/a/
    6XV9dNixtzfUwWdoqLj7HG7A2
    • Docker image / container ίϯςφ࡞੒
    • https://asciinema.org/a/
    06CcXPubAj3RSiMSTy3CZDrfG
    • Jupyter Notebook Λ্ཱͪ͛Δ
    45

    View Slide

  46. σϓϩΠεςʔδʹ͓͚Δ໰୊
    46
    • αʔόߏங΍όονεΫϦϓτ
    Ͱػցֶशͷ݁ՌΛσϓϩΠ͢
    Δඞཁ͕͋Δɻ
    • ػցֶशͷ݁ՌΛεϜʔζʹσ
    ϓϩΠ͢Δʹ͸ɺΑ͍ج൫ʢΠ
    ϯϑϥʣ͕ඞཁ

    View Slide

  47. ػցֶशͷ݁ՌΛσϓϩΠ͢Δίετ
    • ػցֶशνʔϜͷੜ࢈ੑ͸૊৫ͷΠϯϑϥٕ
    ज़ʹґଘ
    • ϓϩμΫγϣϯ؀ڥͰͷαʔόߏஙίετ
    ͕େ͖͍ͱɺػցֶशͷ੒ՌΛαʔϏεʹ
    ൓өͰ͖ͳ͍
    • جຊɿػցֶशνʔϜͷϝϯόࣗ਎͕
    GitHub ΍ GHE ʹϓϧϦΫΤετΛग़͢͜ͱ
    ͰɺϓϩμΫγϣϯ؀ڥʹαʔόΛߏஙͰ͖
    Δ
    47

    View Slide

  48. ฐࣾࣄྫɿσϓϩΠͷޮ཰Խ
    • αʔό؅ཧ
    • ECSͷར༻ʢKubernates΋։࢝ʣ
    • Πϯϑϥ෦ʹΑͬͯ੔උ͞Ε͍ͯΔ
    • ػցֶशνʔϜࣗ਎ͰϓϩμΫγϣϯ؀ڥʹ
    αʔόΛߏஙͰ͖Δ؀ڥ͕ఏڙ͞Ε͍ͯΔ
    • ඞཁͳ࡞ۀɿઃఆϑΝΠϧΛϨϙδτϦʹ௥
    Ճ͢Δ͚ͩʢهड़಺༰ɿೝূɺαʔόͷੑೳ
    etcʣ
    48
    • chie8842 ࢯʹΑͬͯػցֶशϓϩδΣ
    ΫτͷઃఆΛڞ௨Խ͕ਐΜͰ͍Δ
    • খ͞ͳϓϩδΣΫτͰ͋Ε͹ڞ௨ͷઃఆΛ
    ࢖͏
    • ೝূɺதؒσʔλஔ͖৔ etc
    • σϓϩΠϑϩʔͷ؆ૉԽ
    • هड़͕ඞཁͳઃఆ߲໨Λѹॖ

    View Slide

  49. ฐࣾࣄྫɿKelner
    • @_lunardog_ ࢯ͕։ൃ͍ͯ͠ΔOSSϓϩδΣΫτ
    • ര଎Ͱਂ૚ֶशͷϞσϧΛσϓϩΠ͢Δπʔϧ
    • URL: https://github.com/lunardog/kelner
    49

    View Slide

  50. ·ͱΊɿػցֶशνʔϜʹ͓͚ΔΤϯ
    δχΞͷ໾ׂ
    • ػցֶशΛαʔϏεͰར༻͢Δʹ͸έΞ͢Δ෦෼͕୔ࢁ
    • ΞϧΰϦζϜɺσʔλͷऔಘɺίʔυͷ඼࣭ɺσϓϩΠ etc
    • ࣮ݧεςʔδͰ͸ݚڀೳྗ͕ཁٻ͞Εɺίʔυ੔ཧҎ߱Ͱ͸Τϯ
    δχΞϦϯάೳྗ͕ཁٻ
    • ݫີʹ෼ۀମ੍Λங͘ͷ͸೉͍ͨ͠Ίɺ֤ࣗͷೳྗͷ overlap ෦෼
    Λ૿΍͢ͷ͕ॏཁʢϨϏϡʔɺϖΞϓϩͰ஌ݟڞ༗ͳͲʣ
    50

    View Slide

  51. ຊ೔ͷτϐοΫ
    • ػցֶशνʔϜʹ͓͚Διϑτ΢ΣΞΤϯδχΞͷ໾ׂ
    • Webاۀʹ͓͚Δػցֶश෼໺ͷΩϟϦΞܗ੒
    51

    View Slide

  52. νʔϜن໛ʹΑͬͯҟͳΔΩϟϦΞ
    • ಉ͡MLνʔϜͰ΋νʔϜن໛ʹΑͬͯΩϟϦΞܗ੒͸ҟͳΔ
    • ࣗ෼͕ͳΓ͍ͨΤϯδχΞ૾ʹϚον͢Δن໛Λબ୒͢Δͷ͕Α͍
    52

    View Slide

  53. খ͞ͳMLνʔϜͷϝϯό
    • ҰਓͰଟ༷ͳλεΫʹରॲ
    • ػցֶशϓϩδΣΫτͷαΠΫϧ͢΂ͯʹߩݙ͢Δ
    • σʔλऩूɺίʔυ੔ཧɺαʔόߏஙɺϞχλϦϯά
    • ϚϧνελοΫԽ
    • ػցֶशࣗମͷΞϧΰϦζϜΛಥ͖٧ΊΔ࣌ؒ͋·ΓऔΕͳ͍
    53

    View Slide

  54. େ͖ͳMLνʔϜͷϝϯό
    • ෼ۀମ੍ͷ੔උ
    • ઐ໳ʹಛԽͨ͠ܦݧΛੵΊΔ
    • ྫɿ
    • Ϧαʔνϟɿ৽͍͠ΞϧΰϦζϜͷఏҊɺ࿦จ
    • ΠϯϑϥΤϯδχΞɿΤϨΨϯτͳMLσϓϩΠϑϨʔϜϫʔΫͷ
    ಋೖɺ֦ு
    54

    View Slide

  55. ͨͩ͠ɺͲͪΒͷ৔߹Ͱ΋
    Ͳͷϝϯό΋࠷௿ݶ͍࣋ͬͯΔ΂͖ٕज़ελοΫ͸͋Δ
    • Git ͷͪΐͬͱৄ͍͠࢖͍ํ
    • rebaseɺstashɺίϛοτ·ͱΊʢsquashʣɺbisect etc…
    • Ծ૝ԽɿDocker
    • ίʔυͷ඼࣭΁ͷέΞ: ςετۦಈ։ൃ
    • Issue؅ཧɿJiraɺRedmine
    ϝϯόશһ͕Ͱ͖ΔͱϓϩδΣΫτΛ؅ཧ͠΍͍͢
    55

    View Slide

  56. ฐࣾࣄྫɿػցֶशνʔϜΛߏ੒͢Δ
    ϝϯό
    • த͙Β͍ͷن໛ʢ߹ܭ໊̔ʣ
    • େ͖͘෼͚ͯೋछྨͷ໾ׂ͕͋Δ
    1. ػցֶशΤϯδχΞɿػցֶशʴαʔϏε΁ͷΠϯςάϨʔγϣϯ
    ஫ɿϦαʔνϟدΓɺΤϯδχΞدΓͱ֤ࣗͷಘҙ෼໺͸ҧ͏
    2. ΠϯϑϥΤϯδχΞɿػցֶशͷج൫Λ࡞੒
    • ͜ͷଞɺνʔϜ֎ʹ෼ੳج൫ɺશࣾΠϯϑϥνʔϜ͕͋Γଟ͘ͷαϙʔτΛ΋
    Β͍ͬͯΔ
    56

    View Slide

  57. ฐࣾࣄྫɿػցֶशνʔϜϝϯόͷ໾
    ׂ
    • զʑͷن໛Ͱ֤͕ࣗ໾ׂʹϚον͢Δ࢓ࣄ͚ͩΛ͢Δͱ૊৫͕ඇ
    ޮ཰
    • ͔ͳΓྲྀಈతͳ໾ׂ෼୲ɻओۀ຿ͷ໾ׂҎ֎΋ੵۃతʹ୲౰
    • ྫɿΠϯϑϥΛओۀ຿ʹ͢Δϝϯό͕σʔλ෼ੳΛ୲౰
    • ҙਤɿ૊৫಺Ͱ֤ϝϯό͕ߩݙͰ͖Δ෯Λ૿΍͢
    57

    View Slide

  58. ػցֶशʢR&Dʣʹ͓͚ΔΩϟϦΞܗ੒ͷ
    ೉͠͞
    • Πϯϑϥ΍αʔϏε։ൃͱͷҧ͍
    • ػցֶशࣗମ͸ແͯ͘΋αʔϏε͸੒Γཱͭ
    • ػցֶश͸αʔϏεΛΑΓྑ͘͢Δٕज़
    • ձ͕ࣾظ଴͢ΔχʔζͱϚον͢Δ੒ՌΛग़͢Α͏ҙࣝ͠ͳ͍ͱ͓ՙ
    ෺૊৫ʹͳΓ΍͍͢ → νʔϜղମ ʗ(^o^)ʘ
    • ؀ڥʹΑΔधཁͷมಈ͕େ͖͘ɺҰ؏ͨ͠ΩϟϦΞΛங͖ʹ͍͘ɻɻɻ
    58

    View Slide

  59. ػցֶशνʔϜϝϯόͷੜଘઓུ
    • ෆ҆ఆͳ૊৫ͳͷͰɺ͋Δఔ౓͸ੜଘઓུΛҙࣝͨ͠ํ͕Α͍
    • ML/AIधཁ͕ʢҰ࣌తʹʣݮਰͯ͠΋Ձ஋ΛࣦΘͳ͍Α͏ʹඋ͑Δ
    • ํ޲ੑ͸̎ͭ͋ΔʢϚϧνελοΫԽɺҰ఺ಥഁʣ
    59

    View Slide

  60. ੜଘઓུ̍ɿؔ࿈ٕज़ͷशಘ
    • ػցֶशͷεϜʔζͳಋೖʹ͸ଟ༷ͳؔ࿈ٕज़͕ඞཁ
    • ػցֶशʹؔ࿈͢Δٕज़΋शಘͯ͠ϚϧνελοΫԽ
    • ػցֶशҎ֎ʹؔ࿈෼໺Ͱ΋ڧΈΛ͍࣋ͬͯΔͱΑ͍
    • ྫɿtakahi-i ݕࡧΤϯδϯʢSolrɺElasticsearchʣ͕ŧŔŕŪũƄŝſ
    • ػցֶशҎ֎Ͱ΋ձࣾʹߩݙͰ͖ΔΑ͏ʹ
    60

    View Slide

  61. ػցֶशͷपลͰར༻͞ΕΔٕज़ͷҰ෦
    61
    ػցֶश

    View Slide

  62. ετϨʔδʹؔ͢Δٕज़
    ػցֶश
    • ֶशث͕ग़ྗ͢Δ݁Ռ͸ετϨʔδʹอ࣋
    ͞Ε্ͨͰαʔϏεͰར༻͞ΕΔ
    • ݕࡧΤϯδϯɿSolrɺElasticsearch
    • σʔλϕʔεɿMySQLɺPostgreSQL
    • ·ͣ͸εΩʔϚɺςʔϒϧઃܭΛͰ͖ΔΑ
    ͏ʹ
    62

    View Slide

  63. Πϯϑϥʹؔ͢Δٕज़
    63
    ػցֶश
    • ػցֶशͷαʔϏεಋೖΛޮ཰Խ͢Δͷʹར༻Ͱ
    ͖Δଟ༷ͳΠϯϑϥٕज़͕ଘࡏ͢Δ
    • ϫʔΫϑϩʔ؅ཧɿAirflow
    • ߏ੒؅ཧɿAnsibleɺChef
    • αʔό؅ཧɿDockerɺKubernatesɺECS
    • Πϯϑϥ͕ίʔυԽ͞Εͨ؀ڥͩͱɺػցֶशΤ
    ϯδχΞࣗ਎Ͱ͜ͷ͋ͨΓͷ࡞ۀ͕Ͱ͖Δ

    View Slide

  64. αʔϏε։ൃͰར༻͢Δٕज़
    64
    • σʔλϕʔεʹػցֶशͷΞϊςʔγϣϯ݁ՌΛอ࣋ͨ͠ޙɺ
    αʔϏεͰར༻͢ΔͨΊͷΠϯςάϨʔγϣϯ࡞ۀ͕ඞཁ
    • MVCϑϨʔϜϫʔΫʢRailsͳͲʣ
    • ·ͣ͸Ϟσϧɺίϯτϩʔϥ࡞੒Λࣗ෼Ͱ࡞Δ
    • ϑϩϯτͷௐ੔ɿJavaScriptɺCSS
    • ਐԽͷ଎౓͕ૣ͘ΩϟονΞοϓ͕େม͕ͩɺशಘ͢Δͱ
    αʔϏεʹಋೖ͠΍͘͢ͳΔ
    • ྫɿES̒ɺTypeScriptɺ ReactɺVueɺwebpack etc …
    ػցֶश

    View Slide

  65. ෼ੳج൫Ͱར༻͢Δٕज़
    65
    • ॳาɿػցֶशͷσʔλΛ෼ੳج൫͔Βऔಘ͢ΔʢSQLʣ
    • தڃɿଟ༷ͳϏοΫσʔλϑϨʔϜϫʔΫΛ࢖͏
    • PythonɺSQL͚ͩͰͳ͘ɺ෼ࢄϑϨʔϜϫʔΫ
    ʢHadoopɺSparkʣ্ͷίʔυ΋ॻ͘ܦݧΛੵΉ
    • ཧղΛਂΊΔͨΊʹझຯͰΞϧΰϦζϜΛ࣮૷ͯ͠ެ։
    • ࣗ෼͕ੲ࡞ͬͨ΋ͷʢLSH ͷұ࣮૷ʣɿ https://
    github.com/takahi-i/likelike
    • ཧ૝ɿ෼ੳج൫ͷվળʹ΋ߩݙ
    ػցֶश

    View Slide

  66. ஫ҙɿ਺ֶɺ෺ཧֶ෼໺ͷम࢜ɺത͔࢜Βσʔ
    λαΠΤϯςΟετʹస਎͕ਐΜͰ͍Δɻ
    த్൒୺ͳཧղɺΞ΢τϓοτͰੜ͖࢒Δͷ͸
    ݫ͍͠ɻ
    ੜଘઓུ̎ɿػցֶशΛಥ͖٧ΊΔ
    • ෼໺ͱͯ͠ͷ೤͕ऩ·ͬͯ΋τοϓͷधཁ͸
    ৗʹଘࡏ͠ଓ͚Δ
    • ػցֶश෼໺ʹ͓͍ͯɺ௒ҰྲྀͰ͋Δ͜ͱΛ
    ࣔ͢
    • ఆظతʢԿ೥͔ʹҰճʣʹ Top ΧϯϑΝϨ
    ϯεʹ࠾࿥͞ΕΔ
    • NIPSɺICMLɺICCVɺCVPRɺCOLT etc…
    • Kaggle grand master
    66

    View Slide

  67. ·ͱΊ
    • ػցֶशνʔϜʹ͓͚ΔΤϯδχΞͷ໾ׂʹ͍ͭͯղ

    • ϓϩδΣΫτͷ֤εςʔδʢ࣮ݧɺ੔ཧɺσϓϩΠʣ
    ͷ໰୊Λղܾ͢Δ
    • ؔ࿈͢ΔνʔϜɿ෼ੳج൫ɺΠϯϑϥ͕ॏཁͳ໾ׂ
    • ฐࣾͷࣄྫΛ͍͔ͭ͘঺հ
    • Webاۀʹ͓͚Δػցֶश෼໺ͷΩϟϦΞܗ੒ʹ͍ͭ
    ͯ঺հ
    67

    View Slide

  68. ืूɿݚڀ։ൃ෦໳ͷΞϓϦέʔγϣ
    ϯΤϯδχΞ
    • ืूதͰ͢
    • ݚڀͷ੒ՌͱΫοΫύουͷ࣮αʔϏεΛڮ౉͠Λ͢ΔϙδγϣϯͰ͢
    • PoCͰ͸ͳ࣮͘αʔϏεɺΞϓϦέʔγϣϯͷ։ൃ
    • ෼໺ɿػցֶशɺεϚʔτΩονϯ
    • ৄ͘͠͸ͪ͜ΒΛ͓ಡΈ͍ͩ͘͞ ɿhttps://
    cookpad.wd3.myworkdayjobs.com/en-US/jobs/job/Tokyo--Japan/--_R-001087-31
    68

    View Slide

  69. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠
    69

    View Slide