Upgrade to Pro — share decks privately, control downloads, hide ads and more …

データ分析と Docker / Data Analysis with Docker

データ分析と Docker / Data Analysis with Docker

ちゅらデータ社内 LT

Yuki Ishikawa

August 24, 2018
Tweet

More Decks by Yuki Ishikawa

Other Decks in Technology

Transcript

  1. σʔλ෼ੳͱ Docker
    2018.08.24 ͪΎΒσʔλࣾ಺ LT
    @hoto17296

    View Slide

  2. എܠ
    • ࣾ಺ͷ GPU Ϛγϯ͕ Docker ͱ͍͏Կ͔Ͱ

    ಈ͍͍ͯΔΒ͍͕͠ͿͬͪΌ͚Α͘Θ͔Βͳ͍
    • ๭ΧΤϧͷਓ͕΍ͨΒŪƄŕŜŖŪƄŕŜŖݴͬͯΔ

    View Slide

  3. Yuki Ishikawa
    • @hoto17296
    • ๭ΧΤϧͷਓͰ͢
    • ઌिϕϏʔ࢈·Ε·ͨ͠

    View Slide

  4. ࠷ۙ Qiita ʹ
    ॻ͍ͨهࣄ
    Docker
    Docker
    Docker
    Docker
    Docker
    ͍͍ͩͨ Docker

    View Slide

  5. ࠓ೔ͷςʔϚ
    ŪƄŕŜŖ͓͡͞Μ͕ڭ͑ΔŪƄŕŜŖ

    View Slide

  6. ࿩͢͜ͱ
    • Docker ֓ཁ
    • ʮίϯςφʯͱʮΠϝʔδʯ
    • ࣾ಺ GPU αʔό͸Ͳ͏ಈ͍͍ͯΔ͔

    View Slide

  7. ࿩͞ͳ͍͜ͱ
    • Dockerfile ͰͷΠϝʔδͷ࡞Γํ
    • ίϯςφϨδετϦ
    • Volume ͱ Network
    • Docker Compose
    • ΦʔέετϨʔγϣϯ

    View Slide

  8. Docker ֓ཁ

    View Slide

  9. Docker ͷಛ௃ (1/2)
    • ίϯςφܕͷԾ૝Խ
    • ϗετͱΧʔωϧΛڞ༗
    • Linux Ͱ͔͠ಈ͔ͳ͍ (※)
    • ଎͍
    ※ Windows, macOS ͷ Docker ͸ Linux VM ͷ্Ͱಈ͍͍ͯΔ

    View Slide

  10. Docker ͷಛ௃ (2/2)
    • ϙʔλϒϧ
    • ϗετϚγϯʹґଘ͠ͳ͍ͷͰͲ͜Ͱ΋ಈ͘
    • ϩʔΧϧͰ࡞ͬͨΠϝʔδΛͦͷ··ຊ൪؀ڥʹσϓϩΠ
    • ؆୯ʹίϯςφΛ࡞ͬͯյͤΔ
    • ؀ڥߏஙͷίετ͕͔͔Βͳ͍
    • (αʔϏε։ൃ) ॊೈͳσϓϩΠϑϩʔ͕૊ΊΔ
    • (σʔλ෼ੳ) ࣮ݧ؀ڥΛ޷͖ͳ͚ͩ࡞ͬͯյͤΔ

    View Slide

  11. ͱΓ֮͋͑ͣ͑ͯ΄͍͜͠ͱ
    • ʮΠϝʔδʯͱʮίϯςφʯ
    • Πϝʔδɿ ࣮ߦͰ͖ΔϑΝΠϧ (ͷΑ͏ͳ΋ͷ)
    • ίϯςφɿ ࣮ߦதͷϓϩηε (ͷΑ͏ͳ΋ͷ)
    • Docker ʹ͍ͭͯߟ͑Δͱ͖ɺ

    Πϝʔδͷ࿩ͳͷ͔ίϯςφͷ࿩ͳͷ͔

    Λҙࣝͨ͠΄͏͕͍͍

    View Slide

  12. Πϝʔδ 1
    ίϯςφ
    run
    commit
    Πϝʔδ 2
    ʮΠϝʔδ 1ʯ͔Β
    ίϯςφΛ࡞੒࣮ͯ͠ߦ͢Δ
    ίϯςφʹมߋΛՃ͑Δ
    (ϞδϡʔϧΛΠϯετʔϧ͢Δ౳)
    มߋΛՃ͑ͨίϯςφΛ
    ʮΠϝʔδ 2ʯͱ͍͏໊લͰอଘ͢Δ
    ※ ࣮ࡍʹΠϝʔδΛ࡞Δࡍʹ͸ commit Λ௚઀࣮ߦ͢Δ͜ͱ͸ͳ͘ɺ

    ɹ୅ΘΓʹ Dockerfile Λهड़͢Δͷ͕ͩɺͦΕʹ͍ͭͯ͸ࠓճ͸ׂѪ

    View Slide

  13. Πϝʔδͷछྨ
    • ެࣜΠϝʔδ
    • DockerHub Ͱެ։͞Ε͍ͯΔ
    • Ubuntu ͱ͔ Ruby ͱ͔ MySQL ͱ͔
    • ඇެࣜΠϝʔδ
    • ୭͔͕࡞ͬͨΠϝʔδ
    • DockerHub ΍ͦͷଞͷϨδετϦ
    • ϓϥΠϕʔτͳΠϝʔδ
    • ࣗ෼ͰϏϧυͨ͠Πϝʔδ

    View Slide

  14. Πϝʔδ͸มߋࠩ෼ͷੵΈॏͶ
    ubuntu:16.04
    php:7
    scratch
    wordpress:4
    Ubuntu ΛΠϯετʔϧ
    PHP ΛΠϯετʔϧ
    WordPress ΛΠϯετʔϧ
    ↑ ΠϝʔδΛ࢖͏ (ίϯςφΛ࡞Δ) ਓ͸ϨΠϠΛҙࣝ͢Δඞཁ͸ͳ͍
    ʮΑ͘Θ͔ΒΜ͚Ͳ WordPress ͕ಈ͘Πϝʔδʯͱͯ͠࢖͑Δ

    View Slide

  15. ίϯςφͷมߋ͸อଘ͞Εͳ͍
    • ίϯςφΛফͨ͠Βมߋ͸શ෦ফ͑Δ
    • มߋΛӬଓԽ͍ͨ͠৔߹
    • commit ͯ͠৽͍͠ΠϝʔδΛ࡞Δ
    • σʔλΛӬଓԽ͍ͨ͠৔߹ (ϩάͱ͔DBͱ͔)
    • ϘϦϡʔϜΛϚ΢ϯτ͢Δ

    View Slide

  16. ศརͳࣄྫɿ GitLab (1/2)
    • (ࣾ಺ GitLab ͸ Docker Ͱಈ͍ͯΔ)
    • ैདྷͷ GitLab Πϯετʔϧํ๏
    • Ruby, Node, Nginx, PostgreSQL, Redis,

    Sidekiq, GitLab ຊମ ΛΠϯετʔϧ
    • ͦΕͧΕΛద੾ʹಈ࡞͢ΔΑ͏ʹઃఆ
    • ͭΒ͍

    View Slide

  17. ศརͳࣄྫɿ GitLab (2/2)
    • Docker Λ࢖ͬͨ GitLab Πϯετʔϧํ๏
    • GitLab ެࣜΠϝʔδΛ pull ͯ͘͠Δ
    • σʔλΛอଘ͢ΔͨΊͷϘϦϡʔϜΛ༻ҙ
    • ίϯςφΛىಈ͢Δ
    • ؆୯ !!!
    • ʮΑ͘Θ͔ΒΜ͚Ͳ GitLab ͕ಈ͘Πϝʔδʯ

    ͕࢖͑Δ → ϙʔλϏϦςΟ

    View Slide

  18. ࣾ಺ GPU αʔόʹ͍ͭͯ

    View Slide

  19. Docker Ͱ৭ʑಈ͍͍ͯΔ
    • μογϡϘʔυ (80)
    • TensorFlow Notebook (18888)
    • DataScience Notebook (28888)
    • PySpark Notebook (38888)
    • PostgreSQL (5432)

    View Slide

  20. (࠶) ίϯςφͷมߋ͸อଘ͞Εͳ͍
    • ίϯςφΛফͨ͠Βมߋ͸શ෦ফ͑Δ
    • ௥ՃͰϞδϡʔϧΛΠϯετʔϧͯ͠΋

    αʔόΛ࠶ىಈͨ͠Βશ෦ফ͑Δ
    • ॳظઃఆεΫϦϓτ༻ҙ͍ͯͩ͘͠͞
    • ͨͩ͠ Notebook ͸ফ͑ͳ͍
    • ϗετͷϘϦϡʔϜΛϚ΢ϯτ͍ͯ͠Δҝ

    View Slide

  21. ͳΜͰͦΜͳ͜ͱ͢Δඞཁ͕ʁ
    • ʮ؀ڥΛ࠶ݱͰ͖ΔʯΑ͏ʹ͢ΔͨΊ
    • ࢼߦࡨޡͰ؀ڥ͙ͪΌ͙ͪΌʹͯ͠

    ʮ࠶ݱͰ͖ͳ͍͚Ͳ͜ͷ؀ڥͳΒ

    ɹྑ͍෼ੳ݁Ռ͕ग़ͤ·͢ʂʯ

    ͍ͬͯ͏ͷ࠷ѱͰ͸ʁ
    • ʮͲ͏΍ͬͨΒ͜ͷ؀ڥΛ࠶ݱͰ͖Δ͔ʯΛ

    ৗʹҙࣝ͠ͳ͕Β࣮ݧ͠·͠ΐ͏

    View Slide

  22. ྫɿ ๭෼ੳҊ݅
    • GPU αʔόͷ Jupyter Ͱ෼ੳ͍ͯͨ͠
    • ؀ڥΛ࠶ݱ͢ΔεΫϦϓτΛॻ͖ͳ͕Β෼ੳͨ͠
    • Ϟδϡʔϧͷڍಈ͕͓͔͘͠ͳͬͨͱ͖ʹ

    ʮίϯςφյͯ͠࠷ॳ͔Β࡞Γ௚ͯ͠ΈΑ͏ʯ

    Λ͙͢ʹ࣮ߦͯ͠௚ͤͨ
    • ೲ඼෺ʹॳظઃఆεΫϦϓτΛೖΕΔ͚ͩͰ

    ͦΕ͕؀ڥߏஙखॱॻʹͳͬͨ

    View Slide

  23. ·ͱΊ

    View Slide

  24. ·ͱΊ
    • Docker ͸࣮ݧͷ༑
    • ؆୯ʹ࡞ͬͯյͤΔ
    • ؀ڥͷ࠶ݱੑΛ୲อͰ͖Δ
    • ࢖͍͜ͳ͍͖ͯ͠·͠ΐ͏
    • ڭ͑ΔͷͰͳΜͰ΋㘤͍ͯʂʂʂ

    View Slide