データ分析と Docker / Data Analysis with Docker

ちゅらデータ社内 LT

Transcript

1. σʔλ෼ੳͱ Docker
   2018.08.24 ͪΎΒσʔλࣾ಺ LT
   @hoto17296
   

   View Slide

2. എܠ
   • ࣾ಺ͷ GPU Ϛγϯ͕ Docker ͱ͍͏Կ͔Ͱ

   ಈ͍͍ͯΔΒ͍͕͠ͿͬͪΌ͚Α͘Θ͔Βͳ͍
   • ๭ΧΤϧͷਓ͕΍ͨΒŪƄŕŜŖŪƄŕŜŖݴͬͯΔ
   

   View Slide

3. Yuki Ishikawa
   • @hoto17296
   • ๭ΧΤϧͷਓͰ͢
   • ઌिϕϏʔ࢈·Ε·ͨ͠
   

   View Slide

4. ࠷ۙ Qiita ʹ
   ॻ͍ͨهࣄ
   Docker
   Docker
   Docker
   Docker
   Docker
   ͍͍ͩͨ Docker
   

   View Slide

5. ࠓ೔ͷςʔϚ
   ŪƄŕŜŖ͓͡͞Μ͕ڭ͑ΔŪƄŕŜŖ
   

   View Slide

6. ࿩͢͜ͱ
   • Docker ֓ཁ
   • ʮίϯςφʯͱʮΠϝʔδʯ
   • ࣾ಺ GPU αʔό͸Ͳ͏ಈ͍͍ͯΔ͔
   

   View Slide

7. ࿩͞ͳ͍͜ͱ
   • Dockerfile ͰͷΠϝʔδͷ࡞Γํ
   • ίϯςφϨδετϦ
   • Volume ͱ Network
   • Docker Compose
   • ΦʔέετϨʔγϣϯ
   

   View Slide

8. Docker ֓ཁ
   

   View Slide

9. Docker ͷಛ௃ (1/2)
   • ίϯςφܕͷԾ૝Խ
   • ϗετͱΧʔωϧΛڞ༗
   • Linux Ͱ͔͠ಈ͔ͳ͍ (※)
   • ଎͍
   ※ Windows, macOS ͷ Docker ͸ Linux VM ͷ্Ͱಈ͍͍ͯΔ
   

   View Slide

10. Docker ͷಛ௃ (2/2)
    • ϙʔλϒϧ
    • ϗετϚγϯʹґଘ͠ͳ͍ͷͰͲ͜Ͱ΋ಈ͘
    • ϩʔΧϧͰ࡞ͬͨΠϝʔδΛͦͷ··ຊ൪؀ڥʹσϓϩΠ
    • ؆୯ʹίϯςφΛ࡞ͬͯյͤΔ
    • ؀ڥߏஙͷίετ͕͔͔Βͳ͍
    • (αʔϏε։ൃ) ॊೈͳσϓϩΠϑϩʔ͕૊ΊΔ
    • (σʔλ෼ੳ) ࣮ݧ؀ڥΛ޷͖ͳ͚ͩ࡞ͬͯյͤΔ
    

    View Slide

11. ͱΓ֮͋͑ͣ͑ͯ΄͍͜͠ͱ
    • ʮΠϝʔδʯͱʮίϯςφʯ
    • Πϝʔδɿ ࣮ߦͰ͖ΔϑΝΠϧ (ͷΑ͏ͳ΋ͷ)
    • ίϯςφɿ ࣮ߦதͷϓϩηε (ͷΑ͏ͳ΋ͷ)
    • Docker ʹ͍ͭͯߟ͑Δͱ͖ɺ

    Πϝʔδͷ࿩ͳͷ͔ίϯςφͷ࿩ͳͷ͔

    Λҙࣝͨ͠΄͏͕͍͍
    

    View Slide

12. Πϝʔδ 1
    ίϯςφ
    run
    commit
    Πϝʔδ 2
    ʮΠϝʔδ 1ʯ͔Β
    ίϯςφΛ࡞੒࣮ͯ͠ߦ͢Δ
    ίϯςφʹมߋΛՃ͑Δ
    (ϞδϡʔϧΛΠϯετʔϧ͢Δ౳)
    มߋΛՃ͑ͨίϯςφΛ
    ʮΠϝʔδ 2ʯͱ͍͏໊લͰอଘ͢Δ
    ※ ࣮ࡍʹΠϝʔδΛ࡞Δࡍʹ͸ commit Λ௚઀࣮ߦ͢Δ͜ͱ͸ͳ͘ɺ

    ɹ୅ΘΓʹ Dockerfile Λهड़͢Δͷ͕ͩɺͦΕʹ͍ͭͯ͸ࠓճ͸ׂѪ
    

    View Slide

13. Πϝʔδͷछྨ
    • ެࣜΠϝʔδ
    • DockerHub Ͱެ։͞Ε͍ͯΔ
    • Ubuntu ͱ͔ Ruby ͱ͔ MySQL ͱ͔
    • ඇެࣜΠϝʔδ
    • ୭͔͕࡞ͬͨΠϝʔδ
    • DockerHub ΍ͦͷଞͷϨδετϦ
    • ϓϥΠϕʔτͳΠϝʔδ
    • ࣗ෼ͰϏϧυͨ͠Πϝʔδ
    

    View Slide

14. Πϝʔδ͸มߋࠩ෼ͷੵΈॏͶ
    ubuntu:16.04
    php:7
    scratch
    wordpress:4
    Ubuntu ΛΠϯετʔϧ
    PHP ΛΠϯετʔϧ
    WordPress ΛΠϯετʔϧ
    ↑ ΠϝʔδΛ࢖͏ (ίϯςφΛ࡞Δ) ਓ͸ϨΠϠΛҙࣝ͢Δඞཁ͸ͳ͍
    ʮΑ͘Θ͔ΒΜ͚Ͳ WordPress ͕ಈ͘Πϝʔδʯͱͯ͠࢖͑Δ
    

    View Slide

15. ίϯςφͷมߋ͸อଘ͞Εͳ͍
    • ίϯςφΛফͨ͠Βมߋ͸શ෦ফ͑Δ
    • มߋΛӬଓԽ͍ͨ͠৔߹
    • commit ͯ͠৽͍͠ΠϝʔδΛ࡞Δ
    • σʔλΛӬଓԽ͍ͨ͠৔߹ (ϩάͱ͔DBͱ͔)
    • ϘϦϡʔϜΛϚ΢ϯτ͢Δ
    

    View Slide

16. ศརͳࣄྫɿ GitLab (1/2)
    • (ࣾ಺ GitLab ͸ Docker Ͱಈ͍ͯΔ)
    • ैདྷͷ GitLab Πϯετʔϧํ๏
    • Ruby, Node, Nginx, PostgreSQL, Redis,

    Sidekiq, GitLab ຊମ ΛΠϯετʔϧ
    • ͦΕͧΕΛద੾ʹಈ࡞͢ΔΑ͏ʹઃఆ
    • ͭΒ͍
    

    View Slide

17. ศརͳࣄྫɿ GitLab (2/2)
    • Docker Λ࢖ͬͨ GitLab Πϯετʔϧํ๏
    • GitLab ެࣜΠϝʔδΛ pull ͯ͘͠Δ
    • σʔλΛอଘ͢ΔͨΊͷϘϦϡʔϜΛ༻ҙ
    • ίϯςφΛىಈ͢Δ
    • ؆୯ !!!
    • ʮΑ͘Θ͔ΒΜ͚Ͳ GitLab ͕ಈ͘Πϝʔδʯ

    ͕࢖͑Δ → ϙʔλϏϦςΟ
    

    View Slide

18. ࣾ಺ GPU αʔόʹ͍ͭͯ
    

    View Slide

19. Docker Ͱ৭ʑಈ͍͍ͯΔ
    • μογϡϘʔυ (80)
    • TensorFlow Notebook (18888)
    • DataScience Notebook (28888)
    • PySpark Notebook (38888)
    • PostgreSQL (5432)
    

    View Slide

20. (࠶) ίϯςφͷมߋ͸อଘ͞Εͳ͍
    • ίϯςφΛফͨ͠Βมߋ͸શ෦ফ͑Δ
    • ௥ՃͰϞδϡʔϧΛΠϯετʔϧͯ͠΋

    αʔόΛ࠶ىಈͨ͠Βશ෦ফ͑Δ
    • ॳظઃఆεΫϦϓτ༻ҙ͍ͯͩ͘͠͞
    • ͨͩ͠ Notebook ͸ফ͑ͳ͍
    • ϗετͷϘϦϡʔϜΛϚ΢ϯτ͍ͯ͠Δҝ
    

    View Slide

21. ͳΜͰͦΜͳ͜ͱ͢Δඞཁ͕ʁ
    • ʮ؀ڥΛ࠶ݱͰ͖ΔʯΑ͏ʹ͢ΔͨΊ
    • ࢼߦࡨޡͰ؀ڥ͙ͪΌ͙ͪΌʹͯ͠

    ʮ࠶ݱͰ͖ͳ͍͚Ͳ͜ͷ؀ڥͳΒ

    ɹྑ͍෼ੳ݁Ռ͕ग़ͤ·͢ʂʯ

    ͍ͬͯ͏ͷ࠷ѱͰ͸ʁ
    • ʮͲ͏΍ͬͨΒ͜ͷ؀ڥΛ࠶ݱͰ͖Δ͔ʯΛ

    ৗʹҙࣝ͠ͳ͕Β࣮ݧ͠·͠ΐ͏
    

    View Slide

22. ྫɿ ๭෼ੳҊ݅
    • GPU αʔόͷ Jupyter Ͱ෼ੳ͍ͯͨ͠
    • ؀ڥΛ࠶ݱ͢ΔεΫϦϓτΛॻ͖ͳ͕Β෼ੳͨ͠
    • Ϟδϡʔϧͷڍಈ͕͓͔͘͠ͳͬͨͱ͖ʹ

    ʮίϯςφյͯ͠࠷ॳ͔Β࡞Γ௚ͯ͠ΈΑ͏ʯ

    Λ͙͢ʹ࣮ߦͯ͠௚ͤͨ
    • ೲ඼෺ʹॳظઃఆεΫϦϓτΛೖΕΔ͚ͩͰ

    ͦΕ͕؀ڥߏஙखॱॻʹͳͬͨ
    

    View Slide

23. ·ͱΊ
    

    View Slide

24. ·ͱΊ
    • Docker ͸࣮ݧͷ༑
    • ؆୯ʹ࡞ͬͯյͤΔ
    • ؀ڥͷ࠶ݱੑΛ୲อͰ͖Δ
    • ࢖͍͜ͳ͍͖ͯ͠·͠ΐ͏
    • ڭ͑ΔͷͰͳΜͰ΋㘤͍ͯʂʂʂ
    

    View Slide