Docker Compose PaaS の作り方、そして社内に導入した話 / #yapc8oji

YAP(achimon)C::Asia Hachioji 2016 mid in Shinagawa での発表資料です。

https://github.com/hachiojipm/yapcasia-8oji-2016mid-timetable/issues/80

Paus: https://github.com/dtan4/paus

Transcript

1. %PDLFS$PNQPTF1BB4ͷ

   ࡞Γํɺͦͯ͠

   ࣾ಺ʹಋೖͨ͠࿩
   2016-07-03
   YAP(achimon)C::Asia Hachioji 2016 mid in Shinagawa
   Daisuke Fujita (@dtan4)
   

   View Slide

2. Daisuke Fujita / @dtan4
   Engineer, Infrastructure Team

   @ Wantedly, Inc.
   AWS, Terraform / Terraforming, Developer Productivity
   

   View Slide

3. Docker Compose PaaS ͷ࡞Γํɺ

   ͦͯࣾ͠಺ʹಋೖͨ͠࿩
   Why private PaaS?
   Paus: Docker Compose PaaS
   Introduce & Feedbacks
   Latest Updates & Future
   

   View Slide

4. Why private PaaS?
   Problems at Wantedly engineering
   

   View Slide

5. 1ͭͷ QA ؀ڥڞ༗ʹ

   ݶք͕དྷͨ
   ؾܰʹΞϓϦΛσϓϩΠ

   Ͱ͖Δ؀ڥ͕΄͍͠
   

   View Slide

6. 1ͭͷ QA ؀ڥڞ༗ʹ

   ݶք͕དྷͨ
   ؾܰʹΞϓϦΛσϓϩΠ

   Ͱ͖Δ؀ڥ͕΄͍͠
   

   View Slide

7. Deployment environments
   Production QA
   

   View Slide

8. Deployment environments
   Production
   www.wantedly.com ͦͷ΋ͷ
   
   ࠷৽ͷ master Λ deploy ͢Δ
   

   View Slide

9. Deployment environments
   QA
   ಈ࡞νΣοΫɾࣾ಺֬ೝ༻
   
   Production σϓϩΠલʹ

   ඞͣ QA ͰνΣοΫ͢Δ
   
   ϞόΠϧΞϓϦͷςελʔ΋

   ಉ͡ QA ؀ڥʹ઀ଓ
   

   View Slide

10. Problems of QA
    QA
    QA ॱ൪଴ͪ໰୊
    ϞόΠϧςετ΁ͷӨڹ
    

    View Slide

11. Problems of QA
    QA
    QA ॱ൪଴ͪ໰୊
    ΤϯδχΞ͕૿͖͑ͯͨͷͰɺ

    ෳ਺ਓ͕ಉ࣌ʹ QA ʹग़ͨ͘͠
    ͳΔ
    
    => Production ΁ͷϦϦʔε͕

    ஗ΕΔ
    

    View Slide

12. Problems of QA
    QA
    QA ॱ൪଴ͪ໰୊
    

    View Slide

13. Problems of QA
    QA
    ϞόΠϧςετ΁ͷӨڹ
    feature branch σϓϩΠͯ͠

    API ͷڍಈ͕มΘΔ
    
    => ςετ݁ՌʹӨڹ͢Δ
    

    View Slide

14. 1ͭͷ QA ؀ڥڞ༗ʹ

    ݶք͕དྷͨ
    ؾܰʹΞϓϦΛσϓϩΠ

    Ͱ͖Δ؀ڥ͕΄͍͠
    

    View Slide

15. Deploy apps easily
    αʔϏεϦϦʔεʹ͋ͨͬͯɺΠϯϑϥߏஙʹ

    2ਓ೔͘Β͍අ΍͞ΕΔ
    
    • Production... ͸ Kubernetes ೖΕΑ͏ͱͯ͠Δ
    
    ΋ͬͱؾܰʹΞϓϦΛσϓϩΠͰ͖Δ؀ڥ͕΄͍͠
    
    Heroku Έ͍ͨʹ؆୯ʹɺࣗ෼ͨͪͷ޷͖ͳߏ੒Ͱ
    

    View Slide

16. Wantedly ❤ Docker
    Wantedly ͷશαʔϏε͸ 

    Docker on CoreOS ͰՔಇ͍ͯ͠Δ
    • Web app, Elasticsearch, ը૾ม׵αʔόɺ΄͔ॾʑ
    
    ৽نαʔϏε͸ Dockerfile ΛϦϙδτϦʹஔ͍ͯ΋Β͏
    
    ผίϯϙʔωϯτʹґଘ͢Δ৔߹͸ docker-compose.yml ΋
    

    View Slide

17. Docker Compose
    ෳ਺ίϯςφͰߏ੒͞ΕΔΞϓϦέʔγϣϯΛ

    ؆୯ʹ্ཱͪ͛Δπʔϧ
    
    docker-compose.yml ͰαʔϏεఆٛͱ

    ґଘؔ܎Λهड़
    
    docker-compose up Ͱίϯςφىಈ
    

    View Slide

18. Docker Compose
    ʮdocker-compose.yml Ͱߏ੒هड़ͨ͠Β

    ͦΕΛσϓϩΠͰ͖Δ؀ڥ͕΄͍͠…ʯ
    
    ʮdocker-compose up ͯ͘͠ΕΔ͚ͩͰ͍͍ʯ
    

    View Slide

19. 1ͭͷ QA ؀ڥڞ༗ʹ

    ݶք͕དྷͨ
    ؾܰʹΞϓϦΛσϓϩΠ

    Ͱ͖Δ؀ڥ͕΄͍͠
    

    View Slide

20. PaaS Λಋೖ͠Α͏
    

    View Slide

21. Evaluate existing PaaS
    What we need for PaaS
    

    View Slide

22. طଘͷ PaaS ͩͱ

    զʑͷཁٻΛຬͨ͢΋ͷ͕

    ͳ͔ͬͨ
    

    View Slide

23. PaaS in the world
    • Heroku
    
    • Heroku Private Spaces
    
    OSS PaaS
    • dokku / dokku-alt
    
    • Flynn
    
    • Deis
    • Tutum
    
    • DCHQ.io
    
    • Empire
    • Convox
    
    • pool
    
    • …
    

    View Slide

24. PaaS in the world
    Heroku ͩͱ buildpack + addons ߏ੒ ͳͷͰ

    Ͱ͖Δ͜ͱ੍͕ݶ͞ΕΔ
    
    • ΧελϚΠζͨ͠ϛυϧ΢ΣΞͱ͔Λ࢖͑ͳ͍
    
    • e.g. ಠࣗࣙॻΛ૊ΈࠐΜͩ Elasticsearch Docker Image
    

    View Slide

25. PaaS in the world
    ΄ͱΜͲͷ OSS PaaS ͸ Heroku Ϋϩʔϯ
    
    • ݁ہ buildpack ʹറΒΕΔ
    
    

    View Slide

26. طଘͷ PaaS ͩͱ

    զʑͷཁٻΛຬͨ͢΋ͷ͕

    ͳ͔ͬͨ
    

    View Slide

27. PaaS Λ࡞Ζ͏
    

    View Slide

28. Paus
    Docker Compose PaaS
    

    View Slide

29. Paus
    docker-compose.yml ॻ͍ͯ git push ͢Δ͚ͩͰσϓϩΠ
    طଘΠϝʔδ pull ͚ͩͰͳ͘ɺbuild ΋Մೳ
    αϒυϝΠϯׂΓ͋ͯ
    GitHub ΞΧ΢ϯτ࿈ܞ
    for Development / Test
    dtan4/paus
    

    View Slide

30. Paus dtan4/paus
    Inspired by: Raspaas
    http://www.slideshare.net/npsg/2-docker-paas-55540529
    tetsusat/raspaas
    

    View Slide

31. DEMO
    

    View Slide

32. Paus
    https://ja.glosbe.com/ja/id/%E3%82%AF%E3%82%B8%E3%83%A9
    dtan4/paus
    

    View Slide

33. Paus
    https://ja.glosbe.com/ja/id/%E3%82%AF%E3%82%B8%E3%83%A9
    dtan4/paus
    

    View Slide

34. vs. Existing PaaS
    ⭕ Pros
    • ֶशίετ͕গͳ͍
    
    • ߏ଄͕γϯϓϧ
    
    • σϓϩΠ͕खܰ
    ❌ Cons
    • ػೳෆ଍
    

    View Slide

35. –Johnny Appleseed
    “Type a quote here.”
    OS CoreOS
    Data store etcd
    Clustering Docker Swarm
    HTTP Reverse Proxy Vulcand
    git-push entrypoint
    paus-gitreceive

    w/ gitreceive
    Paus Components
    

    View Slide

36. CoreOS
    ίϯςφલఏͷܰྔ Linux Distribution
    
    ͢΂ͯͷΞϓϦέʔγϣϯΛ

    Docker ίϯςφ্Ͱಈ͔͢
    
    Production in Wantedly, Inc.
    
    Update channel: beta

    Reboot strategy: no
    https://coreos.com/assets/images/media/Host-Diagram.png
    

    View Slide

37. etcd
    Raft Λ࠾༻ͨ͠෼ࢄ KVS
    
    CoreOS ͕։ൃ 

    & CoreOS ωΠςΟϒαϙʔτ
    
    Application metadata, Deployments, Routing rules,

    Cluster discovery
    coreos/etcd
    quay.io/coreos/etcd
    https://coreos.com/assets/images/media/5-Machine-Cluster.png
    

    View Slide

38. Docker Swarm
    Docker ͕։ൃ͢Δ Clustering System
    ෳ਺ͷ Docker ϗετΛ·ͱΊͯ 

    1ͭͷڊେͳϦιʔεϓʔϧͱΈͳ͢
    
    Docker Compose ΠϯςάϨʔγϣϯ
    
    Docker Engine 1.12 ʹ౷߹͞ΕΔ
    docker/swarm
    swarm
    

    View Slide

39. Vulcand
    etcd όοΫΤϯυͷ HTTP Reverse Proxy
    Mailgun ͕։ൃ
    
    αʔϏεϦελʔτͳ͠Ͱ

    มߋΛଈ࠲ʹద༻Մೳ
    vulcand/vulcand
    vulcand
    https://camo.githubusercontent.com/b91054e9220422d7cdad498c229ae254b41601f5/687474703a2f2f636f72656f732e636f6d2f6173736574732f696d616765732f6d656469612f76756c63616e2d312d757073747265616d2e706e67
    

    View Slide

40. gitreceive
    git push Λड͚෇͚ͯεΫϦϓτ/ίϚϯυ࣮ߦ
    ໿ 180 ߦͷγΣϧεΫϦϓτ
    
    dokku Ͱ࢖ΘΕͯͨΓɺFlynn ʹຒΊࠐ·ΕͯͨΓ
    progrium/gitreceive
    

    View Slide

41. paus-frontend
    Paus ͷ Web Dashboard
    
    GitHub ΞΧ΢ϯτͰαΠϯΠϯ
    
    ΞϓϦέʔγϣϯͷొ࿥ɺ

    build-time arguments, ؀ڥม਺ͷઃఆ
    
    Alpine Linux + (Go + gin-gonic/gin)
    dtan4/paus-frontend
    quay.io/dtan4/paus-frontend
    

    View Slide

42. paus-frontend dtan4/paus-frontend
    quay.io/dtan4/paus-frontend
    

    View Slide

43. git push ͷड͚ޱ
    docker-compose build 

    + docker-compose up
    etcd ʹϝλσʔλ 

    + Vulcand ઃఆΛอଘ
    
    Ubuntu + gitreceive + receiver (by Go)
    paus-gitreceive dtan4/paus-gitreceive
    quay.io/dtan4/paus-gitreceive
    SFDFJWFS
    XFC EC
    XFC SFEJT
    

    View Slide

44. Deploy phase
    Paus
    QBVTHJUSFDFJWF
    QBVTGSPOUFOE
    XFC EC
    XFC SFEJT
    XFC
    $ git push paus master
    :2222
    

    View Slide

45. Access to application
    Paus
    http://dtan4-rails-ffe7rf8.pausapp.com
    QBVTHJUSFDFJWF
    QBVTGSPOUFOE
    XFC
    
    EC
    XFC
    
    SFEJT
    XFC
    
    :80
    

    View Slide

46. Access to dashboard
    Paus
    http://pausapp.com
    QBVTHJUSFDFJWF
    QBVTGSPOUFOE
    XFC EC
    XFC SFEJT
    XFC
    :80
    

    View Slide

47. Introduce & Feedbacks
    

    View Slide

48. Introduce
    ϓϩτλΠϓ: 1೔
    ։ൃ: 1ϲ݄ऑ
    ΤϯδχΞ޲͚ϋϯζΦϯΛ࣮ࢪ
    progrium/gitreceive
    

    View Slide

49. Feedbacks
    ศར
    ύϑΥʔϚϯε͕ѱ͍

    ϝϞϦॏࢹͰ r3 ࢖ͬͯͨ => c4 ʹஔ͖׵͑
    
    ઃఆ͕໘౗

    ϦϙδτϦʹσϑΥϧτ஋ຒΊࠐΜͩ .env ஔ͍ͯ

    docker-compose.yml Ͱಡ·ͤΔ
    progrium/gitreceive
    

    View Slide

50. Latest Updates & Future
    

    View Slide

51. Deploy from CI
    ʮgit push ͢Δͷ΋໘౗ʯ
    
    GitHub ʹ push ͨ͠Β Paus ʹσϓϩΠʂ

    ؾ͕͍ͭͨΒউखʹσϓϩΠ͞ΕͯΔʂ
    CircleCI ͔Β git push paus
    
    σϓϩΠ਺্ݶΛઃఆͰ͖ΔΑ͏ʹ
    

    View Slide

52. Deploy from CI
    

    View Slide

53. Deploy from CI
    

    View Slide

54. Replace backend
    CoreOS + Swarm Ϋϥελ؅ཧ͕໘౗
    
    Docker Compose ͱີ݁߹
    
    ϩάͷ໰୊
    

    View Slide

55. Replace backend to ECS
    Ϋϥελ؅ཧͱεέδϡʔϦϯάΛ ECS ʹ೚ͤΔ
    API ܦ༝ͰαʔϏε৘ใΛऔಘ
    
    CloudWatch Logs Ͱϩά΋औΕΔ
    
    ΫϥελϝτϦΫεʹج͍ͮͨΦʔτεέʔϧ
    
    => ECS ͸ PaaS backend ͱͯ͠࠷దͰ͸ʁ
    

    View Slide

56. Replace backend to ECS
    ΠϯϑϥνʔϜతʹ͸ Kubernetes ͷ෩ை
    
    ref:
    
    ͏·͍۩߹ʹ΍͍ͬͯ͘
    

    View Slide

57. Recap
    

    View Slide

58. Recap
    ڞ༗ QA ؀ڥʹݶք͕དྷͨ & ؾܰʹσϓϩΠ͔ͨͬͨ͠
    ෳ਺ͷ OSS Λ૊Έ߹Θͤɺࣗ෼ͨͪͷཁٻΛຬͨ͢

    Docker Compose PaaS: Paus Λ࡞ͬͨ
    ࣾ಺ಋೖͯ͠Έͨ΋ͷͷɺվળͷ༨஍͋Γ
    ECS ࢼ͍ͨ͠
    

    View Slide