Docker Compose PaaS の作り方、そして社内に導入した話 / #yapc8oji

Docker Compose PaaS の作り方、そして社内に導入した話 / #yapc8oji

YAP(achimon)C::Asia Hachioji 2016 mid in Shinagawa での発表資料です。

https://github.com/hachiojipm/yapcasia-8oji-2016mid-timetable/issues/80

Paus: https://github.com/dtan4/paus

92ce4587cc8465736433e698b1e50aaa?s=128

Daisuke Fujita

July 03, 2016
Tweet

Transcript

  1. %PDLFS$PNQPTF1BB4ͷ
 ࡞Γํɺͦͯ͠
 ࣾ಺ʹಋೖͨ͠࿩ 2016-07-03 YAP(achimon)C::Asia Hachioji 2016 mid in Shinagawa

    Daisuke Fujita (@dtan4)
  2. Daisuke Fujita / @dtan4 Engineer, Infrastructure Team
 @ Wantedly, Inc.

    AWS, Terraform / Terraforming, Developer Productivity
  3. Docker Compose PaaS ͷ࡞Γํɺ
 ͦͯࣾ͠಺ʹಋೖͨ͠࿩ Why private PaaS? Paus: Docker

    Compose PaaS Introduce & Feedbacks Latest Updates & Future
  4. Why private PaaS? Problems at Wantedly engineering

  5. 1ͭͷ QA ؀ڥڞ༗ʹ
 ݶք͕དྷͨ ؾܰʹΞϓϦΛσϓϩΠ
 Ͱ͖Δ؀ڥ͕΄͍͠

  6. 1ͭͷ QA ؀ڥڞ༗ʹ
 ݶք͕དྷͨ ؾܰʹΞϓϦΛσϓϩΠ
 Ͱ͖Δ؀ڥ͕΄͍͠

  7. Deployment environments Production QA

  8. Deployment environments Production www.wantedly.com ͦͷ΋ͷ ࠷৽ͷ master Λ deploy ͢Δ

  9. Deployment environments QA ಈ࡞νΣοΫɾࣾ಺֬ೝ༻ Production σϓϩΠલʹ
 ඞͣ QA ͰνΣοΫ͢Δ ϞόΠϧΞϓϦͷςελʔ΋


    ಉ͡ QA ؀ڥʹ઀ଓ
  10. Problems of QA QA QA ॱ൪଴ͪ໰୊ ϞόΠϧςετ΁ͷӨڹ

  11. Problems of QA QA QA ॱ൪଴ͪ໰୊ ΤϯδχΞ͕૿͖͑ͯͨͷͰɺ
 ෳ਺ਓ͕ಉ࣌ʹ QA ʹग़ͨ͘͠

    ͳΔ => Production ΁ͷϦϦʔε͕
 ஗ΕΔ
  12. Problems of QA QA QA ॱ൪଴ͪ໰୊

  13. Problems of QA QA ϞόΠϧςετ΁ͷӨڹ feature branch σϓϩΠͯ͠
 API ͷڍಈ͕มΘΔ

    => ςετ݁ՌʹӨڹ͢Δ
  14. 1ͭͷ QA ؀ڥڞ༗ʹ
 ݶք͕དྷͨ ؾܰʹΞϓϦΛσϓϩΠ
 Ͱ͖Δ؀ڥ͕΄͍͠

  15. Deploy apps easily αʔϏεϦϦʔεʹ͋ͨͬͯɺΠϯϑϥߏஙʹ
 2ਓ೔͘Β͍අ΍͞ΕΔ • Production... ͸ Kubernetes ೖΕΑ͏ͱͯ͠Δ

    ΋ͬͱؾܰʹΞϓϦΛσϓϩΠͰ͖Δ؀ڥ͕΄͍͠ Heroku Έ͍ͨʹ؆୯ʹɺࣗ෼ͨͪͷ޷͖ͳߏ੒Ͱ
  16. Wantedly ❤ Docker Wantedly ͷશαʔϏε͸ 
 Docker on CoreOS ͰՔಇ͍ͯ͠Δ

    • Web app, Elasticsearch, ը૾ม׵αʔόɺ΄͔ॾʑ ৽نαʔϏε͸ Dockerfile ΛϦϙδτϦʹஔ͍ͯ΋Β͏ ผίϯϙʔωϯτʹґଘ͢Δ৔߹͸ docker-compose.yml ΋
  17. Docker Compose ෳ਺ίϯςφͰߏ੒͞ΕΔΞϓϦέʔγϣϯΛ
 ؆୯ʹ্ཱͪ͛Δπʔϧ docker-compose.yml ͰαʔϏεఆٛͱ
 ґଘؔ܎Λهड़ docker-compose up Ͱίϯςφىಈ

  18. Docker Compose ʮdocker-compose.yml Ͱߏ੒هड़ͨ͠Β
 ͦΕΛσϓϩΠͰ͖Δ؀ڥ͕΄͍͠…ʯ ʮdocker-compose up ͯ͘͠ΕΔ͚ͩͰ͍͍ʯ

  19. 1ͭͷ QA ؀ڥڞ༗ʹ
 ݶք͕དྷͨ ؾܰʹΞϓϦΛσϓϩΠ
 Ͱ͖Δ؀ڥ͕΄͍͠

  20. PaaS Λಋೖ͠Α͏

  21. Evaluate existing PaaS What we need for PaaS

  22. طଘͷ PaaS ͩͱ
 զʑͷཁٻΛຬͨ͢΋ͷ͕
 ͳ͔ͬͨ

  23. PaaS in the world • Heroku • Heroku Private Spaces

    OSS PaaS • dokku / dokku-alt • Flynn • Deis • Tutum • DCHQ.io • Empire • Convox • pool • …
  24. PaaS in the world Heroku ͩͱ buildpack + addons ߏ੒

    ͳͷͰ
 Ͱ͖Δ͜ͱ੍͕ݶ͞ΕΔ • ΧελϚΠζͨ͠ϛυϧ΢ΣΞͱ͔Λ࢖͑ͳ͍ • e.g. ಠࣗࣙॻΛ૊ΈࠐΜͩ Elasticsearch Docker Image
  25. PaaS in the world ΄ͱΜͲͷ OSS PaaS ͸ Heroku Ϋϩʔϯ

    • ݁ہ buildpack ʹറΒΕΔ
  26. طଘͷ PaaS ͩͱ
 զʑͷཁٻΛຬͨ͢΋ͷ͕
 ͳ͔ͬͨ

  27. PaaS Λ࡞Ζ͏

  28. Paus Docker Compose PaaS

  29. Paus docker-compose.yml ॻ͍ͯ git push ͢Δ͚ͩͰσϓϩΠ طଘΠϝʔδ pull ͚ͩͰͳ͘ɺbuild ΋Մೳ

    αϒυϝΠϯׂΓ͋ͯ GitHub ΞΧ΢ϯτ࿈ܞ for Development / Test dtan4/paus
  30. Paus dtan4/paus Inspired by: Raspaas http://www.slideshare.net/npsg/2-docker-paas-55540529 tetsusat/raspaas

  31. DEMO

  32. Paus https://ja.glosbe.com/ja/id/%E3%82%AF%E3%82%B8%E3%83%A9 dtan4/paus

  33. Paus https://ja.glosbe.com/ja/id/%E3%82%AF%E3%82%B8%E3%83%A9 dtan4/paus

  34. vs. Existing PaaS ⭕ Pros • ֶशίετ͕গͳ͍ • ߏ଄͕γϯϓϧ •

    σϓϩΠ͕खܰ ❌ Cons • ػೳෆ଍
  35. –Johnny Appleseed “Type a quote here.” OS CoreOS Data store

    etcd Clustering Docker Swarm HTTP Reverse Proxy Vulcand git-push entrypoint paus-gitreceive
 w/ gitreceive Paus Components
  36. CoreOS ίϯςφલఏͷܰྔ Linux Distribution ͢΂ͯͷΞϓϦέʔγϣϯΛ
 Docker ίϯςφ্Ͱಈ͔͢ Production in Wantedly,

    Inc. Update channel: beta
 Reboot strategy: no https://coreos.com/assets/images/media/Host-Diagram.png
  37. etcd Raft Λ࠾༻ͨ͠෼ࢄ KVS CoreOS ͕։ൃ 
 & CoreOS ωΠςΟϒαϙʔτ

    Application metadata, Deployments, Routing rules,
 Cluster discovery coreos/etcd quay.io/coreos/etcd https://coreos.com/assets/images/media/5-Machine-Cluster.png
  38. Docker Swarm Docker ͕։ൃ͢Δ Clustering System ෳ਺ͷ Docker ϗετΛ·ͱΊͯ 


    1ͭͷڊେͳϦιʔεϓʔϧͱΈͳ͢ Docker Compose ΠϯςάϨʔγϣϯ Docker Engine 1.12 ʹ౷߹͞ΕΔ docker/swarm swarm
  39. Vulcand etcd όοΫΤϯυͷ HTTP Reverse Proxy Mailgun ͕։ൃ αʔϏεϦελʔτͳ͠Ͱ
 มߋΛଈ࠲ʹద༻Մೳ

    vulcand/vulcand vulcand https://camo.githubusercontent.com/b91054e9220422d7cdad498c229ae254b41601f5/687474703a2f2f636f72656f732e636f6d2f6173736574732f696d616765732f6d656469612f76756c63616e2d312d757073747265616d2e706e67
  40. gitreceive git push Λड͚෇͚ͯεΫϦϓτ/ίϚϯυ࣮ߦ ໿ 180 ߦͷγΣϧεΫϦϓτ dokku Ͱ࢖ΘΕͯͨΓɺFlynn ʹຒΊࠐ·ΕͯͨΓ

    progrium/gitreceive
  41. paus-frontend Paus ͷ Web Dashboard GitHub ΞΧ΢ϯτͰαΠϯΠϯ ΞϓϦέʔγϣϯͷొ࿥ɺ
 build-time arguments,

    ؀ڥม਺ͷઃఆ Alpine Linux + (Go + gin-gonic/gin) dtan4/paus-frontend quay.io/dtan4/paus-frontend
  42. paus-frontend dtan4/paus-frontend quay.io/dtan4/paus-frontend

  43. git push ͷड͚ޱ docker-compose build 
 + docker-compose up etcd

    ʹϝλσʔλ 
 + Vulcand ઃఆΛอଘ Ubuntu + gitreceive + receiver (by Go) paus-gitreceive dtan4/paus-gitreceive quay.io/dtan4/paus-gitreceive SFDFJWFS XFC EC XFC SFEJT
  44. Deploy phase Paus QBVTHJUSFDFJWF QBVTGSPOUFOE XFC EC XFC SFEJT XFC

    $ git push paus master :2222
  45. Access to application Paus http://dtan4-rails-ffe7rf8.pausapp.com QBVTHJUSFDFJWF QBVTGSPOUFOE XFC  EC

    XFC  SFEJT XFC  :80
  46. Access to dashboard Paus http://pausapp.com QBVTHJUSFDFJWF QBVTGSPOUFOE XFC EC XFC

    SFEJT XFC :80
  47. Introduce & Feedbacks

  48. Introduce ϓϩτλΠϓ: 1೔ ։ൃ: 1ϲ݄ऑ ΤϯδχΞ޲͚ϋϯζΦϯΛ࣮ࢪ progrium/gitreceive

  49. Feedbacks ศར ύϑΥʔϚϯε͕ѱ͍
 ϝϞϦॏࢹͰ r3 ࢖ͬͯͨ => c4 ʹஔ͖׵͑ ઃఆ͕໘౗


    ϦϙδτϦʹσϑΥϧτ஋ຒΊࠐΜͩ .env ஔ͍ͯ
 docker-compose.yml Ͱಡ·ͤΔ progrium/gitreceive
  50. Latest Updates & Future

  51. Deploy from CI ʮgit push ͢Δͷ΋໘౗ʯ GitHub ʹ push ͨ͠Β

    Paus ʹσϓϩΠʂ
 ؾ͕͍ͭͨΒউखʹσϓϩΠ͞ΕͯΔʂ CircleCI ͔Β git push paus <branch> σϓϩΠ਺্ݶΛઃఆͰ͖ΔΑ͏ʹ
  52. Deploy from CI

  53. Deploy from CI

  54. Replace backend CoreOS + Swarm Ϋϥελ؅ཧ͕໘౗ Docker Compose ͱີ݁߹ ϩάͷ໰୊

  55. Replace backend to ECS Ϋϥελ؅ཧͱεέδϡʔϦϯάΛ ECS ʹ೚ͤΔ API ܦ༝ͰαʔϏε৘ใΛऔಘ CloudWatch

    Logs Ͱϩά΋औΕΔ ΫϥελϝτϦΫεʹج͍ͮͨΦʔτεέʔϧ => ECS ͸ PaaS backend ͱͯ͠࠷దͰ͸ʁ
  56. Replace backend to ECS ΠϯϑϥνʔϜతʹ͸ Kubernetes ͷ෩ை ref: ͏·͍۩߹ʹ΍͍ͬͯ͘

  57. Recap

  58. Recap ڞ༗ QA ؀ڥʹݶք͕དྷͨ & ؾܰʹσϓϩΠ͔ͨͬͨ͠ ෳ਺ͷ OSS Λ૊Έ߹Θͤɺࣗ෼ͨͪͷཁٻΛຬͨ͢
 Docker

    Compose PaaS: Paus Λ࡞ͬͨ ࣾ಺ಋೖͯ͠Έͨ΋ͷͷɺվળͷ༨஍͋Γ ECS ࢼ͍ͨ͠