Docker をフル活用したインフラの紹介と成長し続けるためのインフラ戦略 / #abejameetup

Docker をフル活用したインフラの紹介と成長し続けるためのインフラ戦略 / #abejameetup

Innovation Meetup ~Startup Infrastructure~ の発表資料です
http://abeja-innovation-meetup.connpass.com/event/36707/

92ce4587cc8465736433e698b1e50aaa?s=128

Daisuke Fujita

August 03, 2016
Tweet

Transcript

  1. %PDLFSΛϑϧ׆༻ͨ͠
 Πϯϑϥͷ঺հ
 ͱ
 ੒௕͠ଓ͚ΔͨΊͷΠϯϑϥઓུ 2016-08-03 Innovation Meetup ~Startup Infrastructure~ Daisuke

    Fujita (@dtan4)
  2. Daisuke Fujita / @dtan4 Engineer, Infrastructure Team
 @ Wantedly, Inc.

    AWS, Terraform / Terraforming, Developer Productivity
  3. https://www.wantedly.com/

  4. 3 @ https://www.wantedly.com/sync

  5. Docker Λϑϧ׆༻ͨ͠
 Πϯϑϥͷ঺հ ੒௕͠ଓ͚ΔͨΊͷ
 Πϯϑϥઓུ

  6. Docker Λϑϧ׆༻ͨ͠ Πϯϑϥͷ঺հ

  7. History of Infrastructure 2011/09- Heroku 2014/08- Docker on AWS 2015/05-

    Terraform 2016/02- CoreOS ૑ۀ
  8. 2011/9~2014/08 Heroku CEO ͕ҰਓͰαʔϏε্ཱͪ͛ɺ࠷ॳ͔Β Ruby on Rails αʔϏε։ൃʹूத͢Δ σϓϩΠ଎͍ɺ
 ϫϯΫϦοΫͰॠ࣌ʹεέʔϧ

    ๛෋ͳ Addons
  9. 2014/08~ Docker on AWS ೔ຊ <-> ถࠃ౦ւ؛ ԟ෮ʹΑΔϨΠςϯγΛແ͍ͨ͘͠ Πϯϑϥઐ೚ͷΤϯδχΞ΋ join

    ͨ͠ ౦ژϦʔδϣϯ͕ଘࡏ͢Δ AWS ΁Ҡߦ Heroku ಉ౳ͷ UX Λఏڙ͢ΔͨΊ Docker Λ࠾༻
  10. 2014/08~ Docker on AWS Capistrano Λ࢖ͬͨಠࣗͷσϓϩΠγεςϜ 
 + Heroku CLI

    ಉ༷ͷΠϯλʔϑΣΠεΛ࣋ͭ CLI EC2 Πϯελϯε (Ubuntu 14.04) ͱ Docker image ͸
 Chef + Packer ͰϓϩϏδϣχϯά
  11. 2015/05~ Terraform Management Console ϙνϙνͯ͠ߏங͢Δͷ͕໘౗ ࡞ۀཤྺΛ࢒͍ͨ͠ 
 Terraform Λར༻ͯ͠ΠϯϑϥߏஙΛίʔυԽ 


    & CI ʹΑΔߏஙࣗಈԽ Terraforming ʹΑΔطଘ AWS, DNSimple ϦιʔεͷίʔυԽ
  12. 2015/05~ Terraform

  13. 2016/02~ CoreOS ϗετϚγϯͷϝϯςφϯεʹඅ΍͢޻਺͕Ͱ͔͍ Chef Λۦஞ͍ͨ͠ 
 @dtan4 ͷϥοϓτοϓͱϓϩϏδϣχϯά༻ΠϯελϯεͰ͔͠ಈ͔ͳ͔ͬͨ 
 ϗετ

    OS Λ CoreOS ʹҠߦ 
 + systemd ʹΑΔαʔϏε؅ཧ
  14. CoreOS ίϯςφલఏͷܰྔ Linux Distribution ͢΂ͯͷΞϓϦέʔγϣϯΛ
 Docker ίϯςφ্Ͱಈ͔͢ Chrome ͱಉ͡ Update

    System Update channel: beta
 Reboot strategy: no https://coreos.com/assets/images/media/Host-Diagram.png
  15. Infrastructure at Wantedly Web, DB, Storage
 
 DNS AWS
 EC2,

    ELB, RDS, Elasticache, S3,
 Route53, VPC, Lambda DNSimple
  16. Infrastructure at Wantedly Logging App Monitoring Infrastucture Monitoring Incident Management

    Logentries New Relic Datadog Pagerduty
  17. Elasticsearch nginx-image-server Web Web Worker Scheduler nginx-image-server Elasticsearch cluster VPC

    Nginx
  18. systemd journald Docker
 Registry dd-agent Web
 Container EC2 Instance

  19. Web (Rails) Docker image ͸2ஈߏ੒ ि1Ͱ intermediate image ΛϏϧυ
 σϓϩΠ࣌ʹ

    web image ΛϏϧυ bundle install git clone wantedly/wantedly Ruby FROM Ubuntu 14.04 rake assets:precompile bundle install git checkout <branch> git fetch FROM intermediate web intermediate
  20. Web (Rails) ࠩ෼ͷΈͷ fetch + bundle install Ͱ
 σϓϩΠΛߴ଎Խ intermediate

    ͸ Chef + Packer bundle install git clone wantedly/wantedly Ruby FROM Ubuntu 14.04 rake assets:precompile bundle install git checkout <branch> git fetch FROM intermediate web intermediate
  21. ެࣜ Elasticsearch image ϕʔεͷ Docker image ֤छϓϥάΠϯ + ಠࣗࣙॻʢਓ໊ etc...ʣ

    --net=host + EC2 discovery ։ൃ؀ڥͰ΋ຊ൪ͱಉ౳ͷ Elasticsearch Λ࢖͑Δ Elasticsearch
  22. Nginx + ngx_small_light ը૾ม׵αʔό S3 ʹอ؅ͨ͠ը૾ΛϦαΠζ / ϒϥʔ / WebP

    ม׵… CloudFront Ͱ഑৴ + Ωϟογϡ nginx-image-server wantedly/nginx-image-server quay.io/wantedly/nginx-image-server Docker Image
 ެ։͍ͯ͠·͢
  23. URL ύϥϝʔλͰಈతʹը૾ม׵ nginx-image-server wantedly/nginx-image-server https://.../
 small_light(dw=978,dh=398,da=s,ds=s,cw=978,ch=398)
 /.../image.png quay.io/wantedly/nginx-image-server

  24. All services on Docker Facebook ࢓ࣄ૬ஊ bot (Python) จॻϓϨϏϡʔ࡞੒αʔό (Golang)

    ࣾ಺ Slack bot (Node.js) ...
 ৽نαʔϏε͸ϦϙδτϦʹ Dockerfile Λஔ͍ͯ΋Β͏
  25. Private Docker Registry (S3 backed) Λհͨ͠
 Docker image ͷ΍ΓऔΓ Deploy

    Web Build server $ docker push $ docker pull $ git fetch $ docker build $ docker run
  26. ϗετϨϕϧͰͷ Blue-Green Deployment Deploy Web Blue Web Green if (-e

    /etc/nginx/switch/blue) { proxy_pass http://blue-container; } if (-e /etc/nginx/switch/green) { proxy_pass http://green-container; }
  27. Heroku ಉ౳ͷΠϯλʔϑΣΠεΛ࣋ͭ CLI (Capistrano) Deploy $ cap -T cap config

    # Display the config vars for an app cap deploy[revision] # Deploy an application cap deploy:diff[ref] # Display diff deployed revision and the revision to cap deploy:rollback # Rollback application cap logs:t[container_name] # Display stream logs for an container cap maintenance:on # Enable maintenance mode cap ps # Display web, worker, scheduler, run type containers cap ps:scale[role,num] # Scale out/in containers cap ps:restart:web[color,chained_task?] # Start web containers cap releases # Display deploy history cap run[command] # Run on-off commands cap scheduler # Display scheduling cap scheduler:clear # Clear scheduler config cap scheduler:update # Update scheduler cap status # Show application status
  28. SSH + Docker ίϯςφܦ༝Ͱ Deploy ࣮ߦ (Sap) Deploy

  29. SSH + Docker ίϯςφܦ༝Ͱ Deploy ࣮ߦ (sap)
 
 CLI ߋ৽࣌ʹ΋֤ࣗͷϚγϯͰΞοϓσʔτͯ͠΋Β͏ඞཁ

    ͕ͳ͍ ֤࡞ۀͷϩάΛ Docker container ϩάͱͯ͠࢒ͤΔ & Ӿཡ
 Ͱ͖Δ Deploy
  30. stdout ʹશϩάΛग़ྗ + journalctl ܦ༝Ͱ Logentries ʹૹ৴ Logging

  31. શϗετʹ Datadog Agent ίϯςφΛ഑ஔ͠ɺ
 ϗετͱ Docker ྆ํͷ؂ࢹΛ࣮ࢪ
 
 Monitoring

  32. ੒௕͠ଓ͚ΔͨΊͷ Πϯϑϥઓུ

  33. https://www.wantedly.com/

  34. 3 @ https://www.wantedly.com/sync

  35. Wantedly Open API ͸ͨΒ͘Λ໘ന͘ʯΛςʔϚʹ͜Ε·ͰαʔϏε։ൃΛల։͖ͯͨ͠ 8BOUFEMZ͕ͦͷՁ஋Λ8BOUFEMZDPNυϝΠϯҎ֎ʹ΋޿͍͖͛ͯ·͢ɻ ੈͷதͷҰਓͰ΋ଟ͘ͷਓ͕ɺ͸ͨΒ͘Λ໘ന͘ʯײ͡ΒΕΔੈͷதʹͳΓ·͢Α͏ʹɻ Ͱ͖Δ͜ͱ ࣗಈೖྗ ΫοΫύουגࣜձࣾ ΫοΫύουגࣜձࣾ

    5݄13೔ʹฐࣾΦϑΟε಺ͷεϖʔεʹͯʮʲୈ17ճʳpotatotips(iOS/Android։ൃTips ڞ༗ձ)ʯ http://connpass.com/event/14143/ Λ։࠵͠·ͨ͠ʂ 18໊ͷൃදऀ͕5෼ؒͷLTʢϥΠτχϯάτʔΫɿΧϯϑΝϨϯεͳͲͰߦΘΕΔ୹͍ ΋ͬͱݟΔ ஑ా୓࢘ ϢʔβʔϑΝʔετਪਐ෦෦௕   ਓ͕ϑΥϩʔத  ਓ͕ϑΥϩʔத ϑΥϩʔ͢Δ ձࣾϑΟʔυϘοΫε ࿩Λฉ͖ʹߦ͘Ϙλϯ ϑΥʔϜࣗಈೖྗϘλϯ Wantedly্ʹ͋ΔʮձࣾϑΟʔυʯ Λɺϒϩά΍ίʔϙϨʔταΠτͳ Ͳɺ޷͖ͳϖʔδʹ؆୯ʹຒΊࠐΉ͜ ͱ͕Ͱ͖·͢ɻखܰͳ࠾༻ϒϥϯσΟ ϯά΍৘ใൃ৴ʹར༻Ͱ͖·͢ɻ ৄ͘͠ ৄ͘͠ ৄ͘͠ ࣗࣾαΠτ౳ͷืूཁ߲ʹɺʮ࿩Λฉ ͖ʹߦ͖͍ͨʯϘλϯΛઃஔ͠ɺ Wantedly্ͷʮ༡ͼʹ͍͘ମݧʯΛࣗ ࣾαΠτͰ΋ల։Ͱ͖·͢ɻ ࠾༻؅ཧγεςϜఏڙاۀ༷౳΁ͷಛ ผఏڙAPIͰ͢ɻީิऀ͕ืूʹΤϯ τϦ͢ΔࡍʹɺWantedly্ͷϓϩϑΟ ʔϧ৘ใΛ׆༻ͯ͠ΤϯτϦ͕Մೳʹ ͳΓ·͢ɻ https://www.wantedly.com/developers
  36. WANTEDLY JOURNAL ͸ͨΒ͘Λ໘ന͘͢ΔϝσΟΞ γΰτͰίίϩΦυϧͬͯͳΜͩΖ͏ʁ 8BOUFMEZ͸ɺγΰτͰίίϩΦυϧਓΛ;΍͍ͨ͠ͱߟ͍͑ͯ·͢ɻ Ͱ͸ɺzγΰτͰίίϩΦυϧzͱ͸Ͳ͏͍͏ঢ়ଶͳͷͰ͠ΐ͏ʁ ͓٬͞Μʹʮ͋Γ͕ͱ͏ʯͱݴΘΕͨͱ͖ ໨ඪͷച্Λୡ੒ͨ͠ͱ͖ νʔϜϝϯόʔͷসإΛݟͨͱ͖ʜ ͦΕ͸ɺਓʹΑͬͯҧ͏Ͱ͠ΐ͏͠ɺ

    ࣌୅΍ࠃʹΑͬͯ΋·ͬͨ͘ҟͳΔͰ͠ΐ͏ɻ ࠓճϦϦʔε͢Δʮ8"/5&%-:+063/"-ʯͰ͸ɺ 8BOUFEMZΛ௨ͯ͠స৬ͨ͠ํ΍ɺීஈ͋·Γݟͨ͜ͱ͕ͳ͍اۀ΁ͷ ΠϯλϏϡʔΛ௨ͯ͠ɺzγΰτͰίίϩΦυϧzͱ͸Կͳͷ͔Λ୳ٻ͠·͢ɻ γΰτΛ୳͍ͯ͠Δਓ΋ɺͦ͏Ͱͳ͍ਓʹͱͬͯ΋ɺ ʮ8"/5&%-:+063/"-ʯ͕ಇ͖ํͷώϯτʹͳΕ͹޾͍Ͱ͢ɻ https://www.wantedly.com/journals
  37. ੈքʹࣗ෼ͷ੠Λൃ৴͠Α͏ https://www.wantedly.com/feed

  38. ੈքʹࣗ෼ͷ੠Λൃ৴͠Α͏ https://www.wantedly.com/tools

  39. https://case.wantedly.com/

  40. Dreams ✨ ৽͍͠αʔϏεΛͲΜͲΜϦϦʔε͍͖͍ͯͨ͠
 ࣄۀͱͯ͠ͷαʔϏεɺϚΠΫϩαʔϏεΞʔΩςΫνϟ ಛੑʹ߹Θͤͯɺݴޠ΍ΞʔΩςΫνϟΛࣗ༝ʹ
 બ୒Ͱ͖ΔΑ͏ʹ͍ͨ͠ গਓ਺ΛอͪͭͭɺαʔϏεΛεέʔϧ͍ͤͨ͞

  41. Problems αʔϏεϦϦʔεͷͨͼʹಉ͡࡞ۀΛ܁Γฦ͢ ߏஙʹ͋ͨͬͯͦΕͳΓͷ޻਺Λཁ͢Δ Wantedly ͷϝΠϯϦϙδτϦ͕ϞϊϦγοΫ

  42. Problems ΠϯϑϥνʔϜ͕αʔϏεϦϦʔεͷϘτϧωοΫʹ
 ͳͬͯ͠·͏ ৽αʔϏε΋݁ہಉ͡ίʔυϕʔεʹࡌͤͯ͠·͏

  43. Solution 1: Kubernetes Kubernetes ΫϥελΛಋೖ͢Δ Dev ͕ΞϓϦέʔγϣϯϨΠϠΛ؅ཧ͢Δ
 Ops ͕ΞϓϦ͕Քಇ͢ΔΫϥελΛ؅ཧ͢Δ
 ʮԿॲͰԿ͕ಈ͍͍ͯΔʯͰ͸ͳ͘ʮԿ͕ಈ͍͍ͯΔ͔ʯ͚ͩ

    ϑϧϚωʔδυͰඞཁͳ΋ͷ (ASG, ELB, DNS, Datadog) ͕
 ࡞ΒΕΔศར͞
  44. Solution 2: API Wantedly ͷ API Λ࡞ΓɺAPI ܦ༝Ͱ͢΂ͯͷΞΫγϣϯΛߦ͏

  45. Solution 2: API React.js ϜʔϒϝϯτʹΑΔ Web ϑϩϯτΤϯυͷ෼཭ Kong Λར༻ͨ͠ೝূ͖ͭ API

    Gateway apig ʹΑΔϞσϧఆ͔ٛΒͷ 
 REST API αʔόɺυΩϡϝϯτͷੜ੒ webmock-proxy ʹΑΔαʔϏεؒ Integration Test ؆ུԽ wantedly/apig wantedly/webmock-proxy
  46. ·ͱΊ

  47. Recap Wantedly Ͱ͸ɺ2೥ऑຊ൪؀ڥΛ Docker Ͱӡ༻͍ͯ͠·͢ Ϣʔβͱ։ൃऀ૒ํͷརศੑΛ௥ٻ্ͨ͠Ͱͷબ୒ 
 ͞ΒͳΔ੒௕ʹ޲͚ɺΑΓॊೈͳΠϯϑϥج൫Λߏஙத

  48. Recap

  49. None