eb_with_multi_container_docker

 eb_with_multi_container_docker

# JAWS-UGおコンテナ支部 #5

- EB with multi-container Docker をproductionで1年間運用して起きたこと

941d3ec0c92d7b43d43c1baa81d4d82d?s=128

Takumi KANZAKI

June 27, 2016
Tweet

Transcript

  1. 2.

    Me • Takumi Kanzaki (@tknzk) • VASILY Inc. Backend Engineer

    • 2015.04~ • iQONAD/ࣾ಺༻੥ٻॻπʔϧ Λ୲౰ ($∀$) • ࠷ۙiQON API ΋୲౰ 2 VASILY Inc. 2016
  2. 5.

    iQONAD • software, middleware • Ruby, Rails, sinatra • MySQL,

    memcached, Redis • infra • EC2, RDS • docker, ElasticBeanstalk (2015.06~) • auto-scaling 5 VASILY Inc. 2016
  3. 7.

    Container ߏ੒ • 4ͭͷcontainerͰՔಇ 1. ຊମͷΞϓϦέʔγϣϯ͕͸͍ͬͨContainer (ad-server) • ruby,supervisord 2.

    httpͷϦΫΤετΛड͚෇͚ΔͨΊͷnginx 3. logίϨΫλͱͯ͠ͷtd-agent 4. ؂ࢹ༻ͷmackerel-agent 7 VASILY Inc. 2016
  4. 9.

    #1 middlewareͷߋ৽ • ruby 2.0 -> 2.2.3 • ϩʔϯν·Ͱ2िؒͰ։ൃΛߦͬͨ͜ͱ͔Βطଘͷ։ൃӡ༻࣮੷ ͷ͋Δruby

    2.0.0 Ͱ։ൃ/ӡ༻͞Ε͍ͯ·ͨ͠ • 2015/11ʹdockerͷڧΈΛ׆͔ͯ͠ɺruby2.2ܥʹҠߦ • ruby2.2 • GGͷվળͷԸܙ 9 VASILY Inc. 2016
  5. 11.

    #1 middlewareͷߋ৽ • ruby 2.0 -> 2.2.3 -> 2.3.1 •

    ࠓ೥ͷ͸͡Ίʹ ruby 2.3.1΁Ҡߦ • ଞͷmiddleware΋ΧδϡΞϧʹ࠷৽΁ • nginx: 1.11.1 • mackerel-agent: 0.31.2 • td-agent: v2.3.1 (fluentd:v0.12.20) 11 VASILY Inc. 2016
  6. 14.

    #2 docker imageංେԽ • ͱ͋ΔλΠϛϯάͷdocker image size • ad_server: 924.6MB

    • nginx: 134.1MB • td-agent: 448.4MB • mackerel-agent: 423.1MB 14 VASILY Inc. 2016
  7. 16.
  8. 18.

    #2-2 docker image࡟ݮ ad-server • ΋ͱ΋ͱ CentOS6Λϕʔεʹbuild • 2ஈߏ੒ •

    ruby, supervisord, mysql Λ૊ΈࠐΜͩimage • ↑ʹad-serverͱͯ͠ඞཁͳGemΛ૊ΈࠐΉ 18 VASILY Inc. 2016
  9. 19.

    #2-2 docker image࡟ݮ ad-server • alpine linuxΛϕʔεʹͯ͠build • ॳظ •

    build͢Δ͚ͩͯ͠૟আΛ͍ͯ͠ͳ͍ঢ়ଶ 19 VASILY Inc. 2016
  10. 21.

    #2-2 docker image࡟ݮ ad-server • ਫ਼ࠪ1 • mysql (mariadb-dev) ͕ͱͯ΋େ͖͍...

    • ෆཁͳόΠφϦΛ࡟আ rm /usr/lib/libmysqld* && \ rm /usr/bin/mysql* 21 VASILY Inc. 2016
  11. 23.

    #2 docker imageංେԽ • ࠷৽ͷ docker image size • ad_server:

    924.6MB -> 342.7MB • nginx: 134.1MB -> 59.63MB • td-agent: 448.4MB -> 430.8MB • mackerel-agent: 423.1MB -> 357.5MB 23 VASILY Inc. 2016
  12. 25.

    #3 base platform • EB w/multi-container docker • Amazon Linux

    ͱ dockerͷ૊Έ߹ΘͤͰෳ਺ 25 VASILY Inc. 2016
  13. 26.

    #3 base platform • EB w/multi-container docker • Amazon Linux

    2015.03 v1.4.0 docker 1.6.0 • Amazon Linux 2015.03 v1.4.1 docker 1.6.0 • Amazon Linux 2015.03 v1.4.6 docker 1.6.2 • Amazon Linux 2015.09 v2.0.8 docker 1.9.1 • Amazon Linux 2016.03 v2.1.0 docker 1.9.1 • Amazon Linux 2016.03 v2.1.1 docker 1.9.1 26 VASILY Inc. 2016
  14. 27.

    #3 base platform • ͍·͸গ͠ݹΊversionΛར༻ • Amazon Linux 2015.09 v2.0.8

    docker 1.9.1 • Amazon Linux 2016.03͸mackerel-agent͕ىಈͰ͖ͳ͍໰୊ • docker͔ͩΒؔ܎ͳ͍ͱࢥ͍ͬͯͨΒىಈͰ͖ͳ͔ͬͨ.. 27 VASILY Inc. 2016
  15. 29.

    #3 base platform • ࠓݟͨΒ, Amazon Linux 2016.03 docker 1.11.1

    ͕Ͱͯͨ • ໌೔ࢼ͍ͨ͠ͱ͓΋͍·͢ 29 VASILY Inc. 2016
  16. 36.

    #5 Time Scheduled-based auto- scaling • ͱ͋ΔΞϓϦ͕pushΛ͏ͭ • spike͢Δ •

    ௨ৗͷauto-scalingͰ͸௥͍͔ͭͳ͍ • pushʹΑΔspike͕ऩ·ͬͨ͋ͱʹىಈ׬ྃ.. • ҙຯͳ͍ 36 VASILY Inc. 2016
  17. 37.

    #5 Time Scheduled-based auto- scaling • time scheduled base auto-scalingΛઃఆ

    • web console͔Β͸ઃఆͰ͖ͳ͍ͷͰaws cliͰߦ͏ 37 VASILY Inc. 2016
  18. 38.

    #5 Time Scheduled-based auto- scaling • time scheduled base auto-scalingΛઃఆ

    • scale out aws autoscaling put-scheduled-update-group-action \ --auto-scaling-group-name awseb-e-XXXXXX \ --scheduled-action-name ScaleOut \ --recurrence "00 10 * * *" \ --desired-capacity 10 \ --min-size 10 \ --max-size 12 38 VASILY Inc. 2016
  19. 39.

    #5 Time Scheduled-based auto- scaling • time scheduled base auto-scalingΛઃఆ

    • scale in aws autoscaling put-scheduled-update-group-action \ --auto-scaling-group-name awseb-e-XXXXXX \ --scheduled-action-name ScaleIn \ --recurrence "30 16 * * *" \ --desired-capacity 6 \ --min-size 6 \ --max-size 8 39 VASILY Inc. 2016
  20. 43.

    #7 Docker build • Dockerfile͸ผrepositoryͰGithubͰ؅ཧ • docker resitory(quay.io)ͷbuild hook͸ະ࠾༻ •

    खݩͷMBP(docker-machine) Ͱdocker build & pushͰӡ༻ • docker build -t quay.io/vasilyjp/ruby:2.3.1-alpine . • docker push quay.io/vasilyjp/ruby:2.3.1-alpine 43 VASILY Inc. 2016
  21. 57.

    #2 mackerel-agentͷΞϥʔτ • base platformΛมߋͨ͠Β͏·͍͘͘Α͏ʹͳͬͨ • Amazon Linux 2015.09 v2.0.8

    docker 1.9.1 • ؀ڥʹΑͬͯtrap͕ಈ͍ͨΓಈ͔ͳ͔ͬͨΓͯ͠Δͱͷ͜ͱ.. • EBͰͷঢ়گ͸Qiitaʹॻ͍ͯ·͢ -> http://qiita.com/tknzk/items/a6b1a9003ff26683b88a 57 VASILY Inc. 2016
  22. 63.

    #4 awsebcli ʹΑΔσϓϩΠʹࣦഊ(Ұ ౓͖Γ) • deploy͸CircleCI͔Β eb deploy ad-server-prod ͰσϓϩΠ

    • applicationΛzipͨ͠ϑΝΠϧΛs3ʹput • ֤instance͕pullͯ͘͠Δ 63 VASILY Inc. 2016
  23. 64.

    #4 awsebcli ʹΑΔσϓϩΠʹࣦഊ(Ұ ౓͖Γ) • ಛఆͷawsebcli͸ s3ʹpush͢ΔϑΝΠϧ໊͕gitͷcommithash 4 ܻͰݻఆ •

    ຊ൪ӡ༻։࢝લʹੜ੒ͨ͠ϑΝΠϧ໊ͱconflictͯ͠deploy͕ί έͨ • ͜ͷ࣌͸ίέ͚͕ͨͩͩͬͨ੒ޭ͢ΔͱσάϨ͢Δ͜ͱʹͳͬ ͨ 64 VASILY Inc. 2016
  24. 67.

    Conclusion • 1೥ؒ EB w/multi-container docker Ͱӡ༻ͯ͠ى͖ͨ͜ͱΛ͓࿩ ͠·ͨ͠ • ӡ༻ϛεʹΑΔো֐΋͋Γ·͕ͨ͠ɺEB΍dockerʹىҼ͢Δো

    ֐͸ͳ͠ • alpine linuxΛproductionͰӡ༻தͰ͢ • iQONຊମͷAPI΋͍ͣΕ͸... 67 VASILY Inc. 2016