Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Container Cluster Strategy for PHP

Container Cluster Strategy for PHP

PHPアプリケーションにおけるDockerコンテナ・クラスター戦略
https://saien.connpass.com/event/69375/
#再演 #phpcon2017

https://github.com/nishigori/phpcon2017-presentation

07fe43bbec550b3230b3a7f9a81de7cc?s=128

Takuya Nishigori

November 02, 2017
Tweet

Transcript

  1. Container Cluster Strategy ~ How design for the PHP web

    applica6on? ~ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 1
  2. Who are you? ɹTwi$er: @_nishigori • VOYAGE GROUP, Inc. SRE͑Μ͡ʹΌʔ

    • ݩʑΞϓϦέʔγϣϯॻ͍ͯͯঃʑʹM/W౳Լʹ߱Γͯͬͨਓ • PHP͸ͻͨ͢Βόʔδϣϯ্͛ͨΓɺOPcache࠷దԽਤͬͨΓ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 2
  3. Agenda • What Docker / Container ? • Orchestra4on •

    Build & Deployment • Capacity Planning (& Monitoring) • History & Future Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 3
  4. What Docker? Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 4

  5. What Docker? Docker is the world's leading so3ware containeriza5on pla7orm.

    -- h$ps:/ /www.docker.com/what-docker Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 5
  6. ʮ͜ΜͯͳΓͥʔ͠ΐΜʯ ʮ΀Βͬͱ;͒ʔΉʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 6

  7. What is the meaning of pla$orm? Kernel1 ͷػೳ΍༷ʑͳϥΠϒϥϦΛར༻ͯ͠ίϯςφΛಈ͔͍ͯ͠Δ containerd &

    runC, rkt, Cgroups, Storage, Namespaces, Networking, Security pivot_root, chroot, etc ... ʹ ͻͱͭͷٕज़͚ͩͰ͸ͳ͍ɾͻͱͭͻͱͭ͸୅ସՄೳ 1 ͜ͷൃදͰ͸Linux্ͷDockerʹ͍ͭͯͷΈݴٴ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 7
  8. What is the meaning of pla$orm? Code as: DockerfileͰContainer ImageΛ

    ͲͷΑ͏ʹߏங͞ΕΔ͔දݱ # Dockerfile FROM nginx:alpine RUN rm -f /etc/nginx/conf.d/*.conf ADD nginx.conf /etc/nginx/nginx.conf RUN nginx -t EXPOSE 80 Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 8
  9. What is the meaning of containerize? ίϯςφΛѻ͏ͨΊͷΠϯλʔϑΣʔε ͕ἧ͍ͬͯΔ $ docker

    pull / push / build / run / image / container / ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 9
  10. What Container? Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 10

  11. What Container? > Package so+ware into standardized units for development,

    shipment and deployment A container image is a lightweight, stand- alone, executable package of a piece of so:ware that includes everything needed to run it: code, run>me, system tools, system libraries, se?ngs. -- h$ps:/ /www.docker.com/what- container Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 11
  12. ʮ͜Μͯͳʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 12

  13. PHPʹ౰ͯ͸ΊͯΈΔ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 13

  14. 2 ... executable package of a piece of so3ware that

    includes everything needed to run it: ... 2 h$ps:/ /www.docker.com/what-container ΑΓҾ༻ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 14
  15. needed to run it: run$me Docker Cluster Strategy - @_nishigori

    #phpcon2017 #࠶ԋ 15
  16. needed to run it: system tools, system libraries libxml /

    libicu / openssl / libcurl / ... needed to run it: se#ngs php.ini / www.conf for php-fpm / ... ports? / memory limit / cpu unit / ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 16
  17. needed to run it: code • Your code • e.g.

    GitHub repository code • Dependencies code • e.g. from composer libraries Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 17
  18. ͋ͳͨͷPHPΞϓϦέʔγϣϯͷ needed run it: * Λ೺Ѳ͠ Container ImageΛ࡞੒͢Δ ᐌ͘: •

    stand-alone ʢcontainer run,me͑͋͞Ε͹େৎ෉ͦ͏ͩʣ • lightweightʢඞཁͳ΋ͷ͔͠ೖ͍ͬͯͳ͍ʣ • executable package ʢඞཁͳ΋ͷ͸શͯἧͬͨͧ͌ʣ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 18
  19. ʮσϞͯ͠ΈΑʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 19

  20. ʮ͜Μͯͳɺ͢͝ʔ͍ʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 20

  21. [nits] Docker Build Pa4ern for PHP • Links container memory

    ... • memory_limit • opcache.memory_consumption • opcache.validate_timestamps=0 • ʢڪΒ͘ʣϦϦʔε͸ৗʹίϯςφͷೖΕସ͑ͳͷͰɺphp fileͷߋ৽೔࣌ΛݟΔඞཁ͸ͳ͍ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 21
  22. Orchestra)on Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 22

  23. What is Orchestra,on • εέδϡʔϦϯά • εέʔϧ • ϓϩϏδϣχϯά •

    ࣗಈσϓϩΠ • Ϋϥελʔ؅ཧ • etc ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 23
  24. Orchestra)on Public cloud provider: • Amazon EC2 Container Service (ECS)

    • Azure Container Service (AKS) • Docker Enterprise Edi<on (Docker EE) • Google Container Service (GKE) Open Source: Docker Swarm / Kuberenes / Marathon / ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 24
  25. ʮ͹ͳͳʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 25

  26. 3 Orchestra)on͸ToolʹΑͬͯ༻ޠɾػೳ͕ҧ͏ͷͰɺ͜Ε͔Β͸ AWS ECSΛྫʹ࿩͠·͢ h"ps:/ /github.com/nishigori/phpcon2017-presenta9on 3 h$ps:/ /aws.amazon.com/ecs/details/ Docker

    Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 26
  27. Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 27

  28. ECS Task • A group of one or more containers

    • ײ֮ͱͯ͠͸ docker-composer up op2onsʹ͍͔ۙ΋ • GKEͰ͍͏Podsͷ֓೦ʹ૬౰ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 28
  29. ECS Service • ECS TaskఆٛͰࢦఆ͞Εͨ਺ͷίϯς φΛ഑ஔ • + AutoScaling •

    Service Load Balancing • ϗετ্ʹίϯςφͷϙʔτΛࣗಈ ׂΓ౰ͯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 29
  30. ECS Cluster • ECS TaskΛ഑ஔ͢ΔίϯςφΠϯελ ϯεͷ࿦ཧάϧʔϓ • EC2 AutoScaling Group

    • EC2 Instance(s) • +AutoScaling Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 30
  31. ECS Cluster Q. Which one should we choose Host OS?

    A. ࣮ʹͨ͘͞Μ͋Δ… (AWS) Amazon ECS Op0mized AMI Container-Op0mized OS (Google) ContainerLinux (CoreOS) Barge OS etc ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 31
  32. ECS Cluster Q. Which one should we choose Host OS?

    • ͨ͘͞Μߟྀ͍ͯ͜͠͏ • Kernel version • Docker version • Launch 3me • etc ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 32
  33. ECS Cluster - Containerͱͷ௼Γ߹͍ • Memory / CPU Reserva1on •

    Memory / CPU Usage docker run --mem=1024m ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 33
  34. Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 34

  35. Deployment Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 35

  36. Deployment - rolling update 1. ৽͍͠ίϯςφͷ্ཱͪ͛ 2. ELB͔ΒαʔϏεΠϯ 3. ݹ͍ίϯςφΛഁغ

    Λ܁Γฦͯ͠શίϯςφΛೖΕସ͍͑ͯ͘ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 36
  37. Deployment - rolling update demo? Docker Cluster Strategy - @_nishigori

    #phpcon2017 #࠶ԋ 37
  38. Deployment for Ops (team) ~ rolling update to the ECS

    Cluster ~ • Drain containers Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 38
  39. Deployment - Canary -- h$ps:/ /mar,nfowler.com/bliki/CanaryRelease.html Docker Cluster Strategy -

    @_nishigori #phpcon2017 #࠶ԋ 39
  40. Deployment - Canary Q. Ͳ͔͜Β੾Γ෼͚Δʁ 1. Load Balancer? 2. ECS

    Cluster? 3. ECS Service? 4. Others? Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 40
  41. Deployment - Canary Case of Docker EE: • Docker EE

    Release Candidate • Customer Zero ਖ਼ʹ޷͖ʹ࢖͑ΑελΠϧ Detail of dockercon EU 2017 - h7ps:/ /europe-2017.dockercon.com/ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 41
  42. Deployment - Container Image Management - ׂѪ - Docker Cluster

    Strategy - @_nishigori #phpcon2017 #࠶ԋ 42
  43. Deployment case of PHP OPcache / APCu ౳ɺαʔϏεΠϯલʹWarmup͍ͨ͠ => ENTRYPOINT

    Ͱఆٛͨ͠εΫϦϓτ಺Ͱ͝ʹΐ͝ʹΐͯ͠Έ Α͏ # Dockerfile FROM xxx # .... ENTRYPOINT ["entrypint-app.sh"] CMD ["--env", "production"] Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 43
  44. Logging Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 44

  45. Logging ίϯςφʹʢجຊʣσʔλ͸࢒͞ͳ͍ log΋ྫ֎Ͱ͸ͳ͍ • Log as data • ΤʔδΣϯτʹΑΔPushܕ •

    fluentd • cloudwatch log agent • ͍ΘΏΔதԝूݖܕʢCentralized loggingʣ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 45
  46. Logging on Dockerfile h"ps:/ /github.com/nginxinc/docker-nginx/blob/3ba04e3/mainline/ stretch/Dockerfile#L91-L93 # forward request and

    error logs to docker log collector RUN ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 46
  47. Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 47

  48. ʮAWS ECS͔͓ͭʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 48

  49. Ψο4 4 ECS͕μϝͬͯݴͬͯΔΜ͡Όͳ͍Yo Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ

    49
  50. ʮGKEʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 50

  51. Ψο5 5 k8s͍͍Αk8s Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ

    51
  52. ʮ…AKEͰ͕Μ̱ʢ͈́ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 52

  53. Ψο6 6 AKE͕μϝͬͯݴͬͯΔΜ͡Όͳ͍Yo Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ

    53
  54. Public Container Service͸पลٕज़΋େࣄ5 • (GCP) Load Balancer / BQ /

    ... • (AWS) S3 / SNS / SQS / ... • ... • SLA? • Using Reserved Instance? 5 k8s͍͍Αk8s Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 54
  55. ࠷ޙʹɺDocker࢙ͱ͜Ε͔Βʢ༧ଌʣΛগ͠ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 55

  56. History 2013.03~ Docker 2014.06~ Docker Hub 2014.06~ Kubernetes (k8s, kube)

    2014.11~ Amazon EC2 Container Service (ECS) 2014.11~ Google Container Engine (GKE) 2014.12~ CoreOS Rocket 2014.12~ Docker Machine / Swarm / Compose 2015.07~ Cloud Native Computing Foundation (CNCF) 2015.08~ VMware vSphere Integrated Containers 2015.09~ Microsoft Azure Container Services (ݱAKS) 2017.03~ Docker Enterprise Edition (Docker EE) 2017.04~ Moby Project 2017.07~ Microsoft Azure Container Instance (ACI) Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 56
  57. History (Orchestra/on) 2013.03~ Docker 2014.06~ Docker Hub 2014.06~ Kubernetes (k8s,

    kube) 2014.11~ Amazon EC2 Container Service (ECS) 2014.11~ Google Container Engine (GKE) 2014.12~ CoreOS Rocket 2014.12~ Docker Machine / Swarm / Compose 2015.07~ Cloud Native Computing Foundation (CNCF) 2015.08~ VMware vSphere Integrated Containers 2015.09~ Microsoft Azure Container Services (ݱAKS) 2017.03~ Docker Enterprise Edition (Docker EE) 2017.04~ Moby Project 2017.07~ Microsoft Azure Container Instance (ACI) Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 57
  58. History (Orchestra/on) • Docker for Win/MacͰKubernetes؀ڥ΋ߏஙՄೳʹ • 2017.11ݱࡏɺϕʔλ൛ͷਃ͠ࠐΈ͕Մೳ • h;ps:/

    /beta.docker.com/ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 58
  59. History (Managed Service) 2013.03~ Docker 2014.06~ Docker Hub 2014.06~ Kubernetes

    (k8s, kube) 2014.11~ Amazon EC2 Container Service (ECS) 2014.11~ Google Container Engine (GKE) 2014.12~ CoreOS Rocket 2014.12~ Docker Machine / Swarm / Compose 2015.07~ Cloud Native Computing Foundation (CNCF) 2015.08~ VMware vSphere Integrated Containers 2015.09~ Microsoft Azure Container Services (ݱAKS) 2017.03~ Docker Enterprise Edition (Docker EE) 2017.04~ Moby Project 2017.07~ Microsoft Azure Container Instance (ACI) Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 59
  60. History (Managed Service) • 2014.05࣌఺ͰGoogle͸20ԯҎ্ͷίϯςφΛՔಇ͍ͯͨ͠ • h.ps:/ /speakerdeck.com/jbeda/containers-at-scale • AWS

    Lambda΋ίϯςφٕज़Λར༻͍ͯ͠Δ • Public Cloudͱͯ͠΋Container Service͸ొ৔͔ͯ͠Βطʹ3೥ Ҏ্ܦ͍ͬͯΔ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 60
  61. History (project / founda4on) 2013.03~ Docker 2014.06~ Docker Hub 2014.06~

    Kubernetes (k8s, kube) 2014.11~ Amazon EC2 Container Service (ECS) 2014.11~ Google Container Engine (GKE) 2014.12~ CoreOS Rocket 2014.12~ Docker Machine / Swarm / Compose 2015.07~ Cloud Native Computing Foundation (CNCF) 2015.08~ VMware vSphere Integrated Containers 2015.09~ Microsoft Azure Container Services (ݱAKS) 2017.03~ Docker Enterprise Edition (Docker EE) 2017.04~ Moby Project 2017.07~ Microsoft Azure Container Instance (ACI) Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 61
  62. History (project / founda4on) Cloud Na)ve Compu)ng Founda)on h2ps:/ /www.cncf.io/

    • AWS͸2017.08~ joined • AWS Lambda౳Ͱ࢖ΘΕ͍ͯΔίϯςφपΓͷϊ΢ϋ΢͕ϑ ΟʔυόοΫ͞ΕΔΜ͡Όͳ͍͔ͱݸਓతʹظ଴ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 62
  63. Thank you By @_nishigori Docker Cluster Strategy - @_nishigori #phpcon2017

    #࠶ԋ 63
  64. btw: Cleanup aws for this presenta3on # https://github.com/nishigori/phpcon2017-presentation $ make

    -C infrastructure destroy Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 64