Container Cluster Strategy for PHP

PHPアプリケーションにおけるDockerコンテナ・クラスター戦略
https://saien.connpass.com/event/69375/
#再演 #phpcon2017

https://github.com/nishigori/phpcon2017-presentation

Transcript

1. Container Cluster Strategy ~ How design for the PHP web applica6on? ~ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 1
2. Who are you? ɹTwi$er: @_nishigori • VOYAGE GROUP, Inc. SRE͑Μ͡ʹΌʔ • ݩʑΞϓϦέʔγϣϯॻ͍ͯͯঃʑʹM/W౳Լʹ߱Γͯͬͨਓ • PHP͸ͻͨ͢Βόʔδϣϯ্͛ͨΓɺOPcache࠷దԽਤͬͨΓ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 2
3. Agenda • What Docker / Container ? • Orchestra4on • Build & Deployment • Capacity Planning (& Monitoring) • History & Future Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 3
4. What Docker? Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 4
5. What Docker? Docker is the world's leading so3ware containeriza5on pla7orm. -- h$ps:/ /www.docker.com/what-docker Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 5
6. ʮ͜ΜͯͳΓͥʔ͠ΐΜʯ ʮ΀Βͬͱ;͒ʔΉʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 6
7. What is the meaning of pla$orm? Kernel1 ͷػೳ΍༷ʑͳϥΠϒϥϦΛར༻ͯ͠ίϯςφΛಈ͔͍ͯ͠Δ containerd & runC, rkt, Cgroups, Storage, Namespaces, Networking, Security pivot_root, chroot, etc ... ʹ ͻͱͭͷٕज़͚ͩͰ͸ͳ͍ɾͻͱͭͻͱͭ͸୅ସՄೳ 1 ͜ͷൃදͰ͸Linux্ͷDockerʹ͍ͭͯͷΈݴٴ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 7
8. What is the meaning of pla$orm? Code as: DockerfileͰContainer ImageΛ ͲͷΑ͏ʹߏங͞ΕΔ͔දݱ # Dockerfile FROM nginx:alpine RUN rm -f /etc/nginx/conf.d/*.conf ADD nginx.conf /etc/nginx/nginx.conf RUN nginx -t EXPOSE 80 Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 8
9. What is the meaning of containerize? ίϯςφΛѻ͏ͨΊͷΠϯλʔϑΣʔε ͕ἧ͍ͬͯΔ $ docker pull / push / build / run / image / container / ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 9
10. What Container? Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 10
11. What Container? > Package so+ware into standardized units for development, shipment and deployment A container image is a lightweight, stand- alone, executable package of a piece of so:ware that includes everything needed to run it: code, run>me, system tools, system libraries, se?ngs. -- h$ps:/ /www.docker.com/what- container Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 11
12. ʮ͜Μͯͳʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 12
13. PHPʹ౰ͯ͸ΊͯΈΔ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 13
14. 2 ... executable package of a piece of so3ware that includes everything needed to run it: ... 2 h$ps:/ /www.docker.com/what-container ΑΓҾ༻ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 14
15. needed to run it: run$me Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 15
16. needed to run it: system tools, system libraries libxml / libicu / openssl / libcurl / ... needed to run it: se#ngs php.ini / www.conf for php-fpm / ... ports? / memory limit / cpu unit / ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 16
17. needed to run it: code • Your code • e.g. GitHub repository code • Dependencies code • e.g. from composer libraries Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 17
18. ͋ͳͨͷPHPΞϓϦέʔγϣϯͷ needed run it: * Λ೺Ѳ͠ Container ImageΛ࡞੒͢Δ ᐌ͘: • stand-alone ʢcontainer run,me͑͋͞Ε͹େৎ෉ͦ͏ͩʣ • lightweightʢඞཁͳ΋ͷ͔͠ೖ͍ͬͯͳ͍ʣ • executable package ʢඞཁͳ΋ͷ͸શͯἧͬͨͧ͌ʣ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 18
19. ʮσϞͯ͠ΈΑʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 19
20. ʮ͜Μͯͳɺ͢͝ʔ͍ʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 20
21. [nits] Docker Build Pa4ern for PHP • Links container memory ... • memory_limit • opcache.memory_consumption • opcache.validate_timestamps=0 • ʢڪΒ͘ʣϦϦʔε͸ৗʹίϯςφͷೖΕସ͑ͳͷͰɺphp fileͷߋ৽೔࣌ΛݟΔඞཁ͸ͳ͍ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 21
22. Orchestra)on Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 22
23. What is Orchestra,on • εέδϡʔϦϯά • εέʔϧ • ϓϩϏδϣχϯά • ࣗಈσϓϩΠ • Ϋϥελʔ؅ཧ • etc ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 23
24. Orchestra)on Public cloud provider: • Amazon EC2 Container Service (ECS) • Azure Container Service (AKS) • Docker Enterprise Edi<on (Docker EE) • Google Container Service (GKE) Open Source: Docker Swarm / Kuberenes / Marathon / ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 24
25. ʮ͹ͳͳʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 25
26. 3 Orchestra)on͸ToolʹΑͬͯ༻ޠɾػೳ͕ҧ͏ͷͰɺ͜Ε͔Β͸ AWS ECSΛྫʹ࿩͠·͢ h"ps:/ /github.com/nishigori/phpcon2017-presenta9on 3 h$ps:/ /aws.amazon.com/ecs/details/ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 26
27. Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 27
28. ECS Task • A group of one or more containers • ײ֮ͱͯ͠͸ docker-composer up op2onsʹ͍͔ۙ΋ • GKEͰ͍͏Podsͷ֓೦ʹ૬౰ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 28
29. ECS Service • ECS TaskఆٛͰࢦఆ͞Εͨ਺ͷίϯς φΛ഑ஔ • + AutoScaling • Service Load Balancing • ϗετ্ʹίϯςφͷϙʔτΛࣗಈ ׂΓ౰ͯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 29
30. ECS Cluster • ECS TaskΛ഑ஔ͢ΔίϯςφΠϯελ ϯεͷ࿦ཧάϧʔϓ • EC2 AutoScaling Group • EC2 Instance(s) • +AutoScaling Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 30
31. ECS Cluster Q. Which one should we choose Host OS? A. ࣮ʹͨ͘͞Μ͋Δ… (AWS) Amazon ECS Op0mized AMI Container-Op0mized OS (Google) ContainerLinux (CoreOS) Barge OS etc ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 31
32. ECS Cluster Q. Which one should we choose Host OS? • ͨ͘͞Μߟྀ͍ͯ͜͠͏ • Kernel version • Docker version • Launch 3me • etc ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 32
33. ECS Cluster - Containerͱͷ௼Γ߹͍ • Memory / CPU Reserva1on • Memory / CPU Usage docker run --mem=1024m ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 33
34. Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 34
35. Deployment Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 35
36. Deployment - rolling update 1. ৽͍͠ίϯςφͷ্ཱͪ͛ 2. ELB͔ΒαʔϏεΠϯ 3. ݹ͍ίϯςφΛഁغ Λ܁Γฦͯ͠શίϯςφΛೖΕସ͍͑ͯ͘ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 36
37. Deployment - rolling update demo? Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 37
38. Deployment for Ops (team) ~ rolling update to the ECS Cluster ~ • Drain containers Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 38
39. Deployment - Canary -- h$ps:/ /mar,nfowler.com/bliki/CanaryRelease.html Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 39
40. Deployment - Canary Q. Ͳ͔͜Β੾Γ෼͚Δʁ 1. Load Balancer? 2. ECS Cluster? 3. ECS Service? 4. Others? Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 40
41. Deployment - Canary Case of Docker EE: • Docker EE Release Candidate • Customer Zero ਖ਼ʹ޷͖ʹ࢖͑ΑελΠϧ Detail of dockercon EU 2017 - h7ps:/ /europe-2017.dockercon.com/ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 41
42. Deployment - Container Image Management - ׂѪ - Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 42
43. Deployment case of PHP OPcache / APCu ౳ɺαʔϏεΠϯલʹWarmup͍ͨ͠ => ENTRYPOINT Ͱఆٛͨ͠εΫϦϓτ಺Ͱ͝ʹΐ͝ʹΐͯ͠Έ Α͏ # Dockerfile FROM xxx # .... ENTRYPOINT ["entrypint-app.sh"] CMD ["--env", "production"] Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 43
44. Logging Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 44
45. Logging ίϯςφʹʢجຊʣσʔλ͸࢒͞ͳ͍ log΋ྫ֎Ͱ͸ͳ͍ • Log as data • ΤʔδΣϯτʹΑΔPushܕ • fluentd • cloudwatch log agent • ͍ΘΏΔதԝूݖܕʢCentralized loggingʣ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 45
46. Logging on Dockerfile h"ps:/ /github.com/nginxinc/docker-nginx/blob/3ba04e3/mainline/ stretch/Dockerfile#L91-L93 # forward request and error logs to docker log collector RUN ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 46
47. Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 47
48. ʮAWS ECS͔͓ͭʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 48
49. Ψο4 4 ECS͕μϝͬͯݴͬͯΔΜ͡Όͳ͍Yo Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 49
50. ʮGKEʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 50
51. Ψο5 5 k8s͍͍Αk8s Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 51
52. ʮ…AKEͰ͕Μ̱ʢ͈́ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 52
53. Ψο6 6 AKE͕μϝͬͯݴͬͯΔΜ͡Όͳ͍Yo Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 53
54. Public Container Service͸पลٕज़΋େࣄ5 • (GCP) Load Balancer / BQ / ... • (AWS) S3 / SNS / SQS / ... • ... • SLA? • Using Reserved Instance? 5 k8s͍͍Αk8s Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 54
55. ࠷ޙʹɺDocker࢙ͱ͜Ε͔Βʢ༧ଌʣΛগ͠ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 55
56. History 2013.03~ Docker 2014.06~ Docker Hub 2014.06~ Kubernetes (k8s, kube) 2014.11~ Amazon EC2 Container Service (ECS) 2014.11~ Google Container Engine (GKE) 2014.12~ CoreOS Rocket 2014.12~ Docker Machine / Swarm / Compose 2015.07~ Cloud Native Computing Foundation (CNCF) 2015.08~ VMware vSphere Integrated Containers 2015.09~ Microsoft Azure Container Services (ݱAKS) 2017.03~ Docker Enterprise Edition (Docker EE) 2017.04~ Moby Project 2017.07~ Microsoft Azure Container Instance (ACI) Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 56
57. History (Orchestra/on) 2013.03~ Docker 2014.06~ Docker Hub 2014.06~ Kubernetes (k8s, kube) 2014.11~ Amazon EC2 Container Service (ECS) 2014.11~ Google Container Engine (GKE) 2014.12~ CoreOS Rocket 2014.12~ Docker Machine / Swarm / Compose 2015.07~ Cloud Native Computing Foundation (CNCF) 2015.08~ VMware vSphere Integrated Containers 2015.09~ Microsoft Azure Container Services (ݱAKS) 2017.03~ Docker Enterprise Edition (Docker EE) 2017.04~ Moby Project 2017.07~ Microsoft Azure Container Instance (ACI) Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 57
58. History (Orchestra/on) • Docker for Win/MacͰKubernetes؀ڥ΋ߏஙՄೳʹ • 2017.11ݱࡏɺϕʔλ൛ͷਃ͠ࠐΈ͕Մೳ • h;ps:/ /beta.docker.com/ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 58
59. History (Managed Service) 2013.03~ Docker 2014.06~ Docker Hub 2014.06~ Kubernetes (k8s, kube) 2014.11~ Amazon EC2 Container Service (ECS) 2014.11~ Google Container Engine (GKE) 2014.12~ CoreOS Rocket 2014.12~ Docker Machine / Swarm / Compose 2015.07~ Cloud Native Computing Foundation (CNCF) 2015.08~ VMware vSphere Integrated Containers 2015.09~ Microsoft Azure Container Services (ݱAKS) 2017.03~ Docker Enterprise Edition (Docker EE) 2017.04~ Moby Project 2017.07~ Microsoft Azure Container Instance (ACI) Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 59
60. History (Managed Service) • 2014.05࣌఺ͰGoogle͸20ԯҎ্ͷίϯςφΛՔಇ͍ͯͨ͠ • h.ps:/ /speakerdeck.com/jbeda/containers-at-scale • AWS Lambda΋ίϯςφٕज़Λར༻͍ͯ͠Δ • Public Cloudͱͯ͠΋Container Service͸ొ৔͔ͯ͠Βطʹ3೥ Ҏ্ܦ͍ͬͯΔ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 60
61. History (project / founda4on) 2013.03~ Docker 2014.06~ Docker Hub 2014.06~ Kubernetes (k8s, kube) 2014.11~ Amazon EC2 Container Service (ECS) 2014.11~ Google Container Engine (GKE) 2014.12~ CoreOS Rocket 2014.12~ Docker Machine / Swarm / Compose 2015.07~ Cloud Native Computing Foundation (CNCF) 2015.08~ VMware vSphere Integrated Containers 2015.09~ Microsoft Azure Container Services (ݱAKS) 2017.03~ Docker Enterprise Edition (Docker EE) 2017.04~ Moby Project 2017.07~ Microsoft Azure Container Instance (ACI) Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 61
62. History (project / founda4on) Cloud Na)ve Compu)ng Founda)on h2ps:/ /www.cncf.io/ • AWS͸2017.08~ joined • AWS Lambda౳Ͱ࢖ΘΕ͍ͯΔίϯςφपΓͷϊ΢ϋ΢͕ϑ ΟʔυόοΫ͞ΕΔΜ͡Όͳ͍͔ͱݸਓతʹظ଴ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 62
63. Thank you By @_nishigori Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 63
64. btw: Cleanup aws for this presenta3on # https://github.com/nishigori/phpcon2017-presentation $ make -C infrastructure destroy Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 64