Container Cluster Strategy for PHP

Slide 1

Slide 1 text

Container Cluster Strategy ~ How design for the PHP web applica6on? ~ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 1

Slide 2

Slide 2 text

Who are you? ɹTwi$er: @_nishigori • VOYAGE GROUP, Inc. SRE͑Μ͡ʹΌʔ • ݩʑΞϓϦέʔγϣϯॻ͍ͯͯঃʑʹM/W౳Լʹ߱Γͯͬͨਓ • PHP͸ͻͨ͢Βόʔδϣϯ্͛ͨΓɺOPcache࠷దԽਤͬͨΓ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 2

Slide 3

Slide 3 text

Agenda • What Docker / Container ? • Orchestra4on • Build & Deployment • Capacity Planning (& Monitoring) • History & Future Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 3

Slide 4

Slide 4 text

What Docker? Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 4

Slide 5

Slide 5 text

What Docker? Docker is the world's leading so3ware containeriza5on pla7orm. -- h$ps:/ /www.docker.com/what-docker Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 5

Slide 6

Slide 6 text

ʮ͜ΜͯͳΓͥʔ͠ΐΜʯ ʮ΀Βͬͱ;͒ʔΉʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 6

Slide 7

Slide 7 text

What is the meaning of pla$orm? Kernel1 ͷػೳ΍༷ʑͳϥΠϒϥϦΛར༻ͯ͠ίϯςφΛಈ͔͍ͯ͠Δ containerd & runC, rkt, Cgroups, Storage, Namespaces, Networking, Security pivot_root, chroot, etc ... ʹ ͻͱͭͷٕज़͚ͩͰ͸ͳ͍ɾͻͱͭͻͱͭ͸୅ସՄೳ 1 ͜ͷൃදͰ͸Linux্ͷDockerʹ͍ͭͯͷΈݴٴ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 7

Slide 8

Slide 8 text

What is the meaning of pla$orm? Code as: DockerﬁleͰContainer ImageΛ ͲͷΑ͏ʹߏங͞ΕΔ͔දݱ # Dockerfile FROM nginx:alpine RUN rm -f /etc/nginx/conf.d/*.conf ADD nginx.conf /etc/nginx/nginx.conf RUN nginx -t EXPOSE 80 Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 8

Slide 9

Slide 9 text

What is the meaning of containerize? ίϯςφΛѻ͏ͨΊͷΠϯλʔϑΣʔε ͕ἧ͍ͬͯΔ $ docker pull / push / build / run / image / container / ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 9

Slide 10

Slide 10 text

What Container? Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 10

Slide 11

Slide 11 text

What Container? > Package so+ware into standardized units for development, shipment and deployment A container image is a lightweight, stand- alone, executable package of a piece of so:ware that includes everything needed to run it: code, run>me, system tools, system libraries, se?ngs. -- h$ps:/ /www.docker.com/what- container Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 11

Slide 12

Slide 12 text

ʮ͜Μͯͳʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 12

Slide 13

Slide 13 text

PHPʹ౰ͯ͸ΊͯΈΔ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 13

Slide 14

Slide 14 text

2 ... executable package of a piece of so3ware that includes everything needed to run it: ... 2 h$ps:/ /www.docker.com/what-container ΑΓҾ༻ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 14

Slide 15

Slide 15 text

needed to run it: run$me Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 15

Slide 16

Slide 16 text

needed to run it: system tools, system libraries libxml / libicu / openssl / libcurl / ... needed to run it: se#ngs php.ini / www.conf for php-fpm / ... ports? / memory limit / cpu unit / ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 16

Slide 17

Slide 17 text

needed to run it: code • Your code • e.g. GitHub repository code • Dependencies code • e.g. from composer libraries Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 17

Slide 18

Slide 18 text

͋ͳͨͷPHPΞϓϦέʔγϣϯͷ needed run it: * Λ೺Ѳ͠ Container ImageΛ࡞੒͢Δ ᐌ͘: • stand-alone ʢcontainer run,me͑͋͞Ε͹େৎ෉ͦ͏ͩʣ • lightweightʢඞཁͳ΋ͷ͔͠ೖ͍ͬͯͳ͍ʣ • executable package ʢඞཁͳ΋ͷ͸શͯἧͬͨͧ͌ʣ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 18

Slide 19

Slide 19 text

ʮσϞͯ͠ΈΑʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 19

Slide 20

Slide 20 text

ʮ͜Μͯͳɺ͢͝ʔ͍ʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 20

Slide 21

Slide 21 text

[nits] Docker Build Pa4ern for PHP • Links container memory ... • memory_limit • opcache.memory_consumption • opcache.validate_timestamps=0 • ʢڪΒ͘ʣϦϦʔε͸ৗʹίϯςφͷೖΕସ͑ͳͷͰɺphp ﬁleͷߋ৽೔࣌ΛݟΔඞཁ͸ͳ͍ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 21

Slide 22

Slide 22 text

Orchestra)on Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 22

Slide 23

Slide 23 text

What is Orchestra,on • εέδϡʔϦϯά • εέʔϧ • ϓϩϏδϣχϯά • ࣗಈσϓϩΠ • Ϋϥελʔ؅ཧ • etc ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 23

Slide 24

Slide 24 text

Orchestra)on Public cloud provider: • Amazon EC2 Container Service (ECS) • Azure Container Service (AKS) • Docker Enterprise Edi

Slide 25

Slide 25 text

ʮ͹ͳͳʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 25

Slide 26

Slide 26 text

3 Orchestra)on͸ToolʹΑͬͯ༻ޠɾػೳ͕ҧ͏ͷͰɺ͜Ε͔Β͸ AWS ECSΛྫʹ࿩͠·͢ h"ps:/ /github.com/nishigori/phpcon2017-presenta9on 3 h$ps:/ /aws.amazon.com/ecs/details/ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 26

Slide 27

Slide 27 text

Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 27

Slide 28

Slide 28 text

ECS Task • A group of one or more containers • ײ֮ͱͯ͠͸ docker-composer up op2onsʹ͍͔ۙ΋ • GKEͰ͍͏Podsͷ֓೦ʹ૬౰ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 28

Slide 29

Slide 29 text

ECS Service • ECS TaskఆٛͰࢦఆ͞Εͨ਺ͷίϯς φΛ഑ஔ • + AutoScaling • Service Load Balancing • ϗετ্ʹίϯςφͷϙʔτΛࣗಈ ׂΓ౰ͯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 29

Slide 30

Slide 30 text

ECS Cluster • ECS TaskΛ഑ஔ͢ΔίϯςφΠϯελ ϯεͷ࿦ཧάϧʔϓ • EC2 AutoScaling Group • EC2 Instance(s) • +AutoScaling Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 30

Slide 31

Slide 31 text

ECS Cluster Q. Which one should we choose Host OS? A. ࣮ʹͨ͘͞Μ͋Δ… (AWS) Amazon ECS Op0mized AMI Container-Op0mized OS (Google) ContainerLinux (CoreOS) Barge OS etc ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 31

Slide 32

Slide 32 text

ECS Cluster Q. Which one should we choose Host OS? • ͨ͘͞Μߟྀ͍ͯ͜͠͏ • Kernel version • Docker version • Launch 3me • etc ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 32

Slide 33

Slide 33 text

ECS Cluster - Containerͱͷ௼Γ߹͍ • Memory / CPU Reserva1on • Memory / CPU Usage docker run --mem=1024m ... Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 33

Slide 34

Slide 34 text

Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 34

Slide 35

Slide 35 text

Deployment Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 35

Slide 36

Slide 36 text

Deployment - rolling update 1. ৽͍͠ίϯςφͷ্ཱͪ͛ 2. ELB͔ΒαʔϏεΠϯ 3. ݹ͍ίϯςφΛഁغ Λ܁Γฦͯ͠શίϯςφΛೖΕସ͍͑ͯ͘ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 36

Slide 37

Slide 37 text

Deployment - rolling update demo? Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 37

Slide 38

Slide 38 text

Deployment for Ops (team) ~ rolling update to the ECS Cluster ~ • Drain containers Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 38

Slide 39

Slide 39 text

Deployment - Canary -- h$ps:/ /mar,nfowler.com/bliki/CanaryRelease.html Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 39

Slide 40

Slide 40 text

Deployment - Canary Q. Ͳ͔͜Β੾Γ෼͚Δʁ 1. Load Balancer? 2. ECS Cluster? 3. ECS Service? 4. Others? Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 40

Slide 41

Slide 41 text

Deployment - Canary Case of Docker EE: • Docker EE Release Candidate • Customer Zero ਖ਼ʹ޷͖ʹ࢖͑ΑελΠϧ Detail of dockercon EU 2017 - h7ps:/ /europe-2017.dockercon.com/ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 41

Slide 42

Slide 42 text

Deployment - Container Image Management - ׂѪ - Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 42

Slide 43

Slide 43 text

Deployment case of PHP OPcache / APCu ౳ɺαʔϏεΠϯલʹWarmup͍ͨ͠ => ENTRYPOINT Ͱఆٛͨ͠εΫϦϓτ಺Ͱ͝ʹΐ͝ʹΐͯ͠Έ Α͏ # Dockerfile FROM xxx # .... ENTRYPOINT ["entrypint-app.sh"] CMD ["--env", "production"] Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 43

Slide 44

Slide 44 text

Logging Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 44

Slide 45

Slide 45 text

Logging ίϯςφʹʢجຊʣσʔλ͸࢒͞ͳ͍ log΋ྫ֎Ͱ͸ͳ͍ • Log as data • ΤʔδΣϯτʹΑΔPushܕ • ﬂuentd • cloudwatch log agent • ͍ΘΏΔதԝूݖܕʢCentralized loggingʣ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 45

Slide 46

Slide 46 text

Logging on Dockerﬁle h"ps:/ /github.com/nginxinc/docker-nginx/blob/3ba04e3/mainline/ stretch/Dockerﬁle#L91-L93 # forward request and error logs to docker log collector RUN ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 46

Slide 47

Slide 47 text

Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 47

Slide 48

Slide 48 text

ʮAWS ECS͔͓ͭʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 48

Slide 49

Slide 49 text

Ψο4 4 ECS͕μϝͬͯݴͬͯΔΜ͡Όͳ͍Yo Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 49

Slide 50

Slide 50 text

ʮGKEʯ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 50

Slide 51

Slide 51 text

Ψο5 5 k8s͍͍Αk8s Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 51

Slide 52

Slide 52 text

ʮ…AKEͰ͕Μ̱ʢ͈́ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 52

Slide 53

Slide 53 text

Ψο6 6 AKE͕μϝͬͯݴͬͯΔΜ͡Όͳ͍Yo Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 53

Slide 54

Slide 54 text

Public Container Service͸पลٕज़΋େࣄ5 • (GCP) Load Balancer / BQ / ... • (AWS) S3 / SNS / SQS / ... • ... • SLA? • Using Reserved Instance? 5 k8s͍͍Αk8s Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 54

Slide 55

Slide 55 text

࠷ޙʹɺDocker࢙ͱ͜Ε͔Βʢ༧ଌʣΛগ͠ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 55

Slide 56

Slide 56 text

History 2013.03~ Docker 2014.06~ Docker Hub 2014.06~ Kubernetes (k8s, kube) 2014.11~ Amazon EC2 Container Service (ECS) 2014.11~ Google Container Engine (GKE) 2014.12~ CoreOS Rocket 2014.12~ Docker Machine / Swarm / Compose 2015.07~ Cloud Native Computing Foundation (CNCF) 2015.08~ VMware vSphere Integrated Containers 2015.09~ Microsoft Azure Container Services (ݱAKS) 2017.03~ Docker Enterprise Edition (Docker EE) 2017.04~ Moby Project 2017.07~ Microsoft Azure Container Instance (ACI) Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 56

Slide 57

Slide 57 text

History (Orchestra/on) 2013.03~ Docker 2014.06~ Docker Hub 2014.06~ Kubernetes (k8s, kube) 2014.11~ Amazon EC2 Container Service (ECS) 2014.11~ Google Container Engine (GKE) 2014.12~ CoreOS Rocket 2014.12~ Docker Machine / Swarm / Compose 2015.07~ Cloud Native Computing Foundation (CNCF) 2015.08~ VMware vSphere Integrated Containers 2015.09~ Microsoft Azure Container Services (ݱAKS) 2017.03~ Docker Enterprise Edition (Docker EE) 2017.04~ Moby Project 2017.07~ Microsoft Azure Container Instance (ACI) Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 57

Slide 58

Slide 58 text

History (Orchestra/on) • Docker for Win/MacͰKubernetes؀ڥ΋ߏஙՄೳʹ • 2017.11ݱࡏɺϕʔλ൛ͷਃ͠ࠐΈ͕Մೳ • h;ps:/ /beta.docker.com/ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 58

Slide 59

Slide 59 text

History (Managed Service) 2013.03~ Docker 2014.06~ Docker Hub 2014.06~ Kubernetes (k8s, kube) 2014.11~ Amazon EC2 Container Service (ECS) 2014.11~ Google Container Engine (GKE) 2014.12~ CoreOS Rocket 2014.12~ Docker Machine / Swarm / Compose 2015.07~ Cloud Native Computing Foundation (CNCF) 2015.08~ VMware vSphere Integrated Containers 2015.09~ Microsoft Azure Container Services (ݱAKS) 2017.03~ Docker Enterprise Edition (Docker EE) 2017.04~ Moby Project 2017.07~ Microsoft Azure Container Instance (ACI) Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 59

Slide 60

Slide 60 text

History (Managed Service) • 2014.05࣌఺ͰGoogle͸20ԯҎ্ͷίϯςφΛՔಇ͍ͯͨ͠ • h.ps:/ /speakerdeck.com/jbeda/containers-at-scale • AWS Lambda΋ίϯςφٕज़Λར༻͍ͯ͠Δ • Public Cloudͱͯ͠΋Container Service͸ొ৔͔ͯ͠Βطʹ3೥ Ҏ্ܦ͍ͬͯΔ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 60

Slide 61

Slide 61 text

History (project / founda4on) 2013.03~ Docker 2014.06~ Docker Hub 2014.06~ Kubernetes (k8s, kube) 2014.11~ Amazon EC2 Container Service (ECS) 2014.11~ Google Container Engine (GKE) 2014.12~ CoreOS Rocket 2014.12~ Docker Machine / Swarm / Compose 2015.07~ Cloud Native Computing Foundation (CNCF) 2015.08~ VMware vSphere Integrated Containers 2015.09~ Microsoft Azure Container Services (ݱAKS) 2017.03~ Docker Enterprise Edition (Docker EE) 2017.04~ Moby Project 2017.07~ Microsoft Azure Container Instance (ACI) Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 61

Slide 62

Slide 62 text

History (project / founda4on) Cloud Na)ve Compu)ng Founda)on h2ps:/ /www.cncf.io/ • AWS͸2017.08~ joined • AWS Lambda౳Ͱ࢖ΘΕ͍ͯΔίϯςφपΓͷϊ΢ϋ΢͕ϑ ΟʔυόοΫ͞ΕΔΜ͡Όͳ͍͔ͱݸਓతʹظ଴ Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 62

Slide 63

Slide 63 text

Thank you By @_nishigori Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 63

Slide 64

Slide 64 text

btw: Cleanup aws for this presenta3on # https://github.com/nishigori/phpcon2017-presentation $ make -C infrastructure destroy Docker Cluster Strategy - @_nishigori #phpcon2017 #࠶ԋ 64