Slide 1

Slide 1 text

© 2016 Mesosphere, Inc. All Rights Reserved. CONTAINERIZATION —A JOURNEY 1 Michael Hausenblas, Distributed Systems Jester | 2016-07-21 | Docker Manchester @mhausenblas

Slide 2

Slide 2 text

© 2016 Mesosphere, Inc. All Rights Reserved. sys admin/SRE appops developer architect QA/test engineer data engineer

Slide 3

Slide 3 text

© 2016 Mesosphere, Inc. All Rights Reserved. THE ENVIRONMENT 3

Slide 4

Slide 4 text

© 2015 Mesosphere, Inc. meBay.com 4 • enabling people to sell and buy stuff online • supports holding auctions as well as special sales events for selected retailers • allows buying and selling via online chat rooms

Slide 5

Slide 5 text

© 2016 Mesosphere, Inc. All Rights Reserved. THE MONOLITH 5

Slide 6

Slide 6 text

© 2016 Mesosphere, Inc. All Rights Reserved. ARCHITECTURE 6

Slide 7

Slide 7 text

© 2016 Mesosphere, Inc. All Rights Reserved. ARCHITECTURE 7 • frontend hosts (Web servers and load balancers) • middleware host, running the RoR app server is • backend hosts are the database servers (MySQL) • two environments (prod, as above and dev with one host per tier and smaller sized machines)

Slide 8

Slide 8 text

© 2016 Mesosphere, Inc. All Rights Reserved. AGILITY & CHALLENGES 8 • rolling out a new version of the monolith • sustain load/scaling • local/dev/test reproducibility • getting paged at 3am when things go belly-up

Slide 9

Slide 9 text

© 2016 Mesosphere, Inc. All Rights Reserved. HERE COMES CONTAINERZ 9

Slide 10

Slide 10 text

© 2016 Mesosphere, Inc. All Rights Reserved. ARCHITECTURE 10

Slide 11

Slide 11 text

© 2016 Mesosphere, Inc. All Rights Reserved. ARCHITECTURE 11 • containerized each component (CI/CD pipeline that generates Docker images) • manually sshing into hosts, using docker run to (re)launch containers • manual, static mapping of containers-to-IP:port

Slide 12

Slide 12 text

© 2016 Mesosphere, Inc. All Rights Reserved. AGILITY & CHALLENGES 12 • effectively using pets approach • better reproducibility • faster roll-outs • still many operational issues (failures, scaling, etc.)

Slide 13

Slide 13 text

© 2016 Mesosphere, Inc. All Rights Reserved. IN FOR A PENNY, IN FOR A POUND 13

Slide 14

Slide 14 text

© 2016 Mesosphere, Inc. All Rights Reserved. ARCHITECTURE 14

Slide 15

Slide 15 text

© 2016 Mesosphere, Inc. All Rights Reserved. ARCHITECTURE 15 • turned the monolith into microservices • only cattle • automatic handling of failures • auto-scaling depending on traffic • canary and blue-green deployments

Slide 16

Slide 16 text

© 2016 Mesosphere, Inc. All Rights Reserved. SOME LESSONS LEARNED 16

Slide 17

Slide 17 text

© 2016 Mesosphere, Inc. All Rights Reserved. CONTAINERS
 17 • don't apply 'VM patterns' • they come and go (implications for monitoring, etc.) • fully automated CI/CD pipeline is essential • usually higher utilization but requires a fair part of automation • needs appops (later more)

Slide 18

Slide 18 text

© 2016 Mesosphere, Inc. All Rights Reserved. REGISTRIES 18 • Docker Hub
 https://hub.docker.com/ • Google Cloud
 https://cloud.google.com/tools/container-registry/ • AWS
 https://aws.amazon.com/ecr/ • CoreOS
 https://quay.io/ • SUSE Portus
 http://port.us.org/ • JFrog Artifactory
 https://www.jfrog.com/artifactory/ • Run your own
 https://docs.docker.com/registry/deploying/

Slide 19

Slide 19 text

© 2016 Mesosphere, Inc. All Rights Reserved. SECRETS 19 Don't bake credentials into images but rather do: $ docker run -d -e API_TOKEN=SECRET somedatabase $ docker run -d -v $(pwd):/fsecret:/fsecret:ro somedatabase Even better: use key-value in-memory stores such as Square's KeyWhiz, HashiCorp's Vault, or Crypt or native solutions

Slide 20

Slide 20 text

© 2016 Mesosphere, Inc. All Rights Reserved. 20 appops The person who writes an app is also the person responsible for operating the app in prod.

Slide 21

Slide 21 text

© 2016 Mesosphere, Inc. All Rights Reserved. 21 It's not about provisioning
 a VM or installing a DC/OS cluster or replacing a faulty HDD …
 
 … this would be on the infrastructure team. appops

Slide 22

Slide 22 text

© 2016 Mesosphere, Inc. All Rights Reserved. 22 appops speakerdeck.com/charity/devops-for-developers-building-an-effective-ops-org-1

Slide 23

Slide 23 text

© 2016 Mesosphere, Inc. All Rights Reserved. TRY IT OUT YOURSELF 23 containerize != Docker image github.com/mhausenblas/marvin

Slide 24

Slide 24 text

© 2016 Mesosphere, Inc. All Rights Reserved. Q & A 24 • @mhausenblas • mhausenblas.info • [email protected] https://dcos.io