Containerization—A Journey

Docker Manchester Meetup, http://www.meetup.com/Docker-Manchester/events/232283804/

Transcript

1. © 2016 Mesosphere, Inc. All Rights Reserved. CONTAINERIZATION —A JOURNEY 1 Michael Hausenblas, Distributed Systems Jester | 2016-07-21 | Docker Manchester @mhausenblas
2. © 2016 Mesosphere, Inc. All Rights Reserved. sys admin/SRE appops developer architect QA/test engineer data engineer
3. © 2016 Mesosphere, Inc. All Rights Reserved. THE ENVIRONMENT 3
4. © 2015 Mesosphere, Inc. meBay.com 4 • enabling people to sell and buy stuff online • supports holding auctions as well as special sales events for selected retailers • allows buying and selling via online chat rooms
5. © 2016 Mesosphere, Inc. All Rights Reserved. THE MONOLITH 5
6. © 2016 Mesosphere, Inc. All Rights Reserved. ARCHITECTURE 6
7. © 2016 Mesosphere, Inc. All Rights Reserved. ARCHITECTURE 7 • frontend hosts (Web servers and load balancers) • middleware host, running the RoR app server is • backend hosts are the database servers (MySQL) • two environments (prod, as above and dev with one host per tier and smaller sized machines)
8. © 2016 Mesosphere, Inc. All Rights Reserved. AGILITY & CHALLENGES 8 • rolling out a new version of the monolith • sustain load/scaling • local/dev/test reproducibility • getting paged at 3am when things go belly-up
9. © 2016 Mesosphere, Inc. All Rights Reserved. HERE COMES CONTAINERZ 9
10. © 2016 Mesosphere, Inc. All Rights Reserved. ARCHITECTURE 10
11. © 2016 Mesosphere, Inc. All Rights Reserved. ARCHITECTURE 11 • containerized each component (CI/CD pipeline that generates Docker images) • manually sshing into hosts, using docker run to (re)launch containers • manual, static mapping of containers-to-IP:port
12. © 2016 Mesosphere, Inc. All Rights Reserved. AGILITY & CHALLENGES 12 • effectively using pets approach • better reproducibility • faster roll-outs • still many operational issues (failures, scaling, etc.)
13. © 2016 Mesosphere, Inc. All Rights Reserved. IN FOR A PENNY, IN FOR A POUND 13
14. © 2016 Mesosphere, Inc. All Rights Reserved. ARCHITECTURE 14
15. © 2016 Mesosphere, Inc. All Rights Reserved. ARCHITECTURE 15 • turned the monolith into microservices • only cattle • automatic handling of failures • auto-scaling depending on traffic • canary and blue-green deployments
16. © 2016 Mesosphere, Inc. All Rights Reserved. SOME LESSONS LEARNED 16
17. © 2016 Mesosphere, Inc. All Rights Reserved. CONTAINERS
 17 • don't apply 'VM patterns' • they come and go (implications for monitoring, etc.) • fully automated CI/CD pipeline is essential • usually higher utilization but requires a fair part of automation • needs appops (later more)
18. © 2016 Mesosphere, Inc. All Rights Reserved. REGISTRIES 18 • Docker Hub
 https://hub.docker.com/ • Google Cloud
 https://cloud.google.com/tools/container-registry/ • AWS
 https://aws.amazon.com/ecr/ • CoreOS
 https://quay.io/ • SUSE Portus
 http://port.us.org/ • JFrog Artifactory
 https://www.jfrog.com/artifactory/ • Run your own
 https://docs.docker.com/registry/deploying/
19. © 2016 Mesosphere, Inc. All Rights Reserved. SECRETS 19 Don't bake credentials into images but rather do: $ docker run -d -e API_TOKEN=SECRET somedatabase $ docker run -d -v $(pwd):/fsecret:/fsecret:ro somedatabase Even better: use key-value in-memory stores such as Square's KeyWhiz, HashiCorp's Vault, or Crypt or native solutions
20. © 2016 Mesosphere, Inc. All Rights Reserved. 20 appops The person who writes an app is also the person responsible for operating the app in prod.
21. © 2016 Mesosphere, Inc. All Rights Reserved. 21 It's not about provisioning
 a VM or installing a DC/OS cluster or replacing a faulty HDD …
 
 … this would be on the infrastructure team. appops
22. © 2016 Mesosphere, Inc. All Rights Reserved. 22 appops speakerdeck.com/charity/devops-for-developers-building-an-effective-ops-org-1
23. © 2016 Mesosphere, Inc. All Rights Reserved. TRY IT OUT YOURSELF 23 containerize != Docker image github.com/mhausenblas/marvin
24. © 2016 Mesosphere, Inc. All Rights Reserved. Q & A 24 • @mhausenblas • mhausenblas.info • michael@dcos.io https://dcos.io