Journey to Docker Production

Evolving Your Infrastructure and Processes - DockerCon 2017 Austin

Transcript

1. Journey to Docker Production Bret Fisher Independent DevOps Sysadmin Docker Captain
2. • Docker Dev+Ops user since 2014 • Used for my co-founded startup • Now, fill time container ops indi consultant • Helped over dozen orgs go prod Who Am I?
3. • You want Docker in production • Containerizing existing infrastructure/apps • Need to break down barriers of getting into Docker production • Often these are "decision barriers", lets make some good defaults • Sometimes it's made-up requirements that aren't required day one • Hopefully I will help you today with all of these! Why Are We Here?
4. • Containerize As-Is • What To Focus On First • Building Prod Layer by Layer • Agile Infrastructure Planning • Other Ways To Make An Impact Agenda
5. Production Docker: A New Hope "Containerize as-is!"
6. New to containers? Limit other changes Sounds obvious, but many container projects start out too big in scope Problems you likely don't need to solve before container day one • Fully automatic CI/CD • Dynamic performance scaling • Containerizing all of an existing solution at once • Service discovery (dynamic updates to LB's, DNS, etc.) • Starting with persistent data Limit Your Simultaneous Innovation
7. • Microservice conversion isn't required • 12 Factor is a horizon we're always chasing • Don't let these ideals delay containerization • Maybe we need a 12 Factor 12-step program Legacy Apps Work In Containers Too At its core, a container is just a process
8. You need a good foundation What To Focus On First
9. is the path to the dark side A Bad Dockerfile...
10. Good Dockerfiles are more important than fancy orchestration • It's your new build and environment documentation • At first, focus on "it works" rather then fancy or lean (or multi-stage) • Use FROM Official distros that are most familiar What To Focus On First: Dockerfiles
11. Dockerfile Production Anti-patterns New teams often have anti-patterns or wrong defaults Lets look at some of these shall we:
12. • Anti: Not storing unique data in volumes Dockerfile Anti-pattern: Trapping Data • Not to be confused with Bind Mounts for sharing data with host
13. • Anti: Letting image builds pull FROM latest Dockerfile Anti-pattern: No Version Pinning • Anti: Letting image builds install latest packages • You don't do this with your code, why do it with dependencies?
14. • Anti: Not changing defaults in container like you would on a VM • e.g. php.ini, mysql.conf.d, java memory Dockerfile Anti-pattern: Leaving Default Config • NOTE: Don't just copy over config files from VM's into image
15. • Anti: Copying in environment config at image build Dockerfile Anti-pattern: Environment Specific • Goal: Single Dockerfile with default ENV's, and overwrite per- environment at run time
16. Building Prod Layer by Layer
17. • Stick with what you know at first • Do some basic performance testing of your existing and planned setup. Even if just defaults with sample data. You will learn lots! • Recent Docker Inc. and HPE whitepaper on MySQL benchmark • (authored by yours truly, and others) • bretfisher.com/dockercon17 Containers-on-VM or Container-on-Bare-Metal Do either, or both. Lots of pros/cons to either
18. • Docker is kernel and storage driver dependent • Innovations/fixes are still happening here • "Minimum" version != "best" version • No pre-existing opinion? Ubuntu 16.04.2 LTS (latest LTS) • Popular, well-tested with Docker • One of latest LTS kernels out there • Wide storage driver support • Get correct Docker for your distro from store.docker.com Linux Distribution/Release Matters
19. • Don't make a decision based on image size (remember it's SIS) • At first: match your existing deployment process • Consider changing to Alpine later, maybe much later Container Base Distribution: Which One?
20. Swarm Architectures
21. • General advice on how to start small and grow big • Based off learnings from • Docker internal testing • Docker reference architectures • Real world deployments • Swarm3k lessons learned Good Defaults: Swarm Architectures
22. • "docker swarm init" done! • This happens every day with VM's, why not with containers • Why bother with swarm? Keeping same workflow across environments, secrets, etc. 1-Node Swarm Baby Swarm
23. • All Managers and Workers • One node can fail • Great when on a very small budget • pet projects • test/CI setups 3-Node Swarm Minimum for HA
24. • All Managers and Workers • Two nodes can fail • My minimum for production that affects business $$ 5-Node Swarm Better High-Availability
25. • 5 dedicated managers • Maybe some workers in DMZ • Anything beyond 5 nodes, stick with 5 Managers and rest workers • Control container placement with labels + constraints 10-Node Swarm Separating Out Managers
26. > docker node update --label-add data-drive=ssd worker1 > docker node update --label-add net-segment=dmz1 worker2 Constraints with Swarm Services > docker service create \ --constraint node.labels.data-drive==ssd mysql
27. • 5 dedicated managers • Resize managers as you grow • Multiple worker subnets on Private/DMZ • Control container placement with labels + constraints 100-Node Swarm Aww she's all grown up!
28. • Different hardware configurations • Different subnets or security groups • Solution: Use node labels with container constraints Not-So-Good Reasons for Multiple Swarms
29. • Geographical boundaries between nodes (e.g. different regions) • Security boundaries for compliance (PCI, etc.) • Personnel boundaries for Swarm management using Docker API • (could mitigate Docker EE UCP, auth plugin) Good Reasons for Multiple Swarms
30. Do You Have An Externally Driven Deadline?
31. • Beware the "Not Implemented Here" Syndrome • Similar to "not invented here" • Opportunities for Outsourcing • Where it can still be a challenge to implement and maintain • And, where the SaaS/On-prem market is mature • For Your Consideration • Image registry • Log aggregation, storage, search, and alerting • 17.05 gets service logs, good for getting started • Monitoring and alerting Outsource Well-Defined Plumbing
32. Other Docker Products that Compliment Docker CE • Swarm/Stacks/Secrets/Routing Mesh • CLI Management Docker for AWS/Azure • Cloud Resource Template • Moby Instance OS Docker EE • DTR/UCP • GUI Mgmt/RBAC/Layer 7 Proxy/Image Cache More Effort More Decisions Less Effort Less Decisions
33. Remember To Be Agile, You'll Build The Next One Better
34. • Already have good infrastructure automation (one click VM)? • Maybe you have great VM autoscale with single-service VM's? • Like the security boundary of the VM OS? You Want Even Less Ops Change? Lets accelerate this even more
35. • Least amount of infrastructure change while letting you: • Run on Dockerfiles recipes rather then Puppet etc. • Improve your Docker management skills • Simplify your VM OS build • Moby Project & LinuxKit will likely make this more popular • Windows is doing similar with Hyper-V Containers • Linux is doing similar with Intel Clear Containers One Container Per VM Why don't we talk about this more?
36. Other Ways To Make An Impact
37. • Use containers for tech support mock environments • Temp test environments, customer demos • Any process where you "wait for human to deploy/configure VM" Other Ways To Make An Impact Docker can be used internally and still add value
38. Thank You! My New Docker Udemy Course 90% off for DockerCon: dockermastery.com slides: bretfisher.com/dockercon17