Journey to Docker Production

Journey to Docker Production

Evolving Your Infrastructure and Processes - DockerCon 2017 Austin


Bret Fisher

April 19, 2017


  1. Journey to Docker Production Bret Fisher Independent DevOps Sysadmin Docker

  2. • Docker Dev+Ops user since 2014 • Used for my

    co-founded startup • Now, fill time container ops indi consultant • Helped over dozen orgs go prod Who Am I?
  3. • You want Docker in production • Containerizing existing infrastructure/apps

    • Need to break down barriers of getting into Docker production • Often these are "decision barriers", lets make some good defaults • Sometimes it's made-up requirements that aren't required day one • Hopefully I will help you today with all of these! Why Are We Here?
  4. • Containerize As-Is • What To Focus On First •

    Building Prod Layer by Layer • Agile Infrastructure Planning • Other Ways To Make An Impact Agenda
  5. Production Docker: A New Hope "Containerize as-is!"

  6. New to containers? Limit other changes Sounds obvious, but many

    container projects start out too big in scope Problems you likely don't need to solve before container day one • Fully automatic CI/CD • Dynamic performance scaling • Containerizing all of an existing solution at once • Service discovery (dynamic updates to LB's, DNS, etc.) • Starting with persistent data Limit Your Simultaneous Innovation
  7. • Microservice conversion isn't required • 12 Factor is a

    horizon we're always chasing • Don't let these ideals delay containerization • Maybe we need a 12 Factor 12-step program Legacy Apps Work In Containers Too At its core, a container is just a process
  8. You need a good foundation What To Focus On First

  9. is the path to the dark side A Bad Dockerfile...

  10. Good Dockerfiles are more important than fancy orchestration • It's

    your new build and environment documentation • At first, focus on "it works" rather then fancy or lean (or multi-stage) • Use FROM Official distros that are most familiar What To Focus On First: Dockerfiles
  11. Dockerfile Production Anti-patterns New teams often have anti-patterns or wrong

    defaults Lets look at some of these shall we:
  12. • Anti: Not storing unique data in volumes Dockerfile Anti-pattern:

    Trapping Data • Not to be confused with Bind Mounts for sharing data with host
  13. • Anti: Letting image builds pull FROM latest Dockerfile Anti-pattern:

    No Version Pinning • Anti: Letting image builds install latest packages • You don't do this with your code, why do it with dependencies?
  14. • Anti: Not changing defaults in container like you would

    on a VM • e.g. php.ini, mysql.conf.d, java memory Dockerfile Anti-pattern: Leaving Default Config • NOTE: Don't just copy over config files from VM's into image
  15. • Anti: Copying in environment config at image build Dockerfile

    Anti-pattern: Environment Specific • Goal: Single Dockerfile with default ENV's, and overwrite per- environment at run time
  16. Building Prod Layer by Layer

  17. • Stick with what you know at first • Do

    some basic performance testing of your existing and planned setup. Even if just defaults with sample data. You will learn lots! • Recent Docker Inc. and HPE whitepaper on MySQL benchmark • (authored by yours truly, and others) • Containers-on-VM or Container-on-Bare-Metal Do either, or both. Lots of pros/cons to either
  18. • Docker is kernel and storage driver dependent • Innovations/fixes

    are still happening here • "Minimum" version != "best" version • No pre-existing opinion? Ubuntu 16.04.2 LTS (latest LTS) • Popular, well-tested with Docker • One of latest LTS kernels out there • Wide storage driver support • Get correct Docker for your distro from Linux Distribution/Release Matters
  19. • Don't make a decision based on image size (remember

    it's SIS) • At first: match your existing deployment process • Consider changing to Alpine later, maybe much later Container Base Distribution: Which One?
  20. Swarm Architectures

  21. • General advice on how to start small and grow

    big • Based off learnings from • Docker internal testing • Docker reference architectures • Real world deployments • Swarm3k lessons learned Good Defaults: Swarm Architectures
  22. • "docker swarm init" done! • This happens every day

    with VM's, why not with containers • Why bother with swarm? Keeping same workflow across environments, secrets, etc. 1-Node Swarm Baby Swarm
  23. • All Managers and Workers • One node can fail

    • Great when on a very small budget • pet projects • test/CI setups 3-Node Swarm Minimum for HA
  24. • All Managers and Workers • Two nodes can fail

    • My minimum for production that affects business $$ 5-Node Swarm Better High-Availability
  25. • 5 dedicated managers • Maybe some workers in DMZ

    • Anything beyond 5 nodes, stick with 5 Managers and rest workers • Control container placement with labels + constraints 10-Node Swarm Separating Out Managers
  26. > docker node update --label-add data-drive=ssd worker1 > docker node

    update --label-add net-segment=dmz1 worker2 Constraints with Swarm Services > docker service create \ --constraint mysql
  27. • 5 dedicated managers • Resize managers as you grow

    • Multiple worker subnets on Private/DMZ • Control container placement with labels + constraints 100-Node Swarm Aww she's all grown up!
  28. • Different hardware configurations • Different subnets or security groups

    • Solution: Use node labels with container constraints Not-So-Good Reasons for Multiple Swarms
  29. • Geographical boundaries between nodes (e.g. different regions) • Security

    boundaries for compliance (PCI, etc.) • Personnel boundaries for Swarm management using Docker API • (could mitigate Docker EE UCP, auth plugin) Good Reasons for Multiple Swarms
  30. Do You Have An Externally Driven Deadline?

  31. • Beware the "Not Implemented Here" Syndrome • Similar to

    "not invented here" • Opportunities for Outsourcing • Where it can still be a challenge to implement and maintain • And, where the SaaS/On-prem market is mature • For Your Consideration • Image registry • Log aggregation, storage, search, and alerting • 17.05 gets service logs, good for getting started • Monitoring and alerting Outsource Well-Defined Plumbing
  32. Other Docker Products that Compliment Docker CE • Swarm/Stacks/Secrets/Routing Mesh

    • CLI Management Docker for AWS/Azure • Cloud Resource Template • Moby Instance OS Docker EE • DTR/UCP • GUI Mgmt/RBAC/Layer 7 Proxy/Image Cache More Effort More Decisions Less Effort Less Decisions
  33. Remember To Be Agile, You'll Build The Next One Better

  34. • Already have good infrastructure automation (one click VM)? •

    Maybe you have great VM autoscale with single-service VM's? • Like the security boundary of the VM OS? You Want Even Less Ops Change? Lets accelerate this even more
  35. • Least amount of infrastructure change while letting you: •

    Run on Dockerfiles recipes rather then Puppet etc. • Improve your Docker management skills • Simplify your VM OS build • Moby Project & LinuxKit will likely make this more popular • Windows is doing similar with Hyper-V Containers • Linux is doing similar with Intel Clear Containers One Container Per VM Why don't we talk about this more?
  36. Other Ways To Make An Impact

  37. • Use containers for tech support mock environments • Temp

    test environments, customer demos • Any process where you "wait for human to deploy/configure VM" Other Ways To Make An Impact Docker can be used internally and still add value
  38. Thank You! My New Docker Udemy Course 90% off for

    DockerCon: slides: