Building Your Docker Swarm Tech Stack

86b88885327486213bf122579d697793?s=47 Bret Fisher
December 04, 2018

Building Your Docker Swarm Tech Stack

Presented at DockerCon 2018 EU, I go through using Docker and the Swarm Orchestrator (a simpler Kuberentes) to stack different tools up from the base OS to a full-featured production server cluster. Also, Sci-Fi. The Video to this deck will be at https://www.bretfisher.com/docker once they are posted.

86b88885327486213bf122579d697793?s=128

Bret Fisher

December 04, 2018
Tweet

Transcript

  1. BRET FISHER Docker Captain, DevOps Dude, Author of Docker Mastery

    Building Your Swarm Tech Stack bretfisher.com/dockercon18 @bretfisher
  2. People ask "Where's my LAMP for container clusters?" Your solutions

    will be a "stack" of infrastructure tools Cloud-native container tools are new We need patterns and examples of full cluster stacks Problem: No Server Tool Lives In Isolation
  3. Build examples of full-ish stacks on different tools Options for

    solo to medium-sized DevOps/Ops teams Use Docker Swarm latest stable as orchestrator Limit cloud vendor lock-in* 4 Goals for Today * Lock-in: A service I can't swap out in my server stack
  4. The Duality of Container Infrastructure “It was the best of

    times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity...” Charles Dickens A Tale of Two Cities, 1859
  5. Clusters are Complex Start small and simple, grow them as

    you grow
  6. EE Platform Architecture Physical Virtualization Public Cloud Platform Security Developer

    Services Registry Services Access Policies App Lifecycle Management Automation & Extensibility Networking Orchestration Storage Container Engine ENTERPRISE EDITION PLATFORM
  7. CE Platform Architecture Public Cloud Overlay Swarm Container Engine COMMUNITY

    EDITION PLATFORM Physical Virtualization Platform Security
  8. None
  9. Sci-Fi! Am I Right!

  10. Sci-Fi! Am I Right!

  11. Epic Battle Royale, on Swarm dogvs.cat

  12. Epic Battle Royale, on Swarm dogvs.cat VS

  13. Epic Battle Royale, on Swarm dogvs.cat Sci-Fi Sounds Edition

  14. dogvs.cat App Services www.dogvs.cat vote.dogvs.cat blog.dogvs.cat result.dogvs.cat

  15. dogvs.cat App Services www.dogvs.cat vote.dogvs.cat blog.dogvs.cat result.dogvs.cat (Stack Files)

  16. dogvs.cat Swarm CE, (Han) Solo Sysadmin

  17. Cloud agnostic, minimal infra Apps auto-recover on node fail Incoming

    TLS Centralized logging Centralized monitoring Healthcheck all containers Infra Requirements of dogvs.cat Performance auto-scaling Self-healing nodes Support serverless functions Services highly available Han Solo Requirements Optional Requirements for Later
  18. 3+ Droplets (Ubuntu 18.04) Block Storage (Volumes) Load Balancer (incoming

    HTTP) Digital Ocean dogvs.cat Services Needed for High Availability
  19. App Services www.dogvs.cat vote.dogvs.cat blog.dogvs.cat result.dogvs.cat

  20. App Services + L7 Proxy www.dogvs.cat vote.dogvs.cat blog.dogvs.cat result.dogvs.cat

  21. App Services + L7 Proxy + Overlay www.dogvs.cat vote.dogvs.cat blog.dogvs.cat

    result.dogvs.cat
  22. App Services + L7 Proxy + Ops www.dogvs.cat vote.dogvs.cat blog.dogvs.cat

    result.dogvs.cat
  23. App Services + L7 Proxy + Ops www.dogvs.cat vote.dogvs.cat blog.dogvs.cat

    result.dogvs.cat
  24. Cluster + External Load Balancer www.dogvs.cat vote.dogvs.cat blog.dogvs.cat result.dogvs.cat

  25. Open Source Stack HW / OS Docker Machine + Digital

    Ocean Runtime Docker CE Orchestration Docker Swarm Networking Docker Swarm Overlay Storage REX-Ray + Digital Ocean Volumes Layer 7 Proxy Traefik + Let's Encrypt Central Logging Elastic ELK Central Monitoring Prometheus + Grafana Swarm GUI Portainer
  26. Deploy Nodes: Docker Machine ./create-servers.sh ./enable-monitoring.sh ./create-swarm.sh

  27. Deploy Storage: RexRay Plug-in docker stack deploy -c stack-rexray.yml rexray

  28. Deploy Proxy Stack: Traefik with Let's Encrypt docker stack deploy

    -c stack-proxy.yml proxy http://www.dogvs.cat:8080/dashboard/
  29. Deploy Ops Stacks: Prometheus + ELK + Portainer docker stack

    deploy -c stack-swarmprom.yml prom docker stack deploy -c stack-elk.yml elk docker stack deploy -c stack-portainer.yml portainer
  30. Our Apps: Voting + Ghost + Static Site docker stack

    deploy -c stack-ghost.yml ghost docker stack deploy -c stack-voting.yml vote docker stack deploy -c stack-menu.yml menu
  31. Deploy Stacks: Maintenance Tasks docker stack deploy -c stack-prune.yml prune

    backups (in stack file with app)
  32. Day Two Operations: Updates stack deploy ∞ micromanage update_config and

    healthcheck tune your limits and reservations
  33. Swarm CE, Trek Tag-Team DevOps dogvs.cat

  34. Create Swarm with Terraform Moving beyond docker-machine for multi-admin

  35. Docker for AWS/Azure *for CE only

  36. Remote Management Bonus 18.09 has SSH endpoints for docker CLI

    !! DOCKER_HOST=ssh://user@server docker ps
  37. Windows Server 2019 Workers Swarm and overlay network feature parity!

  38. Windows Server 2019 Workers Swarm and overlay network feature parity!

  39. Our Apps using Docker App docker-app deploy dogvscat/ghost:0.1.0 docker-app deploy

    dogvscat/voting:0.1.0 docker-app deploy dogvscat/menu:0.1.0 * Go check it out at github.com/docker/app
  40. Security? host setup scanning: Docker Bench image scanning: Aqua Microscanner

    behavior monitoring: Sysdig Falco user namespaces
  41. Swarm Enterprise, Amazonian DevOps dogvscat.biz or

  42. Complex Infrastructure, Harder Deployment "How can I deploy many multi-tier

    app on a many servers, with all the b e l l s a n d w h i s t l e s o f H A orchestration, have load balancing at all levels, with failover and auto recovery?" Amazonian Team
  43. CE Platform Architecture Physical Virtualization Public Cloud Platform Security Networking

    Orchestration Container Engine COMMUNITY EDITION PLATFORM
  44. EE Platform Architecture Physical Virtualization Public Cloud Platform Security Developer

    Services Registry Services Access Policies App Lifecycle Management Automation & Extensibility Networking Orchestration Storage Container Engine ENTERPRISE EDITION PLATFORM
  45. No More One Size Fits All Docker for AWS Docker

    for Azure for CE only
  46. Reference Architecture Automation Tools Ecosystem Integration Docker Certified Infrastructure

  47. DCI AWS

  48. DCI AWS

  49. Docker EE on AWS Stack HW / OS Terraform +

    Ansible + AWS Runtime Docker EE Orchestration Docker Swarm Networking Docker Swarm Overlay Storage Docker Cloudstor EBS/EFS Layer 7 Proxy HTTP Routing Mesh (Interlock+Nginx) Registry Docker EE DTR Central Logging AWS Cloudwatch Logs Central Monitoring AWS Cloudwatch + Telegraph Swarm GUI Docker EE UCP
  50. Summary Infrastructure as code, make everything repeatable No "special" nodes,

    use remote management Grow as you go, assume you'll resize Look for compose files of popular tools to make stacks Don't throw out the good in search of the perfect
  51. Where to Go From Here Automating Docker Enterprise 5:25 this

    room Mission-Critical Migration to Multi-Cluster Kubernetes Wed 2pm room 113 Swarm Intro Workshop Today 4pm room 133 Wed 1:30pm room 133 Swarm Hallway Track Wed 4pm
  52. I'd like to thank the internet

  53. Support the open source you use

  54. PLEASE USE SESSION SURVEY IN EVENTSPACE APP Thanks! ! bretfisher.com/dockercon18

    "Building Your Swarm Tech Stack"