Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Your Docker Swarm Tech Stack

Bret Fisher
December 04, 2018

Building Your Docker Swarm Tech Stack

Presented at DockerCon 2018 EU, I go through using Docker and the Swarm Orchestrator (a simpler Kuberentes) to stack different tools up from the base OS to a full-featured production server cluster. Also, Sci-Fi. The Video to this deck will be at https://www.bretfisher.com/docker once they are posted.

Bret Fisher

December 04, 2018
Tweet

More Decks by Bret Fisher

Other Decks in Technology

Transcript

  1. BRET FISHER
    Docker Captain, DevOps Dude,
    Author of Docker Mastery
    Building Your Swarm Tech Stack
    bretfisher.com/dockercon18
    @bretfisher

    View full-size slide

  2. People ask "Where's my LAMP for container clusters?"
    Your solutions will be a "stack" of infrastructure tools
    Cloud-native container tools are new
    We need patterns and examples of full cluster stacks
    Problem: No Server Tool Lives In Isolation

    View full-size slide

  3. Build examples of full-ish stacks on different tools
    Options for solo to medium-sized DevOps/Ops teams
    Use Docker Swarm latest stable as orchestrator
    Limit cloud vendor lock-in*
    4 Goals for Today
    * Lock-in: A service I can't swap out in my server stack

    View full-size slide

  4. The Duality of Container Infrastructure
    “It was the best of times,
    it was the worst of times,
    it was the age of wisdom,
    it was the age of foolishness,
    it was the epoch of belief,
    it was the epoch of incredulity...”
    Charles Dickens
    A Tale of Two Cities, 1859

    View full-size slide

  5. Clusters are Complex
    Start small and simple, grow them as you grow

    View full-size slide

  6. EE Platform Architecture
    Physical
    Virtualization
    Public Cloud
    Platform
    Security
    Developer
    Services
    Registry
    Services
    Access
    Policies
    App Lifecycle
    Management
    Automation &
    Extensibility
    Networking Orchestration Storage
    Container Engine
    ENTERPRISE EDITION PLATFORM

    View full-size slide

  7. CE Platform Architecture
    Public Cloud
    Overlay Swarm
    Container Engine
    COMMUNITY EDITION PLATFORM
    Physical
    Virtualization
    Platform
    Security

    View full-size slide

  8. Sci-Fi!
    Am I Right!

    View full-size slide

  9. Sci-Fi!
    Am I Right!

    View full-size slide

  10. Epic Battle Royale, on Swarm
    dogvs.cat

    View full-size slide

  11. Epic Battle Royale, on Swarm
    dogvs.cat
    VS

    View full-size slide

  12. Epic Battle Royale, on Swarm
    dogvs.cat
    Sci-Fi Sounds Edition

    View full-size slide

  13. dogvs.cat App Services
    www.dogvs.cat vote.dogvs.cat blog.dogvs.cat
    result.dogvs.cat

    View full-size slide

  14. dogvs.cat App Services
    www.dogvs.cat vote.dogvs.cat blog.dogvs.cat
    result.dogvs.cat
    (Stack Files)

    View full-size slide

  15. dogvs.cat
    Swarm CE,
    (Han) Solo Sysadmin

    View full-size slide

  16. Cloud agnostic, minimal infra
    Apps auto-recover on node fail
    Incoming TLS
    Centralized logging
    Centralized monitoring
    Healthcheck all containers
    Infra Requirements of dogvs.cat
    Performance auto-scaling
    Self-healing nodes
    Support serverless functions
    Services highly available
    Han Solo Requirements Optional Requirements for Later

    View full-size slide

  17. 3+ Droplets (Ubuntu 18.04)
    Block Storage (Volumes)
    Load Balancer (incoming HTTP)
    Digital Ocean dogvs.cat
    Services Needed for High Availability

    View full-size slide

  18. App Services
    www.dogvs.cat vote.dogvs.cat blog.dogvs.cat
    result.dogvs.cat

    View full-size slide

  19. App Services + L7 Proxy
    www.dogvs.cat vote.dogvs.cat blog.dogvs.cat
    result.dogvs.cat

    View full-size slide

  20. App Services + L7 Proxy + Overlay
    www.dogvs.cat vote.dogvs.cat blog.dogvs.cat
    result.dogvs.cat

    View full-size slide

  21. App Services + L7 Proxy + Ops
    www.dogvs.cat vote.dogvs.cat blog.dogvs.cat
    result.dogvs.cat

    View full-size slide

  22. App Services + L7 Proxy + Ops
    www.dogvs.cat vote.dogvs.cat blog.dogvs.cat
    result.dogvs.cat

    View full-size slide

  23. Cluster + External Load Balancer
    www.dogvs.cat vote.dogvs.cat blog.dogvs.cat
    result.dogvs.cat

    View full-size slide

  24. Open Source Stack
    HW / OS Docker Machine + Digital Ocean
    Runtime Docker CE
    Orchestration Docker Swarm
    Networking Docker Swarm Overlay
    Storage REX-Ray + Digital Ocean Volumes
    Layer 7 Proxy Traefik + Let's Encrypt
    Central Logging Elastic ELK
    Central Monitoring Prometheus + Grafana
    Swarm GUI Portainer

    View full-size slide

  25. Deploy Nodes: Docker Machine
    ./create-servers.sh
    ./enable-monitoring.sh
    ./create-swarm.sh

    View full-size slide

  26. Deploy Storage: RexRay Plug-in
    docker stack deploy -c stack-rexray.yml rexray

    View full-size slide

  27. Deploy Proxy Stack: Traefik with Let's Encrypt
    docker stack deploy -c stack-proxy.yml proxy
    http://www.dogvs.cat:8080/dashboard/

    View full-size slide

  28. Deploy Ops Stacks:
    Prometheus + ELK + Portainer
    docker stack deploy -c stack-swarmprom.yml prom
    docker stack deploy -c stack-elk.yml elk
    docker stack deploy -c stack-portainer.yml portainer

    View full-size slide

  29. Our Apps: Voting + Ghost + Static Site
    docker stack deploy -c stack-ghost.yml ghost
    docker stack deploy -c stack-voting.yml vote
    docker stack deploy -c stack-menu.yml menu

    View full-size slide

  30. Deploy Stacks: Maintenance Tasks
    docker stack deploy -c stack-prune.yml prune
    backups (in stack file with app)

    View full-size slide

  31. Day Two Operations: Updates
    stack deploy ∞
    micromanage update_config and healthcheck
    tune your limits and reservations

    View full-size slide

  32. Swarm CE,
    Trek Tag-Team DevOps
    dogvs.cat

    View full-size slide

  33. Create Swarm with Terraform
    Moving beyond docker-machine for multi-admin

    View full-size slide

  34. Docker for AWS/Azure
    *for CE only

    View full-size slide

  35. Remote Management Bonus
    18.09 has SSH endpoints for docker CLI !!
    DOCKER_HOST=ssh://user@server docker ps

    View full-size slide

  36. Windows Server 2019 Workers
    Swarm and overlay network feature parity!

    View full-size slide

  37. Windows Server 2019 Workers
    Swarm and overlay network feature parity!

    View full-size slide

  38. Our Apps using Docker App
    docker-app deploy dogvscat/ghost:0.1.0
    docker-app deploy dogvscat/voting:0.1.0
    docker-app deploy dogvscat/menu:0.1.0
    * Go check it out at github.com/docker/app

    View full-size slide

  39. Security?
    host setup scanning: Docker Bench
    image scanning: Aqua Microscanner
    behavior monitoring: Sysdig Falco
    user namespaces

    View full-size slide

  40. Swarm Enterprise,
    Amazonian DevOps
    dogvscat.biz
    or

    View full-size slide

  41. Complex Infrastructure, Harder Deployment
    "How can I deploy many multi-tier
    app on a many servers, with all the
    b e l l s a n d w h i s t l e s o f H A
    orchestration, have load balancing
    at all levels, with failover and
    auto recovery?"
    Amazonian
    Team

    View full-size slide

  42. CE Platform Architecture
    Physical
    Virtualization
    Public Cloud
    Platform
    Security
    Networking Orchestration
    Container Engine
    COMMUNITY EDITION PLATFORM

    View full-size slide

  43. EE Platform Architecture
    Physical
    Virtualization
    Public Cloud
    Platform
    Security
    Developer
    Services
    Registry
    Services
    Access
    Policies
    App Lifecycle
    Management
    Automation &
    Extensibility
    Networking Orchestration Storage
    Container Engine
    ENTERPRISE EDITION PLATFORM

    View full-size slide

  44. No More One Size Fits All
    Docker for
    AWS
    Docker for
    Azure
    for CE only

    View full-size slide

  45. Reference
    Architecture
    Automation
    Tools
    Ecosystem
    Integration
    Docker Certified Infrastructure

    View full-size slide

  46. Docker EE on AWS Stack
    HW / OS Terraform + Ansible + AWS
    Runtime Docker EE
    Orchestration Docker Swarm
    Networking Docker Swarm Overlay
    Storage Docker Cloudstor EBS/EFS
    Layer 7 Proxy HTTP Routing Mesh (Interlock+Nginx)
    Registry Docker EE DTR
    Central Logging AWS Cloudwatch Logs
    Central Monitoring AWS Cloudwatch + Telegraph
    Swarm GUI Docker EE UCP

    View full-size slide

  47. Summary
    Infrastructure as code, make everything repeatable
    No "special" nodes, use remote management
    Grow as you go, assume you'll resize
    Look for compose files of popular tools to make stacks
    Don't throw out the good in search of the perfect

    View full-size slide

  48. Where to Go From Here
    Automating Docker Enterprise 5:25 this room
    Mission-Critical Migration to Multi-Cluster Kubernetes
    Wed 2pm room 113
    Swarm Intro Workshop
    Today 4pm room 133
    Wed 1:30pm room 133
    Swarm Hallway Track Wed 4pm

    View full-size slide

  49. I'd like to
    thank the
    internet

    View full-size slide

  50. Support
    the open source
    you use

    View full-size slide

  51. PLEASE USE SESSION
    SURVEY IN EVENTSPACE APP
    Thanks! !
    bretfisher.com/dockercon18
    "Building Your Swarm Tech Stack"

    View full-size slide