Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Orchestration for Sysadmins

Orchestration for Sysadmins

Talk I gave with @aluzzardi at DockerCon US 2015

Victor Vieux

June 22, 2015
Tweet

More Decks by Victor Vieux

Other Decks in Technology

Transcript

  1. Orchestration for Sysadmins
    Andrea Luzzardi / Victor Vieux
    Software Engineers @ docker

    View full-size slide

  2. Agenda
    • Introduction to Docker Swarm
    • Integrating Swarm with Mesos
    • Swarm Fault Tolerance: Replication
    • Provisioning Swarm with Docker Machine

    View full-size slide

  3. “Running containers on multiple hosts.”

    View full-size slide

  4. Swarm in a nutshell
    • Exposes several Docker Engines as a single virtual Engine
    • Serves the standard Docker API
    • Extremely easy to get started
    • Batteries included but swappable

    View full-size slide

  5. Timeline
    Oct
    Nov
    Dec
    Jan
    Feb
    Jun
    Proof of Concept
    DockerCon EU
    Open Repository
    First Release Candidate
    Swarm Beta Release
    Global Hack Day
    Open Proposal
    0.2.0 Release
    0.3.0 Release
    Apr

    View full-size slide

  6. Setting up Swarm
    • Create a cluster:
    - $ swarm create
    • Add nodes to a cluster:
    - $ swarm join --advertise= token://
    • Start Swarm:
    - $ swarm manage -H token://
    Or you can use your own etcd, zookeeper or consul

    View full-size slide

  7. Swarm Scheduler : 2 steps
    • Apply filters to exclude nodes
    - ports
    - constraints
    - affinities
    - health
    - dependencies
    • Use a strategy to rank and pick the best node
    - spread
    - binpack
    - random

    View full-size slide

  8. Swarm Scheduler
    • Resource Management
    - Memory, CPU, Network
    $ docker run -m 1g -c 2 -p 80:80 ...
    • Fine-grained tuning
    - Constraints
    $ docker run -e constraint:region==us-east ...
    - Affinities
    $ docker run -e affinities:constainer!=redis* ...
    - Soft Constraints / Affinities

    View full-size slide

  9. “Batteries Included but Swappable”

    View full-size slide

  10. Docker Swarm + Mesos
    • Experimental
    • Docker Swarm act as a Mesos Framework
    • In collaboration with
    • Only one flag to change:
    - $> swarm manage -c mesos-experimental
    • Contributions are welcome:
    - github.com/docker/swarm/tree/master/cluster/mesos

    View full-size slide

  11. Mesos Cluster
    +
    Mesos
    CLI
    Mesos Slaves
    +
    +
    +
    +
    +
    +
    +
    +
    +
    +
    +
    +
    +
    Mesos Masters
    Marathon

    View full-size slide

  12. Mesos Cluster + Docker Swarm
    +
    Mesos
    CLI
    Mesos Slaves
    +
    +
    +
    +
    +
    +
    +
    +
    +
    +
    +
    +
    +
    Mesos Masters
    Marathon
    Docker
    CLI

    View full-size slide

  13. Mesos Cluster + Docker Swarm
    +
    Mesos
    CLI
    Mesos Slaves
    +
    +
    +
    +
    +
    +
    +
    +
    +
    +
    +
    +
    +
    Mesos Masters
    Marathon
    Docker
    CLI
    shipyard
    kitematic compose

    View full-size slide

  14. Swarm Manager Replication

    View full-size slide

  15. Replication
    Docker
    CLI
    Swarm Cluster
    • Swarm Manager is a
    Single Point of Failure
    API Request
    Manage

    View full-size slide

  16. Replication
    Docker
    CLI
    Swarm Cluster
    • Upon failure it
    becomes impossible to
    manage the cluster
    • Cluster continues to
    run un-managed
    API Request
    Manage

    View full-size slide

  17. Replica
    Replication
    Primary
    Docker
    CLI
    Swarm Cluster
    Forward
    API Request
    Manage
    Replica
    • One primary, many replicas
    • Leader Election built on top
    of Consul, etcd, ZooKeeper
    • Replicas forward API calls to
    primary
    • CLI can talk to any instance

    View full-size slide

  18. Replica
    Replication
    Primary
    Docker
    CLI
    Swarm Cluster
    Forward
    API Request
    Manage
    Replica
    • Replicas monitor health
    of primary instance
    • Leader gets re-elected
    upon primary failure

    View full-size slide

  19. Primary
    Replica
    Replication
    Replica
    Docker
    CLI
    Swarm Cluster
    Forward
    API Request
    Manage
    • One replica promoted to
    primary
    • Other replicas notified of
    leadership update
    • CLI can continue talking to
    same instance

    View full-size slide

  20. “From Zero to Docker”

    View full-size slide

  21. Machine in a nutshell
    • Host management for a container centric world
    • Provisions Virtual Machines
    • Installs & Manages Docker Engine
    • Swarm Cluster Provisioning

    View full-size slide

  22. Machine Provisioning
    5 Local Providers
    9 Cloud Providers
    7 Operating Systems

    View full-size slide

  23. Machine Management
    • Install, Configure & Upgrade Docker Engine
    - docker-machine upgrade node-{1,2,3}
    • PKI Management
    - TLS Encryption & Authentication by default
    - CA generation, Key signing and Re-keying
    - Seamless integration: Auto-configure the CLI
    • Bring Your Own Machine - Generic Driver

    View full-size slide

  24. Machine Operations
    • Operate
    - List, Start, Stop and Kill machines
    • Debug
    - docker-machine ssh node-2
    • Script
    - docker-machine toolkit: ip, url, …

    View full-size slide

  25. Machine Clustering
    • Integration with Docker Swarm
    • Provision Swarm management nodes
    • Pre-configure machines to join a Swarm cluster
    • Tweak Swarm options

    View full-size slide

  26. Demo
    https://github.com/aluzzardi/swarm-replication-demo

    View full-size slide

  27. Thank you, Questions?
    Andrea Luzzardi - @aluzzardi
    Victor Vieux - @vieux

    View full-size slide