Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Orchestration for Sysadmins

Orchestration for Sysadmins

Talk I gave with @aluzzardi at DockerCon US 2015

Victor Vieux

June 22, 2015
Tweet

More Decks by Victor Vieux

Other Decks in Technology

Transcript

  1. Agenda • Introduction to Docker Swarm • Integrating Swarm with

    Mesos • Swarm Fault Tolerance: Replication • Provisioning Swarm with Docker Machine
  2. Swarm in a nutshell • Exposes several Docker Engines as

    a single virtual Engine • Serves the standard Docker API • Extremely easy to get started • Batteries included but swappable
  3. Timeline Oct Nov Dec Jan Feb Jun Proof of Concept

    DockerCon EU Open Repository First Release Candidate Swarm Beta Release Global Hack Day Open Proposal 0.2.0 Release 0.3.0 Release Apr
  4. Setting up Swarm • Create a cluster: - $ swarm

    create • Add nodes to a cluster: - $ swarm join --advertise=<node_ip> token://<token> • Start Swarm: - $ swarm manage -H <swarm_ip> token://<token> Or you can use your own etcd, zookeeper or consul
  5. Swarm Scheduler : 2 steps • Apply filters to exclude

    nodes - ports - constraints - affinities - health - dependencies • Use a strategy to rank and pick the best node - spread - binpack - random
  6. Swarm Scheduler • Resource Management - Memory, CPU, Network $

    docker run -m 1g -c 2 -p 80:80 ... • Fine-grained tuning - Constraints $ docker run -e constraint:region==us-east ... - Affinities $ docker run -e affinities:constainer!=redis* ... - Soft Constraints / Affinities
  7. Docker Swarm + Mesos • Experimental • Docker Swarm act

    as a Mesos Framework • In collaboration with • Only one flag to change: - $> swarm manage -c mesos-experimental <mesos_master_url> • Contributions are welcome: - github.com/docker/swarm/tree/master/cluster/mesos
  8. Mesos Cluster + Mesos CLI Mesos Slaves + + +

    + + + + + + + + + + Mesos Masters Marathon
  9. Mesos Cluster + Docker Swarm + Mesos CLI Mesos Slaves

    + + + + + + + + + + + + + Mesos Masters Marathon Docker CLI
  10. Mesos Cluster + Docker Swarm + Mesos CLI Mesos Slaves

    + + + + + + + + + + + + + Mesos Masters Marathon Docker CLI shipyard kitematic compose
  11. BY

  12. Replication Docker CLI Swarm Cluster • Swarm Manager is a

    Single Point of Failure API Request Manage
  13. Replication Docker CLI Swarm Cluster • Upon failure it becomes

    impossible to manage the cluster • Cluster continues to run un-managed API Request Manage
  14. Replica Replication Primary Docker CLI Swarm Cluster Forward API Request

    Manage Replica • One primary, many replicas • Leader Election built on top of Consul, etcd, ZooKeeper • Replicas forward API calls to primary • CLI can talk to any instance
  15. Replica Replication Primary Docker CLI Swarm Cluster Forward API Request

    Manage Replica • Replicas monitor health of primary instance • Leader gets re-elected upon primary failure
  16. Primary Replica Replication Replica Docker CLI Swarm Cluster Forward API

    Request Manage • One replica promoted to primary • Other replicas notified of leadership update • CLI can continue talking to same instance
  17. Machine in a nutshell • Host management for a container

    centric world • Provisions Virtual Machines • Installs & Manages Docker Engine • Swarm Cluster Provisioning
  18. Machine Management • Install, Configure & Upgrade Docker Engine -

    docker-machine upgrade node-{1,2,3} • PKI Management - TLS Encryption & Authentication by default - CA generation, Key signing and Re-keying - Seamless integration: Auto-configure the CLI • Bring Your Own Machine - Generic Driver
  19. Machine Operations • Operate - List, Start, Stop and Kill

    machines • Debug - docker-machine ssh node-2 • Script - docker-machine toolkit: ip, url, …
  20. Machine Clustering • Integration with Docker Swarm • Provision Swarm

    management nodes • Pre-configure machines to join a Swarm cluster • Tweak Swarm options