Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Orchestration for Sysadmins

Orchestration for Sysadmins

Talk I gave with @aluzzardi at DockerCon US 2015

1c23303ba42e0f9c0e017a94e900ea6c?s=128

Victor Vieux

June 22, 2015
Tweet

Transcript

  1. Orchestration for Sysadmins Andrea Luzzardi / Victor Vieux Software Engineers

    @ docker
  2. Agenda • Introduction to Docker Swarm • Integrating Swarm with

    Mesos • Swarm Fault Tolerance: Replication • Provisioning Swarm with Docker Machine
  3. “Running containers on multiple hosts.”

  4. Swarm in a nutshell • Exposes several Docker Engines as

    a single virtual Engine • Serves the standard Docker API • Extremely easy to get started • Batteries included but swappable
  5. Timeline Oct Nov Dec Jan Feb Jun Proof of Concept

    DockerCon EU Open Repository First Release Candidate Swarm Beta Release Global Hack Day Open Proposal 0.2.0 Release 0.3.0 Release Apr
  6. Setting up Swarm • Create a cluster: - $ swarm

    create • Add nodes to a cluster: - $ swarm join --advertise=<node_ip> token://<token> • Start Swarm: - $ swarm manage -H <swarm_ip> token://<token> Or you can use your own etcd, zookeeper or consul
  7. Swarm Scheduler : 2 steps • Apply filters to exclude

    nodes - ports - constraints - affinities - health - dependencies • Use a strategy to rank and pick the best node - spread - binpack - random
  8. Swarm Scheduler • Resource Management - Memory, CPU, Network $

    docker run -m 1g -c 2 -p 80:80 ... • Fine-grained tuning - Constraints $ docker run -e constraint:region==us-east ... - Affinities $ docker run -e affinities:constainer!=redis* ... - Soft Constraints / Affinities
  9. “Batteries Included but Swappable”

  10. Docker Swarm + Mesos • Experimental • Docker Swarm act

    as a Mesos Framework • In collaboration with • Only one flag to change: - $> swarm manage -c mesos-experimental <mesos_master_url> • Contributions are welcome: - github.com/docker/swarm/tree/master/cluster/mesos
  11. Mesos Cluster + Mesos CLI Mesos Slaves + + +

    + + + + + + + + + + Mesos Masters Marathon
  12. Mesos Cluster + Docker Swarm + Mesos CLI Mesos Slaves

    + + + + + + + + + + + + + Mesos Masters Marathon Docker CLI
  13. Mesos Cluster + Docker Swarm + Mesos CLI Mesos Slaves

    + + + + + + + + + + + + + Mesos Masters Marathon Docker CLI shipyard kitematic compose
  14. BY

  15. Swarm Manager Replication

  16. Replication Docker CLI Swarm Cluster • Swarm Manager is a

    Single Point of Failure API Request Manage
  17. Replication Docker CLI Swarm Cluster • Upon failure it becomes

    impossible to manage the cluster • Cluster continues to run un-managed API Request Manage
  18. Replica Replication Primary Docker CLI Swarm Cluster Forward API Request

    Manage Replica • One primary, many replicas • Leader Election built on top of Consul, etcd, ZooKeeper • Replicas forward API calls to primary • CLI can talk to any instance
  19. Replica Replication Primary Docker CLI Swarm Cluster Forward API Request

    Manage Replica • Replicas monitor health of primary instance • Leader gets re-elected upon primary failure
  20. Primary Replica Replication Replica Docker CLI Swarm Cluster Forward API

    Request Manage • One replica promoted to primary • Other replicas notified of leadership update • CLI can continue talking to same instance
  21. “From Zero to Docker”

  22. Machine in a nutshell • Host management for a container

    centric world • Provisions Virtual Machines • Installs & Manages Docker Engine • Swarm Cluster Provisioning
  23. Machine Provisioning 5 Local Providers 9 Cloud Providers 7 Operating

    Systems
  24. Machine Management • Install, Configure & Upgrade Docker Engine -

    docker-machine upgrade node-{1,2,3} • PKI Management - TLS Encryption & Authentication by default - CA generation, Key signing and Re-keying - Seamless integration: Auto-configure the CLI • Bring Your Own Machine - Generic Driver
  25. Machine Operations • Operate - List, Start, Stop and Kill

    machines • Debug - docker-machine ssh node-2 • Script - docker-machine toolkit: ip, url, …
  26. Machine Clustering • Integration with Docker Swarm • Provision Swarm

    management nodes • Pre-configure machines to join a Swarm cluster • Tweak Swarm options
  27. Demo https://github.com/aluzzardi/swarm-replication-demo

  28. Thank you, Questions? Andrea Luzzardi - @aluzzardi Victor Vieux -

    @vieux