Share
Embed
Containers orchestration
An overview of Swarm, Kubernetes, Nomad and Mesos
Laurent Grangeau
December 14, 2016
Transcript
47 slides
-
I am Laurent Grangeau I love to automate things and
run apps at scale. You can find me at @laurentgrangeau Hello! -
Containerization ◉Based on kernel capabilities cgroups namespaces ◉Isolation of processus,
network, io, cpus, memory ◉Like a VM but not a VM -
Containerization ◉LXC (Linux) ◉Docker (Docker Inc.) ◉Rkt (CoreOS) ◉LXD (Ubuntu)
◉They all use the same kernel features -
Orchestration ◉Before, deployment was mainly bash scripts ◉Then came Ansible,
Chef, Puppet Standard way to test and deploy Allow developpers and DevOps to forget about what needs to happen ◉Then containers came We need to forget about what server will host a particular container, or how containers will be restarted, monitored and killed -
Orchestration ◉Composing of architecture, tools, and processes used by humans
to deliver a defined Service. ◉Stitching of software and hardware components together to deliver a defined Service. ◉Connecting and Automating of workflows when applicable to deliver a defined Service. -
Orchestration ◉You might not need it for small apps ◉No
orchestration == manual orchestration Place containers Scale containers Monitor containers Update containers ◉Operate containers at scale ? You need orchestration ! -
Orchestration ◉Cloud services are intended to scale-up arbitrarily and dynamically,
without requiring direct human intervention to do so. ◉Cloud service delivery includes fulfillment assurance and billing. ◉Cloud services delivery entails workflows in various technical and business domains. -
Docker Swarm mode ◉New in Docker 1.12 ◉Docker Inc’s official
solution ◉Major improvement over Swarm ◉Part of core distribution -
Docker Swarm mode core concepts ◉Introduce notion of Service ◉Based
on manager nodes and worker nodes ◉Manager Coordinate via Raft Distribute containers across the cluster No need of extra K/V store like etcd/zookeeper ◉Worker Run containers assigned by a manager -
Docker Swarm mode core concepts ◉Secure by default All communications
are TLS encrypted Certificates are automatically rotated ◉Easy to install and use One command to start a cluster One command to join new nodes ◉Load balancing ◉Service discovery -
Docker Swarm mode ◉Service Fixed numbers of containers are launched
together and kept running ◉Two types of service Replicated : maintain a specified number of containers across the cluster Global : run one instance of a container in each swarm node -
Docker Swarm mode ◉Tasks Each service need one or more
tasks A task is a container plus commands ◉Constraints and labels Control which node a container can be schedule on -
Swarm mode advantages ◉Easy to install ◉Secure by default ◉Secret
management comes with Docker 1.13 ◉Bundled with Docker ◉Have advanced features with enterprise edition UCP (RBAC, enforced security, support, Trusted registry…) ◉Full Docker experience -
Swarm mode disadvantages ◉Some features not yet supported ◉Distributed Application
Bundle still work in progress -
Kubernetes ◉Based on Google’s experience running containers at scale ◉Derived
from Borg See Google’s paper : https://static.googleusercontent.com/media/research.google.c om/fr//pubs/archive/43438.pdf -
Kubernetes ◉Many advanced features like : Load-balancing Secret management RBAC
(Role Based Access Control) … -
Kubernetes core concepts ◉Master A single master handles API calls,
assigns workloads and maintains configurations state ◉Minions Servers that run workloads -
Kubernetes core concepts ◉Pods Units of compute power One or
many containers deployed on the same host Perfom a task Single IP address Flat networking within the pod -
Kubernetes core concepts ◉Services Front end and load balancer for
pods Floating IP address for pods ◉Replication controllers Maintain X replicas of the required pods ◉Labels K/V tags use to identify pods, replication controllers and services -
Kubernetes advantages ◉Easy to install (via kubeadm, but limited to
single master) ◉Advanced features baked-in ◉Have a large community ◉Production grade container orchestration -
Kubernetes disadvantages ◉Harder to get started ◉Extra concepts to learn
(pod, replicaset, …) ◉Extra work to have HA on multiple masters ◉Have to setup etcd, network plugins, DNS servers and certificates authorities -
Nomad concepts ◉Driver Basic means of executing a task Example
: Qemu, Docker, Java, static binaries, … ◉Task Smallest unit of compute Executed by drivers -
Nomad concepts ◉Task group Set of task that must be
run together Run on the same client ◉Job Form of « desired state » Composed of on or more task groups -
Nomad concepts ◉Client Nodes that run Nomad agent Responsible for
registering with the servers, watching for any work to be assigned and executing tasks ◉Server The brain of the cluster Replicate data between each other and perform leader election Manage all jobs and clients, and create tasks allocations -
Nomad advantages ◉Integrated with other Hashicorp tools ◉Can orchestrate more
than Docker ◉Doesn’t depend on external systems or storage ◉Highly available ◉Support multi-datacenter and multi-region configurations -
Nomad disadvantages ◉Have to learn new concepts ◉No native service
discovery (you have to rely on external systems) ◉No load balancing ◉No overlay network ◉No DNS server -
Mesos concepts ◉Master Minimum of three master nodes ◉Slave Run
the tasks passed down by the master ◉Framework Decide what to do with the resources offer to them by Mesos -
Marathon ◉Mesos can’t orchestrate containers alone ◉Have to rely on
another tool Marathon ◉Marathon features Orchestrate docker containers or mesos framework Allow canary style deployments -
Marathon concepts ◉Applications Long running services ◉Features Service discovery (via
dedicated DNS service) Load balancing Constraint management -
Mesos advantages ◉Can orchestrate native mesos containers ◉Can run ten
of thousands of nodes in production -
Mesos disadvantages ◉Significantly harder to setup ◉Steeper learning curve ◉Rely
on external systems (zookeeper) -
Hard to predict a winner Use the framework
that fits most your environment Others schedulers out there (Blox, Diego, etc.) For full Docker experience, Swarm mode is the best choice