Containers orchestration

An overview of Swarm, Kubernetes, Nomad and Mesos

Transcript

1. Containers orchestration An overview of Swarm, Kubernetes, Nomad and Mesos
2. I am Laurent Grangeau I love to automate things and run apps at scale. You can find me at @laurentgrangeau Hello!
3. Containerization 1
4. Containerization ◉Based on kernel capabilities cgroups namespaces ◉Isolation of processus, network, io, cpus, memory ◉Like a VM but not a VM
5. Inspired by Google
6. Containerization ◉LXC (Linux) ◉Docker (Docker Inc.) ◉Rkt (CoreOS) ◉LXD (Ubuntu) ◉They all use the same kernel features
7. What is orchestration ? 2
8. Orchestration ◉Before, deployment was mainly bash scripts ◉Then came Ansible, Chef, Puppet Standard way to test and deploy Allow developpers and DevOps to forget about what needs to happen ◉Then containers came We need to forget about what server will host a particular container, or how containers will be restarted, monitored and killed
9. Orchestration ◉Composing of architecture, tools, and processes used by humans to deliver a defined Service. ◉Stitching of software and hardware components together to deliver a defined Service. ◉Connecting and Automating of workflows when applicable to deliver a defined Service.
10. Why do I need orchestration ? 3
11. Orchestration ◉You might not need it for small apps ◉No orchestration == manual orchestration Place containers Scale containers Monitor containers Update containers ◉Operate containers at scale ? You need orchestration !
12. Orchestration ◉Cloud services are intended to scale-up arbitrarily and dynamically, without requiring direct human intervention to do so. ◉Cloud service delivery includes fulfillment assurance and billing. ◉Cloud services delivery entails workflows in various technical and business domains.
14. Docker Swarm Mode 4
15. Docker Swarm mode ◉New in Docker 1.12 ◉Docker Inc’s official solution ◉Major improvement over Swarm ◉Part of core distribution
16. Docker Swarm mode core concepts ◉Introduce notion of Service ◉Based on manager nodes and worker nodes ◉Manager Coordinate via Raft Distribute containers across the cluster No need of extra K/V store like etcd/zookeeper ◉Worker Run containers assigned by a manager
17. Docker Swarm mode core concepts ◉Secure by default All communications are TLS encrypted Certificates are automatically rotated ◉Easy to install and use One command to start a cluster One command to join new nodes ◉Load balancing ◉Service discovery
18. Docker Swarm mode ◉Service Fixed numbers of containers are launched together and kept running ◉Two types of service Replicated : maintain a specified number of containers across the cluster Global : run one instance of a container in each swarm node
19. Docker Swarm mode ◉Tasks Each service need one or more tasks A task is a container plus commands ◉Constraints and labels Control which node a container can be schedule on
20. Swarm mode advantages ◉Easy to install ◉Secure by default ◉Secret management comes with Docker 1.13 ◉Bundled with Docker ◉Have advanced features with enterprise edition UCP (RBAC, enforced security, support, Trusted registry…) ◉Full Docker experience
21. Swarm mode disadvantages ◉Some features not yet supported ◉Distributed Application Bundle still work in progress
22. Kubernetes 5
23. Kubernetes ◉Based on Google’s experience running containers at scale ◉Derived from Borg See Google’s paper : https://static.googleusercontent.com/media/research.google.c om/fr//pubs/archive/43438.pdf
24. Kubernetes ◉Many advanced features like : Load-balancing Secret management RBAC (Role Based Access Control) …
25. Kubernetes
26. Kubernetes core concepts ◉Master A single master handles API calls, assigns workloads and maintains configurations state ◉Minions Servers that run workloads
27. Kubernetes core concepts ◉Pods Units of compute power One or many containers deployed on the same host Perfom a task Single IP address Flat networking within the pod
28. Kubernetes core concepts ◉Services Front end and load balancer for pods Floating IP address for pods ◉Replication controllers Maintain X replicas of the required pods ◉Labels K/V tags use to identify pods, replication controllers and services
29. Kubernetes advantages ◉Easy to install (via kubeadm, but limited to single master) ◉Advanced features baked-in ◉Have a large community ◉Production grade container orchestration
30. Kubernetes disadvantages ◉Harder to get started ◉Extra concepts to learn (pod, replicaset, …) ◉Extra work to have HA on multiple masters ◉Have to setup etcd, network plugins, DNS servers and certificates authorities
31. Hashicorp Nomad 6
32. Nomad concepts ◉Driver Basic means of executing a task Example : Qemu, Docker, Java, static binaries, … ◉Task Smallest unit of compute Executed by drivers
33. Nomad concepts ◉Task group Set of task that must be run together Run on the same client ◉Job Form of « desired state » Composed of on or more task groups
34. Nomad concepts ◉Client Nodes that run Nomad agent Responsible for registering with the servers, watching for any work to be assigned and executing tasks ◉Server The brain of the cluster Replicate data between each other and perform leader election Manage all jobs and clients, and create tasks allocations
35. Nomad region
36. Nomad datacenter
37. Nomad advantages ◉Integrated with other Hashicorp tools ◉Can orchestrate more than Docker ◉Doesn’t depend on external systems or storage ◉Highly available ◉Support multi-datacenter and multi-region configurations
38. Nomad disadvantages ◉Have to learn new concepts ◉No native service discovery (you have to rely on external systems) ◉No load balancing ◉No overlay network ◉No DNS server
39. Mesos 7
40. Mesos concepts ◉Master Minimum of three master nodes ◉Slave Run the tasks passed down by the master ◉Framework Decide what to do with the resources offer to them by Mesos
41. Marathon ◉Mesos can’t orchestrate containers alone ◉Have to rely on another tool Marathon ◉Marathon features Orchestrate docker containers or mesos framework Allow canary style deployments
42. Marathon concepts ◉Applications Long running services ◉Features Service discovery (via dedicated DNS service) Load balancing Constraint management
43. Mesos advantages ◉Can orchestrate native mesos containers ◉Can run ten of thousands of nodes in production
44. Mesos disadvantages ◉Significantly harder to setup ◉Steeper learning curve ◉Rely on external systems (zookeeper)
45. Conclusion 8
46.  Hard to predict a winner  Use the framework that fits most your environment  Others schedulers out there (Blox, Diego, etc.)  For full Docker experience, Swarm mode is the best choice
47. Any questions ? You can find me at ◉ @laurentgrangeau ◉ laurent.grangeau@gmail.com Thanks!