An Intro to Container Orchestrators Shimojo Lab Seminar (Free Talk) Nov. 29 2017 Keichi Takahashi

What are Container Orchestrators? Cluster management software that enables… ‣ Launches and manages containers on clusters ‣ Scaling and fault recovery of containers ‣ Service discovery and load balancing Examples include ‣ Kubernetes (K8s): Cloud Native Computing Foundation ‣ Docker Swarm: Docker ‣ Nomad: Hashicorp ‣ Marathon: Mesosphere 2

Traditional Web Service Architecture a.k.a. Three-tier architecture 3 Web Server Application Server Database Application Server Load Balancer Database Web Server

Challenges 4 Failure Handling Hardware and software continuously fail in large- scale clusters. Requests must be rerouted appropriately in case of a failure. Traffic Surge Deploying Updates Incoming traffic often fluctuates violently. Server resources must be quickly scaled out to meet the demand. Modern web services are deployed hundreds or thousand times a day. Updates must be applied without stopping the whole service.

Recent Web Service Architecture a.k.a. Microservices architecture ‣ Loosely coupled fine-grained services communicating through RPCs ‣ Heterogenous tech stack, efficient scaling, high fault tolerance, etc. 5 Image Conversion UI Image Conversion Image Conversion Image Conversion Mail Image Conversion Persistency Image Conversion Recommen- dation Image Conversion Job Queue Image Conversion Notification Image Conversion Shopping Cart Image Conversion Inventory Image Conversion Payment

Solution: Container Orchestrators A software that manages containers distributed across a cluster. 6 Container Orchestrator Manage A A B B B C B B B A A C Scale out B B B B A A Hardware Crash Developer/Operator User

Kubernetes (K8s) ‣ The most popular container orchestrator today ‣ Originally designed by Google, donated to the Cloud Native Computing Foundation ‣ Scales up to 5,000 nodes and 300,000 containers ‣ Used in production at: 7 and many more Bloomberg “The Tale of Kubernetes”

Core Control Loop in Kubernetes 8 Diff React Observe Watch changes in the cluster resources Compute diff between actual cluster state and desired cluster state stored in DB Reconcile diff by launching containers, killing containers, etc.

Architecture of Kubernetes 9 kubectl kubelet kube-proxy container container kube-apiserver kube-controller-manager kube-scheduler kubelet kube-proxy container container kubelet kube-proxy container container Master Node Overlay Network Overlay Network etcd

Demo What I will show here: ‣ Start a local Kubernetes cluster using minikube ‣ Create a replicated deployment ‣ Kill a container in the deployment ‣ Scale the deployment 10 kind: Deployment metadata: name: nginx spec: replicas: 3 template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx ports: - containerPort: 80 --- kind: Service metadata: name: nginx-service labels: name: nginx-service spec: ports: - port: 80 targetPort: 80 protocol: TCP selector: app: nginx type: ClusterIP

The Innovation of Container Orchestrators Realized the shift from “how” to “what” ‣ From imperative to declarative ‣ User requests the desired state, whereas the orchestrator executes appropriate procedures to reach the desired state Realized the shift from “fortress” to “weeble” ‣ Embrace mortality and aim for dynamic equilibrium ‣ As long as the cluster can provide services to users as a collective system, the life/death of individual nodes or containers doesn’t matter 11

The Kubernetes Cluster Whack-a-mole Game 12 https://www.youtube.com/watch?v=TyDeZHPJlu8

Wrap-up ‣ A container orchestrator manages containers distributed across a cluster. ‣ It enables faster scaling, fault recovery, and deployment. ‣ It is changing cluster management from “how” to “what” ‣ It is bringing high resiliency to production clusters. 13 https://www.cafereo.co.jp/goods/124082

Further Readings The Tale of Kubernetes (Comic!) ‣ https://www.bloomberg.com/features/2017-kubernetes/ Large-scale cluster management at Google with Borg (Paper) ‣ https://research.google.com/pubs/pub43438.html Scalable Microservices with Kubernetes (MOOC) ‣ https://www.udacity.com/course/scalable-microservices-with- kubernetes--ud615 Whack-a-pod: The Kubernetes cluster whack-a-mole game ‣ https://medium.com/google-cloud/whack-a-pod-359cbfb61662 14