Managing containerized workloads OSCON 2014 Brian Grant Google #google-containers 

Kubernetes κυβερνήτης: Greek for “pilot” or “helmsman of a ship” the open source cluster workload manager from Google ● Manage applications, not machines ● Inspired by Google’s internal systems ○ >10 years experience running containerized apps ○ >2B containers / week ● Extensible & portable ● Could run anywhere ● Apache 2.0 licensed ● Written in Go Today I’m going to talk about Kubernetes, the open source cluster workload manager from Google. Kubernetes raises the level of abstraction, to enable users to focus on their applications rather than on infrastructure. The ideas in Kubernetes were derived from more than 10 years of experience with managing containerized applications at Google. We run more than 2B containers each week. Kubernetes was designed to be extensible and portable so that it could run anywhere. It is written in Go. The last slide has a link to its Github repo. I’m going to motivate the Kubernetes approach to application management using an example. 

Application stack on single host OS DB (mysql) CMS (drupal php-fpm) webserver (nginx) memcache init sshd logger monitor Let’s say you have a simple web application, comprised of a content-management system, webserver, cache, and database. You’ve been running the stack on a single host the traditional way, by installing its packages into the host’s image and plugging into the host’s init mechanism. And now you’re interested in using containers to manage your application in order to facilitate consistency across development, test, and production environments and portability across developer laptops, on prem clusters, and multiple cloud providers. 

Application stack in single container OS DB (mysql) CMS (drupal php-fpm) webserver (nginx) memcache init sshd logger monitor Container sshd supervisor So, you put the entire stack into a single container. In order to keep the components running inside the container, you put them under control of your own supervisor, though you still need to start the container itself using init. Example Dockerfile for this configuration: https://github.com/ricardoamaro/docker-drupal- nginx/blob/master/Dockerfile This works and achieves the portability and consistency goals. However, you update Drupal modules very frequently, and find it’s too disruptive to restart memcached and your database every time you need to do that. 

Application stack in separate containers OS CMS (drupal php-fpm) init sshd logger monitor webserver (nginx) memcache DB (mysql) So, you put each component into a separate container, which also allows you to drop supervisord. To make it easy for the components to find each other, you disable address translation (NAT) and just use host networking. But let’s say you now want to replicate for availability. 

Replicated application stack OS CMS (drupal php-fpm) init sshd webserver (nginx) memcache DB (mysql) OS CMS (drupal php-fpm) init sshd webserver (nginx) memcache DB (mysql) So, you duplicate the entire stack, and set up the database for master-slave replication. But, you realize you can’t easily use this approach to scale out further because it requires reconfiguring your database replication strategy. 

Distributed application stack OS init sshd CMS (drupal) webserver (nginx) OS init sshd OS init sshd DB (mysql) memcache OS init sshd DB (mysql) So you decide to bite the bullet and distribute the application stack across multiple hosts. This means you now have multiple host images to configure, so you’ll probably want to adopt a configuration management system, such as SaltStack. 

OS init sshd CMS (drupal) webserver (nginx) OS init sshd OS init sshd DB (mysql) memcache OS init sshd DB (mysql) OS init sshd memcache OS init sshd CMS (drupal) webserver (nginx) Elastic application stack But it does enable you to scale each tier independently as needed. The real problem is that the management of the application has largely shifted back into the domain of management of virtual or physical hosts, not just for image configuration, but also for provisioning, scaling, deployment, and networking. Furthermore, this approach doesn’t achieve the full benefits promised by containerization, such as: * separation of concerns of developers and operations, * fast startup, and * improved utilization. 

Liberated application stack OS init sshd CMS (drupal) webserver (nginx) DB (mysql) memcache DB (mysql) OS init sshd OS init sshd memcache What we want to do is to liberate the application from the underlying infrastructure and focus on managing the application. Ideally, without having to rewrite it. 

Kubernetes Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Master/Scheduler That is the goal of Kubernetes. Kubernetes schedules containers onto a cluster of hosts. The hosts have homogenous software configurations, which makes them straightforward to manage. 

Pods Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Master/Scheduler CMS WS Actually, Kubernetes doesn’t schedule containers, it schedules Pods. A pod models an application-specific logical host for co- located containers, such as in the Drupal/Nginx example. It provides an explicit abstraction for a common pattern. The containers within the pod share fate. They also share resources, such as filesystem volumes and localhost, which facilitates efficient and private local communication, as well as providing backward compatibility for many applications. 

Scheduled pods Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes - Master/Scheduler CMS WS MC DB CMS WS MC DB MC CMS WS CMS WS DB Compared to just a scheduling affinity mechanism, pods simplify resource sharing and lifecycle management. Compared to running all of the micro-services in a single container under a process supervisor, pods: * decouple their software dependencies, * enable independent updates, and * enable the infrastructure to provide management and introspection of the individual micro-services. For example, the system can probe and restart unresponsive containers, and report the frequency and reasons for failures. These simple features are enormously useful to developers who are iterating on their applications, and want to debug crashes, OOMs, deadlocks, and so on. 

Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes - Master/Scheduler CMS WS MC DB CMS WS MC DB MC CMS WS CMS WS DB CMS WS DB MC Organizing a sea of pods A handful of pods are easy enough to manage and schedule, but what about 10s, 100s, or even 1000s of pods? Users need to be able to perform management operations on sets of related pods, such as to take down an application. The scheduler needs to understand which pods are related to one another in order to make intelligent placement decisions. For example, if you’re replicating a component for availability, it wouldn’t be a good idea to put all replicas on the same host. And clients need to be able to communicate with all instances of a replicated service. 

Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes - Master/Scheduler CMS WS MC DB CMS WS MC DB MC CMS WS CMS WS DB CMS WS DB MC Labels labels: tier: frontend env: production Kubernetes uses key-value labels to identify sets of related pods and other objects. The labels specify identifying metadata about the objects and their roles in the application. For example, typically users deploy multiple applications onto a cluster, each comprised of multiple tiers, which may be broken down into multiple micro-services, and may be running multiple versions, such as in the case of canary instances or rolling updates. Multiple development and test instances may run in the same cluster, or even in the same cluster as production instances. Labels identify these distinguishing properties, and label selectors indicate which label keys and values much match in order to be included in a particular set. 

Replication Controller Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes - Master/Scheduler replicas: 3 template: ... replicaSelector: tier: frontend env: production CMS WS CMS WS CMS WS One case where a label selector is used in Kubernetes is in its Replication Controller, to identify a set of replicas. The responsibility of the replication controller is to ensure that a particular number of replicas belong to the set identified by its label selector. If it doesn’t find enough running, it creates more pods, from a template. If it finds too many, it kills some. This is an example where active controllers are needed in order to maintain the desired state. Programs crash, containers OOM, agents restart, kernels panic, systems reboot, and hosts die. Regardless of the cause of a failure, users want their applications to restart. 

Replication Controller Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes - Master/Scheduler replicas: 4 template: ... replicaSelector: tier: frontend env: production CMS WS CMS WS CMS WS CMS WS By adapting to changes in the desired number of replicas, a replication controller can be used to scale an application up... 

Replication Controller Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes - Master/Scheduler replicas: 1 template: ... replicaSelector: tier: frontend env: production CMS WS ...or down. The replication controller can also facilitate rolling updates. For instance, one could simply update the template and kill the old pods one by one, until they are all replaced by pods created from the updated template. Alternatively, one could create a new replication controller and gradually increase its replica count while decreasing the replica count of the original. Additionally, because the pods created by the replication controller are not owned by it, it’s also possible to remove pods from its control by changing their labels, in which case the controller will just replace them. This is useful for debugging. 

Service Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Machine Host Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes Agent Kubernetes - Master/Scheduler id: web-service port: 9000 selector: tier: frontend env: production Web-Service CMS WS CMS WS CMS WS CMS WS Finally, the service object provides a mechanism for clients to contact other pods. Connections to a service are load-balanced across the pods identified by the service’s label selector. Services can also be used to connect to singleton instances managed by replication controllers. To talk to a service, applications connect to a proxy, which tracks the current addresses of all services so that applications don’t need to. The proxy enables the client to remain oblivious to how many instances there are, whether they are of the same version, where they are scheduled, when they are restarted or rescheduled, and so on. As with replication controllers, the set targeted can be manipulated by changing labels, such as to gradually ramp down or rapidly rollback traffic to the previous release. 

Kubernetes also provides a flat address space, which facilitates self-registration in 3rd-party service discovery systems, such as Eureka. This is an area where we expect to evolve Kubernetes, to improve direct support for naming and to provide additional networking capabilities. 

{ "id": "memcacheController", "desiredState": { "replicas": 3, "replicaSelector": {"name": "memcache"}, "podTemplate": { "desiredState": { "manifest": { "id": "memcache", "containers": [{ "image": "jnagal/memcached", "ports": [{"containerPort": 11211}] }] } }, "labels": {"name": "memcache"} }}, "labels": {"name": "memcache"} } kubecfg.sh -c drupal-repctrl.json create /replicationControllers kubecfg.sh -c drupal-service.json create /services kubecfg.sh -c memcache-repctrl.json create /replicationControllers kubecfg.sh -c memcache-service.json create /services kubecfg.sh -c mysql-repctrl.json create /replicationControllers kubecfg.sh -c mysql-service.json create /services I didn’t prepare a demo, but to run this example, one would need to just create the 4 containers and invoke a half dozen API calls to the Kubernetes master. kubecfg is a simple tool that drives Kubernetes through its master’s REST API. The text on the lower right is an example of a replication controller spec. One could also build a full declarative configuration system on top of this basic mechanism, which would more closely match how we typically manage applications inside Google. 

Github stats ● Stars: >2600 ● Forks: 292 ● Contributors: 53 That’s Kubernetes in a nutshell. There is a lot of interest in Kubernetes from the community. Kubernetes is already one of the most starred container scheduling systems on github. 

Partnerships ● RedHat ● CoreOS ● Mesosphere ● IBM ● Microsoft ● SaltStack ● Docker We also have heavy participation from a number of partners. We have joined forces with: * Others working on container schedulers * Other cloud providers * Infrastructure partners: Docker and SaltStack Together, we’re: * porting Kubernetes to more Linux distributions, cloud platforms, and network environments, * solidifying its APIs, * fleshing out its feature set, * enhancing its process management and scheduling capabilities, * ensuring it’s capable of running production workloads, and * extending it to support new use cases and new kinds of workloads. We want to foster a rich, open ecosystem of systems and 

tools built upon the Kubernetes API, as well as multiple implementations of the API. 

Thank you! Check Kubernetes out at: https://github.com/GoogleCloudPlatform/kubernetes #google-containers on IRC [email protected]