Kubernetes Extensibility

A look at some of the ways Kubernetes is extensible and modular.

Transcript

1. Google Cloud Platform Building Longevity into Kubernetes CoreOS Fest May 9th, 2016 Tim Hockin <thockin@google.com> Senior Staff SW Engineer, Google @thockin
2. Google Cloud Platform Google has been developing and using containers to manage our applications for over 12 years. Images by Connie Zhou
3. Google Cloud Platform That’s a lot of time to accumulate features...
4. Google Cloud Platform That’s a lot of time to accumulate features... The OSS world is far more diverse than Google internally...
5. Google Cloud Platform That’s a lot of time to accumulate features... The OSS world is far more diverse than Google internally... We get to rebuild it all, from scratch, in less than half the time...
6. Google Cloud Platform The chances of satisfying everyone?
7. Google Cloud Platform
8. Google Cloud Platform Modularity Extensibility Pluggability
9. Google Cloud Platform users master nodes apiserver scheduler controller manager kube-proxy kubelet docker cloud provider etcd
10. Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd docker kubelet
11. Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd kube-dns fluentd docker kubelet
12. Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet
13. Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet
14. Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet heapster influxdb
15. Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller influxdb
16. Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb
17. Google Cloud Platform You get the point
18. Google Cloud Platform Modularity Loose coupling is a goal everywhere • simpler • more composable • more extensible Isolate risk with interchangeable parts Sometimes accused of having too many parts -- but most are trivially replaced! Result: A more robust, adaptable system
19. Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider
20. Google Cloud Platform Linked as a library (for now) • needs to be extracted for easier customization Supports major public clouds, and some private clouds Interfaces for things like availability zones, VMs, load-balancers Used by controllers, admission control, Cloud Provider ?
21. Google Cloud Platform users master nodes apiserver scheduler controller manager kube-proxy etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider kubelet docker
22. Google Cloud Platform Not all containers are Docker containers Runtime API abstracts containers and implements pods • Linked as a library, but out-of- process plugin is in devel Docker is just the first implementation In devel: • rkt (CoreOS) • hyper_ (Hyper.sh) Container Runtime Runtime API
23. Google Cloud Platform users master nodes apiserver controller manager kube-proxy etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider kubelet docker scheduler
24. Google Cloud Platform A scheduler is just a program that uses the Kubernetes API Anyone can write their own • e.g. specialized affinity or isolation • customized for your needs • handle only Pods you care about Pods can “opt in” to alternative schedulers with an annotation (alpha) • default scheduler will ignore them Multi-Scheduler Scheduler Scheduler Scheduler ?
25. Google Cloud Platform users master nodes apiserver controller manager kube-proxy etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker scheduler kubelet
26. Google Cloud Platform Network Plugins Introduced in Kubernetes v1.0 • still considered experimental Uses CNI (CoreOS/appc) in v1.1 • simple exec interface • not using Docker libnetwork, • can simply defer to Docker Cluster admins can customize their installs • DHCP, MACVLAN, Flannel, custom... net Plugin Plugin Plugin
27. Google Cloud Platform Storage Plugins Introduced in Kubernetes v1.0 • ~20 plugins in-tree so far Linked as a library • APIs: attach, mount, recycle, provision • not all plugins implement all facets • “flex” plugin for out-of-process plugins • can (soon) pass-through to Docker Usable directly (from a Pod) or through PersistentVolumes abstraction Plugin Plugin Plugin
28. Google Cloud Platform users master nodes apiserver controller manager kube-proxy etcd kube-dns fluentd elasticsearch docker kubelet heapster dashboard influxdb cloud provider docker scheduler kubelet l7-lb-controller
29. Google Cloud Platform Ingress (L7 LB) Services are L3/L4 (IP + port) Many apps are HTTP/HTTPS Ingress maps incoming traffic to backend services • by HTTP host headers • by HTTP URL paths HAProxy, NGINX, AWS and GCE implementations in progress Now with SSL! BETA in Kubernetes v1.2 Client URL Map
30. Google Cloud Platform users master nodes controller manager kube-proxy etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker scheduler kubelet apiserver
31. Google Cloud Platform 3rd Party & API Groups ThirdPartyResource: • store your objects in our API server • CRUD only • creates a new Kind in the API • try ideas or build simple extensions Federated API servers: coming soon • store your objects in your own API server • register with our API, we delegate • more control of validation, errors, etc. • creates a new Kind in the API Pods Services ReplicaSets Your3PR Your API YourAPI API Server
32. Google Cloud Platform users master nodes controller manager etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker scheduler kubelet apiserver kube-proxy
33. Google Cloud Platform Services & kube-proxy Services are an abstraction: a stable IP that fronts N pods Default implementation is iptables, but that is just one way to do it Other implementations are equally valid • “real” load balancers • VIP • SDN-centric solutions • ...let’s get creative! Client Service IP
34. Google Cloud Platform Past-me predicts that future-me is out of time
35. Google Cloud Platform users master nodes controller manager etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker scheduler kubelet kube-proxy apiserver Authorization Authentication Admission
36. Google Cloud Platform users master nodes controller manager etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker kubelet kube-proxy apiserver scheduler Predicates Priorities Extenders
37. Google Cloud Platform users master nodes controller manager kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker kubelet kube-proxy scheduler API storage etcd apiserver
38. Google Cloud Platform users master nodes controller manager kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker kube-proxy scheduler etcd apiserver kubelet Credentials
39. Google Cloud Platform users master nodes controller manager fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker kube-proxy scheduler etcd apiserver kubelet kube-dns Replaceable
40. Google Cloud Platform users master nodes controller manager fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard cloud provider docker kube-proxy scheduler etcd apiserver kubelet kube-dns Replaceable influxdb Replaceable
41. 41 41 Kubernetes is Open https://kubernetes.io Code: github.com/kubernetes/kubernetes Chat: slack.k8s.io Twitter: @kubernetesio open community open design open source open to ideas