Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Kubernetes Extensibility

Kubernetes Extensibility

A look at some of the ways Kubernetes is extensible and modular.

569f10721398d92f5033097ac6d9132c?s=128

Tim Hockin

May 09, 2016
Tweet

Transcript

  1. Google Cloud Platform Building Longevity into Kubernetes CoreOS Fest May

    9th, 2016 Tim Hockin <thockin@google.com> Senior Staff SW Engineer, Google @thockin
  2. Google Cloud Platform Google has been developing and using containers

    to manage our applications for over 12 years. Images by Connie Zhou
  3. Google Cloud Platform That’s a lot of time to accumulate

    features...
  4. Google Cloud Platform That’s a lot of time to accumulate

    features... The OSS world is far more diverse than Google internally...
  5. Google Cloud Platform That’s a lot of time to accumulate

    features... The OSS world is far more diverse than Google internally... We get to rebuild it all, from scratch, in less than half the time...
  6. Google Cloud Platform The chances of satisfying everyone?

  7. Google Cloud Platform

  8. Google Cloud Platform Modularity Extensibility Pluggability

  9. Google Cloud Platform users master nodes apiserver scheduler controller manager

    kube-proxy kubelet docker cloud provider etcd
  10. Google Cloud Platform kubelet users master nodes apiserver scheduler controller

    manager kube-proxy docker cloud provider etcd docker kubelet
  11. Google Cloud Platform kubelet users master nodes apiserver scheduler controller

    manager kube-proxy docker cloud provider etcd kube-dns fluentd docker kubelet
  12. Google Cloud Platform kubelet users master nodes apiserver scheduler controller

    manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet
  13. Google Cloud Platform kubelet users master nodes apiserver scheduler controller

    manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet
  14. Google Cloud Platform kubelet users master nodes apiserver scheduler controller

    manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet heapster influxdb
  15. Google Cloud Platform kubelet users master nodes apiserver scheduler controller

    manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller influxdb
  16. Google Cloud Platform kubelet users master nodes apiserver scheduler controller

    manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb
  17. Google Cloud Platform You get the point

  18. Google Cloud Platform Modularity Loose coupling is a goal everywhere

    • simpler • more composable • more extensible Isolate risk with interchangeable parts Sometimes accused of having too many parts -- but most are trivially replaced! Result: A more robust, adaptable system
  19. Google Cloud Platform kubelet users master nodes apiserver scheduler controller

    manager kube-proxy docker etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider
  20. Google Cloud Platform Linked as a library (for now) •

    needs to be extracted for easier customization Supports major public clouds, and some private clouds Interfaces for things like availability zones, VMs, load-balancers Used by controllers, admission control, Cloud Provider ?
  21. Google Cloud Platform users master nodes apiserver scheduler controller manager

    kube-proxy etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider kubelet docker
  22. Google Cloud Platform Not all containers are Docker containers Runtime

    API abstracts containers and implements pods • Linked as a library, but out-of- process plugin is in devel Docker is just the first implementation In devel: • rkt (CoreOS) • hyper_ (Hyper.sh) Container Runtime Runtime API
  23. Google Cloud Platform users master nodes apiserver controller manager kube-proxy

    etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider kubelet docker scheduler
  24. Google Cloud Platform A scheduler is just a program that

    uses the Kubernetes API Anyone can write their own • e.g. specialized affinity or isolation • customized for your needs • handle only Pods you care about Pods can “opt in” to alternative schedulers with an annotation (alpha) • default scheduler will ignore them Multi-Scheduler Scheduler Scheduler Scheduler ?
  25. Google Cloud Platform users master nodes apiserver controller manager kube-proxy

    etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker scheduler kubelet
  26. Google Cloud Platform Network Plugins Introduced in Kubernetes v1.0 •

    still considered experimental Uses CNI (CoreOS/appc) in v1.1 • simple exec interface • not using Docker libnetwork, • can simply defer to Docker Cluster admins can customize their installs • DHCP, MACVLAN, Flannel, custom... net Plugin Plugin Plugin
  27. Google Cloud Platform Storage Plugins Introduced in Kubernetes v1.0 •

    ~20 plugins in-tree so far Linked as a library • APIs: attach, mount, recycle, provision • not all plugins implement all facets • “flex” plugin for out-of-process plugins • can (soon) pass-through to Docker Usable directly (from a Pod) or through PersistentVolumes abstraction Plugin Plugin Plugin
  28. Google Cloud Platform users master nodes apiserver controller manager kube-proxy

    etcd kube-dns fluentd elasticsearch docker kubelet heapster dashboard influxdb cloud provider docker scheduler kubelet l7-lb-controller
  29. Google Cloud Platform Ingress (L7 LB) Services are L3/L4 (IP

    + port) Many apps are HTTP/HTTPS Ingress maps incoming traffic to backend services • by HTTP host headers • by HTTP URL paths HAProxy, NGINX, AWS and GCE implementations in progress Now with SSL! BETA in Kubernetes v1.2 Client URL Map
  30. Google Cloud Platform users master nodes controller manager kube-proxy etcd

    kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker scheduler kubelet apiserver
  31. Google Cloud Platform 3rd Party & API Groups ThirdPartyResource: •

    store your objects in our API server • CRUD only • creates a new Kind in the API • try ideas or build simple extensions Federated API servers: coming soon • store your objects in your own API server • register with our API, we delegate • more control of validation, errors, etc. • creates a new Kind in the API Pods Services ReplicaSets Your3PR Your API YourAPI API Server
  32. Google Cloud Platform users master nodes controller manager etcd kube-dns

    fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker scheduler kubelet apiserver kube-proxy
  33. Google Cloud Platform Services & kube-proxy Services are an abstraction:

    a stable IP that fronts N pods Default implementation is iptables, but that is just one way to do it Other implementations are equally valid • “real” load balancers • VIP • SDN-centric solutions • ...let’s get creative! Client Service IP
  34. Google Cloud Platform Past-me predicts that future-me is out of

    time
  35. Google Cloud Platform users master nodes controller manager etcd kube-dns

    fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker scheduler kubelet kube-proxy apiserver Authorization Authentication Admission
  36. Google Cloud Platform users master nodes controller manager etcd kube-dns

    fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker kubelet kube-proxy apiserver scheduler Predicates Priorities Extenders
  37. Google Cloud Platform users master nodes controller manager kube-dns fluentd

    elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker kubelet kube-proxy scheduler API storage etcd apiserver
  38. Google Cloud Platform users master nodes controller manager kube-dns fluentd

    elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker kube-proxy scheduler etcd apiserver kubelet Credentials
  39. Google Cloud Platform users master nodes controller manager fluentd elasticsearch

    docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker kube-proxy scheduler etcd apiserver kubelet kube-dns Replaceable
  40. Google Cloud Platform users master nodes controller manager fluentd elasticsearch

    docker kubelet heapster l7-lb-controller dashboard cloud provider docker kube-proxy scheduler etcd apiserver kubelet kube-dns Replaceable influxdb Replaceable
  41. 41 41 Kubernetes is Open https://kubernetes.io Code: github.com/kubernetes/kubernetes Chat: slack.k8s.io

    Twitter: @kubernetesio open community open design open source open to ideas