Tweet
Tweet

Slide 1

Slide 1 text

Google Cloud Platform Building Longevity into Kubernetes CoreOS Fest May 9th, 2016 Tim Hockin Senior Staff SW Engineer, Google @thockin

Slide 2

Slide 2 text

Google Cloud Platform Google has been developing and using containers to manage our applications for over 12 years. Images by Connie Zhou

Slide 3

Slide 3 text

Google Cloud Platform That’s a lot of time to accumulate features...

Slide 4

Slide 4 text

Google Cloud Platform That’s a lot of time to accumulate features... The OSS world is far more diverse than Google internally...

Slide 5

Slide 5 text

Google Cloud Platform That’s a lot of time to accumulate features... The OSS world is far more diverse than Google internally... We get to rebuild it all, from scratch, in less than half the time...

Slide 6

Slide 6 text

Google Cloud Platform The chances of satisfying everyone?

Slide 7

Slide 7 text

Google Cloud Platform

Slide 8

Slide 8 text

Google Cloud Platform Modularity Extensibility Pluggability

Slide 9

Slide 9 text

Google Cloud Platform users master nodes apiserver scheduler controller manager kube-proxy kubelet docker cloud provider etcd

Slide 10

Slide 10 text

Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd docker kubelet

Slide 11

Slide 11 text

Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd kube-dns fluentd docker kubelet

Slide 12

Slide 12 text

Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet

Slide 13

Slide 13 text

Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet

Slide 14

Slide 14 text

Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet heapster influxdb

Slide 15

Slide 15 text

Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller influxdb

Slide 16

Slide 16 text

Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker cloud provider etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb

Slide 17

Slide 17 text

Google Cloud Platform You get the point

Slide 18

Slide 18 text

Google Cloud Platform Modularity Loose coupling is a goal everywhere • simpler • more composable • more extensible Isolate risk with interchangeable parts Sometimes accused of having too many parts -- but most are trivially replaced! Result: A more robust, adaptable system

Slide 19

Slide 19 text

Google Cloud Platform kubelet users master nodes apiserver scheduler controller manager kube-proxy docker etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider

Slide 20

Slide 20 text

Google Cloud Platform Linked as a library (for now) • needs to be extracted for easier customization Supports major public clouds, and some private clouds Interfaces for things like availability zones, VMs, load-balancers Used by controllers, admission control, Cloud Provider ?

Slide 21

Slide 21 text

Google Cloud Platform users master nodes apiserver scheduler controller manager kube-proxy etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider kubelet docker

Slide 22

Slide 22 text

Google Cloud Platform Not all containers are Docker containers Runtime API abstracts containers and implements pods • Linked as a library, but out-of- process plugin is in devel Docker is just the first implementation In devel: • rkt (CoreOS) • hyper_ (Hyper.sh) Container Runtime Runtime API

Slide 23

Slide 23 text

Google Cloud Platform users master nodes apiserver controller manager kube-proxy etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider kubelet docker scheduler

Slide 24

Slide 24 text

Google Cloud Platform A scheduler is just a program that uses the Kubernetes API Anyone can write their own • e.g. specialized affinity or isolation • customized for your needs • handle only Pods you care about Pods can “opt in” to alternative schedulers with an annotation (alpha) • default scheduler will ignore them Multi-Scheduler Scheduler Scheduler Scheduler ?

Slide 25

Slide 25 text

Google Cloud Platform users master nodes apiserver controller manager kube-proxy etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker scheduler kubelet

Slide 26

Slide 26 text

Google Cloud Platform Network Plugins Introduced in Kubernetes v1.0 • still considered experimental Uses CNI (CoreOS/appc) in v1.1 • simple exec interface • not using Docker libnetwork, • can simply defer to Docker Cluster admins can customize their installs • DHCP, MACVLAN, Flannel, custom... net Plugin Plugin Plugin

Slide 27

Slide 27 text

Google Cloud Platform Storage Plugins Introduced in Kubernetes v1.0 • ~20 plugins in-tree so far Linked as a library • APIs: attach, mount, recycle, provision • not all plugins implement all facets • “flex” plugin for out-of-process plugins • can (soon) pass-through to Docker Usable directly (from a Pod) or through PersistentVolumes abstraction Plugin Plugin Plugin

Slide 28

Slide 28 text

Google Cloud Platform users master nodes apiserver controller manager kube-proxy etcd kube-dns fluentd elasticsearch docker kubelet heapster dashboard influxdb cloud provider docker scheduler kubelet l7-lb-controller

Slide 29

Slide 29 text

Google Cloud Platform Ingress (L7 LB) Services are L3/L4 (IP + port) Many apps are HTTP/HTTPS Ingress maps incoming traffic to backend services • by HTTP host headers • by HTTP URL paths HAProxy, NGINX, AWS and GCE implementations in progress Now with SSL! BETA in Kubernetes v1.2 Client URL Map

Slide 30

Slide 30 text

Google Cloud Platform users master nodes controller manager kube-proxy etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker scheduler kubelet apiserver

Slide 31

Slide 31 text

Google Cloud Platform 3rd Party & API Groups ThirdPartyResource: • store your objects in our API server • CRUD only • creates a new Kind in the API • try ideas or build simple extensions Federated API servers: coming soon • store your objects in your own API server • register with our API, we delegate • more control of validation, errors, etc. • creates a new Kind in the API Pods Services ReplicaSets Your3PR Your API YourAPI API Server

Slide 32

Slide 32 text

Google Cloud Platform users master nodes controller manager etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker scheduler kubelet apiserver kube-proxy

Slide 33

Slide 33 text

Google Cloud Platform Services & kube-proxy Services are an abstraction: a stable IP that fronts N pods Default implementation is iptables, but that is just one way to do it Other implementations are equally valid • “real” load balancers • VIP • SDN-centric solutions • ...let’s get creative! Client Service IP

Slide 34

Slide 34 text

Google Cloud Platform Past-me predicts that future-me is out of time

Slide 35

Slide 35 text

Google Cloud Platform users master nodes controller manager etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker scheduler kubelet kube-proxy apiserver Authorization Authentication Admission

Slide 36

Slide 36 text

Google Cloud Platform users master nodes controller manager etcd kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker kubelet kube-proxy apiserver scheduler Predicates Priorities Extenders

Slide 37

Slide 37 text

Google Cloud Platform users master nodes controller manager kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker kubelet kube-proxy scheduler API storage etcd apiserver

Slide 38

Slide 38 text

Google Cloud Platform users master nodes controller manager kube-dns fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker kube-proxy scheduler etcd apiserver kubelet Credentials

Slide 39

Slide 39 text

Google Cloud Platform users master nodes controller manager fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard influxdb cloud provider docker kube-proxy scheduler etcd apiserver kubelet kube-dns Replaceable

Slide 40

Slide 40 text

Google Cloud Platform users master nodes controller manager fluentd elasticsearch docker kubelet heapster l7-lb-controller dashboard cloud provider docker kube-proxy scheduler etcd apiserver kubelet kube-dns Replaceable influxdb Replaceable

Slide 41

Slide 41 text

41 41 Kubernetes is Open https://kubernetes.io Code: github.com/kubernetes/kubernetes Chat: slack.k8s.io Twitter: @kubernetesio open community open design open source open to ideas