try new and interesting ideas • Container-ish: rkt, Containerd, CRI-O • VM-ish: Kata containers, Hyper.sh, gVisor CRI - gRPC based plugins for Kubernetes Docker-specific code was everywhere - making it a plugin made the code better: win-win! Container runtime plugins
private clouds) Built-in cloud-provider API (i.e. send us a PR) is hooked into many core control loops 8 implementations (and huge LOC count), so moving out-of-tree, to separate controllers Cloud providers
less) and virtual load-balancer We ship a default implementation (kube-proxy), but that can be replaced Controller: watch the API server for Services and Endpoints, program the underlying network Services
added • Without changing code or recompiling the system! • With a result that feels “built in” Add custom policy hooks • To built-in and extension APIs • Policy is arbitrary and very customer specific => APIs that add and modify the APIs What does it mean?
of customization Kubernetes provides 2 ways to extend the API • Custom Resource Definitions (CRD) • Extension API Servers (EAS) Both allow dynamic creation of new API “kinds” • Feel native: support kubectl, discovery API, etc. Trade-off: simplicity vs. flexibility Levels of customization
Customized deployments or stateful apps Add new abstractions • Build a custom PaaS that exposes JUST what you need Add entirely new concepts - it doesn’t have to be Kubernetes! What can I do with these?
EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 29d redis ClusterIP None <none> 55555/TCP 1s redis-random-ports NodePort 10.101.242.155 <none> 6379:31248/TCP 1s $ kubectl get deployments NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE redis 1 1 1 1 2s $ kubectl get stacks NAME AGE stackdemo 39s
TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE compose-api ClusterIP 10.110.211.86 <none> 443/TCP 17d $ kubectl get deployments -n docker NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE compose 1 1 1 1 29d compose-api 1 1 1 1 29d
apiserver • Run your own storage • The “main” apiserver is a proxy for your resources • Validation and defaulting built-in CRD: • Write a schema (or not!) for your resource • The “main” apiserver stores and serves your resources • Validation and defaulting as admission webhooks EAP vs. CRD