years of development From infrastructure to APIs, over a dozen extension points I have WAY more material than time! For later reading: https://goo.gl/2qz8jW Let’s talk about extensibility
“one size fits all” for almost anything networking related We need a way for users to customize how Kubernetes consumes networking infra Networking plugins
by CoreOS, now CNCF • Exec interface with stdin/stdout/env API Widely used, including by other projects (e.g. Mesos) Significant vendor uptake (Calico, Weave, Flannel, Contiv) Networking plugins
virtual, block and filesystem • Cloud block devices, FC, iSCSI, NFS, Ceph, Gluster, ... Vendors want their products to integrate well with Kubernetes Storage plugins
try new and interesting ideas • Container-ish: rkt, Containerd, CRI-O • VM-ish: Kata containers, Hyper.sh, gVisor CRI - gRPC based plugins for Kubernetes Docker-specific code was everywhere - making it a plugin made the code better: win-win! Container runtime plugins
private clouds) Built-in cloud-provider API (i.e. send us a PR) is hooked into many core control loops 8 implementations (and huge LOC count), so moving out-of-tree, to separate controllers Cloud providers
less) and virtual load-balancer We ship a default implementation (kube-proxy), but that can be replaced Controller: watch the API server for Services and Endpoints, program the underlying network Services
added • Without changing code or recompiling the system! • With a result that feels “built in” Add custom policy hooks • To built-in and extension APIs • Policy is arbitrary and very customer specific => APIs that add and modify the APIs What does it mean?
of customization Kubernetes provides 2 ways to extend the API • Custom Resource Definitions (CRD) • Extension API Servers (EAS) Both allow dynamic creation of new API “kinds” • Feel native: support kubectl, discovery API, etc. Trade-off: simplicity vs. flexibility Levels of customization
Customized deployments or stateful apps Add new abstractions • Build a custom PaaS that exposes JUST what you need Add entirely new concepts - it doesn’t have to be Kubernetes! What can I do with these?
plane KNative: A PaaS toolkit Kubernetes Metrics API: Adapters for monitoring systems Operators: Software robots to manage complex apps Docker Stacks: A higher-level abstraction Examples
EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 29d redis ClusterIP None <none> 55555/TCP 1s redis-random-ports NodePort 10.101.242.155 <none> 6379:31248/TCP 1s $ kubectl get deployments NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE redis 1 1 1 1 2s $ kubectl get stacks NAME AGE stackdemo 39s
TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE compose-api ClusterIP 10.110.211.86 <none> 443/TCP 17d $ kubectl get deployments -n docker NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE compose 1 1 1 1 29d compose-api 1 1 1 1 29d
apiserver • Run your own storage • The “main” apiserver is a proxy for your resources • Validation and defaulting built-in CRD: • Write a schema (or not!) for your resource • The “main” apiserver stores and serves your resources • Validation and defaulting as admission webhooks EAP vs. CRD
change. Affects writes, not reads. Webhook: The API Server calls your URL, synchronously Run in cluster (via service) or outside (e.g. serverless) Admission webhooks
to any resources Make all the changes (mutating) before doing all the checks (validating) MutatingWebhookConfiguration ValidatingWebhookConfiguration Admission webhooks
thockin
cyberagentdevelopers
aurimas
oracle4engineer
huffon
ymae
yokawasa
freee
zakky
kazzpapa3
ikumatadokoro
notwaldorf
aarron
stephaniewalter
davidbonilla
andyhume
gr2m
deanohume
samlambert
addyosmani
marcelosomers
bkeepers
destraynor
