Kubernetes: One Year Later

Slide 1

Slide 1 text

Google Cloud Platform Kubernetes: One Year Later Container Camp 2016 April 15, 2016 Tim Hockin Senior Staff Software Engineer @thockin

Slide 2

Slide 2 text

Google Cloud Platform Kubernetes Greek for “Helmsman”; also the root of the words “governor” and “cybernetic” • Manages container clusters • Inspired and informed by Google’s experiences and internal systems • Supports multiple cloud and bare-metal environments • Supports multiple container runtimes • 100% Open source, written in Go Manage applications, not machines

Slide 3

Slide 3 text

Google Cloud Platform

Slide 4

Slide 4 text

Google Cloud Platform One Year Later...

Slide 5

Slide 5 text

Google Cloud Platform Velocity 1.0 1.1 1.2 v1.2: - 5k commits, - +50% unique contributors

Slide 6

Slide 6 text

Google Cloud Platform Community Top 0.01% of all Github projects 1200+ external projects based on k8s Companies Contributing Companies Using 800+ unique contributors

Slide 7

Slide 7 text

Google Cloud Platform

Slide 8

Slide 8 text

Google Cloud Platform Kubernetes 101 (accelerated)

Slide 9

Slide 9 text

Google Cloud Platform Containers

Slide 10

Slide 10 text

Google Cloud Platform Containers Pods

Slide 11

Slide 11 text

Google Cloud Platform Volumes

Slide 12

Slide 12 text

Google Cloud Platform Labels

Slide 13

Slide 13 text

Google Cloud Platform Replication Controller

Slide 14

Slide 14 text

Google Cloud Platform Replication Controller

Slide 15

Slide 15 text

Google Cloud Platform Pods Replication Controller

Slide 16

Slide 16 text

Google Cloud Platform Service

Slide 17

Slide 17 text

Google Cloud Platform Service

Slide 18

Slide 18 text

Google Cloud Platform Put it all together Tier: FE Tier: BE Tier: Mid

Slide 19

Slide 19 text

Google Cloud Platform So, What’s New?

Slide 20

Slide 20 text

Google Cloud Platform Multi-Zone Clusters

Slide 21

Slide 21 text

21 Goal: zone-fault tolerance for applications Zero API changes relative to kubernetes ● Create replication controllers, etc. as usual ● Pods get spread across zones Nodes and PersistentVolumes are labelled with their availability zone ● Fully automatic for GKE, GCE, AWS ● Manual for on-premise and other cloud providers (for now) Status: GA in Kubernetes v1.2 User Zone A Zone C Zone B Master Multi-Zone Clusters

Slide 22

Slide 22 text

Google Cloud Platform Deployments

Slide 23

Slide 23 text

Google Cloud Platform Deployments Goal: updates-as-a-service • Rolling update is imperative, client-side Deployment manages replica changes for you • stable object name • updates are configurable, done server-side • kubectl edit or kubectl apply Aggregates stats Can have multiple updates in flight Status: BETA in Kubernetes v1.2 ...

Slide 24

Slide 24 text

Google Cloud Platform Demo

Slide 25

Slide 25 text

Google Cloud Platform DaemonSets

Slide 26

Slide 26 text

Google Cloud Platform DaemonSets Problem: how to run a Pod on every node? • or a subset of nodes Similar to ReplicationController • principle: do one thing, don’t overload “Which nodes?” is a selector Use familiar tools and patterns Status: BETA in Kubernetes v1.2 Pod

Slide 27

Slide 27 text

Google Cloud Platform Demo

Slide 28

Slide 28 text

Google Cloud Platform HorizontalPodAutoscalers

Slide 29

Slide 29 text

Google Cloud Platform HorizontalPodAutoScalers Goal: Automatically scale pods as needed • based on CPU utilization (for now) • custom metrics in Alpha Efficiency now, capacity when you need it Operates within user-defined min/max bounds Set it and forget it Status: GA in Kubernetes v1.2 ... Stats

Slide 30

Slide 30 text

Google Cloud Platform Jobs

Slide 31

Slide 31 text

Google Cloud Platform Jobs Run-to-completion, as opposed to run-forever • Express parallelism vs. required completions • Workflow: restart on failure • Build/test: don’t restart on failure Aggregates success/failure counts Built for batch and big-data work Status: GA in Kubernetes v1.2 ...

Slide 32

Slide 32 text

Google Cloud Platform Demo

Slide 33

Slide 33 text

Google Cloud Platform Secrets

Slide 34

Slide 34 text

Google Cloud Platform Secrets Goal: grant a pod access to a secured something • don’t put secrets in the container image! 12-factor says config comes from the environment • Kubernetes is the environment Manage secrets via the Kubernetes API Inject secrets as virtual volumes into your Pods • late-binding, tmpfs - never touches disk • also available as env vars node API Pod Secret

Slide 35

Slide 35 text

Google Cloud Platform ConfigMaps

Slide 36

Slide 36 text

Google Cloud Platform ConfigMaps Goal: manage app configuration • ...without making overly-brittle container images 12-factor says config comes from the environment • Kubernetes is the environment Manage config via the Kubernetes API Inject config as a virtual volume into your Pods • late-binding, live-updated (atomic) • also available as env vars Status: GA in Kubernetes v1.2 node API Pod Config Map

Slide 37

Slide 37 text

Google Cloud Platform HTTP Load-Balancing

Slide 38

Slide 38 text

Google Cloud Platform Ingress (L7) Many apps are HTTP/HTTPS Services are L3/L4 (IP + port) Ingress maps incoming traffic to backend services • by HTTP host headers • by HTTP URL paths HAProxy, NGINX, AWS and GCE implementations in progress Now with SSL! Status: BETA in Kubernetes v1.2 Client URL Map

Slide 39

Slide 39 text

Google Cloud Platform PersistentVolumes

Slide 40

Slide 40 text

Google Cloud Platform PersistentVolumes A higher-level storage abstraction • insulation from any one cloud environment Admin provisions them, users claim them • NEW: auto-provisioning (alpha in v1.2) Independent lifetime and fate from consumers • lives until user is done with it • can be handed-off between pods Dynamically “scheduled” and managed, like nodes and pods Claim

Slide 41

Slide 41 text

Google Cloud Platform Network Isolation

Slide 42

Slide 42 text

Google Cloud Platform Network Isolation Describe the DAG of your app, enforce it in the network Restrict Pod-to-Pod traffic or across Namespaces Designed by the network SIG • implementations for Calico, OpenShift, Romana, OpenContrail (so far) Status: Alpha in v1.2, expect beta in v1.3

Slide 43

Slide 43 text

Google Cloud Platform Scalability

Slide 44

Slide 44 text

Google Cloud Platform SLO met at >1000 nodes, >30000 pods ● 99% of API calls return in < 1 second ● 99% of pods start in < 5 seconds Pure iptables kube-proxy ● No measurable CPU/throughput/latency hit 4x reduction in kubelet CPU and memory Planned for 1.3 ● Binary-encoded API (protobuf) ● Caching and parallelization in scheduler ● Nodes++ and pods++ Scalability & Performance: v1.2

Slide 45

Slide 45 text

Google Cloud Platform New and coming soon • Cron (scheduled jobs) • Custom metrics • “Apply” a config (more declarative) • Machine-generated Go clients (less deps!) • Volume usage stats • Multi-scheduler support • Node affinity and anti-affinity • More volume types • Out-of-process volume plugin • GUI • Pod hostname and FQDN • Better isolation • Multi-cluster federation • API federation • Private Docker registry • External DNS integration • Volume classes and provisioning • Node fencing • DiY Cloud Provider plugins • More container runtimes (e.g. Rkt, Hyper) • Better auth{n,z} • Big data integrations • Device scheduling (e.g. GPUs)

Slide 46

Slide 46 text

46 46 Kubernetes is Open https://kubernetes.io Code: github.com/kubernetes/kubernetes Chat: slack.k8s.io Twitter: @kubernetesio open community open design open source open to ideas