Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Kubernetes: One Year Later

Kubernetes: One Year Later

A followup to my Kubernetes talk at last year's Container Camp SF


Tim Hockin

April 15, 2016


  1. Google Cloud Platform Kubernetes: One Year Later Container Camp 2016

    April 15, 2016 Tim Hockin <thockin@google.com> Senior Staff Software Engineer @thockin
  2. Google Cloud Platform Kubernetes Greek for “Helmsman”; also the root

    of the words “governor” and “cybernetic” • Manages container clusters • Inspired and informed by Google’s experiences and internal systems • Supports multiple cloud and bare-metal environments • Supports multiple container runtimes • 100% Open source, written in Go Manage applications, not machines
  3. Google Cloud Platform

  4. Google Cloud Platform One Year Later...

  5. Google Cloud Platform Velocity 1.0 1.1 1.2 v1.2: - 5k

    commits, - +50% unique contributors
  6. Google Cloud Platform Community Top 0.01% of all Github projects

    1200+ external projects based on k8s Companies Contributing Companies Using 800+ unique contributors
  7. Google Cloud Platform

  8. Google Cloud Platform Kubernetes 101 (accelerated)

  9. Google Cloud Platform Containers

  10. Google Cloud Platform Containers Pods

  11. Google Cloud Platform Volumes

  12. Google Cloud Platform Labels

  13. Google Cloud Platform Replication Controller

  14. Google Cloud Platform Replication Controller

  15. Google Cloud Platform Pods Replication Controller

  16. Google Cloud Platform Service

  17. Google Cloud Platform Service

  18. Google Cloud Platform Put it all together Tier: FE Tier:

    BE Tier: Mid
  19. Google Cloud Platform So, What’s New?

  20. Google Cloud Platform Multi-Zone Clusters

  21. 21 Goal: zone-fault tolerance for applications Zero API changes relative

    to kubernetes • Create replication controllers, etc. as usual • Pods get spread across zones Nodes and PersistentVolumes are labelled with their availability zone • Fully automatic for GKE, GCE, AWS • Manual for on-premise and other cloud providers (for now) Status: GA in Kubernetes v1.2 User Zone A Zone C Zone B Master Multi-Zone Clusters
  22. Google Cloud Platform Deployments

  23. Google Cloud Platform Deployments Goal: updates-as-a-service • Rolling update is

    imperative, client-side Deployment manages replica changes for you • stable object name • updates are configurable, done server-side • kubectl edit or kubectl apply Aggregates stats Can have multiple updates in flight Status: BETA in Kubernetes v1.2 ...
  24. Google Cloud Platform Demo

  25. Google Cloud Platform DaemonSets

  26. Google Cloud Platform DaemonSets Problem: how to run a Pod

    on every node? • or a subset of nodes Similar to ReplicationController • principle: do one thing, don’t overload “Which nodes?” is a selector Use familiar tools and patterns Status: BETA in Kubernetes v1.2 Pod
  27. Google Cloud Platform Demo

  28. Google Cloud Platform HorizontalPodAutoscalers

  29. Google Cloud Platform HorizontalPodAutoScalers Goal: Automatically scale pods as needed

    • based on CPU utilization (for now) • custom metrics in Alpha Efficiency now, capacity when you need it Operates within user-defined min/max bounds Set it and forget it Status: GA in Kubernetes v1.2 ... Stats
  30. Google Cloud Platform Jobs

  31. Google Cloud Platform Jobs Run-to-completion, as opposed to run-forever •

    Express parallelism vs. required completions • Workflow: restart on failure • Build/test: don’t restart on failure Aggregates success/failure counts Built for batch and big-data work Status: GA in Kubernetes v1.2 ...
  32. Google Cloud Platform Demo

  33. Google Cloud Platform Secrets

  34. Google Cloud Platform Secrets Goal: grant a pod access to

    a secured something • don’t put secrets in the container image! 12-factor says config comes from the environment • Kubernetes is the environment Manage secrets via the Kubernetes API Inject secrets as virtual volumes into your Pods • late-binding, tmpfs - never touches disk • also available as env vars node API Pod Secret
  35. Google Cloud Platform ConfigMaps

  36. Google Cloud Platform ConfigMaps Goal: manage app configuration • ...without

    making overly-brittle container images 12-factor says config comes from the environment • Kubernetes is the environment Manage config via the Kubernetes API Inject config as a virtual volume into your Pods • late-binding, live-updated (atomic) • also available as env vars Status: GA in Kubernetes v1.2 node API Pod Config Map
  37. Google Cloud Platform HTTP Load-Balancing

  38. Google Cloud Platform Ingress (L7) Many apps are HTTP/HTTPS Services

    are L3/L4 (IP + port) Ingress maps incoming traffic to backend services • by HTTP host headers • by HTTP URL paths HAProxy, NGINX, AWS and GCE implementations in progress Now with SSL! Status: BETA in Kubernetes v1.2 Client URL Map
  39. Google Cloud Platform PersistentVolumes

  40. Google Cloud Platform PersistentVolumes A higher-level storage abstraction • insulation

    from any one cloud environment Admin provisions them, users claim them • NEW: auto-provisioning (alpha in v1.2) Independent lifetime and fate from consumers • lives until user is done with it • can be handed-off between pods Dynamically “scheduled” and managed, like nodes and pods Claim
  41. Google Cloud Platform Network Isolation

  42. Google Cloud Platform Network Isolation Describe the DAG of your

    app, enforce it in the network Restrict Pod-to-Pod traffic or across Namespaces Designed by the network SIG • implementations for Calico, OpenShift, Romana, OpenContrail (so far) Status: Alpha in v1.2, expect beta in v1.3
  43. Google Cloud Platform Scalability

  44. Google Cloud Platform SLO met at >1000 nodes, >30000 pods

    • 99% of API calls return in < 1 second • 99% of pods start in < 5 seconds Pure iptables kube-proxy • No measurable CPU/throughput/latency hit 4x reduction in kubelet CPU and memory Planned for 1.3 • Binary-encoded API (protobuf) • Caching and parallelization in scheduler • Nodes++ and pods++ Scalability & Performance: v1.2
  45. Google Cloud Platform New and coming soon • Cron (scheduled

    jobs) • Custom metrics • “Apply” a config (more declarative) • Machine-generated Go clients (less deps!) • Volume usage stats • Multi-scheduler support • Node affinity and anti-affinity • More volume types • Out-of-process volume plugin • GUI • Pod hostname and FQDN • Better isolation • Multi-cluster federation • API federation • Private Docker registry • External DNS integration • Volume classes and provisioning • Node fencing • DiY Cloud Provider plugins • More container runtimes (e.g. Rkt, Hyper) • Better auth{n,z} • Big data integrations • Device scheduling (e.g. GPUs)
  46. 46 46 Kubernetes is Open https://kubernetes.io Code: github.com/kubernetes/kubernetes Chat: slack.k8s.io

    Twitter: @kubernetesio open community open design open source open to ideas