Upgrade to Pro — share decks privately, control downloads, hide ads and more …

KubeCon 2017 - Kernels and Distros

569f10721398d92f5033097ac6d9132c?s=47 Tim Hockin
December 07, 2017

KubeCon 2017 - Kernels and Distros

Our KubeCon 2017 talk. We posit that "Kubernetes Distributions" are A Real Thing, and the the project would be better overall if we start to think about how to manage parts of it more like the Linux kernel and other parts more like a Linux distribution.

569f10721398d92f5033097ac6d9132c?s=128

Tim Hockin

December 07, 2017
Tweet

Transcript

  1. Google Cloud Platform Kubernetes: Kernels & Distros KubeCon 2017, Austin

    December 7, 2017 Tim Hockin <thockin@google.com> @thockin Michael Rubin <mrubin@google.com> @matchstick (c) Google LLC
  2. Google Cloud Platform Where is Kubernetes today?

  3. Google Cloud Platform A fairly large project (~3M LOC) Still

    growing Not a monolithic program A set of cooperating microservices Kubernetes is ...
  4. Google Cloud Platform apiserver scheduler controller manager kube-proxy kubelet etcd

    container runtime
  5. Google Cloud Platform kube-proxy kubelet container runtime elasticsearch heapster influxdb

    fluentd ingress controller kube-dns cloud controller manager apiserver scheduler controller manager etcd CNI driver CSI driver
  6. Google Cloud Platform Extensibility - Enabling out-of-tree Drivers • Network

    (CNI) • Storage (flex, CSI) • Device (e.g. GPU) • Cloud providers Container runtimes Operators / controllers Add-ons • e.g. logging, monitoring
  7. Google Cloud Platform Some assembly required

  8. Google Cloud Platform Some assembly required Must Find Components!

  9. Google Cloud Platform Some assembly required Must Find Components! Download

    Components!
  10. Google Cloud Platform Some assembly required Must Find Components! Download

    Components! Version Skew!
  11. Google Cloud Platform Some assembly required Must Find Components! Download

    Components! Version Skew! Test Hell!
  12. Google Cloud Platform Some assembly required Must Find Components! Download

    Components! Version Skew! Test Hell! Unusable to most users
  13. Google Cloud Platform Comparative analysis

  14. Google Cloud Platform Linux is many projects Decentralized & layered:

    • Installers • Kernel • Bootup • Shells • Tools • Programming Languages • GUIs Roughly “kernel” & “distro”
  15. Google Cloud Platform One big program Lives in its own

    git repo • No tools, add-ons, etc. Releases fairly frequently • 8-10 weeks Has only X.Y versions • Bugs in X.Y get fixed in X.Y+1 • X.Y.Z patch releases managed by community The kernel - Not useful alone
  16. Google Cloud Platform Everything else is developed separately DIY systems

    are not tenable ca. 1992, the concept of “Linux distros” emerged EVERYONE uses a distro Distributions
  17. Google Cloud Platform Distros serve different needs Most users don’t

    care about kernel version, just distro version Generally release slowly • Quarters to years Emphasize their differences • Technical & opinions Distributions
  18. Google Cloud Platform Distributions Distro Value Add Platform support (drivers

    & extensions) Support, security, & testing Packaging & component lifecycle Simple installation
  19. Google Cloud Platform Kubernetes: Kernel or Distribution?

  20. Google Cloud Platform Releases quarterly Includes enough to run most

    clouds • But not all of them • May not be true in the future Contains a bunch of drivers • But not all of them • May not be true in the future Is Kubernetes a kernel or a distro? ?
  21. Google Cloud Platform Integrates some 3rd party add-ons • But

    not many • Does not usually carry patches X.Y.Z releases and medium-term support Lives in a small number of git repos • Highly coupled components Is Kubernetes a kernel or a distro? ?
  22. Google Cloud Platform Platform support (drivers & extensions) Support, security,

    & testing Packaging & component lifecycle Simple installation Kubernetes upstream as a distro Distro Value Add
  23. Google Cloud Platform Other Kubernetes distros Distro Value Add There

    are ALREADY more than 30 Kubernetes distributions! • Clouds • Enterprise vendors • Higher level platforms • Bespoke Platform support (drivers & extensions) Support, security, & testing Packaging & component lifecycle Simple installation
  24. Google Cloud Platform * conformance program Fragmentation risk: conformance

  25. Google Cloud Platform Distros were inevitable.

  26. Google Cloud Platform Distros were inevitable. How do we want

    to organize our project and community?
  27. Google Cloud Platform Others will make distros • No coordination

    or consistency • We will have no say Result: • Fragmentation & politicization • Many options, confusing UX • Over time, converge on 3-4 distros? Option #1: Ignore it
  28. Google Cloud Platform Needs huge non-eng effort Major distraction from

    k8s Others will still do their own • Opinions Result: Probably failure, see option #1 Option #2: One distro to rule them all
  29. Google Cloud Platform Formalize what we already do Focus on

    correctness and stability Others will still do their own • But hopefully based on ours? Result: • Clean up our thinking / processes • Define tools, standards, etc. • Derived distros benefit from staying close Option 3: Find the middle-ground
  30. Google Cloud Platform Concrete Ideas

  31. Google Cloud Platform Pick ONE installer and make it great

    • Which one? Contentious! Or define a manifest format that installers consume? • Think kernel config process and result Installers 39% Please wait...
  32. Google Cloud Platform Formalize “add-ons” • Central repository • Ownership

    • Management mechanism • Tooling ◦ kubectl apt-get update? Start with cluster/addons/... • How to track upstreams? Add-ons
  33. Google Cloud Platform Bound and extract “the kernel” • Still

    multiple binaries • No installer, add-ons, cloud providers Manage it as a single component Release it tick-tock style • Features vs. stability Kernel != distro Manage the kernel OSS K8s
  34. Google Cloud Platform Fork all code into our space •

    Only ship things we control • Carry patches IFF needed Push everything to one repository • No hunting all over the internet Manage the distro Kernel Installers Container Images Addons Components
  35. Google Cloud Platform Distinguish component version from package version •

    e.g. 1.2.3-4 = “4th build of 1.2.3” Release distro every 6-12 months Base deprecation policies on distro releases Manage the distro
  36. Google Cloud Platform A new role - Distro Hero! These

    are just some ideas To pull this off, we need a community • Different skills • Different focus https://commons.wikimedia.org/wiki/File:Placeholder_couple_superhero.png