Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Kubernetes Intro & Update

569f10721398d92f5033097ac6d9132c?s=47 Tim Hockin
September 30, 2015

Kubernetes Intro & Update

From the Kubernetes Meetup #3, 9/30/2015, hosted at Dell in Santa Clara.

569f10721398d92f5033097ac6d9132c?s=128

Tim Hockin

September 30, 2015
Tweet

Transcript

  1. Google confidential │ Do not distribute Kubernetes: Intro & Update

    Kubernetes Meetup 9/30/2015 Tim Hockin <thockin@google.com> Senior Staff Software Engineer @thockin
  2. Google confidential │ Do not distribute Google has been developing

    and using containers to manage our applications for over 10 years. Images by Connie Zhou
  3. Google confidential │ Do not distribute Everything at Google runs

    in containers: • Gmail, Web Search, Maps, ... • MapReduce, batch, ... • GFS, Colossus, ... • Even Google’s Cloud Platform: VMs run in containers!
  4. Google confidential │ Do not distribute Everything at Google runs

    in containers: • Gmail, Web Search, Maps, ... • MapReduce, batch, ... • GFS, Colossus, ... • Even Google’s Cloud Platform: VMs run in containers! We launch over 2 billion containers per week
  5. Google confidential │ Do not distribute But it’s all so

    different! • Deployment • Management, monitoring • Isolation (very complicated!) • Updates • Discovery • Scaling, replication, sets A fundamentally different way of managing applications requires different tooling and abstractions Images by Connie Zhou
  6. Google confidential │ Do not distribute Kubernetes Greek for “Helmsman”;

    also the root of the word “Governor” and “cybernetic” • Container orchestrator • Runs and manages containers • Supports multiple cloud and bare-metal environments • Inspired and informed by Google’s experiences and internal systems • 100% Open source, written in Go Manage applications, not machines
  7. Google confidential │ Do not distribute Pets vs. Cattle

  8. Google confidential │ Do not distribute kubelet UI kubelet CLI

    API users master nodes The 10000 foot view apiserver kubelet scheduler controllers
  9. Google confidential │ Do not distribute UI All you really

    care about API Container Cluster
  10. Google confidential │ Do not distribute Container clusters: A story

    in two parts
  11. Google confidential │ Do not distribute Container clusters: A story

    in two parts 1. Setting up the cluster • Choose a cloud: GCE, AWS, Azure, Rackspace, on-premises, ... • Choose a node OS: CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ... • Provision machines: Boot VMs, install and run kube components, ... • Configure networking: IP ranges for Pods, Services, SDN, ... • Start cluster services: DNS, logging, monitoring, ... • Manage nodes: kernel upgrades, OS updates, hardware failures... Not the easy or fun part, but unavoidable This is where things like Google Container Engine (GKE) really help
  12. Google confidential │ Do not distribute 2. Using the cluster

    • Run Pods & Containers • Replication controllers • Services • Volumes This is the fun part! A distinct set of problems from cluster setup and management Don’t make developers deal with cluster administration! Accelerate development by focusing on the applications, not the cluster Container clusters: A story in two parts
  13. Google confidential │ Do not distribute Networking

  14. Google confidential │ Do not distribute 10.1.1.0/24 172.16.1.1 172.16.1.2 Docker

    networking 10.1.2.0/24 172.16.1.1 10.1.3.0/24 172.16.1.1
  15. Google confidential │ Do not distribute 10.1.1.0/24 172.16.1.1 172.16.1.2 Docker

    networking 10.1.2.0/24 172.16.1.1 10.1.3.0/24 172.16.1.1 NAT NAT NAT NAT NAT
  16. Google confidential │ Do not distribute Kubernetes networking IPs are

    routable • vs docker default private IP Pods can reach each other without NAT • even across nodes No brokering of port numbers • too complex, why bother? This is a fundamental requirement • can be L3 routed • can be underlayed (cloud) • can be overlayed (SDN)
  17. Google confidential │ Do not distribute 10.1.1.0/24 172.16.1.1 172.16.1.2 Kubernetes

    networking 10.1.2.0/24 172.16.1.1 10.1.3.0/24 172.16.1.1
  18. Google confidential │ Do not distribute Pods

  19. Google confidential │ Do not distribute Pods Small group of

    containers & volumes Tightly coupled The atom of scheduling & placement in Kubernetes Shared namespace • share IP address & localhost • share IPC Mortal • can die, cannot be reborn Example: data puller & web server Consumers Content Manager File Puller Web Server Volume Pod
  20. Google confidential │ Do not distribute Volumes Very similar to

    Docker’s concept Pod scoped storage Share the pod’s lifetime & fate Support many types of volume plugins • Empty dir (and tmpfs) • Host path • Git repository • GCE Persistent Disk • AWS Elastic Block Store • iSCSI • NFS • GlusterFS • Ceph File and RBD • Cinder • Secret • ...
  21. Google confidential │ Do not distribute Labels & Selectors

  22. Google confidential │ Do not distribute Arbitrary metadata Attached to

    any API object Generally represent identity Queryable by selectors • think SQL ‘select ... where ...’ The only grouping mechanism • pods under a ReplicationController • pods in a Service • capabilities of a node (constraints) Labels
  23. Google confidential │ Do not distribute App: MyApp Phase: prod

    Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE Selectors
  24. Google confidential │ Do not distribute App: MyApp Phase: prod

    Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp Selectors
  25. Google confidential │ Do not distribute App: MyApp Phase: prod

    Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp, Role = FE Selectors
  26. Google confidential │ Do not distribute App: MyApp Phase: prod

    Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp, Role = BE Selectors
  27. Google confidential │ Do not distribute Selectors App: MyApp Phase:

    prod Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp, Phase = prod
  28. Google confidential │ Do not distribute App: MyApp Phase: prod

    Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp, Phase = test Selectors
  29. Google confidential │ Do not distribute ReplicationControllers

  30. Google confidential │ Do not distribute ReplicationControllers An example of

    control loops Runs out-of-process wrt API server Have 1 job: ensure N copies of a pod • if too few, start new ones • if too many, kill some • grouped by a selector Cleanly layered on top of the core • all access is by public APIs Replicated pods are fungible • No implied order or identity ReplicationController - name = “my-rc” - selector = {“App”: “MyApp”} - podTemplate = { ... } - replicas = 4 API Server How many? 3 Start 1 more OK How many? 4
  31. Google confidential │ Do not distribute Services

  32. Google confidential │ Do not distribute Services A group of

    pods that work together • grouped by a selector Defines access policy • “load balanced” or “headless” Gets a stable virtual IP and port • sometimes called the service portal • also a DNS name VIP is managed by kube-proxy • watches all services • updates iptables when backends change Hides complexity - ideal for non-native apps Virtual IP Client
  33. Google confidential │ Do not distribute Secrets

  34. Google confidential │ Do not distribute Secrets Problem: how to

    grant a pod access to a secured something? • don’t put secrets in the container image! 12-factor says: config comes from the environment • Kubernetes is the environment Manage secrets via the Kubernetes API Inject them as “virtual volumes” into Pods • late-binding • tmpfs - never touches disk node Pod Secret API
  35. Google confidential │ Do not distribute Rolling updates

  36. Google confidential │ Do not distribute Rolling Updates ReplicationController -

    replicas: 3 - selector: - app: MyApp - version: v1 Service - app: MyApp
  37. Google confidential │ Do not distribute Rolling Updates ReplicationController -

    replicas: 3 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 0 - selector: - app: MyApp - version: v2 Service - app: MyApp
  38. Google confidential │ Do not distribute Rolling Updates ReplicationController -

    replicas: 3 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 1 - selector: - app: MyApp - version: v2 Service - app: MyApp
  39. Google confidential │ Do not distribute Rolling Updates ReplicationController -

    replicas: 2 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 1 - selector: - app: MyApp - version: v2 Service - app: MyApp
  40. Google confidential │ Do not distribute Rolling Updates ReplicationController -

    replicas: 2 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 2 - selector: - app: MyApp - version: v2 Service - app: MyApp
  41. Google confidential │ Do not distribute Rolling Updates ReplicationController -

    replicas: 1 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 2 - selector: - app: MyApp - version: v2 Service - app: MyApp
  42. Google confidential │ Do not distribute Rolling Updates ReplicationController -

    replicas: 1 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 3 - selector: - app: MyApp - version: v2 Service - app: MyApp
  43. Google confidential │ Do not distribute Rolling Updates ReplicationController -

    replicas: 0 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 3 - selector: - app: MyApp - version: v2 Service - app: MyApp
  44. Google confidential │ Do not distribute DaemonSets

  45. Google confidential │ Do not distribute DaemonSets Problem: how to

    run a Pod on every node • or a subset of nodes Similar to ReplicationController • principle: do one thing, don’t overload “Which nodes?” is a selector Use familiar tools and patterns Status: EXPERIMENTAL in Kubernetes v1.1
  46. Google confidential │ Do not distribute PersistentVolumes

  47. Google confidential │ Do not distribute PersistentVolumes A higher-level abstraction

    • insulation from any one cloud environment Admin provisions them, users claim them Independent lifetime and fate Can be handed-off between pods and lives until user is done with it Dynamically “scheduled” and managed, like nodes and pods
  48. Google confidential │ Do not distribute PersistentVolumes Cluster Admin

  49. Google confidential │ Do not distribute PersistentVolumes Provision Cluster Admin

    PersistentVolumes
  50. Google confidential │ Do not distribute PersistentVolumes Cluster Admin User

    PersistentVolumes
  51. Google confidential │ Do not distribute PersistentVolumes Cluster Admin User

    PersistentVolumes PVClaim Create
  52. Google confidential │ Do not distribute PersistentVolumes Cluster Admin User

    PersistentVolumes PVClaim Binder
  53. Google confidential │ Do not distribute PersistentVolumes Cluster Admin User

    PersistentVolumes PVClaim Pod Create
  54. Google confidential │ Do not distribute PersistentVolumes Cluster Admin User

    PersistentVolumes PVClaim Pod *
  55. Google confidential │ Do not distribute PersistentVolumes Cluster Admin User

    PersistentVolumes PVClaim Pod Delete *
  56. Google confidential │ Do not distribute PersistentVolumes Cluster Admin User

    PersistentVolumes PVClaim *
  57. Google confidential │ Do not distribute PersistentVolumes Cluster Admin User

    PersistentVolumes PVClaim Pod Create *
  58. Google confidential │ Do not distribute PersistentVolumes Cluster Admin User

    PersistentVolumes PVClaim Pod *
  59. Google confidential │ Do not distribute PersistentVolumes Cluster Admin User

    PersistentVolumes PVClaim Pod Delete *
  60. Google confidential │ Do not distribute PersistentVolumes Cluster Admin User

    PersistentVolumes PVClaim Delete *
  61. Google confidential │ Do not distribute PersistentVolumes Cluster Admin User

    PersistentVolumes Recycler
  62. Google confidential │ Do not distribute New or coming soon

    • Cluster auto-scaling • Jobs (run-to-completion) • Cron • Privileged containers • Graceful termination • Downward API • Simpler deployments • Interactive containers • Network plugins: CNI • Performance++ • Scalability++ (250 in v1.1) • High availability masters • Scheduling • Cluster federation • Easier setup • More volumes • Private registry • L7 load-balancing
  63. Google confidential │ Do not distribute Kubernetes status & plans

    Open sourced in June, 2014 v1.0 in July, 2015 Google Container Engine (GKE) • hosted Kubernetes - don’t think about cluster setup • GA in August, 2015 PaaSes: • RedHat OpenShift, Deis, Stratos Distros: • CoreOS Tectonic, Mirantis Murano (OpenStack), RedHat Atomic, Mesos Driving towards a 1.1 release in O(weeks) • targeting a 3-4 month cadence
  64. Google confidential │ Do not distribute The Goal: Shake things

    up Containers are a new way of working Requires new concepts and new tools Google has a lot of experience... ...but we are listening to the users Workload portability is important!
  65. Google confidential │ Do not distribute Kubernetes is Open -

    open community - open design - open source - open to ideas http://kubernetes.io https://github.com/kubernetes/kubernetes slack: kubernetes twitter: @kubernetesio
  66. Google confidential │ Do not distribute FIN