Tweet
Tweet

Slide 1

Slide 1 text

Google confidential │ Do not distribute Kubernetes: Intro & Update Kubernetes Meetup 9/30/2015 Tim Hockin Senior Staff Software Engineer @thockin

Slide 2

Slide 2 text

Google confidential │ Do not distribute Google has been developing and using containers to manage our applications for over 10 years. Images by Connie Zhou

Slide 3

Slide 3 text

Google confidential │ Do not distribute Everything at Google runs in containers: • Gmail, Web Search, Maps, ... • MapReduce, batch, ... • GFS, Colossus, ... • Even Google’s Cloud Platform: VMs run in containers!

Slide 4

Slide 4 text

Google confidential │ Do not distribute Everything at Google runs in containers: • Gmail, Web Search, Maps, ... • MapReduce, batch, ... • GFS, Colossus, ... • Even Google’s Cloud Platform: VMs run in containers! We launch over 2 billion containers per week

Slide 5

Slide 5 text

Google confidential │ Do not distribute But it’s all so different! • Deployment • Management, monitoring • Isolation (very complicated!) • Updates • Discovery • Scaling, replication, sets A fundamentally different way of managing applications requires different tooling and abstractions Images by Connie Zhou

Slide 6

Slide 6 text

Google confidential │ Do not distribute Kubernetes Greek for “Helmsman”; also the root of the word “Governor” and “cybernetic” • Container orchestrator • Runs and manages containers • Supports multiple cloud and bare-metal environments • Inspired and informed by Google’s experiences and internal systems • 100% Open source, written in Go Manage applications, not machines

Slide 7

Slide 7 text

Google confidential │ Do not distribute Pets vs. Cattle

Slide 8

Slide 8 text

Google confidential │ Do not distribute kubelet UI kubelet CLI API users master nodes The 10000 foot view apiserver kubelet scheduler controllers

Slide 9

Slide 9 text

Google confidential │ Do not distribute UI All you really care about API Container Cluster

Slide 10

Slide 10 text

Google confidential │ Do not distribute Container clusters: A story in two parts

Slide 11

Slide 11 text

Google confidential │ Do not distribute Container clusters: A story in two parts 1. Setting up the cluster • Choose a cloud: GCE, AWS, Azure, Rackspace, on-premises, ... • Choose a node OS: CoreOS, Atomic, RHEL, Debian, CentOS, Ubuntu, ... • Provision machines: Boot VMs, install and run kube components, ... • Configure networking: IP ranges for Pods, Services, SDN, ... • Start cluster services: DNS, logging, monitoring, ... • Manage nodes: kernel upgrades, OS updates, hardware failures... Not the easy or fun part, but unavoidable This is where things like Google Container Engine (GKE) really help

Slide 12

Slide 12 text

Google confidential │ Do not distribute 2. Using the cluster • Run Pods & Containers • Replication controllers • Services • Volumes This is the fun part! A distinct set of problems from cluster setup and management Don’t make developers deal with cluster administration! Accelerate development by focusing on the applications, not the cluster Container clusters: A story in two parts

Slide 13

Slide 13 text

Google confidential │ Do not distribute Networking

Slide 14

Slide 14 text

Google confidential │ Do not distribute 10.1.1.0/24 172.16.1.1 172.16.1.2 Docker networking 10.1.2.0/24 172.16.1.1 10.1.3.0/24 172.16.1.1

Slide 15

Slide 15 text

Google confidential │ Do not distribute 10.1.1.0/24 172.16.1.1 172.16.1.2 Docker networking 10.1.2.0/24 172.16.1.1 10.1.3.0/24 172.16.1.1 NAT NAT NAT NAT NAT

Slide 16

Slide 16 text

Google confidential │ Do not distribute Kubernetes networking IPs are routable • vs docker default private IP Pods can reach each other without NAT • even across nodes No brokering of port numbers • too complex, why bother? This is a fundamental requirement • can be L3 routed • can be underlayed (cloud) • can be overlayed (SDN)

Slide 17

Slide 17 text

Google confidential │ Do not distribute 10.1.1.0/24 172.16.1.1 172.16.1.2 Kubernetes networking 10.1.2.0/24 172.16.1.1 10.1.3.0/24 172.16.1.1

Slide 18

Slide 18 text

Google confidential │ Do not distribute Pods

Slide 19

Slide 19 text

Google confidential │ Do not distribute Pods Small group of containers & volumes Tightly coupled The atom of scheduling & placement in Kubernetes Shared namespace • share IP address & localhost • share IPC Mortal • can die, cannot be reborn Example: data puller & web server Consumers Content Manager File Puller Web Server Volume Pod

Slide 20

Slide 20 text

Google confidential │ Do not distribute Volumes Very similar to Docker’s concept Pod scoped storage Share the pod’s lifetime & fate Support many types of volume plugins • Empty dir (and tmpfs) • Host path • Git repository • GCE Persistent Disk • AWS Elastic Block Store • iSCSI • NFS • GlusterFS • Ceph File and RBD • Cinder • Secret • ...

Slide 21

Slide 21 text

Google confidential │ Do not distribute Labels & Selectors

Slide 22

Slide 22 text

Google confidential │ Do not distribute Arbitrary metadata Attached to any API object Generally represent identity Queryable by selectors • think SQL ‘select ... where ...’ The only grouping mechanism • pods under a ReplicationController • pods in a Service • capabilities of a node (constraints) Labels

Slide 23

Slide 23 text

Google confidential │ Do not distribute App: MyApp Phase: prod Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE Selectors

Slide 24

Slide 24 text

Google confidential │ Do not distribute App: MyApp Phase: prod Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp Selectors

Slide 25

Slide 25 text

Google confidential │ Do not distribute App: MyApp Phase: prod Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp, Role = FE Selectors

Slide 26

Slide 26 text

Google confidential │ Do not distribute App: MyApp Phase: prod Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp, Role = BE Selectors

Slide 27

Slide 27 text

Google confidential │ Do not distribute Selectors App: MyApp Phase: prod Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp, Phase = prod

Slide 28

Slide 28 text

Google confidential │ Do not distribute App: MyApp Phase: prod Role: FE App: MyApp Phase: test Role: FE App: MyApp Phase: prod Role: BE App: MyApp Phase: test Role: BE App = MyApp, Phase = test Selectors

Slide 29

Slide 29 text

Google confidential │ Do not distribute ReplicationControllers

Slide 30

Slide 30 text

Google confidential │ Do not distribute ReplicationControllers An example of control loops Runs out-of-process wrt API server Have 1 job: ensure N copies of a pod • if too few, start new ones • if too many, kill some • grouped by a selector Cleanly layered on top of the core • all access is by public APIs Replicated pods are fungible • No implied order or identity ReplicationController - name = “my-rc” - selector = {“App”: “MyApp”} - podTemplate = { ... } - replicas = 4 API Server How many? 3 Start 1 more OK How many? 4

Slide 31

Slide 31 text

Google confidential │ Do not distribute Services

Slide 32

Slide 32 text

Google confidential │ Do not distribute Services A group of pods that work together • grouped by a selector Defines access policy • “load balanced” or “headless” Gets a stable virtual IP and port • sometimes called the service portal • also a DNS name VIP is managed by kube-proxy • watches all services • updates iptables when backends change Hides complexity - ideal for non-native apps Virtual IP Client

Slide 33

Slide 33 text

Google confidential │ Do not distribute Secrets

Slide 34

Slide 34 text

Google confidential │ Do not distribute Secrets Problem: how to grant a pod access to a secured something? • don’t put secrets in the container image! 12-factor says: config comes from the environment • Kubernetes is the environment Manage secrets via the Kubernetes API Inject them as “virtual volumes” into Pods • late-binding • tmpfs - never touches disk node Pod Secret API

Slide 35

Slide 35 text

Google confidential │ Do not distribute Rolling updates

Slide 36

Slide 36 text

Google confidential │ Do not distribute Rolling Updates ReplicationController - replicas: 3 - selector: - app: MyApp - version: v1 Service - app: MyApp

Slide 37

Slide 37 text

Google confidential │ Do not distribute Rolling Updates ReplicationController - replicas: 3 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 0 - selector: - app: MyApp - version: v2 Service - app: MyApp

Slide 38

Slide 38 text

Google confidential │ Do not distribute Rolling Updates ReplicationController - replicas: 3 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 1 - selector: - app: MyApp - version: v2 Service - app: MyApp

Slide 39

Slide 39 text

Google confidential │ Do not distribute Rolling Updates ReplicationController - replicas: 2 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 1 - selector: - app: MyApp - version: v2 Service - app: MyApp

Slide 40

Slide 40 text

Google confidential │ Do not distribute Rolling Updates ReplicationController - replicas: 2 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 2 - selector: - app: MyApp - version: v2 Service - app: MyApp

Slide 41

Slide 41 text

Google confidential │ Do not distribute Rolling Updates ReplicationController - replicas: 1 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 2 - selector: - app: MyApp - version: v2 Service - app: MyApp

Slide 42

Slide 42 text

Google confidential │ Do not distribute Rolling Updates ReplicationController - replicas: 1 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 3 - selector: - app: MyApp - version: v2 Service - app: MyApp

Slide 43

Slide 43 text

Google confidential │ Do not distribute Rolling Updates ReplicationController - replicas: 0 - selector: - app: MyApp - version: v1 ReplicationController - replicas: 3 - selector: - app: MyApp - version: v2 Service - app: MyApp

Slide 44

Slide 44 text

Google confidential │ Do not distribute DaemonSets

Slide 45

Slide 45 text

Google confidential │ Do not distribute DaemonSets Problem: how to run a Pod on every node • or a subset of nodes Similar to ReplicationController • principle: do one thing, don’t overload “Which nodes?” is a selector Use familiar tools and patterns Status: EXPERIMENTAL in Kubernetes v1.1

Slide 46

Slide 46 text

Google confidential │ Do not distribute PersistentVolumes

Slide 47

Slide 47 text

Google confidential │ Do not distribute PersistentVolumes A higher-level abstraction • insulation from any one cloud environment Admin provisions them, users claim them Independent lifetime and fate Can be handed-off between pods and lives until user is done with it Dynamically “scheduled” and managed, like nodes and pods

Slide 48

Slide 48 text

Google confidential │ Do not distribute PersistentVolumes Cluster Admin

Slide 49

Slide 49 text

Google confidential │ Do not distribute PersistentVolumes Provision Cluster Admin PersistentVolumes

Slide 50

Slide 50 text

Google confidential │ Do not distribute PersistentVolumes Cluster Admin User PersistentVolumes

Slide 51

Slide 51 text

Google confidential │ Do not distribute PersistentVolumes Cluster Admin User PersistentVolumes PVClaim Create

Slide 52

Slide 52 text

Google confidential │ Do not distribute PersistentVolumes Cluster Admin User PersistentVolumes PVClaim Binder

Slide 53

Slide 53 text

Google confidential │ Do not distribute PersistentVolumes Cluster Admin User PersistentVolumes PVClaim Pod Create

Slide 54

Slide 54 text

Google confidential │ Do not distribute PersistentVolumes Cluster Admin User PersistentVolumes PVClaim Pod *

Slide 55

Slide 55 text

Google confidential │ Do not distribute PersistentVolumes Cluster Admin User PersistentVolumes PVClaim Pod Delete *

Slide 56

Slide 56 text

Google confidential │ Do not distribute PersistentVolumes Cluster Admin User PersistentVolumes PVClaim *

Slide 57

Slide 57 text

Google confidential │ Do not distribute PersistentVolumes Cluster Admin User PersistentVolumes PVClaim Pod Create *

Slide 58

Slide 58 text

Google confidential │ Do not distribute PersistentVolumes Cluster Admin User PersistentVolumes PVClaim Pod *

Slide 59

Slide 59 text

Google confidential │ Do not distribute PersistentVolumes Cluster Admin User PersistentVolumes PVClaim Pod Delete *

Slide 60

Slide 60 text

Google confidential │ Do not distribute PersistentVolumes Cluster Admin User PersistentVolumes PVClaim Delete *

Slide 61

Slide 61 text

Google confidential │ Do not distribute PersistentVolumes Cluster Admin User PersistentVolumes Recycler

Slide 62

Slide 62 text

Google confidential │ Do not distribute New or coming soon • Cluster auto-scaling • Jobs (run-to-completion) • Cron • Privileged containers • Graceful termination • Downward API • Simpler deployments • Interactive containers • Network plugins: CNI • Performance++ • Scalability++ (250 in v1.1) • High availability masters • Scheduling • Cluster federation • Easier setup • More volumes • Private registry • L7 load-balancing

Slide 63

Slide 63 text

Google confidential │ Do not distribute Kubernetes status & plans Open sourced in June, 2014 v1.0 in July, 2015 Google Container Engine (GKE) • hosted Kubernetes - don’t think about cluster setup • GA in August, 2015 PaaSes: • RedHat OpenShift, Deis, Stratos Distros: • CoreOS Tectonic, Mirantis Murano (OpenStack), RedHat Atomic, Mesos Driving towards a 1.1 release in O(weeks) • targeting a 3-4 month cadence

Slide 64

Slide 64 text

Google confidential │ Do not distribute The Goal: Shake things up Containers are a new way of working Requires new concepts and new tools Google has a lot of experience... ...but we are listening to the users Workload portability is important!

Slide 65

Slide 65 text

Google confidential │ Do not distribute Kubernetes is Open - open community - open design - open source - open to ideas http://kubernetes.io https://github.com/kubernetes/kubernetes slack: kubernetes twitter: @kubernetesio

Slide 66

Slide 66 text

Google confidential │ Do not distribute FIN