The Container End-Game: An Introduction To Kubernetes And Minikube

Slide 1

Slide 1 text

A presentation by @stuherbert  for @GanbaroDigital The Container End-Game An Introduction To   Kubernetes And Minikube

Slide 2

Slide 2 text

Industry veteran: architect, engineer, leader, manager, mentor F/OSS contributor since 1994 Talking and writing about PHP since 2004 Chief Software Archaeologist Building Quality @GanbaroDigital About Stuart

Slide 3

Slide 3 text

Follow me I do tweet a lot about non-tech stuff though :) @stuherbert

Slide 4

Slide 4 text

@GanbaroDigital ?? ?? Do you use Docker containers for dev work?

Slide 5

Slide 5 text

@GanbaroDigital https://speakerdeck.com/stuartherbert/ docker-for-php-dev-environments

Slide 6

Slide 6 text

@GanbaroDigital ?? ?? How about in Production?

Slide 7

Slide 7 text

@GanbaroDigital

Slide 8

Slide 8 text

@GanbaroDigital Kubernetes is the end-game for Docker containers

Slide 9

Slide 9 text

@GanbaroDigital

Slide 10

Slide 10 text

@GanbaroDigital Minikube gives you (most of) Kubernetes on your dev box*

Slide 11

Slide 11 text

@GanbaroDigital

Slide 12

Slide 12 text

@GanbaroDigital CNCF is helping Kubernetes become the de-facto platform

Slide 13

Slide 13 text

@GanbaroDigital Kubernetes is a paradigm shift

Slide 14

Slide 14 text

@GanbaroDigital It's impossible to talk about Kubernetes without talking about Google.

Slide 15

Slide 15 text

@GanbaroDigital In This Talk 1. Introducing Kubernetes 2. Introducing Minikube 3. Thinking In Kubernetes 4. De-Google-ing Kubernetes

Slide 16

Slide 16 text

@GanbaroDigital In This Talk 1. Introducing Kubernetes 2. Introducing Minikube 3. Thinking In Kubernetes 4. De-Google-ing Kubernetes

Slide 17

Slide 17 text

@GanbaroDigital In This Talk 1. Introducing Kubernetes 2. Introducing Minikube 3. Thinking In Kubernetes 4. De-Google-ing Kubernetes

Slide 18

Slide 18 text

@GanbaroDigital In This Talk 1. Introducing Kubernetes 2. Introducing Minikube 3. Thinking In Kubernetes 4. De-Google-ing Kubernetes

Slide 19

Slide 19 text

@GanbaroDigital This is my experience to date. I'm here to learn from you too.

Slide 20

Slide 20 text

@GanbaroDigital Introducing Kubernetes

Slide 21

Slide 21 text

@GanbaroDigital “ Kubernetes is a DIY platform for managing and running containers in production.

Slide 22

Slide 22 text

@GanbaroDigital That doesn't tell you what it is, only what you can use it for.

Slide 23

Slide 23 text

@GanbaroDigital I ﬁnd it helpful to understand K8S by looking under the hood.

Slide 24

Slide 24 text

@GanbaroDigital Kubernetes Constituents

Slide 25

Slide 25 text

@GanbaroDigital VM

Slide 26

Slide 26 text

@GanbaroDigital VM VM VM VM VM VM

Slide 27

Slide 27 text

@GanbaroDigital VM VM VM VM VM VM

Slide 28

Slide 28 text

@GanbaroDigital VM VM VM VM VM VM Masters

Slide 29

Slide 29 text

@GanbaroDigital VM VM VM VM VM VM Masters Nodes

Slide 30

Slide 30 text

@GanbaroDigital VM VM VM VM VM VM data Masters Nodes

Slide 31

Slide 31 text

@GanbaroDigital VM VM VM VM VM VM data API Masters Nodes

Slide 32

Slide 32 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler Masters Nodes

Slide 33

Slide 33 text

@GanbaroDigital So far, this is very similar to older infrastructure solutions.

Slide 34

Slide 34 text

@GanbaroDigital What comes next is what makes Kubernetes something new.

Slide 35

Slide 35 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler Masters Nodes

Slide 36

Slide 36 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler internal controllers Masters Nodes

Slide 37

Slide 37 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler internal controllers cloud controllers Masters Nodes

Slide 38

Slide 38 text

@GanbaroDigital Pod Pod Pod Pod CNI

Slide 39

Slide 39 text

@GanbaroDigital Pod Pod Pod Pod CNI EBS EBS EBS EBS EBS EBS EBS EBS

Slide 40

Slide 40 text

@GanbaroDigital Pod Pod Pod Pod CNI LB LB EBS EBS EBS EBS EBS EBS EBS EBS

Slide 41

Slide 41 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler internal controllers cloud controllers Masters Nodes

Slide 42

Slide 42 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler internal controllers cloud controllers Masters Nodes Control plane

Slide 43

Slide 43 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd Masters Nodes Control plane internal controllers cloud controllers

Slide 44

Slide 44 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox Masters Nodes Control plane internal controllers cloud controllers

Slide 45

Slide 45 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API Masters Nodes Control plane internal controllers cloud controllers

Slide 46

Slide 46 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking Masters Nodes Control plane internal controllers cloud controllers

Slide 47

Slide 47 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy Masters Nodes Control plane internal controllers cloud controllers

Slide 48

Slide 48 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy Masters Nodes Control plane Workload internal controllers cloud controllers

Slide 49

Slide 49 text

@GanbaroDigital

Slide 50

Slide 50 text

@GanbaroDigital https://kubernetes.io

Slide 51

Slide 51 text

@GanbaroDigital Introducing Minikube

Slide 52

Slide 52 text

@GanbaroDigital Minikube takes most of Kubernetes and sticks it in a single VM on your local machine

Slide 53

Slide 53 text

@GanbaroDigital Scaling Kubernetes Down

Slide 54

Slide 54 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy Masters Nodes Control plane Workload internal controllers cloud controllers

Slide 55

Slide 55 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy Masters Nodes Control plane Workload internal controllers

Slide 56

Slide 56 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd worker API emulated networking proxy Masters Nodes Control plane Workload internal controllers

Slide 57

Slide 57 text

@GanbaroDigital VM VM VM data API Scheduler containerd worker API emulated networking proxy Nodes Control plane Workload internal controllers

Slide 58

Slide 58 text

@GanbaroDigital data API Scheduler containerd worker API emulated networking proxy Control plane Workload internal controllers

Slide 59

Slide 59 text

@GanbaroDigital VM data API Scheduler containerd worker API emulated networking proxy Control plane Workload Boot2Docker internal controllers

Slide 60

Slide 60 text

@GanbaroDigital

Slide 61

Slide 61 text

@GanbaroDigital https://github.com/kubernetes/minikube

Slide 62

Slide 62 text

@GanbaroDigital How Minikube Is Different

Slide 63

Slide 63 text

@GanbaroDigital ?? ?? The things that are missing ... how important are they?

Slide 64

Slide 64 text

@GanbaroDigital We lose the cloud controllers. Because Minikube isn't running inside a cloud environment.

Slide 65

Slide 65 text

@GanbaroDigital That takes away networking and storage that you'll deﬁnitely use on full-fat Kubernetes.

Slide 66

Slide 66 text

@GanbaroDigital We lose any networking that relies on load balancers. This is mostly network access into containers on Kubernetes.

Slide 67

Slide 67 text

@GanbaroDigital Pod Pod Pod Pod CNI LB LB EBS EBS EBS EBS EBS EBS EBS EBS

Slide 68

Slide 68 text

@GanbaroDigital Pod Pod Pod Pod CNI EBS EBS EBS EBS EBS EBS EBS EBS

Slide 69

Slide 69 text

@GanbaroDigital There is a project called MetalLB that can ﬁll the gap. It isn't integrated into Minikube at this time.

Slide 70

Slide 70 text

@GanbaroDigital Pod Pod Pod Pod CNI EBS EBS EBS EBS EBS EBS EBS EBS

Slide 71

Slide 71 text

@GanbaroDigital Pod Pod Pod Pod CNI Ingress EBS EBS EBS EBS EBS EBS EBS EBS

Slide 72

Slide 72 text

@GanbaroDigital Ingress Controller • Works on Minikube • Works on K8S in the cloud • Supports HTTP/HTTPS only • HTTPS is terminated at Ingress

Slide 73

Slide 73 text

@GanbaroDigital We lose attached storage. This affects every container that we attach extra volumes to.

Slide 74

Slide 74 text

@GanbaroDigital Pod Pod Pod Pod CNI Ingress EBS EBS EBS EBS EBS EBS EBS EBS

Slide 75

Slide 75 text

@GanbaroDigital Pod Pod Pod Pod CNI Ingress

Slide 76

Slide 76 text

@GanbaroDigital Pod Pod Pod Pod CNI Ingress EBS HostPath EBS HostPath EBS HostPath EBS HostPath

Slide 77

Slide 77 text

@GanbaroDigital HostPath Volumes • VM folders mounted into containers • When the VM is deleted, all data is lost • Create volumes under /data to keep data between VM reboots

Slide 78

Slide 78 text

@GanbaroDigital Because Minikube runs in a VM, mounting volumes from the host box isn't always an option.

Slide 79

Slide 79 text

@GanbaroDigital And it isn't always reliable :(

Slide 80

Slide 80 text

@GanbaroDigital We also lose the sandbox. Containers may* behave differently as a result. * only if your K8S uses a sandbox

Slide 81

Slide 81 text

@GanbaroDigital Why is all this important to know upfront?

Slide 82

Slide 82 text

@GanbaroDigital You can't use identical deployments on both Minikube and full-fat Kubernetes.

Slide 83

Slide 83 text

@GanbaroDigital Living With Minikube

Slide 84

Slide 84 text

@GanbaroDigital Minikube does its best, but Kubernetes just doesn't scale down nicely today.

Slide 85

Slide 85 text

@GanbaroDigital Minikube is a bit of a RAM and CPU hog.

Slide 86

Slide 86 text

@GanbaroDigital This is a mix of VM overhead and K8S components that burn CPU even when idle.

Slide 87

Slide 87 text

@GanbaroDigital I've had to upgrade my dev box to use Minikube. So has the customer I am working with.

Slide 88

Slide 88 text

@GanbaroDigital

Slide 89

Slide 89 text

@GanbaroDigital

Slide 90

Slide 90 text

@GanbaroDigital Intel NUCs are my secret weapon! 32GB of RAM, NVMe storage, and you can reinstall everything when you screw up.

Slide 91

Slide 91 text

@GanbaroDigital When Minikube Is Useful

Slide 92

Slide 92 text

@GanbaroDigital ?? ?? What is Minikube good for?

Slide 93

Slide 93 text

@GanbaroDigital Use Minikube for faster iteration of your K8S objects.

Slide 94

Slide 94 text

@GanbaroDigital Use Minikube on something like an Intel NUC as a mini K8S server.

Slide 95

Slide 95 text

@GanbaroDigital Stick with Docker Compose for local dev work.

Slide 96

Slide 96 text

@GanbaroDigital If you must host dev work on Kubernetes, spin up a K8S dev cluster on a cloud provider.

Slide 97

Slide 97 text

@GanbaroDigital Thinking in Kubernetes

Slide 98

Slide 98 text

@GanbaroDigital “ Kubernetes is a paradigm shift.

Slide 99

Slide 99 text

@GanbaroDigital Deploying Onto Kubernetes

Slide 100

Slide 100 text

@GanbaroDigital apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80 Kubernetes objects

Slide 101

Slide 101 text

@GanbaroDigital Kubernetes Objects • YAML descriptions • Deﬁne what we want on K8S • Uploaded via kubectl CLI tool

Slide 102

Slide 102 text

@GanbaroDigital VM VM VM data API Scheduler Masters Control plane apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80 internal controllers cloud controllers

Slide 103

Slide 103 text

@GanbaroDigital VM VM VM data API Scheduler internal controllers cloud controllers Masters Control plane apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80

Slide 104

Slide 104 text

Slide 105

Slide 105 text

Slide 106

Slide 106 text

Slide 107

Slide 107 text

@GanbaroDigital VM VM VM data API Scheduler internal controllers cloud controllers Masters Control plane apiVersion: apps/v1 kind: ReplicaSet metadata: name: nginx-deployment-ykzjud ...

Slide 108

Slide 108 text

@GanbaroDigital VM VM VM data API Scheduler internal controllers cloud controllers Masters Control plane apiVersion: apps/v1 kind: ReplicaSet metadata: name: nginx-deployment-ykzjud ...

Slide 109

Slide 109 text

@GanbaroDigital This cycle continues until 'Pods' have been created / updated / deleted in the data store.

Slide 110

Slide 110 text

@GanbaroDigital A Pod is the unit of deployment.

Slide 111

Slide 111 text

@GanbaroDigital A Pod contains 1 (or more) containers.

Slide 112

Slide 112 text

@GanbaroDigital IP addresses are assigned to Pods, not containers.

Slide 113

Slide 113 text

@GanbaroDigital Containers inside the same Pod can talk to each over via localhost:

Slide 114

Slide 114 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy internal controllers cloud controllers Masters Nodes Control plane Workload

Slide 115

Slide 115 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy internal controllers cloud controllers Masters Nodes Control plane Workload

Slide 116

Slide 116 text

@GanbaroDigital VM VM VM VM VM VM data API Scheduler containerd sandbox worker API emulated networking proxy internal controllers cloud controllers Masters Nodes Control plane Workload

Slide 117

Slide 117 text

@GanbaroDigital We don't tell K8S "start this container". We tell K8S "we want these containers running" and K8S makes it happen.

Slide 118

Slide 118 text

@GanbaroDigital ... and keeps it running no matter what until we change our mind.

Slide 119

Slide 119 text

@GanbaroDigital Deployments • Objects describe desired state • K8S updates active state to match • K8S restores active state when things go wrong

Slide 120

Slide 120 text

@GanbaroDigital Deployments • Objects describe desired state • K8S updates active state to match • K8S restores active state when things go wrong

Slide 121

Slide 121 text

@GanbaroDigital Deployments • Objects describe desired state • K8S updates active state to match • K8S restores active state when things go wrong

Slide 122

Slide 122 text

@GanbaroDigital The "desired state ⬌ active state" approach is Inversion of Instruction.

Slide 123

Slide 123 text

@GanbaroDigital The result? Invisible Infrastructure (as far as devs are concerned)

Slide 124

Slide 124 text

@GanbaroDigital Containers Aren't VMs

Slide 125

Slide 125 text

@GanbaroDigital Running on Kubernetes is nothing like running VMs on AWS et al.

Slide 126

Slide 126 text

@GanbaroDigital Running on Kubernetes is nothing like running containers locally.

Slide 127

Slide 127 text

@GanbaroDigital ... but it looks close enough to make this hard to grok.

Slide 128

Slide 128 text

@GanbaroDigital “ Old habits die hard.

Slide 129

Slide 129 text

@GanbaroDigital Let's look at an important example: Backups

Slide 130

Slide 130 text

@GanbaroDigital java /var/lib/myapp Pod

Slide 131

Slide 131 text

@GanbaroDigital java /var/lib/myapp VM tar SSH

Slide 132

Slide 132 text

@GanbaroDigital java /var/lib/myapp Pod tar SSH

Slide 133

Slide 133 text

@GanbaroDigital java /var/lib/myapp Pod tar SSH ✗

Slide 134

Slide 134 text

@GanbaroDigital java /var/lib/myapp Pod tar SSH ✗ ✗

Slide 135

Slide 135 text

@GanbaroDigital Containers are black boxes.

Slide 136

Slide 136 text

@GanbaroDigital You don't SSH into them to take backups.

Slide 137

Slide 137 text

@GanbaroDigital You don't SSH into them to patch them.

Slide 138

Slide 138 text

@GanbaroDigital You don't SSH into them ever.

Slide 139

Slide 139 text

@GanbaroDigital java /var/lib/myapp Container tar Host

Slide 140

Slide 140 text

@GanbaroDigital java /var/lib/myapp Pod tar Host

Slide 141

Slide 141 text

@GanbaroDigital java /var/lib/myapp tar Host ✗ Pod

Slide 142

Slide 142 text

@GanbaroDigital java /var/lib/myapp tar Host ✗ ✗ Pod

Slide 143

Slide 143 text

@GanbaroDigital Only running containers have access to their ﬁlesystems.

Slide 144

Slide 144 text

@GanbaroDigital java /var/lib/myapp Pod

Slide 145

Slide 145 text

@GanbaroDigital java /var/lib/myapp VM stop

Slide 146

Slide 146 text

@GanbaroDigital java /var/lib/myapp VM ✗ stop

Slide 147

Slide 147 text

@GanbaroDigital You can't stop the container's main process to get consistent ﬁlesystem backups.

Slide 148

Slide 148 text

@GanbaroDigital java /var/lib/myapp Pod ✗ stop

Slide 149

Slide 149 text

@GanbaroDigital java /var/lib/myapp Pod ✗ stop

Slide 150

Slide 150 text

@GanbaroDigital java /var/lib/myapp Pod 2

Slide 151

Slide 151 text

@GanbaroDigital Kubernetes knows what we want the cluster to look like: "desired state"

Slide 152

Slide 152 text

@GanbaroDigital When "active state" deviates from "desired state", Kubernetes attempts to put things back.

Slide 153

Slide 153 text

@GanbaroDigital The (portable) solution? Use K8S rollouts to replace the app container with a specialist backup container.

Slide 154

Slide 154 text

@GanbaroDigital "I have a container for that" - the K8S Way

Slide 155

Slide 155 text

@GanbaroDigital Every single practice you already have from traditional infrastructure has to be reinvented.

Slide 156

Slide 156 text

@GanbaroDigital “ Running Kubernetes, you are Alice through the looking glass.

Slide 157

Slide 157 text

@GanbaroDigital Once you're used to the Kubernetes approach, going back to traditional infrastructure just feels wrong.

Slide 158

Slide 158 text

@GanbaroDigital De-Googling Kubernetes

Slide 159

Slide 159 text

@GanbaroDigital We can all adopt Kubernetes thanks to Google.

Slide 160

Slide 160 text

@GanbaroDigital There is still work to do to adapt Kubernetes to the world beyond Google.

Slide 161

Slide 161 text

@GanbaroDigital The Google Bubble

Slide 162

Slide 162 text

@GanbaroDigital VM Minkube

Slide 163

Slide 163 text

@GanbaroDigital VM VM VM VM VM VM Masters Nodes

Slide 164

Slide 164 text

@GanbaroDigital

Slide 165

Slide 165 text

@GanbaroDigital

Slide 166

Slide 166 text

@GanbaroDigital “ Problems change with volume and scale.

Slide 167

Slide 167 text

@GanbaroDigital Google is solving problems you don't have.

Slide 168

Slide 168 text

@GanbaroDigital Google's solutions sometimes don't scale down.

Slide 169

Slide 169 text

@GanbaroDigital A surprising amount of Googlers seem to lack normal company experience.

Slide 170

Slide 170 text

@GanbaroDigital Or maybe Google just has blind spots like anyone else?

Slide 171

Slide 171 text

@GanbaroDigital Most third-party containers don't work out-of-the-box on Kubernetes.

Slide 172

Slide 172 text

@GanbaroDigital

Slide 173

Slide 173 text

@GanbaroDigital Through the CNCF, Kubernetes is evolving to suit a general audience.

Slide 174

Slide 174 text

@GanbaroDigital You absolutely CAN adopt Kubernetes today.

Slide 175

Slide 175 text

@GanbaroDigital You have to adapt if you want to adopt Kubernetes.

Slide 176

Slide 176 text

@GanbaroDigital

Slide 177

Slide 177 text

Thank You How Can We Help You? A presentation by @stuherbert  for @GanbaroDigital