Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Container End-Game: An Introduction To Kubernetes And Minikube

The Container End-Game: An Introduction To Kubernetes And Minikube

More and more people either already have adopted Docker containers for their dev work, or are actively looking into it. Not many of them are using containers in production yet, and are unsure how to go about doing so. After years of uncertainty, the industry is moving towards a common platform for doing just that.

In this talk, Stuart will introduce you to Kubernetes - the de-facto standard approach to running containers in production. He’ll show you how to map the things you know from traditional infrastructure and Docker onto Kubernetes. He’ll also introduce you to minikube - a way to run (most of) Kubernetes on a local machine. He’ll finish by covering some of the areas where Kubernetes needs de-Googling in the future.

Presented at PHP Hampshire on 14th November, 2018.

2c1dc90ff7bf69097a151677624777d2?s=128

Stuart Herbert

November 14, 2018
Tweet

Transcript

  1. A presentation by @stuherbert
 for @GanbaroDigital The Container End-Game An

    Introduction To 
 Kubernetes And Minikube
  2. Industry veteran: architect, engineer, leader, manager, mentor F/OSS contributor since

    1994 Talking and writing about PHP since 2004 Chief Software Archaeologist Building Quality @GanbaroDigital About Stuart
  3. Follow me I do tweet a lot about non-tech stuff

    though :) @stuherbert
  4. @GanbaroDigital ?? ?? Do you use Docker containers for dev

    work?
  5. @GanbaroDigital https://speakerdeck.com/stuartherbert/ docker-for-php-dev-environments

  6. @GanbaroDigital ?? ?? How about in Production?

  7. @GanbaroDigital

  8. @GanbaroDigital Kubernetes is the end-game for Docker containers

  9. @GanbaroDigital

  10. @GanbaroDigital Minikube gives you (most of) Kubernetes on your dev

    box*
  11. @GanbaroDigital

  12. @GanbaroDigital CNCF is helping Kubernetes become the de-facto platform

  13. @GanbaroDigital Kubernetes is a paradigm shift

  14. @GanbaroDigital It's impossible to talk about Kubernetes without talking about

    Google.
  15. @GanbaroDigital In This Talk 1. Introducing Kubernetes 2. Introducing Minikube

    3. Thinking In Kubernetes 4. De-Google-ing Kubernetes
  16. @GanbaroDigital In This Talk 1. Introducing Kubernetes 2. Introducing Minikube

    3. Thinking In Kubernetes 4. De-Google-ing Kubernetes
  17. @GanbaroDigital In This Talk 1. Introducing Kubernetes 2. Introducing Minikube

    3. Thinking In Kubernetes 4. De-Google-ing Kubernetes
  18. @GanbaroDigital In This Talk 1. Introducing Kubernetes 2. Introducing Minikube

    3. Thinking In Kubernetes 4. De-Google-ing Kubernetes
  19. @GanbaroDigital This is my experience to date. I'm here to

    learn from you too.
  20. @GanbaroDigital Introducing Kubernetes

  21. @GanbaroDigital “ Kubernetes is a DIY platform for managing and

    running containers in production.
  22. @GanbaroDigital That doesn't tell you what it is, only what

    you can use it for.
  23. @GanbaroDigital I find it helpful to understand K8S by looking

    under the hood.
  24. @GanbaroDigital Kubernetes Constituents

  25. @GanbaroDigital VM

  26. @GanbaroDigital VM VM VM VM VM VM

  27. @GanbaroDigital VM VM VM VM VM VM

  28. @GanbaroDigital VM VM VM VM VM VM Masters

  29. @GanbaroDigital VM VM VM VM VM VM Masters Nodes

  30. @GanbaroDigital VM VM VM VM VM VM data Masters Nodes

  31. @GanbaroDigital VM VM VM VM VM VM data API Masters

    Nodes
  32. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    Masters Nodes
  33. @GanbaroDigital So far, this is very similar to older infrastructure

    solutions.
  34. @GanbaroDigital What comes next is what makes Kubernetes something new.

  35. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    Masters Nodes
  36. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    internal controllers Masters Nodes
  37. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    internal controllers cloud controllers Masters Nodes
  38. @GanbaroDigital Pod Pod Pod Pod CNI

  39. @GanbaroDigital Pod Pod Pod Pod CNI EBS EBS EBS EBS

    EBS EBS EBS EBS
  40. @GanbaroDigital Pod Pod Pod Pod CNI LB LB EBS EBS

    EBS EBS EBS EBS EBS EBS
  41. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    internal controllers cloud controllers Masters Nodes
  42. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    internal controllers cloud controllers Masters Nodes Control plane
  43. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    containerd Masters Nodes Control plane internal controllers cloud controllers
  44. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    containerd sandbox Masters Nodes Control plane internal controllers cloud controllers
  45. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    containerd sandbox worker API Masters Nodes Control plane internal controllers cloud controllers
  46. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    containerd sandbox worker API emulated networking Masters Nodes Control plane internal controllers cloud controllers
  47. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    containerd sandbox worker API emulated networking proxy Masters Nodes Control plane internal controllers cloud controllers
  48. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    containerd sandbox worker API emulated networking proxy Masters Nodes Control plane Workload internal controllers cloud controllers
  49. @GanbaroDigital

  50. @GanbaroDigital https://kubernetes.io

  51. @GanbaroDigital Introducing Minikube

  52. @GanbaroDigital Minikube takes most of Kubernetes and sticks it in

    a single VM on your local machine
  53. @GanbaroDigital Scaling Kubernetes Down

  54. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    containerd sandbox worker API emulated networking proxy Masters Nodes Control plane Workload internal controllers cloud controllers
  55. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    containerd sandbox worker API emulated networking proxy Masters Nodes Control plane Workload internal controllers
  56. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    containerd worker API emulated networking proxy Masters Nodes Control plane Workload internal controllers
  57. @GanbaroDigital VM VM VM data API Scheduler containerd worker API

    emulated networking proxy Nodes Control plane Workload internal controllers
  58. @GanbaroDigital data API Scheduler containerd worker API emulated networking proxy

    Control plane Workload internal controllers
  59. @GanbaroDigital VM data API Scheduler containerd worker API emulated networking

    proxy Control plane Workload Boot2Docker internal controllers
  60. @GanbaroDigital

  61. @GanbaroDigital https://github.com/kubernetes/minikube

  62. @GanbaroDigital How Minikube Is Different

  63. @GanbaroDigital ?? ?? The things that are missing ... how

    important are they?
  64. @GanbaroDigital We lose the cloud controllers. Because Minikube isn't running

    inside a cloud environment.
  65. @GanbaroDigital That takes away networking and storage that you'll definitely

    use on full-fat Kubernetes.
  66. @GanbaroDigital We lose any networking that relies on load balancers.

    This is mostly network access into containers on Kubernetes.
  67. @GanbaroDigital Pod Pod Pod Pod CNI LB LB EBS EBS

    EBS EBS EBS EBS EBS EBS
  68. @GanbaroDigital Pod Pod Pod Pod CNI EBS EBS EBS EBS

    EBS EBS EBS EBS
  69. @GanbaroDigital There is a project called MetalLB that can fill

    the gap. It isn't integrated into Minikube at this time.
  70. @GanbaroDigital Pod Pod Pod Pod CNI EBS EBS EBS EBS

    EBS EBS EBS EBS
  71. @GanbaroDigital Pod Pod Pod Pod CNI Ingress EBS EBS EBS

    EBS EBS EBS EBS EBS
  72. @GanbaroDigital Ingress Controller • Works on Minikube • Works on

    K8S in the cloud • Supports HTTP/HTTPS only • HTTPS is terminated at Ingress
  73. @GanbaroDigital We lose attached storage. This affects every container that

    we attach extra volumes to.
  74. @GanbaroDigital Pod Pod Pod Pod CNI Ingress EBS EBS EBS

    EBS EBS EBS EBS EBS
  75. @GanbaroDigital Pod Pod Pod Pod CNI Ingress

  76. @GanbaroDigital Pod Pod Pod Pod CNI Ingress EBS HostPath EBS

    HostPath EBS HostPath EBS HostPath
  77. @GanbaroDigital HostPath Volumes • VM folders mounted into containers •

    When the VM is deleted, all data is lost • Create volumes under /data to keep data between VM reboots
  78. @GanbaroDigital Because Minikube runs in a VM, mounting volumes from

    the host box isn't always an option.
  79. @GanbaroDigital And it isn't always reliable :(

  80. @GanbaroDigital We also lose the sandbox. Containers may* behave differently

    as a result. * only if your K8S uses a sandbox
  81. @GanbaroDigital Why is all this important to know upfront?

  82. @GanbaroDigital You can't use identical deployments on both Minikube and

    full-fat Kubernetes.
  83. @GanbaroDigital Living With Minikube

  84. @GanbaroDigital Minikube does its best, but Kubernetes just doesn't scale

    down nicely today.
  85. @GanbaroDigital Minikube is a bit of a RAM and CPU

    hog.
  86. @GanbaroDigital This is a mix of VM overhead and K8S

    components that burn CPU even when idle.
  87. @GanbaroDigital I've had to upgrade my dev box to use

    Minikube. So has the customer I am working with.
  88. @GanbaroDigital

  89. @GanbaroDigital

  90. @GanbaroDigital Intel NUCs are my secret weapon! 32GB of RAM,

    NVMe storage, and you can reinstall everything when you screw up.
  91. @GanbaroDigital When Minikube Is Useful

  92. @GanbaroDigital ?? ?? What is Minikube good for?

  93. @GanbaroDigital Use Minikube for faster iteration of your K8S objects.

  94. @GanbaroDigital Use Minikube on something like an Intel NUC as

    a mini K8S server.
  95. @GanbaroDigital Stick with Docker Compose for local dev work.

  96. @GanbaroDigital If you must host dev work on Kubernetes, spin

    up a K8S dev cluster on a cloud provider.
  97. @GanbaroDigital Thinking in Kubernetes

  98. @GanbaroDigital “ Kubernetes is a paradigm shift.

  99. @GanbaroDigital Deploying Onto Kubernetes

  100. @GanbaroDigital apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app:

    nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80 Kubernetes objects
  101. @GanbaroDigital Kubernetes Objects • YAML descriptions • Define what we

    want on K8S • Uploaded via kubectl CLI tool
  102. @GanbaroDigital VM VM VM data API Scheduler Masters Control plane

    apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80 internal controllers cloud controllers
  103. @GanbaroDigital VM VM VM data API Scheduler internal controllers cloud

    controllers Masters Control plane apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80
  104. @GanbaroDigital VM VM VM data API Scheduler internal controllers cloud

    controllers Masters Control plane apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80
  105. @GanbaroDigital VM VM VM data API Scheduler internal controllers cloud

    controllers Masters Control plane apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80
  106. @GanbaroDigital VM VM VM data API Scheduler internal controllers cloud

    controllers Masters Control plane apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.4 ports: - containerPort: 80
  107. @GanbaroDigital VM VM VM data API Scheduler internal controllers cloud

    controllers Masters Control plane apiVersion: apps/v1 kind: ReplicaSet metadata: name: nginx-deployment-ykzjud ...
  108. @GanbaroDigital VM VM VM data API Scheduler internal controllers cloud

    controllers Masters Control plane apiVersion: apps/v1 kind: ReplicaSet metadata: name: nginx-deployment-ykzjud ...
  109. @GanbaroDigital This cycle continues until 'Pods' have been created /

    updated / deleted in the data store.
  110. @GanbaroDigital A Pod is the unit of deployment.

  111. @GanbaroDigital A Pod contains 1 (or more) containers.

  112. @GanbaroDigital IP addresses are assigned to Pods, not containers.

  113. @GanbaroDigital Containers inside the same Pod can talk to each

    over via localhost:<port-number>
  114. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    containerd sandbox worker API emulated networking proxy internal controllers cloud controllers Masters Nodes Control plane Workload
  115. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    containerd sandbox worker API emulated networking proxy internal controllers cloud controllers Masters Nodes Control plane Workload
  116. @GanbaroDigital VM VM VM VM VM VM data API Scheduler

    containerd sandbox worker API emulated networking proxy internal controllers cloud controllers Masters Nodes Control plane Workload
  117. @GanbaroDigital We don't tell K8S "start this container". We tell

    K8S "we want these containers running" and K8S makes it happen.
  118. @GanbaroDigital ... and keeps it running no matter what until

    we change our mind.
  119. @GanbaroDigital Deployments • Objects describe desired state • K8S updates

    active state to match • K8S restores active state when things go wrong
  120. @GanbaroDigital Deployments • Objects describe desired state • K8S updates

    active state to match • K8S restores active state when things go wrong
  121. @GanbaroDigital Deployments • Objects describe desired state • K8S updates

    active state to match • K8S restores active state when things go wrong
  122. @GanbaroDigital The "desired state ⬌ active state" approach is Inversion

    of Instruction.
  123. @GanbaroDigital The result? Invisible Infrastructure (as far as devs are

    concerned)
  124. @GanbaroDigital Containers Aren't VMs

  125. @GanbaroDigital Running on Kubernetes is nothing like running VMs on

    AWS et al.
  126. @GanbaroDigital Running on Kubernetes is nothing like running containers locally.

  127. @GanbaroDigital ... but it looks close enough to make this

    hard to grok.
  128. @GanbaroDigital “ Old habits die hard.

  129. @GanbaroDigital Let's look at an important example: Backups

  130. @GanbaroDigital java /var/lib/myapp Pod

  131. @GanbaroDigital java /var/lib/myapp VM tar SSH

  132. @GanbaroDigital java /var/lib/myapp Pod tar SSH

  133. @GanbaroDigital java /var/lib/myapp Pod tar SSH ✗

  134. @GanbaroDigital java /var/lib/myapp Pod tar SSH ✗ ✗

  135. @GanbaroDigital Containers are black boxes.

  136. @GanbaroDigital You don't SSH into them to take backups.

  137. @GanbaroDigital You don't SSH into them to patch them.

  138. @GanbaroDigital You don't SSH into them ever.

  139. @GanbaroDigital java /var/lib/myapp Container tar Host

  140. @GanbaroDigital java /var/lib/myapp Pod tar Host

  141. @GanbaroDigital java /var/lib/myapp tar Host ✗ Pod

  142. @GanbaroDigital java /var/lib/myapp tar Host ✗ ✗ Pod

  143. @GanbaroDigital Only running containers have access to their filesystems.

  144. @GanbaroDigital java /var/lib/myapp Pod

  145. @GanbaroDigital java /var/lib/myapp VM stop

  146. @GanbaroDigital java /var/lib/myapp VM ✗ stop

  147. @GanbaroDigital You can't stop the container's main process to get

    consistent filesystem backups.
  148. @GanbaroDigital java /var/lib/myapp Pod ✗ stop

  149. @GanbaroDigital java /var/lib/myapp Pod ✗ stop

  150. @GanbaroDigital java /var/lib/myapp Pod 2

  151. @GanbaroDigital Kubernetes knows what we want the cluster to look

    like: "desired state"
  152. @GanbaroDigital When "active state" deviates from "desired state", Kubernetes attempts

    to put things back.
  153. @GanbaroDigital The (portable) solution? Use K8S rollouts to replace the

    app container with a specialist backup container.
  154. @GanbaroDigital "I have a container for that" - the K8S

    Way
  155. @GanbaroDigital Every single practice you already have from traditional infrastructure

    has to be reinvented.
  156. @GanbaroDigital “ Running Kubernetes, you are Alice through the looking

    glass.
  157. @GanbaroDigital Once you're used to the Kubernetes approach, going back

    to traditional infrastructure just feels wrong.
  158. @GanbaroDigital De-Googling Kubernetes

  159. @GanbaroDigital We can all adopt Kubernetes thanks to Google.

  160. @GanbaroDigital There is still work to do to adapt Kubernetes

    to the world beyond Google.
  161. @GanbaroDigital The Google Bubble

  162. @GanbaroDigital VM Minkube

  163. @GanbaroDigital VM VM VM VM VM VM Masters Nodes

  164. @GanbaroDigital

  165. @GanbaroDigital

  166. @GanbaroDigital “ Problems change with volume and scale.

  167. @GanbaroDigital Google is solving problems you don't have.

  168. @GanbaroDigital Google's solutions sometimes don't scale down.

  169. @GanbaroDigital A surprising amount of Googlers seem to lack normal

    company experience.
  170. @GanbaroDigital Or maybe Google just has blind spots like anyone

    else?
  171. @GanbaroDigital Most third-party containers don't work out-of-the-box on Kubernetes.

  172. @GanbaroDigital

  173. @GanbaroDigital Through the CNCF, Kubernetes is evolving to suit a

    general audience.
  174. @GanbaroDigital You absolutely CAN adopt Kubernetes today.

  175. @GanbaroDigital You have to adapt if you want to adopt

    Kubernetes.
  176. @GanbaroDigital

  177. Thank You How Can We Help You? A presentation by

    @stuherbert
 for @GanbaroDigital