1 Weave Ignite, Kubernetes Cluster API, and all the things! Lucas Käldström - CNCF Ambassador 26th of September, 2019 - Helsinki Image credit: @ashleymcnamara

2 $ whoami Lucas Käldström, freshman Student at Aalto, 20 yo CNCF Ambassador, Certified Kubernetes Administrator and Kubernetes WG/SIG Lead KubeCon Speaker in Berlin, Austin, Copenhagen, Shanghai, Seattle & San Diego KubeCon Keynote Speaker in Barcelona Kubernetes approver and subproject owner (formerly maintainer), active in the community for 4+ years. Got kubeadm to GA. Weave Ignite author, written this summer

3 luxas labs Kubernetes & Cloud Native: - Consulting - Architecture Review - Workshops Main client (so far):

4 Cloud Native Nordics Slack Community Continue the discussions and meet Cloud Natives from Denmark, Sweden, Norway, Finland, and Iceland :) #cloud-native-nordics www.cloudnativenordics.com

5 Cloud Native Nordics is growing!

© 2019 Cloud Native Computing Foundation 6 67 Meetups 4134 Meetup Group Members 3717 Total RSVPs 2001 Unique Attendees Average Meetup RSVPs 55 Cloud Native Nordics Stats https://github.com/cloud-native-nordics/meetups/blob/master/stats.json Unique Speakers 100 72 Unique Sponsors

7

8 Mark your Calendars! - Helsinki: October 24, 2019 at Nokia - How to be (successful at being) wrong - KubeOne: yet another k8s management tool - How to make Kubernetes Real Time for Telcos - Tampere: October 29, 2019 at Eficode - Kubernetes 101 Hands-on Workshop - 4-hour FREE workshop of Intro to k8s-material - Turku: October 31, 2019 at Walkbase - Introduction to the cloud native world - Managing Kubernetes clusters with Rancher - All Meshed Up -- How we use Linkerd

Agenda

10 Agenda - SIG Cluster Lifecycle vision & building blocks - minikube - kubeadm - Cluster API - Weave Ignite - Demo - Architecture - Motivation

History and Context

12 Nodes Control Plane Kubernetes’ high-level component architecture Node 3 OS Container Runtime Kubelet Networking Node 2 OS Container Runtime Kubelet Networking Node 1 OS Container Runtime Kubelet Networking API Server (REST API) Controller Manager (Controller Loops) Scheduler (Bind Pod to Node) etcd (key-value DB, SSOT) User Legend: CNI CRI OCI Protobuf gRPC JSON

13 Setting up a dynamic TLS-secured cluster Nodes Master API Server Controller Manager Scheduler CN=system:kube-controller-manager CN=system:kube-scheduler Kubelet: node-1 HTTPS (6443) Kubelet client O=system:masters Self-signed HTTPS (10250) CN=system:node:node-1 O=system:nodes Kubelet: node-2 (to be joined) Self-signed HTTPS (10250) Bootstrap Token & trusted CA CN=system:node:node-2 O=system:nodes CSR Approver CSR Signer Legend: Logs / Exec calls Normal HTTPS POST CSR SAR Webhook PATCH CSR node-1 CSR node-2 CSR Bootstrap Token CSR=Certificate Signing Request, SAR=Subject Access Review

14 minikube & kubeadm Wanted to simplify the deployment experience Helped create SIG Cluster Lifecycle => co-lead until now Shepherding kubeadm

15 kubeadm = A tool that sets up a minimum viable, best-practice Kubernetes cluster Master 1 Master N Node 1 Node N kubeadm kubeadm kubeadm kubeadm Cloud Provider Load Balancers Monitoring Logging Cluster API Spec Cluster API Cluster API Implementation Addons Kubernetes API Bootstrapping Machines Infrastructure Layer 2 The scope of kubeadm Layer 3 Layer 1

16 kubeadm vs kops or kubespray Two different projects, two different scopes Master 1 Master N Node 1 Node N kubeadm kubeadm kubeadm kubeadm Cloud Provider Load Balancers Monitoring Logging Cluster API Spec Cluster API Cluster API Implementation Addons Kubernetes API Bootstrapping Machines Infrastructure kops

17 Weave Ignite “An open source Virtual Machine (VM) manager with a container UX and built-in GitOps management” - Firecracker MicroVMs & OCI containers to unify containers and VMs. - Works in a GitOps fashion; manages VMs declaratively

Enter the fire

19 Why? Originates from my Finnish conscription time; where I worked on programming tasks We needed to: a) Use open source (no “normal” VM licenses) b) Run legacy applications with “special requirements” c) Integrate with containers

20 How? BUT: Firecracker is HARD to use (for those who happen to not be KVM devs)

21

22

23 Weave Ignite What does it do? Why do you want it? Declarative Infra Next level VM speed

24 # Let's run the weaveworks/ignite-ubuntu docker image as a VM # Use 2 vCPUs and 1GB of RAM, enable automatic SSH access and name it my-vm ignite run weaveworks/ignite-ubuntu \ --cpus 2 \ --memory 1GB \ --ssh \ --name my-vm # List running VMs ignite ps # List Docker (OCI) and kernel images imported into Ignite ignite images ignite kernels # Get the boot logs of the VM ignite logs my-vm # SSH into the VM ignite ssh my-vm Demo!

25 Ways to use Ignite - Docker-like UX - GitOps - (Future) Virtual Kubelet

26 Design & Architecture Use OCI for: a) content distribution b) monitoring processes c) process isolation Device Mapper & DHCP bridges containers + VMs

27 Building VM images (the new way)

28 GitOps all the things! a) Store desired state in Git b) Run an app reconciling desired => actual state c) App writes status back to Git d) Can observe diffs between observed and desired state

29 => gitops-toolkit Prototype to make any app Git-backed Generic framework built upon k8s.io/apimachinery Common code broken out from Ignite

30 Run Kubernetes on top of Ignite VMs - kind - kubeadm / k3s guide - Footloose (docker-compose) - Cluster API providers - => Firekube?

31 “GitOps” for your cluster(s) apiVersion: cluster.k8s.io/v1alpha1 kind: MachineDeployment metadata: name: my-nodes spec: replicas: 3 selector: matchLabels: foo: bar template: metadata: labels: foo: bar spec: providerConfig: value: apiVersion: "baremetalconfig/v1alpha1" kind: "BareMetalProviderConfig" zone: "us-central1-f" machineType: "n1-standard-1" image: "ubuntu-1604-lts" versions: kubelet: 1.14.2 containerRuntime: name: containerd version: 1.2.0 ● With Kubernetes we manage our applications declaratively a. Why not for the cluster itself? ● With the Cluster API, we can declaratively define the desired cluster state a. Operator implementations reconcile the state b. Use Spec & Status like the rest of k8s c. Common management solutions for e.g. upgrades, autoscaling and repair d. Allows for “GitOps” workflows

32 Read the Docs

33 Raspberry Pis! Challenge: Get Firecracker to run on a Raspberry Pi 4. Both Ignite and Firecracker has arm64 binaries, but need GICv2 support. With the GICv2 PR, running VMs on RPi 4 works!

Kubernetes Cluster API

35 kubeadm = A tool that sets up a minimum viable, best-practice Kubernetes cluster Master 1 Master N Node 1 Node N kubeadm kubeadm kubeadm kubeadm Cloud Provider Load Balancers Monitoring Logging Cluster API Spec Cluster API Cluster API Implementation Addons Kubernetes API Bootstrapping Machines Infrastructure Layer 2 The scope of kubeadm Layer 3 Layer 1

36 Cluster API The next step after kubeadm “To make the management of (X) clusters across (Y) providers simple, secure, and configurable.” “How can I manage any number of clusters in a similar fashion to how I manage deployments in Kubernetes?”

37 Cluster API The next step after kubeadm “How do I manage other lifecycle events across that infrastructure (upgrades, deletions, etc.)?” “How can we control all of this via a consistent API across providers?”

38 “GitOps” for your cluster(s) apiVersion: cluster.k8s.io/v1alpha1 kind: MachineDeployment metadata: name: my-nodes spec: replicas: 3 selector: matchLabels: foo: bar template: metadata: labels: foo: bar spec: providerConfig: value: apiVersion: "baremetalconfig/v1alpha1" kind: "BareMetalProviderConfig" zone: "us-central1-f" machineType: "n1-standard-1" image: "ubuntu-1604-lts" versions: kubelet: 1.14.2 containerRuntime: name: containerd version: 1.2.0 ● With Kubernetes we manage our applications declaratively a. Why not for the cluster itself? ● With the Cluster API, we can declaratively define the desired cluster state a. Operator implementations reconcile the state b. Use Spec & Status like the rest of k8s c. Common management solutions for e.g. upgrades, autoscaling and repair d. Allows for “GitOps” workflows

39 Cluster API cluster-addons kubeadm etcdadm Component Config k8s cluster Provisioners We need your help! There is still a lot of work to do in onder to get the full puzzle in place! GA Beta Alpha Pre-Alpha The SIG Cluster Lifecycle Roadmap

Recap

41 Weave Ignite Firecracker SIG Cluster Lifecycle kubeadm Cluster API Raspberry Pi Questions?

Thank you! @luxas on Github @luxas on Kubernetes’ Slack @kubernetesonarm on Twitter lucas@luxaslabs.com