Making virtual machines cloud native with KubeVirt - Marc Sluiter - Red Hat

Kubernetes is nowadays the de-facto standard for orchestrating containers inside and even across clusters. Migrating legacy workloads from virtual machines to containers might be impossible though, be it because lack of manpower, high costs, or some other reason. Ultimately this means you need to manage your virtual machines and your cloud native workloads with two different tools.

Do you? No, luckily you don't! KubeVirt enables you to run your virtual machines as first-class citizen in your Kubernetes cluster, alongside with your containers, sharing the same storage, network, and compute resources. KubeVirt uses Kubernetes' extensibility features to do so.

This session gives you an introduction to these features, shows you how KubeVirt uses them, and ends with a demo of running a virtual machine in Kubernetes.

Transcript

1. Making virtual machines cloud native with KubeVirt Marc Sluiter
2. Marc Sluiter KubeVirt @ Godays.io 2020 About me Marc Sluiter // @slintes Java development since 2001 Docker & Kubernetes user since 2015 First small steps with Go in 2016 Since 2018: • Full time Go development at Red Hat • KubeVirt maintainer (until recently)
3. Marc Sluiter KubeVirt @ Godays.io 2020 Overview • Virtual machines and containers • Kubernetes • KubeVirt • Demo • Q&A
4. Marc Sluiter KubeVirt @ Godays.io 2020 Virtual Machines and Containers Run multiple applications on one host But with Isolation! • VMs ◦ the host’s hypervisor runs complete operating systems with their own kernels etc. ◦ full isolation • Containers ◦ the container runtime runs Linux processes ◦ isolation is done using kernel features (cgroups, namespaces, …) ◦ Docker made using this easier and more popular
5. Marc Sluiter KubeVirt @ Godays.io 2020
6. Marc Sluiter KubeVirt @ Godays.io 2020 Kubernetes • Orchestrates many containers on many nodes (hosts) • Initiated by Google, donated to CNCF • Huge community (contributers and users) • Declarative approach • Continuously compares declared state with observed state and takes action
7. Marc Sluiter KubeVirt @ Godays.io 2020 Kubernetes resources Smallest workload is a Pod • Contains one or more containers • Containers share network and storage • Is ephemeral A pod is typically part of a Deployment • Adds horizontal scaling by running multiple replicas of a Pod • Handles updates of pods
8. Marc Sluiter KubeVirt @ Godays.io 2020 Kubernetes resources (cont.) apiVersion: apps/v1 kind: Deployment metadata: name: deploy-example spec: selector: matchLabels: app: nginx env: prod replicas: 3 strategy: type: RollingUpdate template: <pod template> apiVersion: v1 kind: Pod metadata: name: pod-example labels: app: nginx env: prod spec: containers: - name: nginx image: nginx:stable-alpine ports: - containerPort: 80
9. Marc Sluiter KubeVirt @ Godays.io 2020 Kubernetes resources (cont.) • Service ◦ persistent IP addresses for Pods ◦ load balancing • ConfigMap / Secret ◦ configuration data, will be consumed by Pods • PersistentVolume ◦ abstraction for accessing storage ▪ backed by hostPath, NFS, Ceph, Gluster, and many more ▪ will also be consumed by Pods • Several resources about RBAC (ServiceAccount, Role, RoleBinding,...) • and many more...
10. Marc Sluiter KubeVirt @ Godays.io 2020 Kubernetes components Kube-Apiserver • provides a REST API for CRUD operations on the resources just mentioned • stores current state in a etcd key-value database • entrypoint to Kubernetes for all other components and users Kube-Controller-Manager • decides WHICH pods needs to be started/stopped • e.g. when a node fails the pods will be recreated • or when you scale a deployment
11. Marc Sluiter KubeVirt @ Godays.io 2020 Kubernetes components (cont.) Kube-Scheduler • decides WHERE (on which node) pods will be started • based on resource requests and resource availability (cpu, memory, ...) • based on (anti-)affinity rules Kubelet • runs on every node • actually runs the containers
12. Marc Sluiter KubeVirt @ Godays.io 2020 Kubernetes components (cont.) Kubectl Command line tool for talking to kube-apiserver Examples: kubectl create -f my-deployment.yaml kubectl scale deployment my-deployment --replicas 2 kubectl delete deployment my-deployment
13. Marc Sluiter KubeVirt @ Godays.io 2020
14. Marc Sluiter KubeVirt @ Godays.io 2020 Extending Kubernetes CustomResourceDefinitions • Declare your own Kubernetes resources! • Use a “controller” (a Kubernetes Deployment) for watching your own resource and react by e.g. creating several other Kubernetes resources Aggregated API server • Extend the Kubernetes API Server with your own custom REST endpoints
15. Marc Sluiter KubeVirt @ Godays.io 2020 Great…. But now I have 2 orchestrators?! One for my VMs. One for my containers. There has to be a better solution!
16. Marc Sluiter KubeVirt @ Godays.io 2020 Extending Kubernetes (cont.) How to get Virtual Machines into Kubernetes? • Create a CustomResourceDefinition (CRD) describing a VirtualMachine • Implement a custom controller, which starts a virtual machine when it sees a VirtualMachine CustomResource (CR) But is it that easy?
17. Marc Sluiter KubeVirt @ Godays.io 2020 Extending Kubernetes (cont.) Challenges • How to actually start a VM from a container? • How to integrate a VM with Kubernetes networking and storage? • What works out of the box, what needs to be implemented, where are workarounds needed? Luckily most problems are solved already!
18. Marc Sluiter KubeVirt @ Godays.io 2020 KubeVirt is a Kubernetes Add-on providing a virtualization API
19. Marc Sluiter KubeVirt @ Godays.io 2020 KubeVirt • Open Source, Go • Initiated in 2016 by Red Hat • Contributions by other companies e.g (v)GPU support by Nvidia • CNCF sandbox project since 2019 • Provides an API for running KVM based virtual machines in Kubernetes • Goal: run those VMs alongside with containerized workloads, using the same networks / storage etc.
20. Marc Sluiter KubeVirt @ Godays.io 2020 KubeVirt resources KubeVirt introduces several CRDs for managing virtual machines. The most important ones are: • VirtualMachine (VM): represents a virtual machine, which can be started and stopped • VirtualMachineInstance (VMI): when a VirtualMachine is started, a VirtualMachineInstance is created, which represents the running virtual machine
21. Marc Sluiter KubeVirt @ Godays.io 2020 apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: name: vm-fedora labels: kubevirt.io/vm: vm-fedora spec: running: false template: <vmi template> apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachineInstance metadata: name: vmi-fedora labels: kubevirt.io/vm: vm-fedora spec: domain: devices: disks: - name: containerdisk disk: bus: virtio resources: requests: memory: 2G volumes: - name: containerdisk containerDisk: image: kubevirt/fedora-container-disk-demo
22. Marc Sluiter KubeVirt @ Godays.io 2020 KubeVirt components • KubeVirt implements several controllers dealing with its resources • Implementation uses the Kubernetes client-go library • Every controller basically uses this pattern: a. Watch for changes in declared specification of resources b. Compare the declared state with the observed state of the cluster c. Modify the cluster to reach the desired state d. Update the “status” of the resource e. Repeat until declared state matches observed state “Reconcile loop”
23. Marc Sluiter KubeVirt @ Godays.io 2020 KubeVirt components (cont.) Virt-controller • watches the VM resources in the cluster • creates the VMI resource when VM is started • watches the VMI resources in the cluster • creates the virt-launcher pod ◦ Kubernetes schedules and start the virt-launcher pod • passes ownership of VMI to virt-handler of the assigned node
24. Marc Sluiter KubeVirt @ Godays.io 2020 KubeVirt components (cont.) Virt-handler • runs on every node (Daemonset) • Checks if KVM is available on the node and marks the node accordingly (used for scheduling of the virt-launcher pod) • communicates with the libvirt daemon running in the virt-launcher pod for synchronizing the desired state given in the VMI with the actual state of the virtual machine (CPU/memory resources, devices, …)
25. Marc Sluiter KubeVirt @ Godays.io 2020 KubeVirt components (cont.) Virt-launcher • Runs the virtual machine by ◦ Using libvirtd (API around qemu/KVM) ◦ Libvirt uses the qemu machine emulator/virtualizer ◦ Qemu actually runs the virtual machine ▪ ideally using KVM ▪ software emulation can be enabled (bad performance)
26. Marc Sluiter KubeVirt @ Godays.io 2020 KubeVirt components (cont.) Virt-api • Extends the Kubernetes API as an “aggregated API server” ◦ Additional API endpoints on the VM / VMI resources ◦ offers start / stop / pause / unpause / console / vnc features • Provides and registers webhooks for validating and defaulting of VM / VMI properties
27. Marc Sluiter KubeVirt @ Godays.io 2020 KubeVirt components (cont.) Virt-operator • Special controller for installing, updating and uninstalling the KubeVirt “application” by managing all other KubeVirt components • Watches for the KubeVirt CRD • Such controllers, which handle the lifecycle of “applications” are called operators
28. Marc Sluiter KubeVirt @ Godays.io 2020 KubeVirt (cont.) Virtctl Commandline tool for easy access to the new API endpoints Examples: virtctl start my-vm virtctl console my-vmi virtctl stop my-vm
29. Marc Sluiter KubeVirt @ Godays.io 2020
30. Marc Sluiter KubeVirt @ Godays.io 2020 Demo CLI: https://asciinema.org/a/295577 UI: https://youtu.be/kD1pbxnSLo8
31. Marc Sluiter KubeVirt @ Godays.io 2020 KubeVirt More features • Use file or block PersistentVolumes for writeable vm images • Image upload into PVs using CDI (Containerized Data Importer) • Multiple network interfaces with multus • Live migration / node drain • Pausing
32. Marc Sluiter KubeVirt @ Godays.io 2020 Questions? kubevirt.io // @kubevirt // github.com/kubevirt/kubevirt #virtualization on kubernetes Slack
33. Marc Sluiter KubeVirt @ Godays.io 2020 Thank You! Bare Metal / Virtualization / Containers diagram by Rebecca Dodd / CC BY Kubernetes diagrams and yaml by Bob Killen and Jeff Sica / CC BY This presentation is licensed under a Creative Commons Attribution 4.0 International License. See https://creativecommons.org/licenses/by/4.0/ for more details.