Control Theory, Controllers and Kubernetes: The Holy Trilogy

Transcript

1. Control Theory, Controllers and Kubernetes: The Holy Trilogy Madhav Jivrajani, VMware
2. $ whoami Hi! • Currently a senior @ PES University, Bangalore, India • Been with the Kubernetes community for ~ 1 year • Work on upstream Kubernetes @ VMware • SIG-{ContribEx, API-Machinery, Node, Architecture}
3. Outline • Containerized workloads and orchestration • Can we build systems that can self-assess and self-heal? • Control Theory and PID Control • Where does Kubernetes come in? • Extending Kubernetes • Demo • How does CAPI use CRs?
4. Containerized Workloads and Orchestration
13. What happens if something goes wrong?
15. How do we rectify this and who does it?
16. What if we want to run another app?
19. The answer to these questions is: orchestration!
28. The question that aries now is: how does the orchestrator make decisions?
29. The question that aries now is: how does the orchestrator make decisions? Does a sysadmin monitor the health of the applications and decide what actions to take?
30. The question that aries now is: how does the orchestrator make decisions? Can the orchestrator be intelligent in some form and make these decisions, and if so, what information would it require?
31. The question that aries now is: how does the orchestrator make decisions? Given the information needed, how can the orchestrator keep the system in a stable state?
33. Let’s start with the Control Theory “Hello World” example
47. A Few Terms • The entity that we want to control - System ◦ A system can take inputs and produce outputs. • Where we want to go/our desired state/intent - Set Variable (SV) • Where we currently are/observed state - Process Variable (PV) ◦ System output • How “far” are we currently from our desired state? - Error (e) ◦ e = SP - PV • Who drives the system to where it needs to be? - Controller • A controller that provides system inputs based on system outputs - Closed Loop Controller
49. However, it’s often not this ideal. With the controller applying adjustments, the following questions come to mind: • What if the adjustment applied overshoots or undershoots the SV? ◦ If it does, its likely to cause a thrashing effect. • Can we take past experiences into account and adjust accordingly or in other words, can we compensate? • Can we look at our current error and predict what the error is going to be in the future?
50. Past, Present and Future - PID Controller
51. Past, Present and Future - PID Controller • P - Proportional: Adjust proportional to the error
52. Past, Present and Future - PID Controller • I - Integral: Adjust based on what the current error is and what the error has been in the past
53. Past, Present and Future - PID Controller • D - Derivative: Predict the future error based on rate of change of current error
54. Can we look at our orchestrator as a control system?
55. Is it possible for the orchestrator to self-assess and self-heal?
56. Is it possible for the orchestrator to self-assess and self-heal? Yes.
57. In order to do this, you need two pieces of information: 1. Where do you want to go? 2. Where are you currently?
58. This largely shapes up to be a closed-loop controller.
59. Where does Kubernetes fit in as an orchestrator?
62. Let’s dive a little deeper into how Kubernetes does things using these concepts.
87. All Kubernetes controllers are P controllers!
89. Kubernetes has a few build-in resources such as pod, deployments, replicasets, etc.
90. Each of which are accessed through the Kubernetes API (which is a REST based API)
91. kubectl get pods --namespace mynamespace GET /api/v1/namespaces/mynamespace/pods
92. Other than the built-in resources, Kubernetes allows us to create custom resources. For ex: we can have a custom resource called foo and we can access it through kubectl like any other resource: kubectl get foo GET /apis/{group}/{version}/namespaces/{namespace}/foos
93. Custom Resources are created from Custom Resources Definitions (CRDs). For custom resources created, we can also write custom controllers that have sufficient intelligence baked in to reconcile any state changes.
94. Demo https://github.com/kubernetes/sample-controller/
95. This is extremely powerful for a multitude of reasons, because now you don’t have to look at Kubernetes as only a container orchestration platform.
96. With CRs, Kubernetes can now be looked at as a “Universal Control Plane” or a “Platform For Building Platforms” • With this extensibility, we can now create a custom resource that maybe represents an Infrastructure component, ex - VMs. • We can also write custom controllers, similar to the built-in controllers that Kubernetes has, to try and reconcile state. • With this, we can declaratively manage and provision infrastructure, by using Kubernetes as a base layer.
97. Projects such as Cluster API (CAPI) and Crossplane make use of this extensibility in a similar manner to provision and/or manage infrastructure.
98. For ex: CAPI defines a few CRDs, some of which are: • Machine ◦ Analogous to Kubernetes Pods • MachineDeployment ◦ Analogous to Kubernetes Deployment • MachineSet ◦ Analogous to Kubernetes ReplicaSet
99. https://itnext.io/kubernetes-cluster-creation-on-baremetal-host-using-cluster-api-1c2373230a17
100. References and Resources • Control Theory In Container Fleet Management • PID Loops and The Art of Keeping Systems Stable • Kubernetes Design Principles - Understanding The Why • The Magic of Kubernetes Self-Healing Capabilities • Imperative, Declarative and Kubernetes • Deep Dive into Kubernetes Internals for Builders and Operators • The Cluster API Book
101. Thank you! Twitter: @MadhavJivrajani GitHub: github.com/MadhavJivrajani K8s slack (slack.k8s.io): @madhav