Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Control Theory, Controllers and Kubernetes: The Holy Trilogy

Control Theory, Controllers and Kubernetes: The Holy Trilogy

Madhav Jivrajani

September 11, 2021
Tweet

More Decks by Madhav Jivrajani

Other Decks in Technology

Transcript

  1. Control Theory,
    Controllers and
    Kubernetes:
    The Holy Trilogy
    Madhav Jivrajani, VMware

    View full-size slide

  2. $ whoami
    Hi!
    ● Currently a senior @ PES University, Bangalore, India
    ● Been with the Kubernetes community for ~ 1 year
    ● Work on upstream Kubernetes @ VMware
    ● SIG-{ContribEx, API-Machinery, Node, Architecture}

    View full-size slide

  3. Outline
    ● Containerized workloads and orchestration
    ● Can we build systems that can self-assess and self-heal?
    ● Control Theory and PID Control
    ● Where does Kubernetes come in?
    ● Extending Kubernetes
    ● Demo
    ● How does CAPI use CRs?

    View full-size slide

  4. Containerized Workloads and Orchestration

    View full-size slide

  5. What happens if something goes wrong?

    View full-size slide

  6. How do we rectify this and who does it?

    View full-size slide

  7. What if we want to run another app?

    View full-size slide

  8. The answer to these questions is: orchestration!

    View full-size slide

  9. The question that aries now is: how does the orchestrator
    make decisions?

    View full-size slide

  10. The question that aries now is: how does the orchestrator
    make decisions?
    Does a sysadmin monitor the health of the applications and decide what
    actions to take?

    View full-size slide

  11. The question that aries now is: how does the orchestrator
    make decisions?
    Can the orchestrator be intelligent in some form and make these decisions,
    and if so, what information would it require?

    View full-size slide

  12. The question that aries now is: how does the orchestrator
    make decisions?
    Given the information needed, how can the orchestrator keep the system in
    a stable state?

    View full-size slide

  13. Let’s start with the Control Theory “Hello World” example

    View full-size slide

  14. A Few Terms
    ● The entity that we want to control - System
    ○ A system can take inputs and produce outputs.
    ● Where we want to go/our desired state/intent - Set Variable (SV)
    ● Where we currently are/observed state - Process Variable (PV)
    ○ System output
    ● How “far” are we currently from our desired state? - Error (e)
    ○ e = SP - PV
    ● Who drives the system to where it needs to be? - Controller
    ● A controller that provides system inputs based on system outputs - Closed Loop
    Controller

    View full-size slide

  15. However, it’s often not this ideal. With the controller
    applying adjustments, the following questions come to
    mind:
    ● What if the adjustment applied overshoots or undershoots the SV?
    ○ If it does, its likely to cause a thrashing effect.
    ● Can we take past experiences into account and adjust accordingly or in
    other words, can we compensate?
    ● Can we look at our current error and predict what the error is going to
    be in the future?

    View full-size slide

  16. Past, Present and Future - PID Controller

    View full-size slide

  17. Past, Present and Future - PID Controller
    ● P - Proportional: Adjust proportional to the error

    View full-size slide

  18. Past, Present and Future - PID Controller
    ● I - Integral: Adjust based on what the current error is and what the error
    has been in the past

    View full-size slide

  19. Past, Present and Future - PID Controller
    ● D - Derivative: Predict the future error based on rate of change of
    current error

    View full-size slide

  20. Can we look at our orchestrator as a control system?

    View full-size slide

  21. Is it possible for the orchestrator to self-assess and
    self-heal?

    View full-size slide

  22. Is it possible for the orchestrator to self-assess and
    self-heal?
    Yes.

    View full-size slide

  23. In order to do this, you need two pieces of information:
    1. Where do you want to go?
    2. Where are you currently?

    View full-size slide

  24. This largely shapes up to be a closed-loop controller.

    View full-size slide

  25. Where does Kubernetes fit in as an orchestrator?

    View full-size slide

  26. Let’s dive a little deeper into how Kubernetes does things
    using these concepts.

    View full-size slide

  27. All Kubernetes controllers are P controllers!

    View full-size slide

  28. Kubernetes has a few build-in resources such as pod,
    deployments, replicasets, etc.

    View full-size slide

  29. Each of which are accessed through the Kubernetes API
    (which is a REST based API)

    View full-size slide

  30. kubectl get pods --namespace mynamespace
    GET /api/v1/namespaces/mynamespace/pods

    View full-size slide

  31. Other than the built-in resources, Kubernetes allows us to
    create custom resources.
    For ex: we can have a custom resource called foo and we can access
    it through kubectl like any other resource:
    kubectl get foo
    GET /apis/{group}/{version}/namespaces/{namespace}/foos

    View full-size slide

  32. Custom Resources are created from Custom Resources
    Definitions (CRDs).
    For custom resources created, we can also write custom
    controllers that have sufficient intelligence baked in to
    reconcile any state changes.

    View full-size slide

  33. Demo
    https://github.com/kubernetes/sample-controller/

    View full-size slide

  34. This is extremely powerful for a multitude of reasons,
    because now you don’t have to look at Kubernetes as only
    a container orchestration platform.

    View full-size slide

  35. With CRs, Kubernetes can now be looked at as a
    “Universal Control Plane” or a “Platform For Building
    Platforms”
    ● With this extensibility, we can now create a custom resource that
    maybe represents an Infrastructure component, ex - VMs.
    ● We can also write custom controllers, similar to the built-in
    controllers that Kubernetes has, to try and reconcile state.
    ● With this, we can declaratively manage and provision infrastructure,
    by using Kubernetes as a base layer.

    View full-size slide

  36. Projects such as Cluster API (CAPI) and Crossplane make
    use of this extensibility in a similar manner to provision
    and/or manage infrastructure.

    View full-size slide

  37. For ex: CAPI defines a few CRDs, some of which are:
    ● Machine
    ○ Analogous to Kubernetes Pods
    ● MachineDeployment
    ○ Analogous to Kubernetes Deployment
    ● MachineSet
    ○ Analogous to Kubernetes ReplicaSet

    View full-size slide

  38. https://itnext.io/kubernetes-cluster-creation-on-baremetal-host-using-cluster-api-1c2373230a17

    View full-size slide

  39. References and Resources
    ● Control Theory In Container Fleet Management
    ● PID Loops and The Art of Keeping Systems Stable
    ● Kubernetes Design Principles - Understanding The Why
    ● The Magic of Kubernetes Self-Healing Capabilities
    ● Imperative, Declarative and Kubernetes
    ● Deep Dive into Kubernetes Internals for Builders and Operators
    ● The Cluster API Book

    View full-size slide

  40. Thank you!
    Twitter: @MadhavJivrajani
    GitHub: github.com/MadhavJivrajani
    K8s slack (slack.k8s.io): @madhav

    View full-size slide