Kubernetes Controllers and Reconciliation Patterns

This presentation provides an overview of concepts such as Kubernetes controllers and reconciliation loops. It then builds on those concepts to describe three core patterns that enable automation using Kubernetes controllers.

Transcript

1. EHOSKAS Hossein Kassaei | 2023-02-09 | Ericsson Internal | Page 1 Kubernetes Controllers and Reconciliation Patterns Hossein Kassaei 2023-02-09
2. EHOSKAS Hossein Kassaei | 2023-02-09 | Ericsson Internal | Page 2 Background • Controllers are entities that implement a control loop; a non-terminating loop that regulates the state of a system. Built-in controllers • Initially, controllers were used in the context of the built- in control loops implemented in kube-controller- manager. • Examples: deployment, statefulset, node, replicaset, endpointslice, job, etc. controllers. Custom controllers • To enable extensibility of the Kubernetes API model and leverage its declarative style, custom controllers were introduced later. • Examples: Jetstack cert-manager, Strimzi (for Apache Kafka), Zalando Postgres Operator, etc. • Note: A Kubernetes operator can implement one or multiple controllers.
3. EHOSKAS Hossein Kassaei | 2023-02-09 | Ericsson Internal | Page 3 Controllers watch and reconcile • Continuously watch the desired and actual states and reconcile the two when: - The desired state is changed - The actual state drifts Reconciliation loop reconciliation Desired State Actual State 1 2 1) Watch for changes to the actual state and if there is a change, update the actual state 2) Watch the actual state and if a drift is detected, correct it by bringing the actual state back to the desired state
4. EHOSKAS Hossein Kassaei | 2023-02-09 | Ericsson Internal | Page 4 Pattern 1 reconciliation • Desired state resides outside the Kubernetes cluster • Actual state resides inside the Kubernetes cluster • Desired state is in a Git repository • Actual state is in the API server Example Pattern Desired State Actual State 1. When a k8s manifest (yaml) file changes in the git repo, post the changes to the API server 2. If an API object is deleted or out-of-sync with the source yaml file, recreate or correct it. Reconciliation example Desired State Actual State 1 2 API server GitOps Agent
5. EHOSKAS Hossein Kassaei | 2023-02-09 | Ericsson Internal | Page 5 Pattern 2 reconciliation • Desired and actual states both reside within the Kubernetes cluster • Desired state is in the API server • Actual state is either in or related to the worker nodes Example Pattern Desired State Actual State 1. When the deployment API object is updated (e.g., image or number of replicas changes), update the Pod(s) 2. If the Pod does not exist, (re)create it Reconciliation example Desired State Actual State API server Controller 1 2
6. EHOSKAS Hossein Kassaei | 2023-02-09 | Ericsson Internal | Page 6 Pattern 3 reconciliation • Desired state resides within the Kubernetes cluster • Actual state resides outside the Kubernetes cluster • Desired state is in the API server • Actual state is in the cloud infrastructure Example Pattern Actual State Desired State 1. When a service of type LoadBalancer is created in the API server, configure the cloud network load balancer. Or, when a Cloud SQL custom resource is posted to the API server, provision a Cloud SQL instance with the matching config. 2. If the configuration of the cloud network load balancer has changed, correct it. Or, if the Cloud SQL instance has been accidentally changed or deleted, update or recreate it. Reconciliation example Desired State Actual State 1 2 API server Controller Network Load Balancer Cloud SQL
7. EHOSKAS Hossein Kassaei | 2023-02-09 | Ericsson Internal | Page 7 Patterns can be combined to create powerful automation reconciliation Actual State / Desired State Actual State reconciliation Desired State • For the first control loop, the desired state resides outside the cluster, and the actual state resides inside the cluster. • For the second control loop, the desired state resides inside the cluster (what is actual state for the first control loop), and the actual state also resides inside the cluster Pattern 1 + Pattern 2 1. When a k8s manifest (yaml) file changes in the git repo, post the changes to the API server 2. If an API object is deleted or out-of-sync with the source yaml file, correct it. 3. When an API object (e.g., deployment object) is posted to the API server, create the corresponding Pod(s) 4. If the Pod does not exist, (re)create it Example 1 Actual State / Desired State Actual State API server Kube- Controller- manager 1 2 Desired State GitOps Agent 3 4
8. EHOSKAS Hossein Kassaei | 2023-02-09 | Ericsson Internal | Page 8 Patterns can be combined to create powerful automation reconciliation Actual State / Desired State Actual State reconciliation Desired State • For the first control loop, the desired state resides outside the cluster, and the actual state resides inside the cluster. • For the second control loop, the desired state resides inside the cluster (what is actual state for the first control loop), and the actual state also resides inside the cluster Pattern 1 + Pattern 2 1. When a k8s custom resource (yaml) file changes in the git repo, post the changes to the API server 2. If the custom API object is deleted or out-of-sync with the source yaml file, correct it. 3. When the custom API object (e.g., certificate custom object) is posted to the API server, create the corresponding certificate) 4. If the certificate does not exist, (re)create it Example 2 Actual State / Desired State Actual State API server cert- manager 1 2 Desired State GitOps Agent 3 4 certificate
9. EHOSKAS Hossein Kassaei | 2023-02-09 | Ericsson Internal | Page 9 Patterns can be combined to create powerful automation reconciliation Actual State / Desired State Actual State reconciliation Desired State • For the first control loop, the desired state resides outside the cluster, and the actual state resides inside the cluster. • For the second control loop, the desired state resides inside the cluster (what is actual state for the first control loop), and the actual state resides outside the cluster Pattern 1 + Pattern 3 1. When a k8s resource (yaml) file containing a service of type LoadBalancer is committed to the git repo, post the object to the API server 2. If the service API object is deleted or out-of-sync with the source yaml file, correct it. 3. When the service API object (type LoadBalancer) is posted to the API server, configure the cloud network load balancer 4. If the network load balancer’s configuration is out of sync with the API object, correct it. Example 1 Actual State / Desired State Actual State API server Cloud- controller- manager 1 2 Desired State GitOps Agent 3 4 Network Load Balancer
10. EHOSKAS Hossein Kassaei | 2023-02-09 | Ericsson Internal | Page 10 Patterns can be combined to create powerful automation reconciliation Actual State / Desired State Actual State reconciliation Desired State • For the first control loop, the desired state resides outside the cluster, and the actual state resides inside the cluster. • For the second control loop, the desired state resides inside the cluster (what is actual state for the first control loop), and the actual state resides outside the cluster Pattern 1 + Pattern 3 1. When a k8s resource (yaml) file containing a custom resource of type ‘SQL instance’ is committed to the git repo, post the object to the API server 2. If the custom API object is deleted or out-of-sync with the source yaml file, correct it. 3. When the custom API object (SQL instance) is posted to the API server, create a Cloud SQL instance with configuration 4. If the Cloud SQL’s configuration is out of sync with the custom API object, correct it. Example 2 Actual State / Desired State Actual State API server crossplane provider 1 2 Desired State GitOps Agent 3 4 Cloud SQL