Operator SDK 帶你玩轉 Kubernetes

[CC-BY-SA 4.0]

從觀念角度介紹 Kubernetes 的 Operator Pattern 如何解決大家在管理叢集時繁瑣卻又重複的人工作業，並使用 Operator SDK 來示範如何自己撰寫 Golang Based Operator 來解決問題。

先從 Operator Pattern 介紹整個概念，並舉出幾個例子來讓聽眾理解如何將 Kubernetes Controller 參與其中，以扮演整個 Life-cycle 中負責進行原本重複的人工作業。
舉出幾個常見的 Operator Pattern 並以 Operator SDK 為例，介紹如何如何自己撰寫 Golang Based Operator 搭配 Kubernetes CRD 來擴充 K8s 的功能。

Transcript

1. 2020 Image credit: © COSCUP, CC-BY-SA 4.0
3. 2 DevOps Kubernetes Cluster
4. 3 kubectl apply -f DevOps Kubernetes Cluster
5. 4 kubectl apply -f DevOps Kubernetes Cluster kubectl get ...
6. 5 kubectl apply -f DevOps Kubernetes Cluster kubectl get ... kubectl apply/edit …
7. 6 kubectl apply -f DevOps Kubernetes Cluster kubectl get ... kubectl apply/edit …
8. 7 DevOps Kubernetes Cluster Bot (or scripts)
9. 8 DevOps Kubernetes Cluster Bot (or scripts) bot deploy … kubtctl apply …
10. 9 DevOps Kubernetes Cluster Bot (or scripts) bot deploy … kubtctl apply … kubtctl get …
11. 10 DevOps Kubernetes Cluster Bot (or scripts) bot deploy … kubtctl apply … kubtctl get … kubtctl edit …
12. 11 DevOps Kubernetes Cluster Bot (or scripts) bot deploy … kubtctl apply … kubtctl get … kubtctl edit … bot update …
13. 12 DevOps Kubernetes Cluster Bot (or scripts) bot deploy … kubtctl apply … kubtctl get … kubtctl edit … bot update … alert
14. 13 DevOps Kubernetes Cluster Bot (or scripts) bot deploy … kubtctl apply … kubtctl get … kubtctl edit … bot update … alert https://www.botkube.io/
15. 14 DevOps Kubernetes Cluster Bot (or scripts) bot deploy … kubtctl apply … kubtctl get … kubtctl edit … bot update … alert K8s Cluster 1 K8s Cluster 2 Communicate Channel https://www.botkube.io/
16. 15 DevOps Kubernetes Cluster Bot (or scripts) bot deploy … kubtctl apply … kubtctl get … kubtctl edit … bot update … alert https://github.com/learnk8s/xlskubectl
17. 16 DevOps Kubernetes Cluster Bot (or scripts)
18. 17 DevOps Kubernetes Cluster Bot (or scripts)
19. 18 DevOps Kubernetes Cluster Bot (or scripts) https://kubernetes.io/docs/concepts/overview/components/
20. 19 DevOps Kubernetes Cluster Bot (or scripts) https://kubernetes.io/docs/concepts/overview/components/
21. 20 DevOps Kubernetes Cluster Bot (or scripts) ……
22. 21 DevOps Kubernetes Cluster Bot (or scripts) …… Database Controller Changed event Commit changes
23. 22 DevOps Kubernetes Cluster Bot (or scripts) …… Database Controller Changed event Commit changes
24. 23 DevOps Kubernetes Cluster Bot (or scripts) …… Database Controller Changed event Commit changes Extendible?
27. 26 https://coreos.com/blog/introducing-operators.html
28. 27 https://coreos.com/blog/introducing-operators.html
29. 28 https://coreos.com/blog/introducing-operators.html !
34. 33 by
35. 34 by
36. 35 by
37. 36 by Basic Install Automated application provisioning and configuration management
38. 37 by Seamless Upgrades Patch and minor version upgrades supported
39. 38 by Full Lifecycle App lifecycle, storage lifecycle (backup, failure recovery)
40. 39 by Deep Insights Metrics, alerts, log processing and workload analysis
41. 40 by Auto Pilot Horizontal/vertical scaling, auto config tuning, abnormal detection, scheduling tuning
42. 41 by
43. 42 by
44. 43 by
45. 44 by
47. 46 kubernetes-sigs controller-runtime https://book.kubebuilder.io/ kubernetes-sigs kustomize
52. 51 Queue Client Reconciler Cache Scheme Manager Controller
53. 52 Queue Client Reconciler Cache Scheme Manager Controller
54. 53 Queue Client Reconciler Cache Scheme Manager Controller
55. 54 Queue Client Reconciler Cache Scheme Manager Controller
56. 55 Queue Client Reconciler Cache Scheme Manager Controller
58. 57 apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata name: crontabs.v1.example.com spec: names: kind: CronTab ...
59. 58 YAML
60. 59 YAML // Memcached is the Schema for the memcacheds API // +k8s:openapi-gen=true // +kubebuilder:subresource:status // +kubebuilder:resource:path=memcached,scope=Namespaced, shortName=mc // +kubebuilder:printcolumn:name="Size",type=integer, JSONPath=`.spec.size", description="Size of the memcached deployment" // +kubebuilder:printcolumn:name="Nodes",type=string, JSONPath=".status.nodes[*]"
61. 60 YAML // Memcached is the Schema for the memcacheds API // +k8s:openapi-gen=true // +kubebuilder:subresource:status // +kubebuilder:resource:path=memcached,scope=Namespaced, shortName=mc // +kubebuilder:printcolumn:name="Size",type=integer, JSONPath=`.spec.size", description="Size of the memcached deployment" // +kubebuilder:printcolumn:name="Nodes",type=string, JSONPath=".status.nodes[*]" // +kubebuilder:rbac:groups=cache.example.com, resources=memcacheds, verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=cache.example.com, resources=memcacheds/status, verbs=get;update;patch // +kubebuilder:rbac:groups=apps, resources=deployments, verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=core, resources=pods, verbs=get;list;
62. 61 YAML
64. 63 Queue Reconciler
65. Queue Reconciler 64 // add new Controller to mgr with r as the reconcile.Reconciler func add(mgr manager.Manager, r reconcile.Reconciler) error { return ctrl.NewControllerManagedBy(mgr). Named("memcached-controller"). For(&cachev1alpha1.Memcached{}). Owns(&appsv1.Deployment{}). Owns(&corev1.Service{}). Complete(r) }
66. Queue Reconciler 65 // add new Controller to mgr with r as the reconcile.Reconciler func add(mgr manager.Manager, r reconcile.Reconciler) error { return ctrl.NewControllerManagedBy(mgr). Named("memcached-controller"). For(&cachev1alpha1.Memcached{}). Owns(&appsv1.Deployment{}). Owns(&corev1.Service{}). Complete(r) } Watch Memcached changes and enqueue Memcached
67. Queue Reconciler 66 // add new Controller to mgr with r as the reconcile.Reconciler func add(mgr manager.Manager, r reconcile.Reconciler) error { return ctrl.NewControllerManagedBy(mgr). Named("memcached-controller"). For(&cachev1alpha1.Memcached{}). Owns(&appsv1.Deployment{}). Owns(&corev1.Service{}). Complete(r) } Watch Deployment changes and enqueue Memcached owner Watch Memcached changes and enqueue Memcached
68. 67 Queue Reconciler
69. Queue Reconciler 68 // add new Controller to mgr with r as the reconcile.Reconciler func add(mgr manager.Manager, r reconcile.Reconciler) error { return ctrl.NewControllerManagedBy(mgr). Named("memcached-controller"). For(&cachev1alpha1.Memcached{}). Owns(&appsv1.Deployment{}). Owns(&corev1.Service{}). WithEventFilter(&predicate.ResourceVersionChangedPredicate{}). Complete(r) }
70. Queue Reconciler 69 // add new Controller to mgr with r as the reconcile.Reconciler func add(mgr manager.Manager, r reconcile.Reconciler) error { return ctrl.NewControllerManagedBy(mgr). Named("memcached-controller"). For(&cachev1alpha1.Memcached{}). Owns(&appsv1.Deployment{}). Owns(&corev1.Service{}). WithEventFilter(&predicate.ResourceVersionChangedPredicate{}). Complete(r) } // Or ... predicate.NewPredicateFuncs( func(meta metav1.Object, object runtime.Object) bool { // filtering... return true })
71. Queue Reconciler 70 // More powerful predicate.Funcs{ CreateFunc: func(e event.CreateEvent) bool { // ... }, UpdateFunc: func(e event.UpdateEvent) bool { // ... }, DeleteFunc: func(e event.DeleteEvent) bool { // ... }, GenericFunc: func(e event.GenericEvent) bool { // ... }, }
72. Queue Reconciler 71
73. Queue Controller 72
74. Queue Controller 73 // The Controller will requeue the Request to be processed again // if the returned error is non-nil or Result.Requeue is true, // otherwise upon completion will remove the work from the queue. func (r *Reconciler) Reconcile(request reconcile.Request) ( reconcile.Result, error) { // ... }
75. Queue Controller 74 // The Controller will requeue the Request to be processed again // if the returned error is non-nil or Result.Requeue is true, // otherwise upon completion will remove the work from the queue. func (r *Reconciler) Reconcile(request reconcile.Request) ( reconcile.Result, error) { // ... } Reconcile
76. Queue Controller 75 Reconcile // Fetch the Memcached instance memcached := &cachev1alpha1.Memcached{} err := r.client.Get(context.TODO(), request.NamespacedName, memcached) if err != nil { if errors.IsNotFound(err) { reqLogger.Info("Ignoring since object must be deleted.") return reconcile.Result{}, nil } reqLogger.Error(err, "Failed to get Memcached.") return reconcile.Result{}, err }
77. Queue Controller 76 Reconcile // Fetch the Memcached instance memcached := &cachev1alpha1.Memcached{} err := r.client.Get(context.TODO(), request.NamespacedName, memcached) if err != nil { if errors.IsNotFound(err) { reqLogger.Info("Ignoring since object must be deleted.") return reconcile.Result{}, nil } reqLogger.Error(err, "Failed to get Memcached.") return reconcile.Result{}, err } Fetch Resource
78. Queue Controller 77 Reconcile Fetch Resource // Check if the Deployment already exists, if not create a new one deployment := &appsv1.Deployment{} err = r.client.Get(context.TODO(), types.NamespacedName{ Name: memcached.Name, Namespace: memcached.Namespace, }, deployment) if err != nil && errors.IsNotFound(err) { dep := r.deploymentForMemcached(memcached) err = r.client.Create(context.TODO(), dep) if err != nil { return reconcile.Result{}, err } return reconcile.Result{Requeue: true}, nil } else if err != nil { return reconcile.Result{}, err }
79. Queue Controller 78 Reconcile Fetch Resource Create Deployment Fetch Deployment // Check if the Deployment already exists, if not create a new one deployment := &appsv1.Deployment{} err = r.client.Get(context.TODO(), types.NamespacedName{ Name: memcached.Name, Namespace: memcached.Namespace, }, deployment) if err != nil && errors.IsNotFound(err) { dep := r.deploymentForMemcached(memcached) err = r.client.Create(context.TODO(), dep) if err != nil { return reconcile.Result{}, err } return reconcile.Result{Requeue: true}, nil } else if err != nil { return reconcile.Result{}, err }
80. Queue Controller 79 Reconcile Fetch Resource Create Deployment Fetch Deployment // Check if the Deployment already exists, if not create a new one deployment := &appsv1.Deployment{} err = r.client.Get(context.TODO(), types.NamespacedName{ Name: memcached.Name, Namespace: memcached.Namespace, }, deployment) if err != nil && errors.IsNotFound(err) { dep := r.deploymentForMemcached(memcached) err = r.client.Create(context.TODO(), dep) if err != nil { return reconcile.Result{}, err } return reconcile.Result{Requeue: true}, nil } else if err != nil { return reconcile.Result{}, err }
81. Queue Controller 80 Reconcile Fetch Resource Create Deployment Fetch Deployment // Check if the Deployment already exists, if not create a new one deployment := &appsv1.Deployment{} err = r.client.Get(context.TODO(), types.NamespacedName{ Name: memcached.Name, Namespace: memcached.Namespace, }, deployment) if err != nil && errors.IsNotFound(err) { dep := r.deploymentForMemcached(memcached) err = r.client.Create(context.TODO(), dep) if err != nil { return reconcile.Result{}, err } return reconcile.Result{Requeue: true}, nil } else if err != nil { return reconcile.Result{}, err } // deploymentForMemcached returns a memcached Deployment object func (r *Reconciler) deploymentForMemcached(m *cachev1alpha1.Memcached) *appsv1.Deployment { dep := &appsv1.Deployment{ /* ... */ } // Set Memcached instance as the owner of the Deployment. controllerutil.SetControllerReference(m, dep, r.scheme) return dep }
82. Queue Controller 81 Reconcile Fetch Resource Create Deployment Fetch Deployment // Check if the Deployment already exists, if not create a new one deployment := &appsv1.Deployment{} err = r.client.Get(context.TODO(), types.NamespacedName{ Name: memcached.Name, Namespace: memcached.Namespace, }, deployment) if err != nil && errors.IsNotFound(err) { dep := r.deploymentForMemcached(memcached) err = r.client.Create(context.TODO(), dep) if err != nil { return reconcile.Result{}, err } return reconcile.Result{Requeue: true}, nil } else if err != nil { return reconcile.Result{}, err } // deploymentForMemcached returns a memcached Deployment object func (r *Reconciler) deploymentForMemcached(m *cachev1alpha1.Memcached) *appsv1.Deployment { dep := &appsv1.Deployment{ /* ... */ } // Set Memcached instance as the owner of the Deployment. controllerutil.SetControllerReference(m, dep, r.scheme) return dep } Deployment ownerRef:
83. Queue Controller 82 Reconcile Fetch Resource Create Deployment Fetch Deployment // Check if the Deployment already exists, if not create a new one deployment := &appsv1.Deployment{} err = r.client.Get(context.TODO(), types.NamespacedName{ Name: memcached.Name, Namespace: memcached.Namespace, }, deployment) if err != nil && errors.IsNotFound(err) { dep := r.deploymentForMemcached(memcached) err = r.client.Create(context.TODO(), dep) if err != nil { return reconcile.Result{}, err } return reconcile.Result{Requeue: true}, nil } else if err != nil { return reconcile.Result{}, err } // deploymentForMemcached returns a memcached Deployment object func (r *Reconciler) deploymentForMemcached(m *cachev1alpha1.Memcached) *appsv1.Deployment { dep := &appsv1.Deployment{ /* ... */ } // Set Memcached instance as the owner of the Deployment. controllerutil.SetControllerReference(m, dep, r.scheme) return dep } Deployment ownerRef: Memcached
84. Queue Controller 83 Reconcile Fetch Resource Create Deployment Fetch Deployment // Check if the Deployment already exists, if not create a new one deployment := &appsv1.Deployment{} err = r.client.Get(context.TODO(), types.NamespacedName{ Name: memcached.Name, Namespace: memcached.Namespace, }, deployment) if err != nil && errors.IsNotFound(err) { dep := r.deploymentForMemcached(memcached) err = r.client.Create(context.TODO(), dep) if err != nil { return reconcile.Result{}, err } return reconcile.Result{Requeue: true}, nil } else if err != nil { return reconcile.Result{}, err }
85. Queue Controller 84 Reconcile Fetch Resource Create Deployment Fetch Deployment // Check if the Deployment already exists, if not create a new one deployment := &appsv1.Deployment{} err = r.client.Get(context.TODO(), types.NamespacedName{ Name: memcached.Name, Namespace: memcached.Namespace, }, deployment) if err != nil && errors.IsNotFound(err) { dep := r.deploymentForMemcached(memcached) err = r.client.Create(context.TODO(), dep) if err != nil { return reconcile.Result{}, err } return reconcile.Result{Requeue: true}, nil } else if err != nil { return reconcile.Result{}, err }
86. Queue Controller 85 Reconcile Fetch Resource Create Deployment Fetch Deployment // Ensure the deployment size is the same as the spec size := memcached.Spec.Size if *deployment.Spec.Replicas != size { deployment.Spec.Replicas = &size err = r.client.Update(context.TODO(), deployment) if err != nil { return reconcile.Result{}, err } }
87. Queue Controller 86 Reconcile Fetch Resource Create Deployment Fetch Deployment // Ensure the deployment size is the same as the spec size := memcached.Spec.Size if *deployment.Spec.Replicas != size { deployment.Spec.Replicas = &size err = r.client.Update(context.TODO(), deployment) if err != nil { return reconcile.Result{}, err } } Reconcile Deployment
88. Queue Controller 87 Reconcile Fetch Resource Create Deployment Fetch Deployment Reconcile Deployment // Update the Memcached status with the pod names // List the pod names for this memcached's deployment podNames := getPodNames(podList.Items) // Update status.Nodes if needed if !reflect.DeepEqual(podNames, memcached.Status.Nodes) { memcached.Status.Nodes = podNames err := r.client.Status().Update(context.TODO(), memcached) if err != nil { return reconcile.Result{}, err } }
89. Queue Controller 88 Reconcile Fetch Resource Create Deployment Fetch Deployment Reconcile Deployment // Update the Memcached status with the pod names // List the pod names for this memcached's deployment podNames := getPodNames(podList.Items) // Update status.Nodes if needed if !reflect.DeepEqual(podNames, memcached.Status.Nodes) { memcached.Status.Nodes = podNames err := r.client.Status().Update(context.TODO(), memcached) if err != nil { return reconcile.Result{}, err } } Update Resource Status
90. Queue Controller 89 Reconcile Fetch Resource Create Deployment Fetch Deployment Reconcile Deployment Update Resource Status
92. 91 Deployment ownerRef: Memcached
93. 92 Deployment ownerRef: Memcached
94. 93 Deployment ownerRef: Memcached ReplicaSet ownerRef: Pod ownerRef:
95. 94 Deployment ownerRef: Memcached ReplicaSet ownerRef: Pod ownerRef:
96. Deployment ownerRef: Memcached ReplicaSet ownerRef: Pod ownerRef: 95
97. Deployment ownerRef: Memcached ReplicaSet ownerRef: Pod ownerRef: 96
98. Deployment ownerRef: Memcached ReplicaSet ownerRef: Pod ownerRef: 97 // This is the equivalent of calling // Watches(&source.Kind{Type: <ForType-forInput>}, // &handler.EnqueueRequestForOwner{ // OwnerType: apiType, IsController: true, // }) func (blder *Builder) Owns(object runtime.Object, opts ...OwnsOption) *Builder { input := OwnsInput{object: object} for _, opt := range opts { opt.ApplyToOwns(&input) } blder.ownsInput = append(blder.ownsInput, input) return blder } https://github.com/kubernetes-sigs/controller-runtime/blob/v0.6.1/pkg/builder/controller.go
99. 98 // This is the equivalent of calling // Watches(&source.Kind{Type: <ForType-forInput>}, // &handler.EnqueueRequestForOwner{ // OwnerType: apiType, IsController: true, // }) func (blder *Builder) Owns(object runtime.Object, opts ...OwnsOption) *Builder { input := OwnsInput{object: object} for _, opt := range opts { opt.ApplyToOwns(&input) } blder.ownsInput = append(blder.ownsInput, input) return blder } https://github.com/kubernetes-sigs/controller-runtime/blob/v0.6.1/pkg/builder/controller.go
100. Deployment ownerRef: Memcached 99 // This is the equivalent of calling // Watches(&source.Kind{Type: <ForType-forInput>}, // &handler.EnqueueRequestForOwner{ // OwnerType: apiType, IsController: true, // }) func (blder *Builder) Owns(object runtime.Object, opts ...OwnsOption) *Builder { input := OwnsInput{object: object} for _, opt := range opts { opt.ApplyToOwns(&input) } blder.ownsInput = append(blder.ownsInput, input) return blder } https://github.com/kubernetes-sigs/controller-runtime/blob/v0.6.1/pkg/builder/controller.go func (blder *Builder) doWatch() error { // ... // Watches the managed types for _, own := range blder.ownsInput { src := &source.Kind{Type: own.object} hdler := &handler.EnqueueRequestForOwner{ OwnerType: blder.forInput.object, IsController: true, } allPredicates := append([]predicate.Predicate(nil), blder.globalPredicates...) allPredicates = append(allPredicates, own.predicates...) if err := blder.ctrl.Watch(src, hdler, allPredicates...); err != nil { return err } } // ... }
101. 100 // This is the equivalent of calling // Watches(&source.Kind{Type: <ForType-forInput>}, // &handler.EnqueueRequestForOwner{ // OwnerType: apiType, IsController: true, // }) func (blder *Builder) Owns(object runtime.Object, opts ...OwnsOption) *Builder { input := OwnsInput{object: object} for _, opt := range opts { opt.ApplyToOwns(&input) } blder.ownsInput = append(blder.ownsInput, input) return blder } https://github.com/kubernetes-sigs/controller-runtime/blob/v0.6.1/pkg/builder/controller.go func (blder *Builder) doWatch() error { // ... // Watches the managed types for _, own := range blder.ownsInput { src := &source.Kind{Type: own.object} hdler := &handler.EnqueueRequestForOwner{ OwnerType: blder.forInput.object, IsController: true, } allPredicates := append([]predicate.Predicate(nil), blder.globalPredicates...) allPredicates = append(allPredicates, own.predicates...) if err := blder.ctrl.Watch(src, hdler, allPredicates...); err != nil { return err } } // ... }
102. 101 // This is the equivalent of calling // Watches(&source.Kind{Type: <ForType-forInput>}, // &handler.EnqueueRequestForOwner{ // OwnerType: apiType, IsController: true, // }) func (blder *Builder) Owns(object runtime.Object, opts ...OwnsOption) *Builder { input := OwnsInput{object: object} for _, opt := range opts { opt.ApplyToOwns(&input) } blder.ownsInput = append(blder.ownsInput, input) return blder } https://github.com/kubernetes-sigs/controller-runtime/blob/v0.6.1/pkg/handler/enqueue_owner.go func (blder *Builder) doWatch() error { // ... // Watches the managed types for _, own := range blder.ownsInput { src := &source.Kind{Type: own.object} hdler := &handler.EnqueueRequestForOwner{ OwnerType: blder.forInput.object, IsController: true, } allPredicates := append([]predicate.Predicate(nil), blder.globalPredicates...) allPredicates = append(allPredicates, own.predicates...) if err := blder.ctrl.Watch(src, hdler, allPredicates...); err != nil { return err } } // ... } type EnqueueRequestForOwner struct { // ... } func (e *EnqueueRequestForOwner) Create( evt event.CreateEvent, q workqueue.RateLimitingInterface) { for _, req := range e.getOwnerReconcileRequest(evt.Meta) { q.Add(req) } } func (e *EnqueueRequestForOwner) Update( evt event.UpdateEvent, q workqueue.RateLimitingInterface) { for _, req := range e.getOwnerReconcileRequest(evt.MetaOld) { q.Add(req) } for _, req := range e.getOwnerReconcileRequest(evt.MetaNew) { q.Add(req) } } // ...Delete & Generic
103. 102 https://github.com/kubernetes-sigs/controller-runtime/blob/v0.6.1/pkg/handler/enqueue_owner.go func (blder *Builder) doWatch() error { // ... // Watches the managed types for _, own := range blder.ownsInput { src := &source.Kind{Type: own.object} hdler := &handler.EnqueueRequestForOwner{ OwnerType: blder.forInput.object, IsController: true, } allPredicates := append([]predicate.Predicate(nil), blder.globalPredicates...) allPredicates = append(allPredicates, own.predicates...) if err := blder.ctrl.Watch(src, hdler, allPredicates...); err != nil { return err } } // ... } type EnqueueRequestForOwner struct { // ... } func (e *EnqueueRequestForOwner) Create( evt event.CreateEvent, q workqueue.RateLimitingInterface) { for _, req := range e.getOwnerReconcileRequest(evt.Meta) { q.Add(req) } } func (e *EnqueueRequestForOwner) Update( evt event.UpdateEvent, q workqueue.RateLimitingInterface) { for _, req := range e.getOwnerReconcileRequest(evt.MetaOld) { q.Add(req) } for _, req := range e.getOwnerReconcileRequest(evt.MetaNew) { q.Add(req) } } // ...Delete & Generic
104. 103 https://github.com/kubernetes-sigs/controller-runtime/blob/v0.6.1/pkg/handler/enqueue_owner.go func (blder *Builder) doWatch() error { // ... // Watches the managed types for _, own := range blder.ownsInput { src := &source.Kind{Type: own.object} hdler := &handler.EnqueueRequestForOwner{ OwnerType: blder.forInput.object, IsController: true, } allPredicates := append([]predicate.Predicate(nil), blder.globalPredicates...) allPredicates = append(allPredicates, own.predicates...) if err := blder.ctrl.Watch(src, hdler, allPredicates...); err != nil { return err } } // ... } type EnqueueRequestForOwner struct { // ... } func (e *EnqueueRequestForOwner) Create( evt event.CreateEvent, q workqueue.RateLimitingInterface) { for _, req := range e.getOwnerReconcileRequest(evt.Meta) { q.Add(req) } } func (e *EnqueueRequestForOwner) Update( evt event.UpdateEvent, q workqueue.RateLimitingInterface) { for _, req := range e.getOwnerReconcileRequest(evt.MetaOld) { q.Add(req) } for _, req := range e.getOwnerReconcileRequest(evt.MetaNew) { q.Add(req) } } // ...Delete & Generic // getOwnerReconcileRequest looks at object and returns a slice of // reconcile.Request to reconcile owners of object that match e.OwnerType. func (e *EnqueueRequestForOwner) getOwnerReconcileRequest( object metav1.Object ) []reconcile.Request { var result []reconcile.Request for _, ref := range e.getOwnersReferences(object) { refGV, err := schema.ParseGroupVersion(ref.APIVersion) // ... if ref.Kind == e.groupKind.Kind && refGV.Group == e.groupKind.Group { // Match found - add a Request for the object referred to request := reconcile.Request{NamespacedName: types.NamespacedName{ Name: ref.Name, }} // ... result = append(result, request) } } return result }
105. Deployment ownerRef: Memcached ReplicaSet ownerRef: Pod ownerRef: 104 Watch Deployment changes and enqueue Memcached owner Watch Memcached changes and enqueue Memcached https://github.com/kubernetes-sigs/controller-runtime/blob/v0.6.1/pkg/handler/enqueue_owner.go type EnqueueRequestForOwner struct { // ... } func (e *EnqueueRequestForOwner) Create( evt event.CreateEvent, q workqueue.RateLimitingInterface) { for _, req := range e.getOwnerReconcileRequest(evt.Meta) { q.Add(req) } } func (e *EnqueueRequestForOwner) Update( evt event.UpdateEvent, q workqueue.RateLimitingInterface) { for _, req := range e.getOwnerReconcileRequest(evt.MetaOld) { q.Add(req) } for _, req := range e.getOwnerReconcileRequest(evt.MetaNew) { q.Add(req) } } // ...Delete & Generic // getOwnerReconcileRequest looks at object and returns a slice of // reconcile.Request to reconcile owners of object that match e.OwnerType. func (e *EnqueueRequestForOwner) getOwnerReconcileRequest( object metav1.Object ) []reconcile.Request { var result []reconcile.Request for _, ref := range e.getOwnersReferences(object) { refGV, err := schema.ParseGroupVersion(ref.APIVersion) // ... if ref.Kind == e.groupKind.Kind && refGV.Group == e.groupKind.Group { // Match found - add a Request for the object referred to request := reconcile.Request{NamespacedName: types.NamespacedName{ Name: ref.Name, }} // ... result = append(result, request) } } return result }
106. Deployment ownerRef: Memcached ReplicaSet ownerRef: Pod ownerRef: 105 Watch Deployment changes and enqueue Memcached owner Watch Memcached changes and enqueue Memcached
107. Deployment ownerRef: Memcached ReplicaSet ownerRef: Pod ownerRef: 106 Watches(&source.Kind{Type: &appsv1.Deployment{}}, &handler.EnqueueRequestForOwner{OwnerType: &cachev1alpha1.Memcached{}, IsController: true}) Watches(&source.Kind{Type: &cachev1alpha1.Memcached{}}, &handler.EnqueueRequestForObject{})
108. Deployment ownerRef: Memcached ReplicaSet ownerRef: Pod ownerRef: 107
109. Deployment ownerRef: Memcached ReplicaSet ownerRef: Pod ownerRef: 108 map := handler.ToRequestsFunc( func(obj handler.MapObject) []reconcile.Request { var result []reconcile.Request for _, ref := range getOwnersReferences(obj) { for _, rref := range getOwnersReferences(ref) { for _, rrref := range getOwnersReferences(rref) { if isMemcachedObj(rrref) { result = append(result, reconcile.Request{ NamespacedName: types.NamespacedName{ Name: rrref.Name, }, }) } } } } return result })
110. Deployment ownerRef: Memcached ReplicaSet ownerRef: Pod ownerRef: 109 // add new Controller to mgr with r as the reconcile.Reconciler func add(mgr manager.Manager, r reconcile.Reconciler) error { return ctrl.NewControllerManagedBy(mgr). Named("memcached-controller"). For(&cachev1alpha1.Memcached{}). Owns(&appsv1.Deployment{}). Owns(&corev1.Service{}). Watches( &source.Kind{Type: &corev1.Pod{}}, &handler.EnqueueRequestsFromMapFunc{ ToRequests: map, }). Complete(r) }
111. 110 Deployment ownerRef: Memcached ReplicaSet ownerRef: Pod ownerRef:
112. 111 Deployment ownerRef: Memcached ReplicaSet ownerRef: Pod ownerRef:
113. 112 Deployment ownerRef: template: Memcached ReplicaSet ownerRef: template: Pod ownerRef: label:
114. 113 Deployment ownerRef: template: Memcached ReplicaSet ownerRef: template: Pod ownerRef: label:
115. 114 Deployment ownerRef: template: Memcached ReplicaSet ownerRef: template: Pod ownerRef: label: // add new Controller to mgr with r as the reconcile.Reconciler func add(mgr manager.Manager, r reconcile.Reconciler) error { return ctrl.NewControllerManagedBy(mgr). Named("memcached-controller"). For(&cachev1alpha1.Memcached{}). Owns(&appsv1.Deployment{}). Owns(&corev1.Service{}). Watches( &source.Kind{Type: &corev1.Pod{}}, &enqueueRequestsFromLabel{}, &builder.WithPredicates(&hasMemcachedLabelFilter{}), ). Complete(r) }
116. 115 Deployment ownerRef: template: Memcached ReplicaSet ownerRef: template: Pod ownerRef: label: // add new Controller to mgr with r as the reconcile.Reconciler func add(mgr manager.Manager, r reconcile.Reconciler) error { return ctrl.NewControllerManagedBy(mgr). Named("memcached-controller"). For(&cachev1alpha1.Memcached{}). Owns(&appsv1.Deployment{}). Owns(&corev1.Service{}). Watches( &source.Kind{Type: &corev1.Pod{}}, &enqueueRequestsFromLabel{}, &builder.WithPredicates(&hasMemcachedLabelFilter{}), ). Complete(r) }
118. 117 Service FirewallRule ownerRef:
119. 118 Service FirewallRule ownerRef:
120. 119 Service FirewallRule ownerRef:
121. 120 FirewallRule ownerRef:
122. 121 FirewallRule ownerRef:
129. 128 FirewallRule
131. 130 FirewallRule
132. 131 FirewallRule Finalizers:
133. 132 FirewallRule Finalizers: DeletionTimestamp: FirewallRule Finalizers: DeletionTimestamp:
134. 133 FirewallRule Finalizers: DeletionTimestamp: Reconcile Fetch Resource Create Deployment Fetch Deployment Reconcile Deployment Update Resource Status
135. 134 FirewallRule Finalizers: DeletionTimestamp: Reconcile Fetch Resource Create Deployment Fetch Deployment Reconcile Deployment Update Resource Status Add Finalizer Perform Deletion
136. 135 FirewallRule Finalizers: DeletionTimestamp: Reconcile Fetch Resource Create Deployment Fetch Deployment Reconcile Deployment Update Resource Status Add Finalizer Perform Deletion const FinalizerName = "finalizer.example.com/firewall" func (r *Reconciler) Reconcile(request reconcile.Request) ( reconcile.Result, error) { // Fetch FirewallRule... if rule.GetDeletionTimestamp() != nil { // If we have not done deletion yet if contain(rule.Finalizers, FinalizerName) { // Perform deletion... return reconcile.Result{}, r.removeFinalizer(rule) } } else { // If we haven't add finalizer yet if !contain(rule.Finalizers, FinalizerName) { r.addFinalizer(rule) } } // ... }
137. 136 FirewallRule Finalizers: DeletionTimestamp: Reconcile Fetch Resource Create Deployment Fetch Deployment Reconcile Deployment Update Resource Status Add Finalizer Perform Deletion const FinalizerName = "finalizer.example.com/firewall" func (r *Reconciler) Reconcile(request reconcile.Request) ( reconcile.Result, error) { // Fetch FirewallRule... if rule.GetDeletionTimestamp() != nil { // If we have not done deletion yet if contain(rule.Finalizers, FinalizerName) { // Perform deletion... return reconcile.Result{}, r.removeFinalizer(rule) } } else { // If we haven't add finalizer yet if !contain(rule.Finalizers, FinalizerName) { r.addFinalizer(rule) } } // ... } Need to be deleted
138. 137 FirewallRule Finalizers: DeletionTimestamp: Reconcile Fetch Resource Create Deployment Fetch Deployment Reconcile Deployment Update Resource Status Add Finalizer Perform Deletion const FinalizerName = "finalizer.example.com/firewall" func (r *Reconciler) Reconcile(request reconcile.Request) ( reconcile.Result, error) { // Fetch FirewallRule... if rule.GetDeletionTimestamp() != nil { // If we have not done deletion yet if contain(rule.Finalizers, FinalizerName) { // Perform deletion... return reconcile.Result{}, r.removeFinalizer(rule) } } // ... } Not finalized yet
139. 138 FirewallRule Finalizers: DeletionTimestamp: Reconcile Fetch Resource Create Deployment Fetch Deployment Reconcile Deployment Update Resource Status Add Finalizer Perform Deletion const FinalizerName = "finalizer.example.com/firewall" func (r *Reconciler) Reconcile(request reconcile.Request) ( reconcile.Result, error) { // Fetch FirewallRule... if rule.GetDeletionTimestamp() != nil { // If we have not done deletion yet if contain(rule.Finalizers, FinalizerName) { // Perform deletion... return reconcile.Result{}, r.removeFinalizer(rule) } } // ... } Remember to remove finalizer
140. 139 FirewallRule Finalizers: DeletionTimestamp: Reconcile Fetch Resource Create Deployment Fetch Deployment Reconcile Deployment Update Resource Status Add Finalizer Perform Deletion const FinalizerName = "finalizer.example.com/firewall" func (r *Reconciler) Reconcile(request reconcile.Request) ( reconcile.Result, error) { // Fetch FirewallRule... else { // If we haven't add finalizer yet if !contain(rule.Finalizers, FinalizerName) { r.addFinalizer(rule) } } // ... } Not add finalizer yet
141. 140 FirewallRule Finalizers: DeletionTimestamp: Reconcile Fetch Resource Create Deployment Fetch Deployment Reconcile Deployment Update Resource Status Add Finalizer Perform Deletion const FinalizerName = "finalizer.example.com/firewall" func (r *Reconciler) Reconcile(request reconcile.Request) ( reconcile.Result, error) { // Fetch FirewallRule... else { // If we haven't add finalizer yet if !contain(rule.Finalizers, FinalizerName) { r.addFinalizer(rule) } } // ... } Remember to add finalizer
142. 141 Reconcile Fetch Resource Create Deployment Fetch Deployment Reconcile Deployment Update Resource Status Add Finalizer Perform Deletion
144. 143 StatefulSet replicas: 3 vcTemplate:
145. 144 Pod 1 Pod 2 Pod 3 StatefulSet replicas: 3 vcTemplate: PVC 1 PVC 2 PVC 3 PV 1 PV 2 PV 3
146. 145 Pod 1 Pod 2 Pod 3 StatefulSet replicas: 3 vcTemplate: PVC 1 PVC 2 PVC 3 PV 1 PV 2 PV 3 A B C D E F
147. 146 Pod 1 Pod 2 Pod 3 StatefulSet replicas: 2 vcTemplate: PVC 1 PVC 2 PVC 3 PV 1 PV 2 PV 3 A B C D E F
148. 147 Pod 1 Pod 2 StatefulSet replicas: 2 vcTemplate: PVC 1 PVC 2 PVC 3 PV 1 PV 2 PV 3 A B C D E F
149. 148 Pod 1 Pod 2 StatefulSet replicas: 2 vcTemplate: PVC 1 PVC 2 PVC 3 PV 1 PV 2 PV 3 A B C D E F
150. 149 Pod 1 Pod 2 StatefulSet replicas: 2 vcTemplate: PVC 1 PVC 2 PVC 3 PV 1 PV 2 PV 3 A B C D E F
151. 150 Pod 1 Pod 2 StatefulSet replicas: 2 vcTemplate: PVC 1 PVC 2 PVC 3 PV 1 PV 2 PV 3 A B C D E F Drainer Drainer
152. 151 Pod 1 Pod 2 StatefulSet replicas: 2 vcTemplate: PVC 1 PVC 2 PVC 3 PV 1 PV 2 PV 3 A B C D E F Drainer Drainer
153. 152 Pod 1 Pod 2 StatefulSet replicas: 2 vcTemplate: PVC 1 PVC 2 PVC 3 PV 1 PV 2 PV 3 A B C D E F
154. 153 Pod 1 Pod 2 StatefulSet replicas: 2 vcTemplate: PVC 1 PVC 2 PV 1 PV 2 A B C D E F
159. š 158
160. 2020 Image credit: © COSCUP, CC-BY-SA 4.0