Developing Kubernetes Operators with Golang and Operator SDK

GoKonf; November23rd, 2024

Transcript

1. X: @systemcraftsman | 🦋: @systemcraftsman.com Developing Kubernetes Operators with Golang and Operator SDK Aykut M. Bulgu Principal Software Engineer @ Red Hat @SystemCraftsman
2. X: @systemcraftsman | 🦋: @systemcraftsman.com About Me #oc apply -f aykutbulgu.yaml apiVersion: redhat/v6 kind: Principal Software Engineer metadata: name: Aykut Bulgu namespace: Red Hat Ansible Engineering (previously Kafka) annotations: twitter: @SystemCraftsman bsky: @SystemCraftsman.com email: - aykut@systemcraftsman.com - abulgu@redhat.com ex-organizer: Software Craftsmanship Turkey founder: System Craftsman labels: married: yes children: 2 interests: - basketball/nba/rockets - gamification - stoicism spec: replicas: 1 containers: - image: aykut:latest
3. X: @systemcraftsman | 🦋: @systemcraftsman.com Schedule • What is Kubernetes? ◦ Container Orchestration ◦ Kubernetes Architecture ◦ The Controller Pattern • Extending Kubernetes ◦ Controllers ◦ Operators • Developing Operators ◦ Kubernetes Clients ◦ Operator Development Frameworks • Developing Operators with Golang ◦ Kubebuilder ◦ Operator SDK • Demo: KubeGame
4. X: @systemcraftsman | 🦋: @systemcraftsman.com What is Kubernetes? and why do we need it?
5. X: @systemcraftsman | 🦋: @systemcraftsman.com Container Orchestration but…
6. X: @systemcraftsman | 🦋: @systemcraftsman.com Container Orchestration $ docker build -t app:v1 .
7. X: @systemcraftsman | 🦋: @systemcraftsman.com $ docker build -t app:v1 . $ docker run app:v1 Container Orchestration
8. X: @systemcraftsman | 🦋: @systemcraftsman.com ? Container Orchestration
9. X: @systemcraftsman | 🦋: @systemcraftsman.com $ docker run app/frontend:v1 link-to-backend $ docker run app/frontend:v1 link-to-backend $ docker run app/backend:v1 link-to-db-cache-messaging $ docker run app/backend:v1 link-to-db-cache-messaging $ docker run app/database:v1 $ docker run app/cache:v1 link-to-db $ docker run app/messaging:v1 Container Orchestration
10. X: @systemcraftsman | 🦋: @systemcraftsman.com $ docker-compose up Container Orchestration
11. X: @systemcraftsman | 🦋: @systemcraftsman.com We need more than just containers kubernetes ? Scheduling Decide where to deploy containers Lifecycle and health Keep containers running despite failures Discovery Find other containers on the network Monitoring Visibility into running containers Security Control who can do what Scaling Scale containers up and down Persistence Survive data beyond container lifecycle Aggregation Compose apps from multiple containers
12. X: @systemcraftsman | 🦋: @systemcraftsman.com What is Kubernetes? Kubernetes is an open-source system for automating deployment, operations, and scaling of containerized applications across multiple hosts kubernetes
13. X: @systemcraftsman | 🦋: @systemcraftsman.com Kubernetes Architecture and a quick look at the concepts.
14. X: @systemcraftsman | 🦋: @systemcraftsman.com Kubernetes Architecture
15. X: @systemcraftsman | 🦋: @systemcraftsman.com Declarative Model https://itnext.io/kubernetes-for-dummies-life-of-a-pod-fc8158e27aa YAML file Kubernetes
16. X: @systemcraftsman | 🦋: @systemcraftsman.com Kubernetes Architecture https://itnext.io/kubernetes-for-dummies-life-of-a-pod-fc8158e27aa
17. X: @systemcraftsman | 🦋: @systemcraftsman.com The Controller Pattern
18. X: @systemcraftsman | 🦋: @systemcraftsman.com The Control Loop In robotics and automation, a control loop is a non-terminating loop that regulates the state of a system. In Kubernetes, controllers are control loops that watch the state of your cluster, then make or request changes where needed.
19. X: @systemcraftsman | 🦋: @systemcraftsman.com The Controller Pattern Actual Desired Action 2 replicas 3 replicas Scale up v1.0 v2.0 Update Object Deleted Delete Observe Analyze Act
20. X: @systemcraftsman | 🦋: @systemcraftsman.com Extending Kubernetes
21. X: @systemcraftsman | 🦋: @systemcraftsman.com Extending Kubernetes 1. Kubectl plugins 2. API Access extensions 3. API extensions 4. Scheduling extensions 5. Controllers and Operators 6. Network plugins 7. Device and Storage plugins https://kubernetes.io/docs/concepts/extend-kubernetes/
22. X: @systemcraftsman | 🦋: @systemcraftsman.com Controllers
23. X: @systemcraftsman | 🦋: @systemcraftsman.com Controllers Observe Analyze Act Reconciliation Loop
24. X: @systemcraftsman | 🦋: @systemcraftsman.com Controllers apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:stable-alpine ports: - containerPort: 80
25. X: @systemcraftsman | 🦋: @systemcraftsman.com Developing Controllers
26. X: @systemcraftsman | 🦋: @systemcraftsman.com Developing Controllers You can develop Controllers with any language, such as: To manage Kubernetes resources by using: • Labels • Annotations • ConfigMaps
27. X: @systemcraftsman | 🦋: @systemcraftsman.com A Controller Written in Bash apiVersion : apps/v1 kind: Deployment ... name: config-watcher-controller ... spec: serviceAccountName : config-watcher-controller containers : - name: kubeapi-proxy image: k8spatterns/kubeapi-proxy - name: config-watcher image: k8spatterns/curl-jq env: - name: WATCH_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace command: - "sh" - "/watcher/config-watcher-controller.sh" volumeMounts : ... https://k8spatterns.io/
28. X: @systemcraftsman | 🦋: @systemcraftsman.com Operators
29. X: @systemcraftsman | 🦋: @systemcraftsman.com Operators Operator extends Controller • Labels • Annotations • ConfigMaps Custom Resources
30. X: @systemcraftsman | 🦋: @systemcraftsman.com Operators apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:stable-alpine ports: - containerPort: 80 Prometheus YAML ? ?
31. X: @systemcraftsman | 🦋: @systemcraftsman.com Custom Resource Definition https://k8spatterns.io/
32. X: @systemcraftsman | 🦋: @systemcraftsman.com Operators’ Controller apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:stable-alpine ports: - containerPort: 80 Prometheus YAML Prometheus Controller
33. X: @systemcraftsman | 🦋: @systemcraftsman.com When to use which? https://k8spatterns.io/
34. X: @systemcraftsman | 🦋: @systemcraftsman.com Developing Operators
35. X: @systemcraftsman | 🦋: @systemcraftsman.com Options for Developing Operators • Calling the API directly • Using the Clients • Using Operator Development Frameworks
36. X: @systemcraftsman | 🦋: @systemcraftsman.com Pure Bash (Again!)
37. X: @systemcraftsman | 🦋: @systemcraftsman.com Kubernetes Clients
38. X: @systemcraftsman | 🦋: @systemcraftsman.com Python Client https://blog.palark.com/writing-a-kubernetes-operator-in-python-without-frameworks-and-sdk/
39. X: @systemcraftsman | 🦋: @systemcraftsman.com Fabric8 Client (Java)
40. X: @systemcraftsman | 🦋: @systemcraftsman.com Operator Development Frameworks • Kubebuilder • Operator SDK • Metacontroller • Charmed Operator SDK • KubeOps
41. X: @systemcraftsman | 🦋: @systemcraftsman.com Developing Operators with Golang
42. X: @systemcraftsman | 🦋: @systemcraftsman.com Why Golang?
43. X: @systemcraftsman | 🦋: @systemcraftsman.com Golang is faster than the fastest! https://medium.com/swlh/cloud-native-java-vs-golang-2a72c0531b05
44. X: @systemcraftsman | 🦋: @systemcraftsman.com Java Developers, keep trying! :P
45. X: @systemcraftsman | 🦋: @systemcraftsman.com Operator Development Tools for Golang • Kubebuilder • Operator SDK • Metacontroller • Charmed Operator SDK • KubeOps
46. X: @systemcraftsman | 🦋: @systemcraftsman.com Kubebuilder
47. X: @systemcraftsman | 🦋: @systemcraftsman.com Kubebuilder $ kubebuilder init … Kubebuilder is a framework for building Kubernetes APIs using custom resource definitions (CRDs).
48. X: @systemcraftsman | 🦋: @systemcraftsman.com Kubebuilder as a Library https://github.com/kubernetes-sigs/kubebuilder
49. X: @systemcraftsman | 🦋: @systemcraftsman.com Operator SDK
50. X: @systemcraftsman | 🦋: @systemcraftsman.com Operator SDK
51. X: @systemcraftsman | 🦋: @systemcraftsman.com Operator SDK - GitHub
52. X: @systemcraftsman | 🦋: @systemcraftsman.com Operator SDK • Ansible • Helm • Go • Java (Quarkus) Java Operator SDK (with Quarkus Extension)
53. X: @systemcraftsman | 🦋: @systemcraftsman.com Operator SDK
54. X: @systemcraftsman | 🦋: @systemcraftsman.com Operator SDK $ operator-sdk init … … --plugins helm … --plugins ansible … --plugins quarkus
55. X: @systemcraftsman | 🦋: @systemcraftsman.com Why Use Operator SDK? • Built-in Scaffolding • Leverages Kubebuilder • Rich CLI • Operator Lifecycle Manager (OLM) Integration • Built-in Testing and Validation • Helm, Ansible, and Java (Quarkus) Support • Comprehensive Documentation and Community
56. X: @systemcraftsman | 🦋: @systemcraftsman.com Key Benefits for Golang Developers • Rapid Development: ◦ Faster project setup with less boilerplate code. • Reusability: ◦ Scaffolded code aligns with Kubernetes best practices, reducing maintenance overhead • Improved Debugging and Testing: ◦ Pre-built utilities for simulating Kubernetes clusters locally. • Production-ready Features: ◦ OLM integration and packaging make it easier to distribute Operators.
57. X: @systemcraftsman | 🦋: @systemcraftsman.com Example - Grafana Operator https://github.com/grafana/grafana-operator
58. X: @systemcraftsman | 🦋: @systemcraftsman.com Initializing the Operator
59. X: @systemcraftsman | 🦋: @systemcraftsman.com Creating the API and Reconcilers
60. X: @systemcraftsman | 🦋: @systemcraftsman.com Creating the API and Reconcilers
61. X: @systemcraftsman | 🦋: @systemcraftsman.com MyCustomResource Type
62. X: @systemcraftsman | 🦋: @systemcraftsman.com MyCustomResource Reconciler
63. X: @systemcraftsman | 🦋: @systemcraftsman.com Demo
64. X: @systemcraftsman | 🦋: @systemcraftsman.com Ready Player One
65. X: @systemcraftsman | 🦋: @systemcraftsman.com KubeGame: A Gamification Operator oasis-postgres Oasis oasis-postgres • Archaide • Chthonia • Incipio • Middle-earth
66. X: @systemcraftsman | 🦋: @systemcraftsman.com Talk is Cheap, Let’s See the Demo!
67. X: @systemcraftsman | 🦋: @systemcraftsman.com Demo on GitHub
68. X: @systemcraftsman | 🦋: @systemcraftsman.com Tribute to a Master https://github.com/vigo/dinozorus Uğur “vigo” Özyılmazel
69. X: @systemcraftsman | 🦋: @systemcraftsman.com Thank You! Xwitter: @systemcraftsman 🦋Bluesky: @systemcraftsman.com Email: aykut@systemcraftsman.com Linkedin: https://www.linkedin.com/in/mabulgu/ Website: https://www.systemcraftsman.com/ GitHub: https://github.com/mabulgu