Developing Kubernetes Operators with Java Operator SDK

Slide 1

Slide 1 text

@systemcraftsman Developing Kubernetes Operators with Java Operator SDK Aykut M. Bulgu Principal Software Engineer @ Red Hat @SystemCraftsman

Slide 2

Slide 2 text

@systemcraftsman Schedule ● What is Kubernetes? ○ Container Orchestration ○ Kubernetes Architecture ○ The Controller Pattern ● Extending Kubernetes ○ Controllers ○ Operators ● Developing Operators ○ Kubebuilder ○ Operator SDK ● Developing Operators with Java ○ Java Operator SDK ○ Quarkus Extension ● Demo

Slide 3

Slide 3 text

@systemcraftsman About Me #oc apply -f aykutbulgu.yaml apiVersion: redhat/v5 kind: Principal Software Engineer metadata: name: Aykut Bulgu namespace: Red Hat Kafka Engineering annotations: twitter: @systemcraftsman email: - aykut@systemcraftsman.com - abulgu@redhat.com organizer: Software Craftsmanship Turkey founder: System Craftsman labels: married: yes children: 2 interests: - tech (mostly kafka) - aikido - gamification - stoicism spec: replicas: 1 containers: - image: aykut:latest

Slide 4

Slide 4 text

@systemcraftsman What is Kubernetes? and why do we need it?

Slide 5

Slide 5 text

@systemcraftsman Container Orchestration but…

Slide 6

Slide 6 text

@systemcraftsman Container Orchestration $ docker build -t app:v1 .

Slide 7

Slide 7 text

@systemcraftsman $ docker build -t app:v1 . $ docker run app:v1 Container Orchestration

Slide 8

Slide 8 text

@systemcraftsman ? Container Orchestration

Slide 9

Slide 9 text

@systemcraftsman $ docker build -t app/frontend:v1 . $ docker build -t app/backend:v1 . $ docker build -t app/database:v1 . $ docker build -t app/cache:v1 . $ docker build -t app/messaging:v1 . Container Orchestration

Slide 10

Slide 10 text

@systemcraftsman ? Container Orchestration

Slide 11

Slide 11 text

@systemcraftsman $ docker run app/frontend:v1 link-to-backend $ docker run app/frontend:v1 link-to-backend $ docker run app/backend:v1 link-to-db-cache-messaging $ docker run app/backend:v1 link-to-db-cache-messaging $ docker run app/database:v1 $ docker run app/cache:v1 link-to-db $ docker run app/messaging:v1 Container Orchestration

Slide 12

Slide 12 text

@systemcraftsman $ docker-compose up Container Orchestration

Slide 13

Slide 13 text

@systemcraftsman We need more than just containers kubernetes ? Scheduling Decide where to deploy containers Lifecycle and health Keep containers running despite failures Discovery Find other containers on the network Monitoring Visibility into running containers Security Control who can do what Scaling Scale containers up and down Persistence Survive data beyond container lifecycle Aggregation Compose apps from multiple containers

Slide 14

Slide 14 text

@systemcraftsman What is Kubernetes? Kubernetes is an open-source system for automating deployment, operations, and scaling of containerized applications across multiple hosts kubernetes

Slide 15

Slide 15 text

@systemcraftsman Kubernetes Architecture and a quick look at the concepts.

Slide 16

Slide 16 text

@systemcraftsman Kubernetes Architecture

Slide 17

Slide 17 text

@systemcraftsman Declarative Model https://itnext.io/kubernetes-for-dummies-life-of-a-pod-fc8158e27aa YAML ﬁle Kubernetes

Slide 18

Slide 18 text

@systemcraftsman Kubernetes Architecture https://itnext.io/kubernetes-for-dummies-life-of-a-pod-fc8158e27aa

Slide 19

Slide 19 text

@systemcraftsman The Control Loop In robotics and automation, a control loop is a non-terminating loop that regulates the state of a system. In Kubernetes, controllers are control loops that watch the state of your cluster, then make or request changes where needed.

Slide 20

Slide 20 text

@systemcraftsman The Controller Pattern

Slide 21

Slide 21 text

@systemcraftsman The Controller Pattern Actual Desired Action 2 replicas 3 replicas Scale up v1.0 v2.0 Update Object Deleted Delete Observe Analyze Act

Slide 22

Slide 22 text

@systemcraftsman Extending Kubernetes

Slide 23

Slide 23 text

@systemcraftsman Extending Kubernetes 1. Kubectl plugins 2. API Access extensions 3. API extensions 4. Scheduling extensions 5. Controllers and Operators 6. Network plugins 7. Device and Storage plugins https://kubernetes.io/docs/concepts/extend-kubernetes/

Slide 24

Slide 24 text

@systemcraftsman Controllers

Slide 25

Slide 25 text

@systemcraftsman Controllers Observe Analyze Act Reconciliation Loop

Slide 26

Slide 26 text

@systemcraftsman Controllers apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:stable-alpine ports: - containerPort: 80

Slide 27

Slide 27 text

@systemcraftsman Developing Controllers

Slide 28

Slide 28 text

@systemcraftsman Developing Controllers You can develop Controllers with any language, such as: To manage Kubernetes resources by using: ● Labels ● Annotations ● ConﬁgMaps

Slide 29

Slide 29 text

@systemcraftsman A Controller Written in Bash apiVersion : apps/v1 kind: Deployment ... name: config-watcher-controller ... spec: serviceAccountName : config-watcher-controller containers : - name: kubeapi-proxy image: k8spatterns/kubeapi-proxy - name: config-watcher image: k8spatterns/curl-jq env: - name: WATCH_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace command: - "sh" - "/watcher/config-watcher-controller.sh" volumeMounts : ... https://k8spatterns.io/

Slide 30

Slide 30 text

@systemcraftsman Operators

Slide 31

Slide 31 text

@systemcraftsman Operators Operator extends Controller ● Labels ● Annotations ● ConﬁgMaps Custom Resources

Slide 32

Slide 32 text

@systemcraftsman Operators apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:stable-alpine ports: - containerPort: 80 Prometheus YAML ?

Slide 33

Slide 33 text

@systemcraftsman Custom Resource Deﬁnition https://k8spatterns.io/

Slide 34

Slide 34 text

@systemcraftsman Operators’ Controller apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:stable-alpine ports: - containerPort: 80 Prometheus YAML Prometheus Controller

Slide 35

Slide 35 text

@systemcraftsman When to use which? https://k8spatterns.io/

Slide 36

Slide 36 text

@systemcraftsman Developing Operators

Slide 37

Slide 37 text

@systemcraftsman Operator Development Frameworks ● Kubebuilder ● Operator SDK ● Metacontroller ● Charmed Operator SDK ● KubeOps

Slide 38

Slide 38 text

@systemcraftsman Kubebuilder

Slide 39

Slide 39 text

@systemcraftsman Kubebuilder $ kubebuilder init … Kubebuilder is a framework for building Kubernetes APIs using custom resource deﬁnitions (CRDs).

Slide 40

Slide 40 text

@systemcraftsman Kubebuilder is Also a Library https://github.com/kubernetes-sigs/kubebuilder

Slide 41

Slide 41 text

@systemcraftsman Operator SDK

Slide 42

Slide 42 text

@systemcraftsman Operator SDK ● Ansible ● Helm ● Go

Slide 43

Slide 43 text

@systemcraftsman Operator SDK

Slide 44

Slide 44 text

@systemcraftsman Operator SDK $ operator-sdk init … … --plugins helm … --plugins ansible

Slide 45

Slide 45 text

@systemcraftsman Operator SDK ● Ansible ● Helm ● Go ● Java (Quarkus) Java Operator SDK (with Quarkus Extension)

Slide 46

Slide 46 text

@systemcraftsman Developing Operators with Java

Slide 47

Slide 47 text

@systemcraftsman Developing Operators with Java ● Fabric8 Kubernetes Client ● Java Operator SDK

Slide 48

Slide 48 text

@systemcraftsman Fabric8 Kubernetes Client

Slide 49

Slide 49 text

@systemcraftsman Fabric8 Kubernetes Client

Slide 50

Slide 50 text

@systemcraftsman An Example: Strimzi Kafka Operator https://strimzi.io/

Slide 51

Slide 51 text

@systemcraftsman An Example: Strimzi Kafka Operator https://strimzi.io/

Slide 52

Slide 52 text

@systemcraftsman Java Operator SDK

Slide 53

Slide 53 text

@systemcraftsman Java Operator SDK https://javaoperatorsdk.io/ Java Operator SDK is a higher level framework and related tooling to support writing Kubernetes Operators in Java. It makes it easy to implement best practices and patterns for an Operator. It provides a controller runtime, support for testing operators, and related tooling. In addition to that implementing conversion hooks and dynamic admission controllers are supported as a separate projects.

Slide 54

Slide 54 text

@systemcraftsman Java Operator SDK https://github.com/java-operator-sdk/java-operator-sdk ● Wraps Fabric8 and conﬁgures it for listening to changes on the speciﬁed Custom Resources, thus hiding the boilerplate code required for this. ● Provides a clean interface to implement the reconciliation loop for a particular resource type. ● Schedules change events to be executed in an efficient manner. Filtering obsolete events and executing unrelated events in parallel. ● Retry failed reconciliation attempts.

Slide 55

Slide 55 text

@systemcraftsman Java Operator SDK https://blog.container-solutions.com/cloud-native-java-infrastructure-automation-with-kubernetes-operators

Slide 56

Slide 56 text

@systemcraftsman Reconciliation in Java Operator SDK https://blog.container-solutions.com/kubernetes-operators-explained

Slide 57

Slide 57 text

@systemcraftsman Reconciliation in Java Operator SDK # Reconciliation execution is always triggered by an event. # Events typically come from a primary resource, most of the time a custom resource. Reconciler implementations are associated with a given resource type. # No concurrent reconciliation happens for any given resource. # If an exception is thrown during execution, then it schedules a retry. # If new events are received during the controller execution, then a new reconciliation is scheduled. # If the reconcilier instructed the SDK to reschedule a reconciliation at a later date, then a timer event with the speciﬁed delay is scheduled.

Slide 58

Slide 58 text

@systemcraftsman Reconciliation in Java Operator SDK Controller Level Eventing Level Reconciliation Level An Operator is a set of independent controllers. The Controller class, however, is an internal class managed by the framework itself and usually shouldn’t interacted with directly by end users. It manages all the processing units involved with reconciling a single type of Kubernetes resource.

Slide 59

Slide 59 text

@systemcraftsman Reconciliation in Java Operator SDK public class MyCustomResourceReconciler implements Reconciler { }

Slide 60

Slide 60 text

@systemcraftsman Reconciliation in Java Operator SDK

Slide 61

Slide 61 text

@systemcraftsman Other Features of Java Operator SDK ● Finalizer Support ● Support for Well Known (non-custom) Kubernetes Resources ● Max Interval Between Reconciliations ● Automatic Retries on Error ● Reconciliation Rate Limiting ● Dynamically Changing Target Namespaces ● Automatic Generation of CRDs ● …and many more at https://javaoperatorsdk.io/docs/features

Slide 62

Slide 62 text

@systemcraftsman Quarkus Extension and the Operator SDK Plugin😎

Slide 63

Slide 63 text

@systemcraftsman Quarkus Operator SDK https://github.com/quarkiverse/quarkus-operator-sdk

Slide 64

Slide 64 text

@systemcraftsman Why Quarkus?

Slide 65

Slide 65 text

@systemcraftsman Quarkus Improves Memory Utilization

Slide 66

Slide 66 text

@systemcraftsman Quarkus Improves Startup Time

Slide 67

Slide 67 text

@systemcraftsman Quarkus can compete with Golang https://medium.com/swlh/cloud-native-java-vs-golang-2a72c0531b05

Slide 68

Slide 68 text

@systemcraftsman Quarkus Operator SDK Features ● Automatically generates a main class ● Provides CDI for the Kubernetes client ● Automatically generates CRDs for all CustomResource implementations used by reconcilers ● Provides a bundle generator ● All Quarkus features such as: ○ The Quarkus Dev mode ○ Native binary generation ○ CDI ○ All MicroProﬁle Spec beneﬁts (SmallRye is the implementation) ■ Health Check ■ Reactive Messaging ■ … ● For more: https://quarkiverse.github.io/quarkiverse-docs/quarkus-operator-sdk

Slide 69

Slide 69 text

@systemcraftsman Using the Quarkus Extension

Slide 70

Slide 70 text

@systemcraftsman Using the Quarkus Extension or Create a Quarkus project +

Slide 71

Slide 71 text

@systemcraftsman Operator SDK for Java ● Ansible ● Helm ● Go ● Java (Quarkus) Java Operator SDK (with Quarkus Extension)

Slide 72

Slide 72 text

@systemcraftsman Operator SDK Java Plugin https://github.com/operator-framework/java-operator-plugins

Slide 73

Slide 73 text

@systemcraftsman Operator SDK for Java $ operator-sdk init … … --plugins quarkus

Slide 74

Slide 74 text

@systemcraftsman Creating an API

Slide 75

Slide 75 text

@systemcraftsman The Generated Reconciler Class

Slide 76

Slide 76 text

@systemcraftsman Demo

Slide 77

Slide 77 text

@systemcraftsman Ready Player One

Slide 78

Slide 78 text

@systemcraftsman KubeGame: A Gamiﬁcation Operator oasis-postgres Oasis oasis-postgres ● Archaide ● Chthonia ● Incipio ● Middle-earth

Slide 79

Slide 79 text

@systemcraftsman Talk is Cheap, Let’s See the Demo!

Slide 80

Slide 80 text

@systemcraftsman Demo on GitHub

Slide 81

Slide 81 text

@systemcraftsman Who is Using the Java Operator SDK? ● Keycloak Operator - JOSDK+Quarkus - https://github.com/keycloak/keycloak ● Strimzi Access Operator - JOSDK - https://github.com/strimzi/kafka-access-operator ● Apache Flink Kubernetes Operator - JOSDK - https://github.com/apache/flink-kubernetes-operator ● Airflow Dag Operator - JOSDK+Quarkus - https://github.com/cdmikechen/airflow-dag-operator

Slide 82

Slide 82 text

@systemcraftsman Contact & Social ● Twitter: @systemcraftsman ● Email: aykut@systemcraftsman.com ● Linkedin: https://www.linkedin.com/in/mabulgu/ ● Website: https://www.systemcraftsman.com/

Slide 83

Slide 83 text

@systemcraftsman Thank You! www.SystemCraftsman.com