Developing Kubernetes Operators with Java Operator SDK

@systemcraftsman Developing Kubernetes Operators with Java Operator SDK Aykut M.
Bulgu Principal Software Engineer @ Red Hat @SystemCraftsman

@systemcraftsman Schedule • What is Kubernetes? ◦ Container Orchestration ◦
Kubernetes Architecture ◦ The Controller Pattern • Extending Kubernetes ◦ Controllers ◦ Operators • Developing Operators ◦ Kubebuilder ◦ Operator SDK • Developing Operators with Java ◦ Java Operator SDK ◦ Quarkus Extension • Demo

@systemcraftsman About Me #oc apply -f aykutbulgu.yaml apiVersion: redhat/v5 kind:
Principal Software Engineer metadata: name: Aykut Bulgu namespace: Red Hat Kafka Engineering annotations: twitter: @systemcraftsman email: - [email protected] - [email protected] organizer: Software Craftsmanship Turkey founder: System Craftsman labels: married: yes children: 2 interests: - tech (mostly kafka) - aikido - gamification - stoicism spec: replicas: 1 containers: - image: aykut:latest

@systemcraftsman What is Kubernetes? and why do we need it?

@systemcraftsman Container Orchestration but…

@systemcraftsman Container Orchestration $ docker build -t app:v1 .

@systemcraftsman $ docker build -t app:v1 . $ docker run
app:v1 Container Orchestration

@systemcraftsman ? Container Orchestration

@systemcraftsman $ docker build -t app/frontend:v1 . $ docker build
-t app/backend:v1 . $ docker build -t app/database:v1 . $ docker build -t app/cache:v1 . $ docker build -t app/messaging:v1 . Container Orchestration

@systemcraftsman ? Container Orchestration

@systemcraftsman $ docker run app/frontend:v1 link-to-backend $ docker run app/frontend:v1
link-to-backend $ docker run app/backend:v1 link-to-db-cache-messaging $ docker run app/backend:v1 link-to-db-cache-messaging $ docker run app/database:v1 $ docker run app/cache:v1 link-to-db $ docker run app/messaging:v1 Container Orchestration

@systemcraftsman $ docker-compose up Container Orchestration

@systemcraftsman We need more than just containers kubernetes ? Scheduling
Decide where to deploy containers Lifecycle and health Keep containers running despite failures Discovery Find other containers on the network Monitoring Visibility into running containers Security Control who can do what Scaling Scale containers up and down Persistence Survive data beyond container lifecycle Aggregation Compose apps from multiple containers

@systemcraftsman What is Kubernetes? Kubernetes is an open-source system for
automating deployment, operations, and scaling of containerized applications across multiple hosts kubernetes

@systemcraftsman Kubernetes Architecture and a quick look at the concepts.

@systemcraftsman Kubernetes Architecture

@systemcraftsman Declarative Model https://itnext.io/kubernetes-for-dummies-life-of-a-pod-fc8158e27aa YAML ﬁle Kubernetes

@systemcraftsman Kubernetes Architecture https://itnext.io/kubernetes-for-dummies-life-of-a-pod-fc8158e27aa

@systemcraftsman The Control Loop In robotics and automation, a control
loop is a non-terminating loop that regulates the state of a system. In Kubernetes, controllers are control loops that watch the state of your cluster, then make or request changes where needed.

@systemcraftsman The Controller Pattern

@systemcraftsman The Controller Pattern Actual Desired Action 2 replicas 3
replicas Scale up v1.0 v2.0 Update Object Deleted Delete Observe Analyze Act

@systemcraftsman Extending Kubernetes

@systemcraftsman Extending Kubernetes 1. Kubectl plugins 2. API Access extensions
3. API extensions 4. Scheduling extensions 5. Controllers and Operators 6. Network plugins 7. Device and Storage plugins https://kubernetes.io/docs/concepts/extend-kubernetes/

@systemcraftsman Controllers

@systemcraftsman Controllers Observe Analyze Act Reconciliation Loop

@systemcraftsman Controllers apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels:
app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:stable-alpine ports: - containerPort: 80

@systemcraftsman Developing Controllers

@systemcraftsman Developing Controllers You can develop Controllers with any language,
such as: To manage Kubernetes resources by using: • Labels • Annotations • ConﬁgMaps

@systemcraftsman A Controller Written in Bash apiVersion : apps/v1 kind:
Deployment ... name: config-watcher-controller ... spec: serviceAccountName : config-watcher-controller containers : - name: kubeapi-proxy image: k8spatterns/kubeapi-proxy - name: config-watcher image: k8spatterns/curl-jq env: - name: WATCH_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace command: - "sh" - "/watcher/config-watcher-controller.sh" volumeMounts : ... https://k8spatterns.io/

@systemcraftsman Operators

@systemcraftsman Operators Operator extends Controller • Labels • Annotations •
ConﬁgMaps Custom Resources

@systemcraftsman Operators apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels:
app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:stable-alpine ports: - containerPort: 80 Prometheus YAML ?

@systemcraftsman Custom Resource Deﬁnition https://k8spatterns.io/

@systemcraftsman Operators’ Controller apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment
labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:stable-alpine ports: - containerPort: 80 Prometheus YAML Prometheus Controller

@systemcraftsman When to use which? https://k8spatterns.io/

@systemcraftsman Developing Operators

@systemcraftsman Operator Development Frameworks • Kubebuilder • Operator SDK •
Metacontroller • Charmed Operator SDK • KubeOps

@systemcraftsman Kubebuilder

@systemcraftsman Kubebuilder $ kubebuilder init … Kubebuilder is a framework
for building Kubernetes APIs using custom resource deﬁnitions (CRDs).

@systemcraftsman Kubebuilder is Also a Library https://github.com/kubernetes-sigs/kubebuilder

@systemcraftsman Operator SDK

@systemcraftsman Operator SDK • Ansible • Helm • Go

@systemcraftsman Operator SDK

@systemcraftsman Operator SDK $ operator-sdk init … … --plugins helm
… --plugins ansible

@systemcraftsman Operator SDK • Ansible • Helm • Go •
Java (Quarkus) Java Operator SDK (with Quarkus Extension)

@systemcraftsman Developing Operators with Java

@systemcraftsman Developing Operators with Java • Fabric8 Kubernetes Client •
Java Operator SDK

@systemcraftsman Fabric8 Kubernetes Client

@systemcraftsman An Example: Strimzi Kafka Operator https://strimzi.io/

@systemcraftsman Java Operator SDK

@systemcraftsman Java Operator SDK https://javaoperatorsdk.io/ Java Operator SDK is a
higher level framework and related tooling to support writing Kubernetes Operators in Java. It makes it easy to implement best practices and patterns for an Operator. It provides a controller runtime, support for testing operators, and related tooling. In addition to that implementing conversion hooks and dynamic admission controllers are supported as a separate projects.

@systemcraftsman Java Operator SDK https://github.com/java-operator-sdk/java-operator-sdk • Wraps Fabric8 and conﬁgures
it for listening to changes on the speciﬁed Custom Resources, thus hiding the boilerplate code required for this. • Provides a clean interface to implement the reconciliation loop for a particular resource type. • Schedules change events to be executed in an efficient manner. Filtering obsolete events and executing unrelated events in parallel. • Retry failed reconciliation attempts.

@systemcraftsman Java Operator SDK https://blog.container-solutions.com/cloud-native-java-infrastructure-automation-with-kubernetes-operators

@systemcraftsman Reconciliation in Java Operator SDK https://blog.container-solutions.com/kubernetes-operators-explained

@systemcraftsman Reconciliation in Java Operator SDK # Reconciliation execution is
always triggered by an event. # Events typically come from a primary resource, most of the time a custom resource. Reconciler implementations are associated with a given resource type. # No concurrent reconciliation happens for any given resource. # If an exception is thrown during execution, then it schedules a retry. # If new events are received during the controller execution, then a new reconciliation is scheduled. # If the reconcilier instructed the SDK to reschedule a reconciliation at a later date, then a timer event with the speciﬁed delay is scheduled.

@systemcraftsman Reconciliation in Java Operator SDK Controller Level Eventing Level
Reconciliation Level An Operator is a set of independent controllers. The Controller class, however, is an internal class managed by the framework itself and usually shouldn’t interacted with directly by end users. It manages all the processing units involved with reconciling a single type of Kubernetes resource.

@systemcraftsman Reconciliation in Java Operator SDK public class MyCustomResourceReconciler implements
Reconciler<MyCustomResource> { }

@systemcraftsman Reconciliation in Java Operator SDK

@systemcraftsman Other Features of Java Operator SDK • Finalizer Support
• Support for Well Known (non-custom) Kubernetes Resources • Max Interval Between Reconciliations • Automatic Retries on Error • Reconciliation Rate Limiting • Dynamically Changing Target Namespaces • Automatic Generation of CRDs • …and many more at https://javaoperatorsdk.io/docs/features

@systemcraftsman Quarkus Extension and the Operator SDK Plugin😎

@systemcraftsman Quarkus Operator SDK https://github.com/quarkiverse/quarkus-operator-sdk

@systemcraftsman Why Quarkus?

@systemcraftsman Quarkus Improves Memory Utilization

@systemcraftsman Quarkus Improves Startup Time

@systemcraftsman Quarkus can compete with Golang https://medium.com/swlh/cloud-native-java-vs-golang-2a72c0531b05

@systemcraftsman Quarkus Operator SDK Features • Automatically generates a main
class • Provides CDI for the Kubernetes client • Automatically generates CRDs for all CustomResource implementations used by reconcilers • Provides a bundle generator • All Quarkus features such as: ◦ The Quarkus Dev mode ◦ Native binary generation ◦ CDI ◦ All MicroProﬁle Spec beneﬁts (SmallRye is the implementation) ▪ Health Check ▪ Reactive Messaging ▪ … • For more: https://quarkiverse.github.io/quarkiverse-docs/quarkus-operator-sdk

@systemcraftsman Using the Quarkus Extension

@systemcraftsman Using the Quarkus Extension or Create a Quarkus project
+

@systemcraftsman Operator SDK for Java • Ansible • Helm •
Go • Java (Quarkus) Java Operator SDK (with Quarkus Extension)

@systemcraftsman Operator SDK Java Plugin https://github.com/operator-framework/java-operator-plugins

@systemcraftsman Operator SDK for Java $ operator-sdk init … …
--plugins quarkus

@systemcraftsman Creating an API

@systemcraftsman The Generated Reconciler Class

@systemcraftsman Demo

@systemcraftsman Ready Player One

@systemcraftsman KubeGame: A Gamiﬁcation Operator oasis-postgres Oasis oasis-postgres • Archaide
• Chthonia • Incipio • Middle-earth

@systemcraftsman Talk is Cheap, Let’s See the Demo!

@systemcraftsman Demo on GitHub

@systemcraftsman Who is Using the Java Operator SDK? • Keycloak
Operator - JOSDK+Quarkus - https://github.com/keycloak/keycloak • Strimzi Access Operator - JOSDK - https://github.com/strimzi/kafka-access-operator • Apache Flink Kubernetes Operator - JOSDK - https://github.com/apache/flink-kubernetes-operator • Airflow Dag Operator - JOSDK+Quarkus - https://github.com/cdmikechen/airflow-dag-operator

@systemcraftsman Contact & Social • Twitter: @systemcraftsman • Email: [email protected]
• Linkedin: https://www.linkedin.com/in/mabulgu/ • Website: https://www.systemcraftsman.com/

@systemcraftsman Thank You! www.SystemCraftsman.com

Developing Kubernetes Operators with Java Opera...

Developing Kubernetes Operators with Java Operator SDK

More Decks by Aykut Bulgu

Other Decks in Technology

Featured

Transcript