Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Developing Kubernetes Operators with Java Operator SDK

Developing Kubernetes Operators with Java Operator SDK

Developing Kubernetes Operators with Java Operator SDK - JavaDay Istanbul 2023

Aykut Bulgu

May 07, 2023
Tweet

More Decks by Aykut Bulgu

Other Decks in Technology

Transcript

  1. @systemcraftsman
    Developing Kubernetes Operators
    with
    Java Operator SDK
    Aykut M. Bulgu
    Principal Software Engineer @ Red Hat
    @SystemCraftsman

    View Slide

  2. @systemcraftsman
    Schedule
    ● What is Kubernetes?
    ○ Container Orchestration
    ○ Kubernetes Architecture
    ○ The Controller Pattern
    ● Extending Kubernetes
    ○ Controllers
    ○ Operators
    ● Developing Operators
    ○ Kubebuilder
    ○ Operator SDK
    ● Developing Operators with Java
    ○ Java Operator SDK
    ○ Quarkus Extension
    ● Demo

    View Slide

  3. @systemcraftsman
    About Me #oc apply -f aykutbulgu.yaml
    apiVersion: redhat/v5
    kind: Principal Software Engineer
    metadata:
    name: Aykut Bulgu
    namespace: Red Hat Kafka Engineering
    annotations:
    twitter: @systemcraftsman
    email:
    - [email protected]
    - [email protected]
    organizer: Software Craftsmanship Turkey
    founder: System Craftsman
    labels:
    married: yes
    children: 2
    interests:
    - tech (mostly kafka)
    - aikido
    - gamification
    - stoicism
    spec:
    replicas: 1
    containers:
    - image: aykut:latest

    View Slide

  4. @systemcraftsman
    What is Kubernetes?
    and why do we need it?

    View Slide

  5. @systemcraftsman
    Container Orchestration
    but…

    View Slide

  6. @systemcraftsman
    Container Orchestration
    $ docker build -t app:v1 .

    View Slide

  7. @systemcraftsman
    $ docker build -t app:v1 .
    $ docker run app:v1
    Container Orchestration

    View Slide

  8. @systemcraftsman
    ?
    Container Orchestration

    View Slide

  9. @systemcraftsman
    $ docker build -t app/frontend:v1 .
    $ docker build -t app/backend:v1 .
    $ docker build -t app/database:v1 .
    $ docker build -t app/cache:v1 .
    $ docker build -t app/messaging:v1 .
    Container Orchestration

    View Slide

  10. @systemcraftsman
    ?
    Container Orchestration

    View Slide

  11. @systemcraftsman
    $ docker run app/frontend:v1 link-to-backend
    $ docker run app/frontend:v1 link-to-backend
    $ docker run app/backend:v1 link-to-db-cache-messaging
    $ docker run app/backend:v1 link-to-db-cache-messaging
    $ docker run app/database:v1
    $ docker run app/cache:v1 link-to-db
    $ docker run app/messaging:v1
    Container Orchestration

    View Slide

  12. @systemcraftsman
    $ docker-compose up
    Container Orchestration

    View Slide

  13. @systemcraftsman
    We need more than just containers
    kubernetes
    ? Scheduling
    Decide where to deploy containers
    Lifecycle and health
    Keep containers running despite failures
    Discovery
    Find other containers on the network
    Monitoring
    Visibility into running containers
    Security
    Control who can do what
    Scaling
    Scale containers up and down
    Persistence
    Survive data beyond container lifecycle
    Aggregation
    Compose apps from multiple containers

    View Slide

  14. @systemcraftsman
    What is Kubernetes?
    Kubernetes is an open-source
    system for automating deployment,
    operations, and scaling of
    containerized applications across
    multiple hosts
    kubernetes

    View Slide

  15. @systemcraftsman
    Kubernetes Architecture
    and a quick look at the concepts.

    View Slide

  16. @systemcraftsman
    Kubernetes Architecture

    View Slide

  17. @systemcraftsman
    Declarative Model
    https://itnext.io/kubernetes-for-dummies-life-of-a-pod-fc8158e27aa
    YAML file
    Kubernetes

    View Slide

  18. @systemcraftsman
    Kubernetes Architecture
    https://itnext.io/kubernetes-for-dummies-life-of-a-pod-fc8158e27aa

    View Slide

  19. @systemcraftsman
    The Control Loop
    In robotics and automation, a control loop
    is a non-terminating loop that regulates
    the state of a system.
    In Kubernetes, controllers are control
    loops that watch the state of your cluster,
    then make or request changes where
    needed.

    View Slide

  20. @systemcraftsman
    The Controller Pattern

    View Slide

  21. @systemcraftsman
    The Controller Pattern
    Actual Desired Action
    2 replicas 3 replicas Scale up
    v1.0 v2.0 Update
    Object Deleted Delete
    Observe
    Analyze
    Act

    View Slide

  22. @systemcraftsman
    Extending Kubernetes

    View Slide

  23. @systemcraftsman
    Extending Kubernetes
    1. Kubectl plugins
    2. API Access extensions
    3. API extensions
    4. Scheduling extensions
    5. Controllers and Operators
    6. Network plugins
    7. Device and Storage plugins
    https://kubernetes.io/docs/concepts/extend-kubernetes/

    View Slide

  24. @systemcraftsman
    Controllers

    View Slide

  25. @systemcraftsman
    Controllers
    Observe
    Analyze
    Act
    Reconciliation Loop

    View Slide

  26. @systemcraftsman
    Controllers
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: nginx-deployment
    labels:
    app: nginx
    spec:
    replicas: 3
    selector:
    matchLabels:
    app: nginx
    template:
    metadata:
    labels:
    app: nginx
    spec:
    containers:
    - name: nginx
    image: nginx:stable-alpine
    ports:
    - containerPort: 80

    View Slide

  27. @systemcraftsman
    Developing Controllers

    View Slide

  28. @systemcraftsman
    Developing Controllers
    You can develop Controllers with any language, such as:
    To manage Kubernetes resources by using:
    ● Labels
    ● Annotations
    ● ConfigMaps

    View Slide

  29. @systemcraftsman
    A Controller Written in Bash
    apiVersion
    : apps/v1
    kind: Deployment
    ...
    name: config-watcher-controller
    ...
    spec:
    serviceAccountName
    : config-watcher-controller
    containers
    :
    - name: kubeapi-proxy
    image: k8spatterns/kubeapi-proxy
    - name: config-watcher
    image: k8spatterns/curl-jq
    env:
    - name: WATCH_NAMESPACE
    valueFrom:
    fieldRef:
    fieldPath: metadata.namespace
    command:
    - "sh"
    - "/watcher/config-watcher-controller.sh"
    volumeMounts
    :
    ...
    https://k8spatterns.io/

    View Slide

  30. @systemcraftsman
    Operators

    View Slide

  31. @systemcraftsman
    Operators
    Operator extends Controller
    ● Labels
    ● Annotations
    ● ConfigMaps
    Custom Resources

    View Slide

  32. @systemcraftsman
    Operators
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: nginx-deployment
    labels:
    app: nginx
    spec:
    replicas: 3
    selector:
    matchLabels:
    app: nginx
    template:
    metadata:
    labels:
    app: nginx
    spec:
    containers:
    - name: nginx
    image: nginx:stable-alpine
    ports:
    - containerPort: 80
    Prometheus
    YAML
    ?

    View Slide

  33. @systemcraftsman
    Custom Resource Definition
    https://k8spatterns.io/

    View Slide

  34. @systemcraftsman
    Operators’ Controller
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: nginx-deployment
    labels:
    app: nginx
    spec:
    replicas: 3
    selector:
    matchLabels:
    app: nginx
    template:
    metadata:
    labels:
    app: nginx
    spec:
    containers:
    - name: nginx
    image: nginx:stable-alpine
    ports:
    - containerPort: 80
    Prometheus
    YAML
    Prometheus
    Controller

    View Slide

  35. @systemcraftsman
    When to use which?
    https://k8spatterns.io/

    View Slide

  36. @systemcraftsman
    Developing Operators

    View Slide

  37. @systemcraftsman
    Operator Development Frameworks
    ● Kubebuilder
    ● Operator SDK
    ● Metacontroller
    ● Charmed Operator SDK
    ● KubeOps

    View Slide

  38. @systemcraftsman
    Kubebuilder

    View Slide

  39. @systemcraftsman
    Kubebuilder
    $ kubebuilder init …
    Kubebuilder is a framework for building Kubernetes APIs using custom resource definitions (CRDs).

    View Slide

  40. @systemcraftsman
    Kubebuilder is Also a Library
    https://github.com/kubernetes-sigs/kubebuilder

    View Slide

  41. @systemcraftsman
    Operator SDK

    View Slide

  42. @systemcraftsman
    Operator SDK
    ● Ansible
    ● Helm
    ● Go

    View Slide

  43. @systemcraftsman
    Operator SDK

    View Slide

  44. @systemcraftsman
    Operator SDK
    $ operator-sdk init …
    … --plugins helm
    … --plugins ansible

    View Slide

  45. @systemcraftsman
    Operator SDK
    ● Ansible
    ● Helm
    ● Go
    ● Java (Quarkus)
    Java Operator SDK
    (with Quarkus Extension)

    View Slide

  46. @systemcraftsman
    Developing Operators
    with Java

    View Slide

  47. @systemcraftsman
    Developing Operators with Java
    ● Fabric8 Kubernetes Client
    ● Java Operator SDK

    View Slide

  48. @systemcraftsman
    Fabric8 Kubernetes Client

    View Slide

  49. @systemcraftsman
    Fabric8 Kubernetes Client

    View Slide

  50. @systemcraftsman
    An Example: Strimzi Kafka Operator
    https://strimzi.io/

    View Slide

  51. @systemcraftsman
    An Example: Strimzi Kafka Operator
    https://strimzi.io/

    View Slide

  52. @systemcraftsman
    Java Operator SDK

    View Slide

  53. @systemcraftsman
    Java Operator SDK
    https://javaoperatorsdk.io/
    Java Operator SDK is a higher level framework and
    related tooling to support writing Kubernetes
    Operators in Java. It makes it easy to implement
    best practices and patterns for an Operator.
    It provides a controller runtime, support for testing
    operators, and related tooling. In addition to that
    implementing conversion hooks and dynamic
    admission controllers are supported as a separate
    projects.

    View Slide

  54. @systemcraftsman
    Java Operator SDK
    https://github.com/java-operator-sdk/java-operator-sdk
    ● Wraps Fabric8 and configures it for listening
    to changes on the specified Custom
    Resources, thus hiding the boilerplate code
    required for this.
    ● Provides a clean interface to implement the
    reconciliation loop for a particular resource
    type.
    ● Schedules change events to be executed in an
    efficient manner. Filtering obsolete events and
    executing unrelated events in parallel.
    ● Retry failed reconciliation attempts.

    View Slide

  55. @systemcraftsman
    Java Operator SDK
    https://blog.container-solutions.com/cloud-native-java-infrastructure-automation-with-kubernetes-operators

    View Slide

  56. @systemcraftsman
    Reconciliation in Java Operator SDK
    https://blog.container-solutions.com/kubernetes-operators-explained

    View Slide

  57. @systemcraftsman
    Reconciliation in Java Operator SDK
    # Reconciliation execution is always triggered by an event.
    # Events typically come from a primary resource, most of the time a custom
    resource. Reconciler implementations are associated with a given resource type.
    # No concurrent reconciliation happens for any given resource.
    # If an exception is thrown during execution, then it schedules a retry.
    # If new events are received during the controller execution, then a new
    reconciliation is scheduled.
    # If the reconcilier instructed the SDK to reschedule a reconciliation at a later
    date, then a timer event with the specified delay is scheduled.

    View Slide

  58. @systemcraftsman
    Reconciliation in Java Operator SDK
    Controller Level
    Eventing Level
    Reconciliation
    Level
    An Operator is a set of
    independent controllers. The
    Controller class, however, is an
    internal class managed by the
    framework itself and usually
    shouldn’t interacted with
    directly by end users. It
    manages all the processing
    units involved with reconciling
    a single type of Kubernetes
    resource.

    View Slide

  59. @systemcraftsman
    Reconciliation in Java Operator SDK
    public class MyCustomResourceReconciler implements
    Reconciler {
    }

    View Slide

  60. @systemcraftsman
    Reconciliation in Java Operator SDK

    View Slide

  61. @systemcraftsman
    Other Features of Java Operator SDK
    ● Finalizer Support
    ● Support for Well Known (non-custom) Kubernetes Resources
    ● Max Interval Between Reconciliations
    ● Automatic Retries on Error
    ● Reconciliation Rate Limiting
    ● Dynamically Changing Target Namespaces
    ● Automatic Generation of CRDs
    ● …and many more at https://javaoperatorsdk.io/docs/features

    View Slide

  62. @systemcraftsman
    Quarkus Extension
    and the Operator SDK Plugin😎

    View Slide

  63. @systemcraftsman
    Quarkus Operator SDK
    https://github.com/quarkiverse/quarkus-operator-sdk

    View Slide

  64. @systemcraftsman
    Why Quarkus?

    View Slide

  65. @systemcraftsman
    Quarkus Improves Memory Utilization

    View Slide

  66. @systemcraftsman
    Quarkus Improves Startup Time

    View Slide

  67. @systemcraftsman
    Quarkus can compete with Golang
    https://medium.com/swlh/cloud-native-java-vs-golang-2a72c0531b05

    View Slide

  68. @systemcraftsman
    Quarkus Operator SDK Features
    ● Automatically generates a main class
    ● Provides CDI for the Kubernetes client
    ● Automatically generates CRDs for all CustomResource implementations used by reconcilers
    ● Provides a bundle generator
    ● All Quarkus features such as:
    ○ The Quarkus Dev mode
    ○ Native binary generation
    ○ CDI
    ○ All MicroProfile Spec benefits (SmallRye is the implementation)
    ■ Health Check
    ■ Reactive Messaging
    ■ …
    ● For more: https://quarkiverse.github.io/quarkiverse-docs/quarkus-operator-sdk

    View Slide

  69. @systemcraftsman
    Using the Quarkus Extension

    View Slide

  70. @systemcraftsman
    Using the Quarkus Extension
    or
    Create a
    Quarkus project
    +

    View Slide

  71. @systemcraftsman
    Operator SDK for Java
    ● Ansible
    ● Helm
    ● Go
    ● Java (Quarkus)
    Java Operator SDK
    (with Quarkus Extension)

    View Slide

  72. @systemcraftsman
    Operator SDK Java Plugin
    https://github.com/operator-framework/java-operator-plugins

    View Slide

  73. @systemcraftsman
    Operator SDK for Java
    $ operator-sdk init …
    … --plugins quarkus

    View Slide

  74. @systemcraftsman
    Creating an API

    View Slide

  75. @systemcraftsman
    The Generated Reconciler Class

    View Slide

  76. @systemcraftsman
    Demo

    View Slide

  77. @systemcraftsman
    Ready Player One

    View Slide

  78. @systemcraftsman
    KubeGame: A Gamification Operator
    oasis-postgres
    Oasis
    oasis-postgres
    ● Archaide
    ● Chthonia
    ● Incipio
    ● Middle-earth

    View Slide

  79. @systemcraftsman
    Talk is Cheap,
    Let’s See the Demo!

    View Slide

  80. @systemcraftsman
    Demo on GitHub

    View Slide

  81. @systemcraftsman
    Who is Using the Java Operator SDK?
    ● Keycloak Operator - JOSDK+Quarkus - https://github.com/keycloak/keycloak
    ● Strimzi Access Operator - JOSDK - https://github.com/strimzi/kafka-access-operator
    ● Apache Flink Kubernetes Operator - JOSDK - https://github.com/apache/flink-kubernetes-operator
    ● Airflow Dag Operator - JOSDK+Quarkus - https://github.com/cdmikechen/airflow-dag-operator

    View Slide

  82. @systemcraftsman
    Contact & Social
    ● Twitter: @systemcraftsman
    ● Email: [email protected]
    ● Linkedin: https://www.linkedin.com/in/mabulgu/
    ● Website: https://www.systemcraftsman.com/

    View Slide

  83. @systemcraftsman
    Thank You!
    www.SystemCraftsman.com

    View Slide