Slide 1

Slide 1 text

KUBERNETES OPERATORS Roland Huß, Red Hat, @ro14nd DevOpsGathering - Bochum - 2019-03-13

Slide 2

Slide 2 text

KUBERNETES OPERATORS // @RO14ND Kubernetes 2

Slide 3

Slide 3 text

KUBERNETES OPERATORS // @RO14ND Kubernetes Controllers 3 ● Kubernetes: Declaratige orchestration platform ● Based on resource objects for declaring target state ● Reconciliation: ○ Observe current state ○ Analyse and compare against declared state ○ Bring current state closer to declared state Observe - Analyse - Act

Slide 4

Slide 4 text

KUBERNETES OPERATORS // @RO14ND Observe - Analyze - Act 4

Slide 5

Slide 5 text

KUBERNETES OPERATORS // @RO14ND Custom Controller 5 ● Watches Kubernetes Resources ● Enhances platform behaviour or introduces new feautures ● Regular Kubernetes application (Deployments, Pod) ● Running permanently in the background ● Common Custom Controller triggers: ○ Labels ○ Annotations ○ ConfigMaps

Slide 6

Slide 6 text

KUBERNETES OPERATORS // @RO14ND Operators 6 ● Operator IS-A Controller ● Custom domain modelled as Custom Resource Definition ● Semantically: Operator automates operational tasks and is codified knowledge ● Technically: Operator = Controller + CRD ● CoreOS pioneered the Operator movement An Operator is a Kubernetes Controller that understands two domains: Kubernetes and something else. By combining knowledge of both areas, it can automate tasks that usually require a human operator that understands both domains. — Jimmy Zelinskie https://github.com/kubeflow/tf-operator/issues/300#issuecomment-357527937

Slide 7

Slide 7 text

KUBERNETES OPERATORS // @RO14ND Controller Operator Spectrum 7

Slide 8

Slide 8 text

KUBERNETES OPERATORS // @RO14ND Custom Resource Definition 8 apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: prometheuses.monitoring.coreos.com spec: group: monitoring.coreos.com names: kind: Prometheus plural: prometheuses scope: Namespaced version: v1 validation: openAPIV3Schema: ....

Slide 9

Slide 9 text

KUBERNETES OPERATORS // @RO14ND Custom Resource 9 apiVersion: monitoring.coreos.com/v1 kind: Prometheus metadata: name: prometheus spec: serviceMonitorSelector: matchLabels: team: frontend resources: requests: memory: 400Mi status: phase: Installed

Slide 10

Slide 10 text

KUBERNETES OPERATORS // @RO14ND CRD Classification 10 ● Installation CRDs ○ Installing and operating applications ○ Backup and Restore ○ Monitoring and self-healing ○ Example: Prometheus for installing Prometheus & components ● Application CRDs ○ Application specific domain concepts ○ Example: ServiceMonitor for registering Kubernetes service to be scraped by Prometheus

Slide 11

Slide 11 text

KUBERNETES OPERATORS // @RO14ND KUBERNETES PATTERNS 11 https://kubernetes-patterns.io

Slide 12

Slide 12 text

KUBERNETES OPERATORS // @RO14ND Operator Installation Options 12 One Operator per Namespace One Operator watching all Namespaces system namespaces tenant namespaces MongoDB object MongoDB object MongoDB object MongoDB object Operator system namespaces tenant namespaces object object object object

Slide 13

Slide 13 text

KUBERNETES OPERATORS // @RO14ND https://github.com/operator-framework/awesome-operators 13

Slide 14

Slide 14 text

KUBERNETES OPERATORS // @RO14ND 14

Slide 15

Slide 15 text

KUBERNETES OPERATORS // @RO14ND 15

Slide 16

Slide 16 text

KUBERNETES OPERATORS // @RO14ND Operator Development 16 ● Operator can be implemented in any language ● Frameworks: ○ Operator Framework (Golang, Helm, Ansible) ■ https://github.com/operator-framework ○ Kubebuilder (Golang) ■ https://github.com/kubernetes-sigs/kubebuilder ○ Metacontroller (Language agnostic) ■ https://metacontroller.app/ ○ jvm-operators (Java, Groovy, Kotlin, ….) ■ https://github.com/jvm-operators

Slide 17

Slide 17 text

KUBERNETES OPERATORS // @RO14ND Kubebuilder 17 ● https://github.com/kubernetes-sigs/kubebuilder ● Scaffolding for Golang based Operators ● Mutliple CRDs within one project ● Works directly with Kubernetes API

Slide 18

Slide 18 text

KUBERNETES OPERATORS // @RO14ND Metacontroller 18 ● https://metacontroller.app/ ●

Slide 19

Slide 19 text

KUBERNETES OPERATORS // @RO14ND jvm-controllers 19 ● https://github.com/jvm-operators ● Toolkit for creating Kubernetes and OpenShift in JVM languages ● Supports CRD and ConfigMaps for lifecycle management ● Callback based API called on CRD lifecyle events ● Plan: Switching to a Quarkus based architecture for creating native operators

Slide 20

Slide 20 text

KUBERNETES OPERATORS // @RO14ND Operator Framework 20 For Builders: ● Easily create operators on Kubernetes via a common method ● Provide standardized set of tools to build consistent operators For Consumers: ● Consume of cloud-native applications more secure and easier ● Keep installed operators up to date for security reasons and app lifecycle management

Slide 21

Slide 21 text

KUBERNETES OPERATORS // @RO14ND Operator Framework 21 https://github.com/operator-framework Build Operators Install, update, and manage Operators Operator usage reporting

Slide 22

Slide 22 text

KUBERNETES OPERATORS // @RO14ND Operator SDK 22 ● Framework and Tookit for creating Operators ● Skaffolding of a project skeleton ● Scorecard check ● Uses controller-runtime under the hood ○ https://github.com/kubernetes-sigs/controller-runtime ● Modes ○ Golang ○ Ansible ○ Helm

Slide 23

Slide 23 text

KUBERNETES OPERATORS // @RO14ND Operator Maturity Model 23 Phase I Phase II Phase III Phase IV Phase V Basic Install Automated application provisioning and configuration management Seamless Upgrades Patch and minor version upgrades supported Full Lifecycle App lifecycle, storage lifecycle (backup, failure recovery) Deep Insights Metrics, alerts, log processing and workload analysis Auto Pilot Horizontal/vertical scaling, auto config tuning, abnormal detection, scheduling tuning

Slide 24

Slide 24 text

KUBERNETES OPERATORS // @RO14ND DEMO 24

Slide 25

Slide 25 text

KUBERNETES OPERATORS // @RO14ND Operation Lifecycle Manager (OLM) 25 ● OLM: Operators for managing lifecycle or Operators ● “Operator Operator” ● Main components: ○ olm-operator: Managing operator deployments ○ catalog-operator: Managing subscriptions to channels ○ OperatorGroups for implementing multi-tenancy of operartors ● Installs packages from OperatorHub.io ● Included OOTB in OpenShift 4 ● Main CRD: ClusterServiceVersion

Slide 26

Slide 26 text

KUBERNETES OPERATORS // @RO14ND ClusterServiceVersion 26 ● Metadata (name, description, version, links, labels, icon, ...) ● Operator Installation ○ Type: Deployment ■ Set of service accounts / required permissions ■ Set of deployments ● CRDs ○ Type ○ Owned - managed by this service ○ Required - must exist in the cluster for this service to run ○ Resources - a list of resources that the Operator interacts with ○ Descriptors - annotate CRD spec and status fields to provide semantic information

Slide 27

Slide 27 text

KUBERNETES OPERATORS // @RO14ND Operator Framework Summary 27 OPERATOR SDK KUBERNETES OPERATOR + = + OPERATOR LIFECYCLE MANAGER + OPERATOR METERING DEVELOPER ADMINISTRATOR USER “create new operator” scaffolding custom logic metadata package “make available to” “list packages” “subscribe to channel” subscription namespace operator instance managed application “collect metrics” “Create application”

Slide 28

Slide 28 text

THANK YOU @ro14nd