Slide 1

Slide 1 text

Crossplane Providers and OLM Repackaging Openshift Commons Briefing Krish Chowdhary Red Hat 1

Slide 2

Slide 2 text

2 - SWE @ Red Hat Office of the CTO - Provider-AWS Maintainer - Provider-In-Cluster Maintainer - Kubernetes SIG-Storage Member - Twitter: @krishchow_ - Github: krishchow Krish Chowdhary

Slide 3

Slide 3 text

Intro - What is a Crossplane provider? - Where are providers? - When should I use or create a provider? - Why would I need to create a provider? - Who creates & maintains providers? - What are the main components? - OLM Repackaging 3 Agenda

Slide 4

Slide 4 text

4 What?

Slide 5

Slide 5 text

What is Crossplane 5 Let’s quickly go over Crossplane Crossplane allows you to manage the entire lifecycle, you can update and delete external resources Observing Managing Specific Providers expose information for the resources through Kubernetes fields on the CRs and Events. Provisioning The Crossplane project allows you to provision cloud resources from within your Kubernetes Cluster Context

Slide 6

Slide 6 text

Context “ Providers are packages that enable Crossplane to provision infrastructure on an external service. They bring CRDs (i.e. managed resources) that map one-to-one to external infrastructure resources, as well as controllers to manage the life-cycle of those resources. “ 6 - Crossplane Docs

Slide 7

Slide 7 text

Context 7 ▸ Providers are similar to Operators, but you utilize the Crossplane Operator for installing and management ▸ Providers use a lot of the same tooling as Operators, such as kube-builder, controller-runtime and controller-tools. For this reason, it is commonly considered as an opinionated form of an Operator. ▸ The main conceptual difference is that providers are designed to reference some external* resource ▸ For a developer, the nice thing is that you don’t need to worry about RBAC, deployment and bootstrapping since tooling exists What are providers, really? * This is kind of a lie, providers can really manage any resource, and there are providers designed to handle in cluster resources as well!

Slide 8

Slide 8 text

8 Where?

Slide 9

Slide 9 text

Where are providers located? Crossplane providers are open source, and available on Github under the Crossplane or Crossplane-Contrib organizations. Some examples are: Provider-SQL - Repo Orchestrates SQL servers by creating Users, Grants, Roles, etc. Provider-Helm - Repo Manage and deploy Helm charts using custom resources Provider-AWS - Repo Provision resources on AWS such as databases (RDS) or k8s clusters (EKS) 9 Where

Slide 10

Slide 10 text

10 When?

Slide 11

Slide 11 text

When When does it make sense to create a provider? 11 When should a provider be created rather than using an existing one? Providers are designed to interface with external APIs, and manage the CRUD lifecycle for these resources. Consume External Resources There is an emphasis on high fidelity for managed resources, you should create a provider if your resources are well defined and granular. High Fidelity Resources If you want to use the powerful composition engine, the best way* is to create a provider which manages your resources Utilize Compositions * Technically you could utilize any resource within a composition, but the recommended approach is utilize a provider: via 1730

Slide 12

Slide 12 text

12 Why?

Slide 13

Slide 13 text

Crossplane is a CNCF Sandbox project, the project and many providers are open-source, anyone can contribute. ▸ Shared development and maintenance of common resources ▸ Cloud vendors can expose their APIs in Kubernetes through a common interface ▸ Streamlined development and consumption, Crossplane handles a lot of the messy parts. Why 13 Why should you create or contribute to a provider?

Slide 14

Slide 14 text

Context ▸ Proxy Issues: We will see this shortly, but integral to the Crossplane Operator is pulling an OCI image with metadata. This can cause issues with cluster running behind a proxy. ▸ Credential sharing: Similar to above, if you are utilizing a private container registry, you will need to separately supply credentials to Crossplane and the Kubelet. ▸ Resource limitations: You may not always want to install the Crossplane Operator for security or resource limitations. With OLM repackaging you can choose to just install the providers you need, and you can directly provision resources. 14 Why might we want to repackage a provider?

Slide 15

Slide 15 text

15 Who?

Slide 16

Slide 16 text

Who is creating and maintaining providers? Who 16 There are many contributors from organizations like Upbound, Red Hat, Squiz and Accenture Vendors Organizations Engineers from vendors like Alibaba, IBM, AWS and Equinix contribute to the development of their providers Community The Crossplane operator related providers are open source, many contributors are active members of the community

Slide 17

Slide 17 text

17 Provider & OLM Components

Slide 18

Slide 18 text

- One of the most popular Crossplane providers, with support for dozens of AWS resources - Also, a great example of provider to dissect :) 18 Components Provider Primer - AWS

Slide 19

Slide 19 text

- A common sample Operator used in guides and examples - Also, a great example of the standard structure of an OLM Operator 19 Components OLM Primer - Memcached

Slide 20

Slide 20 text

Components 20 Memcached Operator Structure Core Components ▸ api/ - contains the API definitions for the Memcached Operator Crossplane Provider Structure ▸ apis/ - analogous to the “api” folder in Memcached Operator. ▸ controllers/ - contains the logic and tests for the controllers/reconcilers (CRUD) ▸ pkg/ - controllers and clients used by the provider, similar to the “controller” folder. ▸ config/ + bundle/ - manifests for the deployment of the operator and the related RBAC, as well as CRDs from the API ▸ package/ + cluster/ - contain the Dockerfiles used, and the CRDs generated from the API

Slide 21

Slide 21 text

TL;DR - Mainly semantic differences at this point, however the last point is key. 21 Components

Slide 22

Slide 22 text

Components 22 Memcached Operator Structure Core Components ▸ api/ - contains the API definitions for the Memcached Operator Crossplane Provider Structure ▸ apis/ - analogous to the “api” folder in Memcached Operator. ▸ controllers/ - contains the logic and tests for the controllers/reconcilers (CRUD) ▸ pkg/ - controllers and clients used by the provider, similar to the “controller” folder. ▸ config/ + bundle/ - manifests for the deployment of the operator and the related RBAC, as well as CRDs from the API ▸ package/ + cluster/ - contain the Dockerfiles used, and the CRDs generated from the API How do I actually deploy a provider? There’s no Deployment, ClusterRole or anything else that I can use!

Slide 23

Slide 23 text

- One of the most popular ways to install a Crossplane provider 23 Aside on Deployment Provider CRD apiVersion: pkg.crossplane.io/v1 kind: Provider metadata: name: provider-aws spec: package: crossplane/provider-aws:alpha

Slide 24

Slide 24 text

- Installed the same way as providers, but the configuration exposes XRDs - Users install providers through the use of dependencies. 24 Aside on Deployment Configuration CRD apiVersion: meta.pkg.crossplane.io/v1 kind: Configuration metadata: name: my-favorite-configuration spec: crossplane: version: ">=v1.0.0" dependsOn: - provider: crossplane/provider-aws version: ">=v0.14.0"

Slide 25

Slide 25 text

25 The Delta

Slide 26

Slide 26 text

Where are our missing components related to deploying the provider? 26 Components

Slide 27

Slide 27 text

Components 27 1. https://github.com/tokibi/undocker - What does our package actually contain? - With a tool like undocker we can actually browse the contents, without digging through the build process1. - What we find is interesting, the image we reference contains only CRDs, metadata, and a reference to the actual controller within a YAML file The Provider Package $ docker pull crossplane/provider-aws:v0.18.1 v0.18.1: Pulling from crossplane/provider-aws $ docker save crossplane/provider-aws:v0.18.1 | undocker -o provider-aws $ tree provider-aws provider-aws └── package.yaml $ tail provider-aws/package.yaml -n 143 apiVersion: meta.pkg.crossplane.io/v1alpha1 kind: Provider metadata: name: provider-aws ... spec: controller: image: crossplane/provider-aws-controller:v0.18.1

Slide 28

Slide 28 text

Components 28 For brevity, we are not covering the Crossplane Operator in detail, relevant code for this logic can be found here with the relevant controller focusing on the provider revisions It follows that there are two separate steps, with our metadata image, and then the actual controller. The rest of the puzzle The main Crossplane Operator parses the package.yaml, installs CRDs and configures RBAC for the provider at runtime The Crossplane Operator then also creates the deployment with the tag from spec.controller.image in the package.yaml

Slide 29

Slide 29 text

Components 29 For brevity, we are not covering the Crossplane Operator in detail, relevant code for this logic can be found here with the relevant controller focusing on the provider revisions It follows that there are two separate steps, with our metadata image, and then the actual controller. The rest of the puzzle The main Crossplane Operator parses the package.yaml, installs CRDs and configures RBAC for the provider at runtime The Crossplane Operator then also creates the deployment with the tag from spec.controller.image in the package.yaml This is notable because OLM opts to define RBAC and deployment manifests at build time

Slide 30

Slide 30 text

30 OLM Repackaging

Slide 31

Slide 31 text

- Dockerfile - Makefile - PROJECT.boilerplate.txt - config/ - gen_rbac.sh - This script creates an `rbac.go` file, which contains kubebuilder annotations that generate the RBAC - gen_project.sh - This script generates our top-level PROJECT file (metadata) 31 Aside on Deployment Configuration CRD $ git clone [email protected]:redhat-et/olm-repackage.git $ tree olm-repackage . ├── Dockerfile ├── Makefile ├── PROJECT.boilerplate.txt ├── README.md ├── config/ │ ├── certmanager/ │ ├── crd/ # contains generated CRDs │ ├── default/ │ ├── manager/ # contains deployment │ ├── manifests/ # ClusterServiceVersion (csv) │ ├── prometheus/ │ ├── rbac/ # contains generated RBAC │ ├── samples/ │ └── scorecard/ ├── gen_project.sh └── gen_rbac.sh Again for brevity some resources in sub-folders have been omitted, the contents of the config directory mirrors contains all of the required manifests

Slide 32

Slide 32 text

Repackaging 32 Not included: Go-lang, Docker/Podman 1. https://github.com/mikefarah/yq 2. https://sdk.operatorframework.io/docs/building-operators/golang/installation/ 3. Kustomize and controller-gen are installed by targets in the Makefile Kustomize is a tool used to customize Kubernetes resources through configuration patches The Operator-SDK CLI tool is used to handle the generation and validation of deployment manifests Operator-SDK2 controller-gen is a tool used to generate CRDs, RBAC and code through marker comments in Go controller-gen3 yq is a tool similar to jq, except it allows you to query fields in a YAML document yq1 Kustomize3 Repackage Tooling

Slide 33

Slide 33 text

33 Setup all of our dependencies, these were all seen on the previous slide. Make sure to also to have credentials setup for a container registry (like Quay.io) Repackaging Provider-OLM Repackaging Part 1 - Setup Clone your target repository - for this example, we will be using the provider-aws Clone the olm-repackage repository, then copy the contents into the root of the provider repository.

Slide 34

Slide 34 text

Repackaging 34

Slide 35

Slide 35 text

35 Lastly, we want to build our bundle. Although the newly created Operator has been pushed, we still need to create the bundle, which adds all of our manifests and metadata to the single package which is installed by the Operator Lifecycle Manager (OLM) Repackaging Provider-OLM Repackaging Part 2 - Build Add the appropriate environment variables, and build the Operator. This also runs all of our code gen, and afterwards we can push it up to our container registry.

Slide 36

Slide 36 text

Repackaging 36

Slide 37

Slide 37 text

37 Running the Provider-AWS Operator

Slide 38

Slide 38 text

linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHat 38 Linkedin: krishchow Twitter: @krishchow_ Github: krishchow Or just ping me on Slack :) Thank you