Openshift Commons Briefing: Crossplane Providers and OLM Repackaging

Crossplane is a project that strives to bring cloud infrastructure, services, and applications closer to your Kubernetes cluster in order to create a hybrid control plane. This goal is primarily achieved through the use of providers, which are standalone controllers for a specific API group. The Crossplane project itself manages the lifecycle of these providers, from installation to cleanup. In this briefing, Red Hat's Krish Chowdhary will briefly discuss what providers are, the Crossplane architecture as well as how we can repackage a provider installation via the Operator Lifecycle Manager (OLM).

Transcript

1. Crossplane Providers and OLM Repackaging
   Openshift Commons
   Briefing
   Krish Chowdhary
   Red Hat
   1
   

   View Slide

2. 2
   - SWE @ Red Hat Office of the CTO
   - Provider-AWS Maintainer
   - Provider-In-Cluster Maintainer
   - Kubernetes SIG-Storage Member
   - Twitter: @krishchow_
   - Github: krishchow
   Krish Chowdhary
   

   View Slide

3. Intro
   - What is a Crossplane provider?
   - Where are providers?
   - When should I use or create a provider?
   - Why would I need to create a provider?
   - Who creates & maintains providers?
   - What are the main components?
   - OLM Repackaging
   3
   Agenda
   

   View Slide

4. 4
   What?
   

   View Slide

5. What is Crossplane
   5
   Let’s quickly go over Crossplane
   Crossplane allows you to
   manage the entire lifecycle,
   you can update and delete
   external resources
   Observing
   Managing
   Specific Providers expose
   information for the
   resources through
   Kubernetes fields on the CRs
   and Events.
   Provisioning
   The Crossplane project
   allows you to provision cloud
   resources from within your
   Kubernetes Cluster
   Context
   

   View Slide

6. Context
   “ Providers are packages that enable
   Crossplane to provision infrastructure on an
   external service. They bring CRDs (i.e.
   managed resources) that map one-to-one
   to external infrastructure resources, as well
   as controllers to manage the life-cycle of
   those resources. “
   6
   - Crossplane Docs
   

   View Slide

7. Context
   7
   ▸ Providers are similar to Operators, but you utilize the Crossplane Operator for
   installing and management
   ▸ Providers use a lot of the same tooling as Operators, such as kube-builder,
   controller-runtime and controller-tools. For this reason, it is commonly considered
   as an opinionated form of an Operator.
   ▸ The main conceptual difference is that providers are designed to reference some
   external* resource
   ▸ For a developer, the nice thing is that you don’t need to worry about RBAC,
   deployment and bootstrapping since tooling exists
   What are providers, really?
   * This is kind of a lie, providers can really manage any resource, and there are providers designed to handle in cluster resources as well!
   

   View Slide

8. 8
   Where?
   

   View Slide

9. Where are providers located?
   Crossplane providers are open source, and available on Github under the
   Crossplane or Crossplane-Contrib organizations. Some examples are:
   Provider-SQL - Repo
   Orchestrates SQL servers by creating Users, Grants, Roles, etc.
   Provider-Helm - Repo
   Manage and deploy Helm charts using custom resources
   Provider-AWS - Repo
   Provision resources on AWS such as databases (RDS) or k8s clusters (EKS)
   9
   Where
   

   View Slide

10. 10
    When?
    

    View Slide

11. When
    When does it make sense
    to create a provider?
    11
    When should a provider be
    created rather than using an
    existing one?
    Providers are designed to interface with external
    APIs, and manage the CRUD lifecycle for these
    resources.
    Consume External Resources
    There is an emphasis on high fidelity for
    managed resources, you should create a provider
    if your resources are well defined and granular.
    High Fidelity Resources
    If you want to use the powerful composition
    engine, the best way* is to create a provider
    which manages your resources
    Utilize Compositions
    * Technically you could utilize any resource within a composition, but the recommended approach is utilize a provider: via 1730
    

    View Slide

12. 12
    Why?
    

    View Slide

13. Crossplane is a CNCF Sandbox project, the
    project and many providers are open-source,
    anyone can contribute.
    ▸ Shared development and maintenance of
    common resources
    ▸ Cloud vendors can expose their APIs in
    Kubernetes through a common interface
    ▸ Streamlined development and consumption,
    Crossplane handles a lot of the messy parts.
    Why
    13
    Why should you create or contribute to a provider?
    

    View Slide

14. Context
    ▸ Proxy Issues: We will see this shortly, but integral to the Crossplane Operator is
    pulling an OCI image with metadata. This can cause issues with cluster running
    behind a proxy.
    ▸ Credential sharing: Similar to above, if you are utilizing a private container registry,
    you will need to separately supply credentials to Crossplane and the Kubelet.
    ▸ Resource limitations: You may not always want to install the Crossplane Operator
    for security or resource limitations. With OLM repackaging you can choose to just
    install the providers you need, and you can directly provision resources.
    14
    Why might we want to repackage a provider?
    

    View Slide

15. 15
    Who?
    

    View Slide

16. Who is creating and maintaining providers?
    Who
    16
    There are many contributors
    from organizations like
    Upbound, Red Hat, Squiz and
    Accenture
    Vendors
    Organizations
    Engineers from vendors like
    Alibaba, IBM, AWS and Equinix
    contribute to the
    development of their
    providers
    Community
    The Crossplane operator
    related providers are open
    source, many contributors are
    active members of the
    community
    

    View Slide

17. 17
    Provider & OLM
    Components
    

    View Slide

18. - One of the most
    popular Crossplane
    providers, with
    support for dozens
    of AWS resources
    - Also, a great
    example of provider
    to dissect :)
    18
    Components
    Provider Primer - AWS
    

    View Slide

19. - A common sample
    Operator used in
    guides and examples
    - Also, a great example of
    the standard structure
    of an OLM Operator
    19
    Components
    OLM Primer - Memcached
    

    View Slide

20. Components
    20
    Memcached Operator Structure
    Core Components
    ▸ api/ - contains the API definitions for the
    Memcached Operator
    Crossplane Provider Structure
    ▸ apis/ - analogous to the “api” folder in
    Memcached Operator.
    ▸ controllers/ - contains the logic and
    tests for the controllers/reconcilers (CRUD)
    ▸ pkg/ - controllers and clients used by the
    provider, similar to the “controller” folder.
    ▸ config/ + bundle/ - manifests for the
    deployment of the operator and the related
    RBAC, as well as CRDs from the API
    ▸ package/ + cluster/ - contain the
    Dockerfiles used, and the CRDs generated
    from the API
    

    View Slide

21. TL;DR - Mainly semantic
    differences at this point,
    however the last point is
    key.
    21
    Components
    

    View Slide

22. Components
    22
    Memcached Operator Structure
    Core Components
    ▸ api/ - contains the API definitions for the
    Memcached Operator
    Crossplane Provider Structure
    ▸ apis/ - analogous to the “api” folder in
    Memcached Operator.
    ▸ controllers/ - contains the logic and
    tests for the controllers/reconcilers (CRUD)
    ▸ pkg/ - controllers and clients used by the
    provider, similar to the “controller” folder.
    ▸ config/ + bundle/ - manifests for the
    deployment of the operator and the
    related RBAC, as well as CRDs from the API
    ▸ package/ + cluster/ - contain the
    Dockerfiles used, and the CRDs generated
    from the API
    How do I actually deploy a provider?
    There’s no Deployment,
    ClusterRole or anything else that I
    can use!
    

    View Slide

23. - One of the most
    popular ways to
    install a
    Crossplane
    provider
    23
    Aside on Deployment
    Provider CRD
    apiVersion: pkg.crossplane.io/v1
    kind: Provider
    metadata:
    name: provider-aws
    spec:
    package: crossplane/provider-aws:alpha
    

    View Slide

24. - Installed the
    same way as
    providers, but the
    configuration
    exposes XRDs
    - Users install
    providers
    through the use
    of dependencies.
    24
    Aside on Deployment
    Configuration CRD
    apiVersion: meta.pkg.crossplane.io/v1
    kind: Configuration
    metadata:
    name: my-favorite-configuration
    spec:
    crossplane:
    version: ">=v1.0.0"
    dependsOn:
    - provider: crossplane/provider-aws
    version: ">=v0.14.0"
    

    View Slide

25. 25
    The Delta
    

    View Slide

26. Where are our missing
    components related to
    deploying the provider?
    26
    Components
    

    View Slide

27. Components
    27
    1. https://github.com/tokibi/undocker
    - What does our package actually
    contain?
    - With a tool like undocker we can
    actually browse the contents, without
    digging through the build process1.
    - What we find is interesting, the image
    we reference contains only CRDs,
    metadata, and a reference to the
    actual controller within a YAML file
    The Provider Package
    $ docker pull crossplane/provider-aws:v0.18.1
    v0.18.1: Pulling from crossplane/provider-aws
    $ docker save crossplane/provider-aws:v0.18.1 |
    undocker -o provider-aws
    $ tree provider-aws
    provider-aws
    └── package.yaml
    $ tail provider-aws/package.yaml -n 143
    apiVersion: meta.pkg.crossplane.io/v1alpha1
    kind: Provider
    metadata:
    name: provider-aws
    ...
    spec:
    controller:
    image: crossplane/provider-aws-controller:v0.18.1
    

    View Slide

28. Components
    28
    For brevity, we are not covering the Crossplane Operator in detail, relevant code for this logic can be found here with the relevant
    controller focusing on the provider revisions
    It follows that there are two
    separate steps, with our
    metadata image, and then
    the actual controller.
    The rest of the puzzle
    The main Crossplane
    Operator parses the
    package.yaml, installs CRDs
    and configures RBAC for
    the provider at runtime
    The Crossplane Operator then
    also creates the deployment
    with the tag from
    spec.controller.image in
    the package.yaml
    

    View Slide

29. Components
    29
    For brevity, we are not covering the Crossplane Operator in detail, relevant code for this logic can be found here with the relevant
    controller focusing on the provider revisions
    It follows that there are two
    separate steps, with our
    metadata image, and then
    the actual controller.
    The rest of the puzzle
    The main Crossplane
    Operator parses the
    package.yaml, installs CRDs
    and configures RBAC for
    the provider at runtime
    The Crossplane Operator then
    also creates the deployment
    with the tag from
    spec.controller.image in
    the package.yaml
    This is notable because OLM opts to define RBAC and
    deployment manifests at build time
    

    View Slide

30. 30
    OLM
    Repackaging
    

    View Slide

31. - Dockerfile
    - Makefile
    - PROJECT.boilerplate.txt
    - config/
    - gen_rbac.sh - This script
    creates an `rbac.go` file, which
    contains kubebuilder
    annotations that generate the
    RBAC
    - gen_project.sh - This script
    generates our top-level
    PROJECT file (metadata)
    31
    Aside on Deployment
    Configuration CRD
    $ git clone git@github.com:redhat-et/olm-repackage.git
    $ tree olm-repackage
    .
    ├── Dockerfile
    ├── Makefile
    ├── PROJECT.boilerplate.txt
    ├── README.md
    ├── config/
    │ ├── certmanager/
    │ ├── crd/ # contains generated CRDs
    │ ├── default/
    │ ├── manager/ # contains deployment
    │ ├── manifests/ # ClusterServiceVersion (csv)
    │ ├── prometheus/
    │ ├── rbac/ # contains generated RBAC
    │ ├── samples/
    │ └── scorecard/
    ├── gen_project.sh
    └── gen_rbac.sh
    Again for brevity some resources in sub-folders have been omitted, the contents of the config directory
    mirrors contains all of the required manifests
    

    View Slide

32. Repackaging
    32
    Not included: Go-lang, Docker/Podman
    1. https://github.com/mikefarah/yq
    2. https://sdk.operatorframework.io/docs/building-operators/golang/installation/
    3. Kustomize and controller-gen are installed by targets in the Makefile
    Kustomize is a tool used to
    customize Kubernetes resources
    through configuration patches
    The Operator-SDK CLI tool is used
    to handle the generation and
    validation of deployment manifests
    Operator-SDK2
    controller-gen is a tool used to
    generate CRDs, RBAC and code
    through marker comments in Go
    controller-gen3
    yq is a tool similar to jq, except it
    allows you to query fields in a YAML
    document
    yq1
    Kustomize3
    Repackage Tooling
    

    View Slide

33. 33
    Setup all of our
    dependencies, these were
    all seen on the previous
    slide. Make sure to also to
    have credentials setup for a
    container registry (like
    Quay.io)
    Repackaging
    Provider-OLM Repackaging
    Part 1 - Setup
    Clone your target
    repository - for this
    example, we will be using
    the provider-aws
    Clone the olm-repackage
    repository, then copy the
    contents into the root of the
    provider repository.
    

    View Slide

34. Repackaging
    34
    

    View Slide

35. 35
    Lastly, we want to build our bundle. Although
    the newly created Operator has been
    pushed, we still need to create the bundle,
    which adds all of our manifests and metadata
    to the single package which is installed by the
    Operator Lifecycle Manager (OLM)
    Repackaging
    Provider-OLM Repackaging
    Part 2 - Build
    Add the appropriate environment variables,
    and build the Operator. This also runs all of
    our code gen, and afterwards we can push it
    up to our container registry.
    

    View Slide

36. Repackaging
    36
    

    View Slide

37. 37
    Running the Provider-AWS Operator
    

    View Slide

38. linkedin.com/company/red-hat
    youtube.com/user/RedHatVideos
    facebook.com/redhatinc
    twitter.com/RedHat
    38
    Linkedin: krishchow
    Twitter: @krishchow_
    Github: krishchow
    Or just ping me on Slack :)
    Thank you
    

    View Slide