Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to Crossplane at VSHN

Introduction to Crossplane at VSHN

VSHN and I have been a fan of Crossplane since its very early days in 2019. Since then the project has matured a lot and is now used in production by VSHN and many others. In mid 2020 Crossplane became a CNCF Sandbox project and lately applied to be promoted to be a CNCF Incubation project. It's time for an introduction to Crossplane, why it matters to VSHN and talk about our production usage.

The talk can be viewed here: https://youtu.be/68RCw4ECVpI
A longer blog post is available here: https://www.vshn.ch/blog/crossplane-the-control-plane-of-the-future

Tobias Brunner

April 29, 2021
Tweet

More Decks by Tobias Brunner

Other Decks in Technology

Transcript

  1. VSHN – The DevOps Company
    Tobias Brunner
    Introduction to
    Crossplane
    Hi, I’m Tobias.
    In the next minutes I’ll introduce Crossplane to you and
    will show you around. At the end of this talk you will
    know what it’s all about and how Crossplane could help
    you.
    Speaker notes
    1

    View full-size slide

  2. VSHN – The DevOps Company
    • Kubernetes Operator
    • Declarative Infrastructure Management
    • Abstraction for Self-Service APIs
    • Packaging and distribution of custom
    infrastructure
    • Started in 2018, CNCF Sandbox project
    since 2020, move to Incubation requested
    What?
    ⏭ Crossplane is an open source Kubernetes Operator
    which enables to assemble infrastructure from multiple
    vendors in a
    ⏭ declarative way.
    ⏭ It exposes higher level self-service APIs for users of
    the platform to consume in an abstracted way, without
    having to write any code besides YAML.
    ⏭ All these customized infrastructure description can
    be packaged, easily distributed and shared.
    ⏭ Crossplane has been invented by the maintainers of
    the well known storage operator Rook in 2018. It has
    been donated to the Cloud Native Computing
    Foundation in mid 2020 and recently applied to be
    promoted to become an incubation level project.
    There are a few key features, from which I will introduce
    the three most important ones to you in the next slides.
    Speaker notes
    2

    View full-size slide

  3. VSHN – The DevOps Company
    • Building block for upstream API
    • Managed Resource: Provider speci c
    & opinionated CRDs
    • XRM - Crossplane Resource Model: E.g.
    spec.forProvider
    • Of cial provider: AWS, GCP, Azure,
    Rook, Alibaba
    • More under the
    GitHub org
    Crossplane Providers ✨
    crossplane-contrib
    Let’s start with the Crossplane providers.
    ⏭ They are the building blocks for talking to upstream
    APIs, like AWS, Azure, Google Cloud or even Helm.
    ⏭ And they provide an abstraction layer by bringing
    provider specific custom resources with an opinionated
    structure. Such provider specific resources are called
    managed resources. From a users perspective they are
    just standard Kubernetes custom resources.
    ⏭ But with the opinionated Crossplane resource
    model, each of these custom resources look similar.
    For example, each custom resource has a field called
    "spec dot forProvider" which contains the detailed
    configuration of the upstream resource. Most of the
    time it reflects all possible options which are available
    in the third-party API. This allows full flexibility per
    supported API.
    ⏭ Besides the official Crossplane providers, a lot more
    ⏭ can be found under the crossplane-contrib GitHub
    organization. This is also the place where new providers
    are born and already contains a lot of very good
    providers.
    Speaker notes
    3

    View full-size slide

  4. VSHN – The DevOps Company
    Composite Resource De nition (XRD)
    API de nition, opinionated CRD - XRM
    Composition
    Collection of resources
    Composite Resource (XR)
    Cluster-scope instance of XRD,
    selected Composition
    Composite Resource Claim (XRC)
    Namespace-scoped instance, maps to
    XR
    Crossplane Compositions
    🥏 The next key feature are the Crossplane compositions.
    This feature allows for custom and opinionated
    infrastructure definitions which can be easily composed
    and provided to the user of the platform in a self-
    service way.
    This feature consists of four object types which we will
    now getting to know.
    • The first one are Composite resource definitions -
    called XRD - which define the custom API definitions.
    With them the operator of the platform describes the
    opinionated services which are then made available
    to the user. XRDs in the background generate
    Crossplane-flavored Kubernetes custom resource
    definitions which adhere to the Crossplane Resource
    Model.
    • The Compositions actually implement the custom
    infrastructure which is defined in XRDs. There can be
    multiple implementations per XRD, which the user
    then can choose from. One could for example be a
    service which is tuned for production, the other one
    might have options enabled which are great for the
    development environment. The user then choses
    which Composition should be selected when creating
    the service instance.
    • A composite resource is the cluster-scope object to
    instantiate a custom service, implemented by the
    selected composition.
    • And with composite resource claims the user of the
    platform specifies the infrastructure needs on a
    namespace scope. This can be done for example
    together with the application deployment.
    By splitting cluster-scope and namespace-scope
    objects, security constraints can be easily implemented
    Speaker notes
    4

    View full-size slide

  5. VSHN – The DevOps Company
    • Distribute Providers and con gurations
    • Packaged as opinionated OCI image
    • Con guration versioning
    • RBAC handling
    • Dependency management
    • Packaging with Crossplane CLI
    Crossplane Packaging 📦
    And the last important feature are the Crossplane
    packages.
    ⏭ With them, provider installation and configuration as
    well as custom infrastructures can be easily distributed
    and shared.
    ⏭ These packages are OCI compliant images which
    contain just a bunch of Crossplane YAML and can
    therefore be hosted on any OCI compliant registry.
    ⏭ Crossplane packages allow for configuration
    versioning,
    ⏭ RBAC handling and
    ⏭ dependency
    management.
    ⏭ Packaging can be done manually, but much better
    via the Crossplane CLI.
    Speaker notes
    5

    View full-size slide

  6. VSHN – The DevOps Company
    • CLI Tool vs. Control-Plane
    • State le handling vs. Reconciliation
    • Full infra vs. pieces of it
    • Crossplane can use Terraform providers (alpha) 💪
    Comparison to Terraform ⇄
    Some of you might wonder what the differences are
    between Crossplane and Terraform as it looks like they
    are mostly doing the same at first sight.
    ⏭ The main difference is that Terraform is designed to
    be a CLI tool which acts only on invocation. Crossplane
    on the other hand is an always running controller which
    acts on change.
    ⏭ To know what’s the current state is, Terraform
    tracks it in a state file. This complicates collaboration
    on infrastructure and full automation a lot, mainly
    because of state file locking mechanisms needed. If
    the actual state differs to the one in the state file, bad
    things may happen. Manual changes to the
    infrastructure are only detected on the next CLI
    invocation while Crossplane actively reconciles the
    infrastructure all the time and makes manual changes
    impossible to stay undetected.
    ⏭ Terraform always acts on the full infrastructure it
    takes care of, where Crossplane is able to only
    configure pieces of it, making it much less invasive for
    small changes.
    ⏭ Lately Crossplane started an initiative to be able to
    re-use Terraform providers as Crossplane providers
    which could be a huge boost of available providers. The
    VMware provider which was released just lately is
    actually based on the Terraform provider.
    This doesn’t mean that Terraform is bad, it just behaves
    very differently to do a similar job. As Crossplane is a
    control-plane and Terraform is designed to work with
    control-planes, Crossplane could even be managed with
    Terraform.
    Speaker notes
    6

    View full-size slide

  7. VSHN – The DevOps Company
    • Well-Known API
    • Same language as app
    deployment
    • Tooling re-use (GitOps,
    Project Syn)
    • One place to de ne
    everything
    • Reconciliation loop, prevent
    con g drift
    • RBAC to protect
    infrastructure management,
    still allowing self-service
    Why do we care? ♥
    The previously introduced concepts and features are
    the reasons why we at VSHN and I personally care a lot
    about Crossplane.
    But there are more reasons:
    ⏭ We can leverage the well-known Kubernetes API.

    It’s the same language we also use to deploy
    applications.
    ⏭ This allows to reuse already existing
    tooling like GitOps or CI/CD Pipelines.
    ⏭ And the
    infrastructure and deployment needs of an application
    is defined on the same places.
    ⏭ As Crossplane runs a reconciliation loop all the
    time, configuration drift isn’t possible anymore. The
    control-plane is always active, looking for changes on
    the upstream configuration. If it doesn’t match the
    declared configuration, it simply makes sure to adhere
    to it.
    ⏭ And last but not least, RBAC allows to protect the
    infrastructure provisioning and configuration but
    allowing self-service in an easy way.
    Speaker notes
    7

    View full-size slide

  8. VSHN – The DevOps Company
    Kubernetes Clusters
    Helm
    "App Cluster" / appc-app
    Crossplane
    "Service Cluster" / appc-service1
    Service deployments (via Helm)
    Composite
    Resource
    Developer
    Crossplane Con guration
    CompositeResourceDefinition
    redisinstances.syn.tools
    Composition
    redis-small
    API Definition
    Creates
    RedisInstance
    redis1
    Helm Release
    redis1
    Kubernetes
    API
    Service
    Consumer
    Live Demo 1 🐚
    Let’s see how Crossplane looks like in real-life.
    In this first live-demo we will have a look at various
    Crossplane specific Kubernetes objects and provision a
    Redis service.
    The left diagram shows the demo Kubernetes clusters
    in use. The top one is the one we’ll be looking at in the
    demo. It runs Crossplane and it is the cluster in which
    we’ll be deploying Redis. The actual service then runs
    on the bottom cluster, the service cluster. Provisioning
    happens via Helm.
    The right diagram shows the Crosplane objects involved
    in this demo. There is a Redis instance XRD, a
    Composition which implements this XRD and we’ll be
    creating a RedisInstance object to actually deploy the
    instance.
    Let’s dive in…
    Speaker notes
    8

    View full-size slide

  9. VSHN – The DevOps Company
    • Internal marketplace on Cloudfoundry
    • Integration with Open Service Broker API
    • Open Source Crossplane Service Broker
    • Crossplane Helm Provider
    • Huge environment
    Crossplane Service Broker
    🧩
    github.com/vshn/crossplane-service-broker
    After this first live-demo, I want to tell you more about
    our first use case we’ve implemented with Crossplane.
    ⏭ It is an internal marketplace on Cloudfoundry to
    enable self-service service provisioning for development
    and service teams.
    ⏭ The integration to this marketplace had to be done
    with the Open Service Broker API as this is the API
    which is tightly integrated in the Cloudfoundry world.
    ⏭ To achieve this integration we developed a bridge
    between Crossplane and the Open Service Broker API.
    The two concepts match very well. Our Crossplane
    Service Broker is available as an Open Source tool on
    GitHub.
    ⏭ As the infrastructure of the customer doesn’t
    provide any cloud services, we are provisioning the
    actual services using the Crossplane Helm Provider on
    service clusters.
    ⏭ This project already pushed the boundaries of
    Crossplane a lot, we’re talking about more than 1000
    managed services.
    Speaker notes
    9

    View full-size slide

  10. VSHN – The DevOps Company
    Kubernetes Clusters
    Open Service Broker API
    "Service Consumer" / appc-consumer1
    Kubernetes Service Catalog (SVCAT)
    Helm
    "Control Plane" / appc-control
    Crossplane & Crossplane Service Broker
    "Service Cluster" / appc-service1
    Service deployments (via Helm)
    Service
    Provisioning
    and
    Binding
    Developer
    Crossplane Con guration
    Crossplane Service Broker
    CompositeResourceDefinition
    redisinstances.syn.tools
    Composition
    redis-small
    API Definition
    Creates
    RedisInstance
    redis1
    Service Plan Service Instance
    Helm Release
    redis1
    Open
    Service
    Broker
    API
    Service
    Consumer
    Provision
    Deprovision
    Service Offering
    Live Demo 2 🐚
    In this second demo we will see the service provisioning
    of the same Redis service as we’ve seen in the first
    demo, this time using the Open Service Broker API with
    the Kubernetes Service Catalog.
    This time we have three Kubernetes clusters for the
    demo. The top one represents the end-user which has
    the Kubernetes Service Catalog installed which
    connects to the Crossplane Service Broker.
    The control cluster in the middle runs Crossplane and
    the Crossplane Service Broker and services are once
    again deployed via Helm on the Service cluster.
    The diagram on the right is nearly the same as in the
    fist demo. This time it shows the mapping between
    Crossplane objects and Open Service Broker API
    constructs.
    Let’s dive in…
    That’s it for the demo part.
    Speaker notes
    10

    View full-size slide

  11. VSHN – The DevOps Company
    • Crossplane is …
    • … the only Control Plane …
    • … you ever need again.
    Conclusion
    🥳 In the last few minutes I gave you an overview of
    Crossplane. Of course there’s a lot more to it, the best
    way to learn it is to try it yourself.
    And all I can say is:
    ⏭ Crossplane is
    ⏭ the only
    Control Plane
    ⏭ you ever need again.
    → at least until something better appears
    Speaker notes
    11

    View full-size slide

  12. VSHN – The DevOps Company
    vshn.timer
    Shameless self-marketing
    Tobias Brunner –
    VSHN AG – Neugasse 10 – CH-8005 Zürich – +41 44 545 53 00 – –
    Thanks!
    www.vshn.ch/blog/vshn-timer
    tobru.ch/about-newsletter/
    [email protected]
    vshn.ch [email protected]
    Thanks a lot for listening and don’t hesitate to contact
    me if you have any questions or want to know more.
    And if you’re interested in Cloud Native news and tools,
    you’re invited to follow the vshn.timer and subscribe to
    my personal newsletter.
    Speaker notes
    12

    View full-size slide