Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Migrating the GitLab–Kubernetes Integration from Certificates to the Agent

Migrating the GitLab–Kubernetes Integration from Certificates to the Agent

Presentation shown during the third Switzerland GitLab Meetup, on April 27th, 2022.

Adrian Kosmaczewski

April 27, 2022
Tweet

More Decks by Adrian Kosmaczewski

Other Decks in Technology

Transcript

  1. VSHN – The DevOps Company
    Wednesday April 27th, 2022
    Adrian Kosmaczewski, Developer Relations, VSHN
    The New GitLab
    Kubernetes Agent
    Switzerland GitLab Meetup
    Welcome to the third Switzerland GitLab Meetup. Today
    we’re going to talk about a fundamental piece of
    machinery for GitOps engineers working with GitLab: the
    new GitLab Kubernetes Agent, replacing the old
    certificate-based integration, to be discontinued soon.
    Speaker notes
    1

    View Slide

  2. VSHN – The DevOps Company

    The integration of GitLab and Kubernetes is a critical
    one. More and more developers are using it to
    automatically deploy their applications to Kubernetes
    clusters from the confort of their Git client, in a typical
    GitOps fashion: git push and a few minutes later, your
    app is published.
    Speaker notes
    2

    View Slide

  3. VSHN – The DevOps Company
    To integrate GitLab and Kubernetes, since GitLab 10.4
    (released in 2018) we were using certificates; that is,
    you had to give the URL of the Kubernetes API of your
    cluster, provide a few secrets, and boom, your GitLab
    can talk to your Kubernetes.
    But this is all about to change right now.
    Speaker notes
    3

    View Slide

  4. VSHN – The DevOps Company
    The New GitLab Kubernetes Agent
    Let us talk today about the new way of integrating
    GitLab and Kubernetes, using the brand new GitLab
    Kubernetes Agent, which completely replaces the old
    certificate-based integration.
    Speaker notes
    4

    View Slide

  5. VSHN – The DevOps Company
    (Source: about.gitlab.com/blog)
    The deprecation notice of the certificate-based
    mechanism, and its replacement with the new GitLab
    Kubernetes Agent, was announced in a blog post in
    November last year.
    Speaker notes
    5

    View Slide

  6. VSHN – The DevOps Company
    (Source: about.gitlab.com/blog)
    The list of breaking changes in GitLab 15.0 contains a
    special section called "Configure" with precisely a
    section explaining how the SaaS certificate-based
    integration with Kubernetes will be phased out
    completely.
    This is huge news for both SaaS and self-hosted users
    of GitLab.
    Speaker notes
    6

    View Slide

  7. VSHN – The DevOps Company
    1. It requires direct access to the K8s API
    2. Many valuable features required giving cluster admin rights to
    GitLab
    3. New trend: pull based deployments & CI/CD (Argo CD, anyone?)
    4. Polling… polling… polling.
    Issues with Certificates
    Why would GitLab perform such a groundbreaking
    change? It turns out that the old certificate-based
    mechanism had some shortcomings:
    It required complete and direct access to the
    Kubernetes API, which, well, can be considered a
    security risk by some organizations.
    To access many advanced features, you had to
    provide GitLab with cluster admin rights. Again, not
    something that would put a smile on a security
    engineer.
    There’s a new trend in the CI/CD space, which is
    "pull-based deployments", similar to what Argo CD is
    offering, and GitLab wants to be a part of that market
    too.
    In general, the certificate-based mechanism relies in
    polling, which as we all know, is not the greatest of
    ideas in Computer Science.
    Speaker notes
    7

    View Slide

  8. VSHN – The DevOps Company
    Deprecated since version 14.5
    GitLab.com: removed in version 15.0
    Self-hosted: disabled in 15.0, removed in 15.6
    Kubernetes version: minimum 1.20 (released December 2020,
    supported until February 2022)
    Deprecation & Requirements
    How and when will the old certificate mechanism be
    replaced by the GitLab Kubernetes Agent?
    The certificate mechanism has been already marked as
    deprecated since version 14.5, and will be completely
    removed in GitLab SaaS in version 15.0. For self-
    hosted users, it will be disabled in 15.0 and completely
    removed in 15.6, so you have some time to migrate to
    the agent.
    Finally, to be able to use the GitLab Kubernetes Agent,
    you must use a version of Kubernetes equal or higher
    to 1.20.
    Speaker notes
    8

    View Slide

  9. VSHN – The DevOps Company
     Not all certificate-based features will be available with Agents!
    Important!
    A very important thing to know is that this is effectively
    a breaking change. Many features available with the
    certificate-based mechanism will not be available with
    the new GitLab Kubernetes Agent!
    For example, GitLab Managed Apps (GMA) were
    deprecated in GitLab 14.0, and the agent for
    Kubernetes does not support them.
    Speaker notes
    9

    View Slide

  10. VSHN – The DevOps Company
    Clusters configured through code
    Based on regular K8s RBAC rules
    Scaling is trivial: one agent, one environment
    Agent connections can be shared to other groups and projects
    Also supports push-based "classic" CI/CD deployments
    Agent supports pull-based GitOps deployments
    Bidirectional communication opens door to new features
    Advantages
    However, not all is bad news, since the new Agent
    brings some quite interesting advantages to the table:
    You can configure your clusters using code, including
    a configuration file in your projects
    It uses regular Kubernetes RBAC rules
    There is a one-to-one relationship between agents
    and environments
    Agent connections can be shared among projects
    Agents supports the classic CI/CD experience offered
    by GitLab today
    But it also support GitOps pull-based deployments
    And it offers a bidirectional communication, opening
    the door to new features in the future.
    Speaker notes
    10

    View Slide

  11. VSHN – The DevOps Company
    Integrate a cluster, located behind a firewall or NAT, with GitLab
    Access API endpoints in a cluster in real time
    Enable real-time features by pushing information about events
    happening in a cluster
    Enable a cache of Kubernetes objects through informers, kept up-
    to-date with very low latency
    …in general, removing polling
    Major Goals of the Agent
    To reach such goals, GitLab built the Kubernetes Agent
    with some very specific ideas in mind:
    It should be possible to integrate clusters behind
    firewalls or NATs
    It should be possible to access API endpoints in real
    time
    It should be possible to enable real-time features,
    getting live information about what’s going on in the
    cluster.
    It should be possible to cache information about
    Kubernetes objects in GitLab, kept updated with very
    low latency
    And in general, we should get rid of polling
    altogether!
    Speaker notes
    11

    View Slide

  12. VSHN – The DevOps Company
    Kubernetes cluster
    GitLab
    gRPC bidirectional streaming
    gRPC gRPC
    REST API
    agentk
    kas
    GitLab RoR Gitaly
    Architecture
    (Source: gitlab.com/gitlab-org)
    This diagram, taken from the technical architecture of
    the agent, shows the main architecture ideas of the
    agent, and how it works.
    GitLab RoR is the main GitLab application. It uses
    gRPC to talk to kas.
    agentk is the GitLab Agent. It keeps a connection
    established to a kas instance, waiting for requests to
    process. It may also actively send information about
    things happening in the cluster.
    kas is the GitLab Agent Server, and is responsible
    for:
    Accepting requests from agentk.
    Authentication of requests from agentk by
    querying GitLab RoR.
    Fetching agent’s configuration from a
    corresponding Git repository by querying Gitaly.
    Matching incoming requests from GitLab RoR
    with existing connections from
    the right agentk,
    forwarding requests to it and forwarding
    responses back.
    (Optional) Sending notifications through
    ActionCable for events received from agentk.
    Polling manifest repositories for GitOps support
    by communicating with Gitaly.
    Speaker notes
    12

    View Slide

  13. VSHN – The DevOps Company
    1. Minimize network traffic
    2. The client-server relationship is inverted
    3. GitLab will add more logic into kas rather than into agentk
    Design Principles
    These major design principles of the agent are driven by
    two major requirements:
    1. This is because the connection must be initiated
    from inside the Kubernetes cluster to bypass any
    firewall or NAT the cluster may be located behind
    2. agentk should be kept streamlined and small to
    minimize the need for upgrades.
    Speaker notes
    13

    View Slide

  14. VSHN – The DevOps Company
    agentk and kas are written in Go instead of Ruby
    Each agentk connects to an instance of kas
    Connections are kept open
    Multiple agents per K8s cluster allowed…
    … each running under (potentially) a separate ServiceAccount
    … with different privileges!
    Each kas instance tracks the agents connected to it in a Redis
    instance
    Agent names follow RFC 1123 (alphanumeric, –, 63 chars max)
    Interesting Tidbits
    Would you like to know more about the GitLab
    Kubernetes Agent? Here go some interesting bits of
    information for nerds.
    Speaker notes
    14

    View Slide

  15. VSHN – The DevOps Company
    kas
    Kubernetes 3
    Kubernetes 2
    Kubernetes 1
    register connected agents
    register connected agents
    lookup agent
    gRPC
    gRPC
    gRPC
    gRPC
    gRPC to any kas kas 1
    kas 2
    kas 3
    agentk 3, Pod1
    agentk 2, Pod1
    agentk 1, Pod1
    agentk 1, Pod2
    GitLab Rails
    Redis
    (Source: gitlab.com/gitlab-org)
    And this other diagram also shows how the agentk and
    kas components work together.
    Speaker notes
    15

    View Slide

  16. VSHN – The DevOps Company
    GitOps Workflow: pull
    "Classic" CI/CD Workflow: push
    New Workflows
    So, as we mentioned previously, the new GitLab
    Kubernetes Agent opens up the door to pull-based
    deployments, similar to what Argo CD provides, but fully
    integrated in GitLab.
    Speaker notes
    16

    View Slide

  17. VSHN – The DevOps Company
     Start with the CI/CD Workflow!
    Agent not exposed to the internet
    Does not require full cluster-admin access
    Migrating to the Agent
    Now the big question: should you migrate to the new
    GitLab Kubernetes Agent?
    Well, unless you are using self-hosted GitLab, you won’t
    have much of a choice, and even so, the writing’s on
    the wall.
    You might want to start using the "Classic" CI/CD
    push-based workflow, which offers a few advantages to
    the old certificate-based mechanism.
    Speaker notes
    17

    View Slide

  18. VSHN – The DevOps Company
    1. In your GitLab project create a file:
    .gitlab/agents//config.yaml
    Contains GitOps Workflow configuration
    Empty for CI/CD Workflow!
    2. Install the GitLab agent in your cluster
    Using helm
    3. Update your .gitlab-ci.yml by selecting the proper context
    4. Run your pipeline!
    How to Use?
    How to use the GitLab Kubernetes Agent? It is a very
    short and simple process:
    Create a configuration file in your GitLab project and
    push it to the repo.
    Install the agent in your cluster
    Update your .gitlab-ci.yml file
    And run your pipeline!
    Speaker notes
    18

    View Slide

  19. VSHN – The DevOps Company
    SaaS GitLab
    Exoscale SKS Cluster (Kubernetes 1.23.5)
    "Classic" push-based CI/CD workflow
    Demo!
    In this demo I’m going to show you how to connect a
    project located in SaaS GitLab to an Exoscale SKS
    Cluster, running Kubernetes 1.23.5. I will use the
    "Classic" push-based workflow, which is the simplest
    possible approach to get started.
    Speaker notes
    19

    View Slide

  20. VSHN – The DevOps Company

    So as I’ve said previously, the GitLab to Kubernetes
    connectivity has changed completely…
    Speaker notes
    20

    View Slide

  21. VSHN – The DevOps Company


    … featuring now a full back and forth communication
    mechanism, based on a small piece of software running
    in your cluster.
    Speaker notes
    21

    View Slide

  22. VSHN – The DevOps Company
    Official Documentation
    Announcement Blog Post
    Breaking changes in GitLab 15.0
    More Information
    docs.gitlab.com/ee/user/clusters/agent/
    about.gitlab.com/blog/2021/11/15/deprecating-the-cert-based-
    kubernetes-integration/
    about.gitlab.com/blog/2022/04/18/gitlab-releases-15-breaking-
    changes/#configure
    If you would like to learn more about the GitLab
    Kubernetes Agent, here are some links for you.
    Speaker notes
    22

    View Slide

  23. VSHN – The DevOps Company
    Discussion about APPUiO Cloud
    Agent-less GitLab integration with OpenShift
    About APPUiO Cloud
    github.com/appuio/appuio-cloud-community/discussions/41
    www.vshn.ch/en/blog/agent-less-gitlab-integration-with-
    openshift/
    In the context of our product APPUiO Cloud, which is an
    "OpenShift Projects as a Service" offering, we are
    currently evaluating options regarding the new GitLab
    Kubernetes Agent. In the meantime, my colleague
    Christian Cremer has written an article with a simple
    hack to get your GitLab instance of choice connected to
    your Kubernetes clusters in the simplest way possible,
    and I strongly recommend that you check it out.
    Speaker notes
    23

    View Slide

  24. VSHN – The DevOps Company
    Adrian Kosmaczewski, Developer Relations, VSHN –
    VSHN AG – Neugasse 10 – CH-8005 Zürich – +41 44 545 53 00 – –
    Thanks!
    [email protected]
    vshn.ch [email protected]
    Thanks a lot for your attention! Do not hesitate to
    contact VSHN if you want to start using the GitLab
    Kubernetes Agent in your own projects.
    Speaker notes
    24

    View Slide