Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How Containerized Pipelines Can Boost Your CI/CD by Nico Meisenzahl

How Containerized Pipelines Can Boost Your CI/CD by Nico Meisenzahl

DevOps Gathering

March 11, 2020
Tweet

More Decks by DevOps Gathering

Other Decks in Programming

Transcript

  1. Nico Meisenzahl • Senior Cloud & DevOps Consultant at white

    duck • Docker Community Leader & GitLab Hero • loves Kubernetes, DevOps and Cloud © white duck GmbH 2020 Phone: +49 8031 230159 0 Email: [email protected] Twitter: @nmeisenzahl LinkedIn: https://www.linkedin.com/in/nicomeisenzahl Blog: https://meisenzahl.org
  2. Agenda • containerized pipeline – why? • pipeline workload on

    Kubernetes with GitLab CI/CD • image builds on Kubernetes with Kaniko • cloud-native pipelines with Tekton © white duck GmbH 2020
  3. Containerized Pipelines – Why? • for the same reasons why

    you should use containers • isolation • dependencies • scalability • immutability • example: your new project needs version X all others still require Y • you can include any kind of build / deploy dependency • NodeJS, .NET Core, Go, Terraform, Ansible… you name it © white duck GmbH 2020
  4. Pipeline job image • contains everything a single pipeline job

    needs • binaries, libraries, tools, ... • provide all external dependencies • use a pipeline to build/rebuild it periodically (security fixes!) • you should define fix versions for your dependencies • use caching to speed up your builds © white duck GmbH 2020
  5. Kubernetes vs. docker run • every pipeline job runs in

    a container • based on an image with all requirements for this single job • Build host with Docker daemon (or any other container solution) • GitLab Runner Kubernetes executor • integrates your CI/CD with Kubernetes • runs a pod per job • containing a container with the defined image along with some service containers • allows you to share your compute and scale your pipelines © white duck GmbH 2020
  6. GitLab Runner Kubernetes executor • runs itself in a pod

    • needs to be deployed in your Kubernetes Cluster • automatable Helm deployment • schedules job pods • build steps of a pipeline job • prepare → creates pod with build and service containers • pre-build → clones repo, restore cache, download artifacts • build → user build steps • post-build → creates caches and upload artifacts © white duck GmbH 2020
  7. Image builds on Kubernetes with Kaniko • any Docker-in-Docker solution

    has issues • exposing Docker socket • mounting /var/lib/docker • privileged mode • image builds without the need of any privileges or dependencies • runs in a container (gcr.io/kaniko-project/executor) • use build caching to speed up your pipeline • layer caching (layers get pushed pushed to a registry) • base image caching (local mount point) © white duck GmbH 2020
  8. Cloud-native pipelines with Tekton • moves your whole CI/CD into

    Kubernetes • uses containers as their building blocks • based on CRDs and Controllers • Tekton Pipelines emerged out of the Knative build project • Continuous Delivery Fundation Graduated project • „CI/CD framework for Kubernetes” • Jenkins X pipelines are based on Tekton • contributions by Google, IBM, RedHat, Cloudbees, TriggerMesh, … © white duck GmbH 2020
  9. Cloud-native pipelines with Tekton • Tekton Triggers can be used

    to call a pipeline • push, issue, webhook, …. • Tekton also provides a CLI and Dashboard • get started • https://github.com/tektoncd/pipeline/blob/master/docs/tutorial.md • https://github.com/tektoncd/catalog © white duck GmbH 2020
  10. Questions? Slides: https://www.slideshare.net/nmeisenzahl Demo: https://gitlab.com/groups/containerized-cicd Nico Meisenzahl (Senior Cloud &

    DevOps Consultant) Phone: +49 8031 230159 0 Email: [email protected] Twitter: @nmeisenzahl LinkedIn: https://www.linkedin.com/in/nicomeisenzahl Blog: https://meisenzahl.org © white duck GmbH 2020