.NET Day 2023: Continuous Deployment Showdown: Traditional CI/CD vs. GitOps

Slide 1

Slide 1 text

Continuous Deployment Showdown: Traditional CI/CD vs. GitOps Marc Müller Principal Consultant [email protected] @muellermarc www.4tecture.ch

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

Slide Download https://www.4tecture.ch/events/dotnetday2023

Slide 4

Slide 4 text

Agenda ▪ Intro ▪ Setup ▪ Implementing the Deployment ▪ Staging ▪ Automatic Updates ▪ Cluster Setup with Azure Pipelines ▪ Conclusion

Slide 5

Slide 5 text

Intro

Slide 6

Slide 6 text

Once upon a time… Source: https://pixabay.com/de/illustrations/ai-generiert-mann-designer-7974535/, https://imgflip.com/i/7x3amn, https://imgflip.com/i/7x3apt, https://imgflip.com/i/7x3at0 * pretty similar

Slide 7

Slide 7 text

Our playground… ▪ ASP.NET Core Application ▪ Dockerized ▪ Kubernetes deployment environment

Slide 8

Slide 8 text

Kubernetes Deployment Deployment Pod Service Pod Pod Ingress Secret Secret Secret ConfigMap HPA ReplicaSet dapr component Sidecar Pod

Slide 9

Slide 9 text

We focus on deployment …this is the build

Slide 10

Slide 10 text

Kustomize ▪ Standalone tool ▪ for customizing Kubernetes objects ▪ by using a declarative configuration language ▪ Built-in in kubectl since 1.14 ▪ kustomization.yaml → procecces resources ▪ Bases and overlays ▪ Patches, ConfigMaps and Secrets, Variable substitution

Slide 11

Slide 11 text

Helm ▪ Package Manger for Kubernetes ▪ Define, install and upgrade Kubernetes applications ▪ Configuration based on «Charts» Source: https://helm.sh/

Slide 12

Slide 12 text

Helm Features ▪ Manage Complexity ▪ Charts describe even the most complex apps ▪ Provide repeatable application installation ▪ Serve as a single point of authority ▪ Easy Updates ▪ Take the pain out of updates ▪ In-place upgrades ▪ Custom hooks. ▪ Simple Sharing ▪ Versioning ▪ Easy to share, and host on public or private servers ▪ Rollbacks ▪ Use helm rollback to roll back to an older version of a release with ease. Source: https://helm.sh/

Slide 13

Slide 13 text

Source: https://imgflip.com/i/7xal0f

Slide 14

Slide 14 text

GitOps ▪ Declarative Configuration: ▪ System state is described in a declarative manner, usually using files like YAML. ▪ Version-Controlled System State: ▪ Desired system state is stored in a version control system, typically Git. ▪ Git serves as the single source of truth. ▪ Automated Delivery: ▪ Changes in the Git repository trigger automatic system updates. ▪ Software Agents Ensure System Convergence: ▪ Tools continuously monitor and ensure the actual system state matches the desired state in the Git repository. ▪ Any discrepancies lead the system to self-correct to match the desired state. ▪ Infrastructure as Code (IaC): ▪ Infrastructure setup and configuration are defined and version-controlled as code. ▪ Enhanced Visibility and Traceability: ▪ All changes are tracked, auditable, and can be rolled back using Git's capabilities. ▪ Reduced Manual Intervention: ▪ Direct changes to production are discouraged. ▪ Changes are made via pull requests or commits to the repository.

Slide 15

Slide 15 text

GitOps By following these principles, GitOps aims to make operations and deployment more consistent, repeatable, and secure.

Slide 16

Slide 16 text

Setup

Slide 17

Slide 17 text

General Setup Deployment Target (k8s) Config (Git) Artifacts (Container Registry) Deployer App Container Image Helm Chart

Slide 18

Slide 18 text

Azure Pipelines

Slide 19

Slide 19 text

Base Infrastructure CD Service CD Pipeline Design Base Infrastructure Template Resource Template Resource Template Dev/Test Prod Service CI Build Stage PR Stage Testing PreProd Prod Compile Service DB Schema Compile System Tests Infrastructure Artifacts Pipeline Artifacts Task / Job Templates Task / Job Templates Resource Groups, vNets, VMs, Azure SQL, CosmosDB, … App Deployment, DB instance, DB Schema, Managed Identities, Storage, … Deplyoment Verification

Slide 20

Slide 20 text

Azure DevOps & AKS CI / CD Azure DevOps CI Pipeline Azure DevOps CD Pipeline Container Registry Build App Test Build Container Repo Configuration Helm Upgrade AKS (Cluster) Helm Chart Package Push Cluster Pull container image Release Pull Helm chart

Slide 21

Slide 21 text

Configuration as Code ▪ Pipelines, Templates, Variables / Values files stored in Git ▪ Do not store secrets in Git! ▪ Mono-Repo / Repo per Service ▪ Staging is implemented by ▪ Feature Branches → PR → PR Deployment with QA → PR Approval / Integration ▪ Main Branch → Pre Production → Production (checks and approvals, deployment rings)

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

helm upgrade --install azdodeploymentagent \ .\charts\deploymentagent \ --namespace azdoagent \ --create-namespace \ --set agent.pool=k8sdemodeployment\ --set agent.token=xxxx

Slide 25

Slide 25 text

Flux

Slide 26

Slide 26 text

FluxCD ▪ Open-source GitOps toolkit for Kubernetes ▪ Part of the CNCF ▪ Monitors Git repositories ▪ Automatically applies the changes to the cluster

Slide 27

Slide 27 text

Flux Source Controller

Slide 28

Slide 28 text

Flux Kustomize Controller

Slide 29

Slide 29 text

Flux Helm Controller

Slide 30

Slide 30 text

Flux Notification Controller

Slide 31

Slide 31 text

Flux Image Update Automation

Slide 32

Slide 32 text

Setup Flux ▪ Install the CLI ▪ Installation in k8s ▪ flux bootstrap ▪ flux install ▪ Add kustomizations, sources, … Deployment Target (k8s) Config (Git) Artifacts (Container Registry) Flux Controller Flux Controller Flux Controller Flux Controller

Slide 33

Slide 33 text

Implementing the Deployment

Slide 34

Slide 34 text

Azure Pipelines

Slide 35

Slide 35 text

CD Pipeline with CI Resource

Slide 36

Slide 36 text

Define Versioning

Slide 37

Slide 37 text

Deployment Job

Slide 38

Slide 38 text

Deployment

Slide 39

Slide 39 text

Deployment

Slide 40

Slide 40 text

Configuration

Slide 41

Slide 41 text

Flux

Slide 42

Slide 42 text

Kustomization

Slide 43

Slide 43 text

Helm Repository

Slide 44

Slide 44 text

Helm Release

Slide 45

Slide 45 text

Configuration

Slide 46

Slide 46 text

Unique Resource Names

Slide 47

Slide 47 text

Staging

Slide 48

Slide 48 text

Azure Pipelines

Slide 49

Slide 49 text

Pipeline Stages

Slide 50

Slide 50 text

Environments

Slide 51

Slide 51 text

Dedicated Settings per Stage

Slide 52

Slide 52 text

Dedicated Settings per Stage

Slide 53

Slide 53 text

Flux

Slide 54

Slide 54 text

Structuring Best Practices ▪ Use base configuration with overlays ▪ Overlay config maps / secrets ▪ Overlay objects with patches - apps - application1 - base - overlays - dev - prod - application2 - base - overlays - dev - prod

Slide 55

Slide 55 text

Structure

Slide 56

Slide 56 text

Overlay Kustomization

Slide 57

Slide 57 text

Overlay Values

Slide 58

Slide 58 text

Patch Release

Slide 59

Slide 59 text

Add dedicated objects

Slide 60

Slide 60 text

Automatic Updates

Slide 61

Slide 61 text

Azure Pipelines

Slide 62

Slide 62 text

Out-of-the box: triggers and branches ▪ Pipeline Triggers on Source Update ▪ Use a branching concept and PR workflow ▪ PR deployments ▪ Checks and Approvals

Slide 63

Slide 63 text

Flux

Slide 64

Slide 64 text

GitOps ▪ Git Repo defines what is deployed ▪ No update without new commit Repo Updates ▪ Task after successful CI ▪ Listen for new images / Helm charts

Slide 65

Slide 65 text

Flux Image Automation ▪ Monitor Container Registry ▪ Analyze updates based on policy ▪ Change configuration – commit change in GIT repo ▪ Standard GitOps flow is triggered

Slide 66

Slide 66 text

Image Repository

Slide 67

Slide 67 text

Image Policy

Slide 68

Slide 68 text

Image Update Automation

Slide 69

Slide 69 text

No content

Slide 70

Slide 70 text

Updates

Slide 71

Slide 71 text

Updates

Slide 72

Slide 72 text

Cluster Setup with Azure Pipelines

Slide 73

Slide 73 text

Deploy Cluster Services ▪ Cluster setup has a lot of Helm chart deployments ▪ Configuration as Code ▪ Pipeline automation on configuration object ▪ Runtime Parameters / Parameters support objects ▪ Define your cluster configuration as object

Slide 74

Slide 74 text

Parameter with all charts

Slide 75

Slide 75 text

No content

Slide 76

Slide 76 text

Add and update Helm Repos

Slide 77

Slide 77 text

Install helm charts

Slide 78

Slide 78 text

Conclusion

Slide 79

Slide 79 text

Comparison Pipelines ▪ Everything in Git ▪ Versatile ▪ Full DevOps Lifecycle ▪ End-to-end Traceability ▪ Integrated Test Automation Flux ▪ Everything in Git ▪ K8s Deployments ▪ Focus on Continuous Deployment ▪ Independent / Various Sources

Slide 80

Slide 80 text

Conclusion ▪ Both provide a state-of-the-art deployment ▪ Git is the source of truth ▪ We use ▪ Azure Pipelines: Classic Development to Deployment (CI/CD) ▪ Flux: disconnected from development / prod cluster deployment

Slide 81

Slide 81 text

Q & A

Slide 82

Slide 82 text

Thank you for your attention! If you have any questions do not hesitate to contact us: 4tecture GmbH Marc Müller Industriestrasse 25 Principal Consultant CH-8604 Volketswil +41 44 508 37 00 [email protected] [email protected] @muellermarc www.4tecture.ch www.powerofdevops.com