GitOps - Continuous Deployment for Kubernetes

GitOps is an alternative way to run deployments on Kubernetes. It leverages tools that are used by developers. Deployments are approved by Pull Requests and automatically rolled out to the desired cluster. GitOps can be used in variety of different use cases like multiple environments and co-located clusters.

Transcript

1. Continuous Deployment for Kubernetes
2. Waldemar Biller Software Architect, Manager 10+ years in IT Java, Spring, DDD, Kubernetes, Vue, Flutter, ... Email: wb@biller.digital Twitter: @wbiller
3. CI / CD Developer Code Repository Continuous Integration Container Registry kubectl / helm Push Build & Test Publish D eploy R un Icons: CC BY 3.0
4. 3:03
5. CI / CD Developer Code Repository Continuous Integration Container Registry kubectl / helm Push Build & Test Publish D eploy R un Icons: CC BY 3.0 Where is your state? What’s your MTTR in case of a complete outage? Why does your CI system has access to the cluster?
6. The single source of truth
7. The entire system is described declaratively Icons: CC BY 3.0
8. The canonical desired system state is versioned in Git Icons: CC BY 3.0
9. Approved changes to the desired state are automatically applied to the system Icons: CC BY 3.0
10. Software agents ensure correctness and alert on divergence Icons: CC BY 3.0
11. GitOps Compatibility State Repository Declaratively Automatic Rollouts Notifications Keel https://keel.sh Can be activated only via annotations No Exposes Webhooks to trigger deployment Slack, HipChat, etc. ArgoCD https://argoproj.github.io/ Relies on Helm, Ksonnet, Kubernetes, etc. State is held in the application Updates have to triggered manually through UI No Flux CD https://www.fluxcd.io/ Relies on Kubernetes, CRD for Helm Charts, Integrations available Yes Compares current state and desired state and triggers deployment Yes, via Fluxcloud Icons: CC BY 3.0
12. Flux CD GitOps in Practice
13. Developer Code Repository Continuous Integration Container Registry Ops State Repository State Sync Deployment Sync Icons: CC BY 3.0 CI / CD with GitOps Push Build & Test Publish Update Pull Watch / Download Notify
14. Developer Code Repository Continuous Integration Container Registry Ops State Repository State Sync Deployment Sync Icons: CC BY 3.0 CI / CD with GitOps Push Build & Test Publish Update Pull Watch / Download Notify
15. Developer Code Repository Continuous Integration Container Registry Ops State Repository State Sync Deployment Sync Icons: CC BY 3.0 CI / CD with GitOps Push Build & Test Publish Update Pull Watch / Download Notify
16. Workflow Push Code to Code Repository Push the code the Code Repository and follow usual workflow with PRs and Reviews. Build & Test Build the Code and execute the Unit and Integration Tests. Publish Publish the Docker Image to the Container Registry. Optionally the Helm Chart can be published to a Chart Museum. Notify or create PR directly Optionally notify Ops about the new version, so they can roll-out the application Push State changes Push the state changes to the State Repository and follow the usual workflow with PRs and Reviews. Sync State Watch the State Repository and sync on changes. Download Images and Deploy Download the new Docker Image and run the deployment Developer Operations
17. Developer Code Repository Continuous Integration Container Registry Ops State Repository State Sync Deployment Sync Icons: CC BY 3.0 CI / CD with GitOps Push Build & Test Publish Update Pull Watch / Download Notify
18. Workflow Watch the Container Registry Watch the Container Registry for newer versions and optionally download the new images. Update State Update the state definition to match the image version. Automatic
19. Automation Deployments are always executed when state repository is updated Automation is activated by annotations: flux.weave.works/automated: "true" Supports SemVer, RegEx and Glob Patterns: flux.weave.works/tag.fluxcloud: semver:^0.34.0 flux.weave.works/tag.fluxcloud: glob:0.34.* flux.weave.works/tag.fluxcloud: regex:dev-.*
20. Installation helm repo add fluxcd https://charts.fluxcd.io helm upgrade -i flux \ --set helmOperator.create=true \ --set helmOperator.createCRD=true \ --set git.url=git@github.com:wbiller/state-repository \ --set git.path=development \ --set git.branch=master \ --namespace flux \ fluxcd/flux fluxctl identity --k8s-fwd-ns flux
21. Custom Resources HelmRelease (Chart) spec: chart: repository: # Chart Museum URL name: # Chart name version: # Chart version releaseName: # Name to use for release values: # Overwrites for chart variables valuesFrom: # Sources for overwrites configMapKeyRef: name: key: optional: secretKeyRef: name: key: optional: externalSourceRef: url: optional: HelmRelease (Git) spec: chart: git: # Repository URL path: # Path to chart branch: # Branch to use releaseName: # Name to use for release values: # Overwrites for chart variables valuesFrom: # Sources for overwrites configMapKeyRef: name: key: optional: secretKeyRef: name: key: optional: externalSourceRef: url: optional:
22. Helm Charts apiVersion: flux.weave.works/v1beta1 kind: HelmRelease metadata: name: nginx-ingress spec: chart: repository: https://kubernetes-charts.storage.googleapis.com/ name: nginx-ingress version: 1.24.3 releaseName: nginx-ingress values: controller: config: server-tokens: "false" use-forwarded-headers: "true"
23. Separate Repositories preview production
24. Separate Branches master preview production
25. preview production Separate Directories development
26. Directory Structure . ├── README.md ├── common │ ├── cert-manager-issuers-crd.yaml │ ├── letsencrypt-prod-ClusterIssuer.yaml │ ├── monitoring │ │ ├── prometheus-HelmRelease.yaml │ │ └── prometheus-operator-HelmRelease.yaml │ ├── monitoring-ns.yaml │ ├── ext-system │ │ ├── cert-manager-HelmRelease.yaml │ │ ├── flux-HelmRelease.yaml │ │ ├── nginx-ingress-HelmRelease.yaml │ └── ext-system-ns.yaml ├── dev │ ├── application │ │ └── application-HelmRelease.yaml └── test └── application └── application-HelmRelease.yaml
27. Manifest factorization Integration with Kustomize to patch base definitions for specific targets/envs Requires additional parameters for fluxd apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: webapp namespace: webapp spec: rules: - host: dev.example.com http: paths: - backend: serviceName: webapp servicePort: 80 path: / namespace: webapp bases: - ../base patchesStrategicMerge: - ingress.yaml apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: webapp namespace: webapp spec: rules: - host: www.example.com http: paths: - backend: serviceName: webapp servicePort: 80 path: /app base/ingress.yaml prod/kustomization.yaml prod/ingress.yaml
28. Multiple Environments path: common, preview path: common, production State Repository Preview Production
29. Blue/Green Migrations State Repository branch: production branch: production-new
30. Co-Located Cluster Europe Americas State Repository path: common, production path: common, production
31. Manage Clusters with GitOps Cluster API allows creation of new clusters from within Kubernetes Custom Resource Definitions for Clusters, Machines and Kubeadm Config Current list of implementations: AWS, Azure, GCP, Bare Metal, OpenStack, etc. See https://cluster-api.sigs.k8s.io/user/quick-start.html
32. State Repository Management Cluster New Cluster Read cluster definition Read desired state Deploy cluster Cluster Management
33. Sealed Secrets Prevent leaking clear text secrets into version control Provides certificate that’s created inside the cluster Secrets are encrypted using the certificate and decrypted in the cluster Provides a kubeseal CLI Create a intermediate secret in JSON kubectl -n ext-system create secret generic token --from-literal=token=token --dry-run -o json > token.json Encrypt the secret kubeseal --controller-namespace=ext-system --controller-name=sealed-secrets \ --format=yaml < token.json > token-sealedsecret.yaml
34. Thank you! Waldemar Biller Email: wb@biller.digital Twitter: @wbiller