Continuous Deployment for Kubernetes 

Waldemar Biller Software Architect, Manager 10+ years in IT Java, Spring, DDD, Kubernetes, Vue, Flutter, ... Email: [email protected] Twitter: @wbiller 

CI / CD Developer Code Repository Continuous Integration Container Registry kubectl / helm Push Build & Test Publish D eploy R un Icons: CC BY 3.0 

3:03 

CI / CD Developer Code Repository Continuous Integration Container Registry kubectl / helm Push Build & Test Publish D eploy R un Icons: CC BY 3.0 Where is your state? What’s your MTTR in case of a complete outage? Why does your CI system has access to the cluster? 

The single source of truth 

The entire system is described declaratively Icons: CC BY 3.0 

The canonical desired system state is versioned in Git Icons: CC BY 3.0 

Approved changes to the desired state are automatically applied to the system Icons: CC BY 3.0 

Software agents ensure correctness and alert on divergence Icons: CC BY 3.0 

GitOps Compatibility State Repository Declaratively Automatic Rollouts Notifications Keel https://keel.sh Can be activated only via annotations No Exposes Webhooks to trigger deployment Slack, HipChat, etc. ArgoCD https://argoproj.github.io/ Relies on Helm, Ksonnet, Kubernetes, etc. State is held in the application Updates have to triggered manually through UI No Flux CD https://www.fluxcd.io/ Relies on Kubernetes, CRD for Helm Charts, Integrations available Yes Compares current state and desired state and triggers deployment Yes, via Fluxcloud Icons: CC BY 3.0 

Flux CD GitOps in Practice 

Developer Code Repository Continuous Integration Container Registry Ops State Repository State Sync Deployment Sync Icons: CC BY 3.0 CI / CD with GitOps Push Build & Test Publish Update Pull Watch / Download Notify 

Developer Code Repository Continuous Integration Container Registry Ops State Repository State Sync Deployment Sync Icons: CC BY 3.0 CI / CD with GitOps Push Build & Test Publish Update Pull Watch / Download Notify 

Developer Code Repository Continuous Integration Container Registry Ops State Repository State Sync Deployment Sync Icons: CC BY 3.0 CI / CD with GitOps Push Build & Test Publish Update Pull Watch / Download Notify 

Workflow Push Code to Code Repository Push the code the Code Repository and follow usual workflow with PRs and Reviews. Build & Test Build the Code and execute the Unit and Integration Tests. Publish Publish the Docker Image to the Container Registry. Optionally the Helm Chart can be published to a Chart Museum. Notify or create PR directly Optionally notify Ops about the new version, so they can roll-out the application Push State changes Push the state changes to the State Repository and follow the usual workflow with PRs and Reviews. Sync State Watch the State Repository and sync on changes. Download Images and Deploy Download the new Docker Image and run the deployment Developer Operations 

Developer Code Repository Continuous Integration Container Registry Ops State Repository State Sync Deployment Sync Icons: CC BY 3.0 CI / CD with GitOps Push Build & Test Publish Update Pull Watch / Download Notify 

Workflow Watch the Container Registry Watch the Container Registry for newer versions and optionally download the new images. Update State Update the state definition to match the image version. Automatic 

Automation Deployments are always executed when state repository is updated Automation is activated by annotations: flux.weave.works/automated: "true" Supports SemVer, RegEx and Glob Patterns: flux.weave.works/tag.fluxcloud: semver:^0.34.0 flux.weave.works/tag.fluxcloud: glob:0.34.* flux.weave.works/tag.fluxcloud: regex:dev-.* 

Installation helm repo add fluxcd https://charts.fluxcd.io helm upgrade -i flux \ --set helmOperator.create=true \ --set helmOperator.createCRD=true \ --set [email protected]:wbiller/state-repository \ --set git.path=development \ --set git.branch=master \ --namespace flux \ fluxcd/flux fluxctl identity --k8s-fwd-ns flux 

Custom Resources HelmRelease (Chart) spec: chart: repository: # Chart Museum URL name: # Chart name version: # Chart version releaseName: # Name to use for release values: # Overwrites for chart variables valuesFrom: # Sources for overwrites configMapKeyRef: name: key: optional: secretKeyRef: name: key: optional: externalSourceRef: url: optional: HelmRelease (Git) spec: chart: git: # Repository URL path: # Path to chart branch: # Branch to use releaseName: # Name to use for release values: # Overwrites for chart variables valuesFrom: # Sources for overwrites configMapKeyRef: name: key: optional: secretKeyRef: name: key: optional: externalSourceRef: url: optional: 

Helm Charts apiVersion: flux.weave.works/v1beta1 kind: HelmRelease metadata: name: nginx-ingress spec: chart: repository: https://kubernetes-charts.storage.googleapis.com/ name: nginx-ingress version: 1.24.3 releaseName: nginx-ingress values: controller: config: server-tokens: "false" use-forwarded-headers: "true" 

Separate Repositories preview production 

Separate Branches master preview production 

preview production Separate Directories development 

Directory Structure . ├── README.md ├── common │ ├── cert-manager-issuers-crd.yaml │ ├── letsencrypt-prod-ClusterIssuer.yaml │ ├── monitoring │ │ ├── prometheus-HelmRelease.yaml │ │ └── prometheus-operator-HelmRelease.yaml │ ├── monitoring-ns.yaml │ ├── ext-system │ │ ├── cert-manager-HelmRelease.yaml │ │ ├── flux-HelmRelease.yaml │ │ ├── nginx-ingress-HelmRelease.yaml │ └── ext-system-ns.yaml ├── dev │ ├── application │ │ └── application-HelmRelease.yaml └── test └── application └── application-HelmRelease.yaml 

Manifest factorization Integration with Kustomize to patch base definitions for specific targets/envs Requires additional parameters for fluxd apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: webapp namespace: webapp spec: rules: - host: dev.example.com http: paths: - backend: serviceName: webapp servicePort: 80 path: / namespace: webapp bases: - ../base patchesStrategicMerge: - ingress.yaml apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: webapp namespace: webapp spec: rules: - host: www.example.com http: paths: - backend: serviceName: webapp servicePort: 80 path: /app base/ingress.yaml prod/kustomization.yaml prod/ingress.yaml 

Multiple Environments path: common, preview path: common, production State Repository Preview Production 

Blue/Green Migrations State Repository branch: production branch: production-new 

Co-Located Cluster Europe Americas State Repository path: common, production path: common, production 

Manage Clusters with GitOps Cluster API allows creation of new clusters from within Kubernetes Custom Resource Definitions for Clusters, Machines and Kubeadm Config Current list of implementations: AWS, Azure, GCP, Bare Metal, OpenStack, etc. See https://cluster-api.sigs.k8s.io/user/quick-start.html 

State Repository Management Cluster New Cluster Read cluster definition Read desired state Deploy cluster Cluster Management 

Sealed Secrets Prevent leaking clear text secrets into version control Provides certificate that’s created inside the cluster Secrets are encrypted using the certificate and decrypted in the cluster Provides a kubeseal CLI Create a intermediate secret in JSON kubectl -n ext-system create secret generic token --from-literal=token=token --dry-run -o json > token.json Encrypt the secret kubeseal --controller-namespace=ext-system --controller-name=sealed-secrets \ --format=yaml < token.json > token-sealedsecret.yaml 

Thank you! Waldemar Biller Email: [email protected] Twitter: @wbiller