Progressive Delivery in the Kubernetes Era - Voxxed Days Luxembourg

Slide 1

Slide 1 text

Next-gen CI/CD with Gitops and Progressive Delivery Kevin Dubois Principal Developer Advocate, Red Hat

Slide 2

Slide 2 text

Kevin Dubois ● Principal Developer Advocate at Red Hat ● 18+ years of experience building and delivering software applications ● Speak English, Dutch, French, Italian ● Passionate about improving Dev Experience with Open Source @[email protected] youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois @kevindubois.com

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Developer Flow Outer loop Inner loop Pull/Merge Request Production Build / Package Code Push Debug Code Review Build Deploy Security Tests Compliance Inner loop Outer loop Developer Test

Slide 6

Slide 6 text

Today, let’s focus on the “Outer Loop” The outer loop consists of the larger team processes that your code flows through on its way to the cluster: code reviews, integration tests, security and compliance. It needs be transparent and fast for developers from all teams. Outer Loop Development

Slide 7

Slide 7 text

Love Thy Mono Every 4 months Every week/day/hour

Slide 8

Slide 8 text

CI - CD - CD Build Test Security Checks Release Deploy Stage Deploy Prod Continuous Integration Continuous Delivery Continuous Deployment Manual Auto

Slide 9

Slide 9 text

Continuous Delivery… of a racing game :)

Slide 10

Slide 10 text

The application Push to give energy windmill Kafka Topic 2.Sends the interaction Dashboard: Green Energy Nickname Team Push/Tap to generate energy Cars that needs energy Two teams competing (top 5 players) First wins

Slide 11

Slide 11 text

Architecture 3: Generate power (REST) Game Dashboard 1: Assign player Name & Team (REST) 6: Update dashboard (SSE) 2: Increment player cluster counter 4: Send power event 5: Receive power events

Slide 12

Slide 12 text

YOU PLAY! Scan the QR Code with your phone to play

Slide 13

Slide 13 text

What if we added a new feature?

Slide 14

Slide 14 text

Dev Ops Friday | 4:45 PM Wall of confusion

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

Developer Flow Outer loop Inner loop Pull/Merge Request Production Build / Package Code Push Debug Code Review Build Deploy Security Tests Compliance Inner loop Outer loop Developer Test

Slide 17

Slide 17 text

Cloud-Native CI/CD Containers Built for container apps and runs on Kubernetes Designed with microservices and distributed teams in mind DevOps Serverless Runs serverless with no CI/CD engine to manage and maintain

Slide 18

Slide 18 text

Why Cloud-Native CI/CD? Traditional CI/CD Cloud-Native CI/CD Designed for Virtual Machines Designed for Containers and Kubernetes Require IT Ops for CI engine maintenance Pipeline as a service with no Ops overhead Plugins shared across CI engine Pipelines fully isolated from each other Plugin dependencies with undefined update cycles Everything lifecycled as container images No interoperability with Kubernetes resources Native Kubernetes resources Admin manages persistence Platform manages persistence Config baked into CI engine container Configured via Kubernetes ConfigMaps Declarative !

Slide 19

Slide 19 text

Tekton is a Graduated Continuous Delivery Foundation project and follows the OpenSSF best practices. Contributions from Google, Red Hat, Cloudbees, IBM, Elastic, Puppet, and many more An open-source project for providing a set of shared and standard components for building Kubernetes-style CI/CD systems https://tekton.dev

Slide 20

Slide 20 text

Step • Runs commands within container(builder image) • Mounts volumes, uses env vars • Eg. ‘mvn test’ or ‘git clone’ Task • A list of steps that are executed in sequential order • Takes inputs, outputs parameters Task Run • Runs a individual Task Pipeline • List of tasks defined to run in a certain order • Takes inputs, outputs parameters Pipeline Run • Runs a Pipeline Typed Decoupled Cloud Native Declarative Tekton Concepts

Slide 21

Slide 21 text

apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: wind-turbine-pipeline spec: params: - name: MANIFESTS_GIT_REPO type: string tasks: - name: git-clone params: - name: url value: $(params.GIT_REPO) workspaces: - name: output workspace: source workspaces: - name: source

Slide 22

Slide 22 text

Tekton Hub Search, discover and install Tekton Tasks hub.tekton.dev

Slide 23

Slide 23 text

Tekton CLI(tkn) •List and Describe • Pipeline • Resource • Task • Task Run • Pipeline Run •View logs • Task Run • Pipeline Run •https://github.com/tektoncd/cli

Slide 24

Slide 24 text

Gitops

Slide 25

Slide 25 text

What is GitOps? Treat everything as code Git is the single source of truth Operations through Git workflows

Slide 26

Slide 26 text

CI/CD Engines Jenkins Spinnaker Tekton Concourse CI …... CI/CD versus GitOps Desired State Cluster State Observe State Take Action GitOps Engines ACM, ArgoCD, FluxCD Razee, Faros Desired State Cluster State

Slide 27

Slide 27 text

ArgoCD Sync Monitor Detect drift Take action Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes. Cluster and application configuration versioned in Git Automatically syncs configuration from Git to clusters Drift detection, visualization and correction

Slide 28

Slide 28 text

Let’s deploy our new feature in a Modern, Automated, Gitops way!

Slide 29

Slide 29 text

Live Coding

Slide 30

Slide 30 text

Source Git Repository Image Registry CI GitOps Application Delivery Model

Slide 31

Slide 31 text

Source Git Repository Image Registry CI Config Git Repository Kubernetes CD Pull Request / Commit Push Pull GitOps Application Delivery Model

Slide 32

Slide 32 text

GitOps Application Delivery Model Push Pull Pull Request Source Git Repository Image Registry Config Git Repository Kubernetes Deploy Monitor Detect drift CD Take action

Slide 33

Slide 33 text

V2 Scan the QR Code with your phone to play

Slide 34

Slide 34 text

By default: a big bang / all or nothing release

Slide 35

Slide 35 text

Progressive Delivery

Slide 36

Slide 36 text

What is Progressive Delivery? Build Test Security Checks Release Deploy Stage Deploy Prod Continuous Integration Continuous Delivery Continuous Deployment Manual Auto

Slide 37

Slide 37 text

What is Progressive Delivery? ● No Big Bang ● Deploy != Release ● Metrics ● Subset of Users

Slide 38

Slide 38 text

Why Progressive Delivery? ● Decreases Downtime ● Limits the Tragedy ● Deploy & Release to Production faster ● Less mocking or setting up unreliable ‘fake’ services

Slide 39

Slide 39 text

Delivery Techniques

Slide 40

Slide 40 text

Blue Green Deployment ● All Or Nothing ● Quick Rollback

Slide 41

Slide 41 text

Canary Releases ● Small Percentage ● Increase depending on metrics

Slide 42

Slide 42 text

Dark Launches ● Mirroring Traffic ● Dark Canaries ● Feature Flags

Slide 43

Slide 43 text

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

The New Pyramid?

Slide 46

Slide 46 text

How to accomplish Progressive Delivery

Slide 47

Slide 47 text

Accomplishing Progressive Delivery with

Slide 48

Slide 48 text

Blue - Green apiVersion: v1 kind: Service metadata: name: my-service labels: app: mystuff spec: ports: - name: http port: 8000 selector: inservice: mypods type: LoadBalancer apiVersion: apps/v1 kind: Deployment metadata: name: mynode-deployment spec: replicas: 1 selector: matchLabels: app: mynode template: metadata: labels: app: mynode spec: containers: - name: mynode image: quay.io/rhdevelopers/mynode:v1 ports: - containerPort : 8000 kubectl label pod -l app=mynode inservice=mypods

Slide 49

Slide 49 text

Canary Releases kubectl scale deployment myapp-v1 --replicas=3 kubectl scale deployment myapp-v2 --replicas=1

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

Controlling Microservices with a Service Mesh Code Independent (Polyglot) • Intelligent Routing and Load-Balancing • Smarter Canary Releases • Dark Launch • Chaos: Fault Injection • Resilience: Circuit Breakers • Observability & Telemetry: Metrics and Tracing • Security: Encryption & Authorization • Fleet wide policy enforcement 51

Slide 52

Slide 52 text

Istio Architecture Control Plane The data plane is composed of a set of intelligent proxies (Envoy) deployed as sidecars. These proxies mediate and control all network communication between microservices. They also collect and report telemetry on all mesh traffic. The control plane manages and conﬁgures the proxies to route traffic. Data Plane

Slide 53

Slide 53 text

Pod Container JVM Service A Sidecar Container Pod Container JVM Service C Sidecar Container Pod Container JVM Service B Sidecar Container The sidecar intercepts all network traffic

Slide 54

Slide 54 text

Canary Release apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: recommendation spec: hosts: - recommendation http: - route: - destination: host: recommendation subset: version-v1 weight: 75 - destination: host: recommendation subset: version-v2 weight: 25

Slide 55

Slide 55 text

Shadowing Traffic apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: recommendation spec: hosts: - recommendation http: - route: - destination: host: recommendation subset: version-v1 mirror: host: recommendation subset: version-v2

Slide 56

Slide 56 text

Dark Canary apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: recommendation spec: hosts: - recommendation http: - match: - headers: end-user: exact: Alexandra route: - destination: host: recommendation subset: version-v2 - route: - destination: host: recommendation subset: version-v1

Slide 57

Slide 57 text

Automated Progressive Delivery

Slide 58

Slide 58 text

Argo Rollouts

Slide 59

Slide 59 text

Argo Rollouts ArgoCD detects change rollout Monitors Data

Slide 60

Slide 60 text

Rolling out automatically apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: name: rollouts-demo labels: app: rollouts-demo spec: strategy: canary: steps: - setWeight: 20 - pause: duration: "1m" - setWeight: 50 - pause: duration: "2m" canaryService: rollouts-demo-canary stableService: rollouts-demo-backend trafficRouting: istio: virtualService: name: rollout-vsvc routes: - primary …

Slide 61

Slide 61 text

“Smart” Progressive Delivery Based on Metrics

Slide 62

Slide 62 text

Metrics Based Rollouts strategy: canary: analysis: args: - name: service-name value: rollouts-demo-canary.canary.svc.cluster.local templates: - templateName: success-rate canaryService: rollouts-demo-canary stableService: rollouts-demo-stable trafficRouting: istio: virtualService: name: rollout-vsvc routes: - primary steps: - setWeight: 30 - pause: { duration: 20s } - setWeight: 40 - pause: { duration: 10s } - setWeight: 60 - pause: { duration: 10s } - setWeight: 80 - pause: { duration: 5s } - setWeight: 90 - pause: { duration: 5s } - setWeight: 100 - pause: { duration: 5s }

Slide 63

Slide 63 text

apiVersion: argoproj.io/v1alpha1 kind: AnalysisTemplate metadata: name: success-rate spec: args: - name: service-name metrics: - name: success-rate interval: 10s successCondition: len(result) == 0 || result[0] >= 0.95 failureLimit: 2 provider: prometheus: address: https://internal:[email protected]:9090 query: | sum(irate(istio_requests_total{ reporter="source", destination_service=~"{{args.service-name}}", response_code!~"5.*"}[30s]) ) Metrics Based Rollouts

Slide 64

Slide 64 text

Progressive Delivery Demo @kevindubois 64 🚀🚀🚀

Slide 65

Slide 65 text

Final Notes ● State is always hard, start with stateless ● Step by Step ● Embrace GitOps ● If you haven’t automatically destroyed something by mistake, you aren’t automating enough ● Demos ○ https://dn.dev/istio-tutorial ○ https://github.com/kdubois/progressive-delivery ○ https://github.com/redhat-developer-demos/bubbles-progressive-delivery ○ github.com/redhat-developer-demos/quinoa-wind-turbine 65

Slide 66

Slide 66 text

Start exploring in the OpenShift Sandbox. Learn containers, Kubernetes, and OpenShift in your browser. developers.redhat.com/developer-sandbox Try Red Hat's products and technologies without setup or configuration.

Slide 67

Slide 67 text

https://red.ht/gitops-cookbook

Slide 68

Slide 68 text

68 Learn OpenShift and all Cloud Native for free in our web based learning portal Resources Try OpenShift GitOps on our learning portal Red Hat Developers Deep Dive Join our workshops to explore Argo CD, Tekton, Kubernetes, Helm and more! Sign up at developers.redhat.com Find out more about Red Hat’s products and what it offers developers GitOps ebooks Find out more on GitOps with ebooks https://developers.redhat.com/e-books Get more about GitOps on Red Hat Developer Portal!

Slide 69

Slide 69 text

Join Red Hat Developer. Build here. Go anywhere. facebook.com/RedHatDeveloper youtube.com/RedHatDevelopers twitter.com/rhdevelopers linkedin.com/showcase/red-hat-developer Thank you!