Next-gen CI/CD with Gitops and Progressive Delivery | Jfokus

Transcript

1. @kevindubois Next-gen CI/CD with Gitops and Progressive Delivery Kevin Dubois Principal Developer Advocate, Red Hat @kevindubois
2. @kevindubois Kevin Dubois ★ Java Champion ★ Principal Developer Advocate at Red Hat ★ Based in Belgium 󰎐 ★ Speak English, Dutch, French, Italian ★ Open Source Contributor (Quarkus, Camel, Knative, ..) ★ Community Organizer (BeJUG, BeCNCF) @kevindubois@mastodon.social youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois @kevindubois.com
3. @kevindubois How it started…
4. @kevindubois
5. @kevindubois
6. @kevindubois CI / CD
7. @kevindubois CI / CD Build Test Security Checks Release Deploy Stage Deploy Prod Continuous Integration Continuous Delivery Manual
8. @kevindubois Every 4 months Every week/day/hour
9. @kevindubois Continuous Developer Flow Outer loop Inner loop Pull/Merge Request Production Build / Package Code Push Debug Code Review Build Deploy Security Tests Compliance Inner loop Outer loop Developer Test
10. @kevindubois CI - CD - CD Build Test Security Checks Release Deploy Stage Deploy Prod Continuous Integration Continuous Delivery Continuous Deployment Manual Auto
11. @kevindubois Let’s play a little game :)
12. @kevindubois The application Push to give energy windmill Kafka Topic 2.Sends the interaction Dashboard: Green Energy Nickname Team Push/Tap to generate energy Cars that need energy Two teams competing (top 5 players) First wins
13. @kevindubois Architecture 3: Generate power (REST) Game Dashboard 1: Assign player Name & Team (REST) 6: Update dashboard (SSE) 2: Increment player cluster counter 4: Send power event 5: Receive power events
14. @kevindubois 14 Team Köttbullar
15. @kevindubois vs … 15
16. @kevindubois 16
17. @kevindubois 17
18. @kevindubois Team Surströmming
19. @kevindubois YOU PLAY! Scan the QR Code with your phone to play
20. @kevindubois What if we added a new feature?
21. @kevindubois Dev Ops Friday | 4:45 PM Wall of confusion
22. @kevindubois Live Coding
23. @kevindubois
24. @kevindubois Developer Flow Outer loop Inner loop Pull/Merge Request Production Build / Package Code Push Debug Code Review Build Deploy Security Tests Compliance Inner loop Outer loop Developer Test
25. @kevindubois Serverless CI Containers Built for container apps and runs on Kubernetes Designed with microservices and distributed teams in mind DevOps Serverless Runs serverless with no CI/CD engine to manage and maintain
26. @kevindubois Why Cloud-Native CI/CD? Traditional CI/CD Cloud-Native CI/CD Designed for Virtual Machines Designed for Containers and Kubernetes Require IT Ops for CI engine maintenance Pipeline as a service with no Ops overhead Plugins shared across CI engine Pipelines fully isolated from each other Plugin dependencies with undefined update cycles Everything lifecycled as container images No interoperability with Kubernetes resources Native Kubernetes resources Admin manages persistence Platform manages persistence Config baked into CI engine container Configured via Kubernetes ConfigMaps Declarative !
27. @kevindubois Tekton is a Graduated Continuous Delivery Foundation project and follows the OpenSSF best practices. Contributions from Google, Red Hat, Cloudbees, IBM, Elastic, Puppet, and many more An open-source project for providing a set of shared and standard components for building Kubernetes-style CI/CD systems https://tekton.dev
28. @kevindubois Step • Runs commands within container(builder image) • Mounts volumes, uses env vars • Eg. ‘mvn test’ or ‘git clone’ Task • A list of steps that are executed in sequential order • Takes inputs, outputs parameters Task Run • Runs a individual Task Pipeline • List of tasks defined to run in a certain order • Takes inputs, outputs parameters Pipeline Run • Runs a Pipeline Typed Decoupled Cloud Native Declarative Tekton Concepts
29. @kevindubois apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: wind-turbine-pipeline spec: params: - name: MANIFESTS_GIT_REPO type: string tasks: - name: git-clone params: - name: url value: $(params.GIT_REPO) workspaces: - name: output workspace: source workspaces: - name: source
30. @kevindubois Tekton Hub Search, discover and install Tekton Tasks hub.tekton.dev
31. @kevindubois Tekton CLI(tkn) •List and Describe • Pipeline • Resource • Task • Task Run • Pipeline Run •View logs • Task Run • Pipeline Run •https://github.com/tektoncd/cli
32. @kevindubois Gitops
33. @kevindubois What is GitOps? Treat everything as code Git is the single source of truth Operations through Git workflows
34. @kevindubois CI/CD Engines Jenkins Spinnaker Tekton Concourse CI …... CI/CD versus GitOps Desired State Cluster State Observe State Take Action GitOps Engines ACM, ArgoCD, FluxCD Razee, Faros Desired State Cluster State
35. @kevindubois ArgoCD Sync Monitor Detect drift Take action Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes. Cluster and application configuration versioned in Git Automatically syncs configuration from Git to clusters Drift detection, visualization and correction
36. @kevindubois Source Git Repository Image Registry CI GitOps Application Delivery Model
37. @kevindubois Source Git Repository Image Registry CI Config Git Repository Kubernetes CD Pull Request / Commit Push Pull GitOps Application Delivery Model
38. @kevindubois GitOps Application Delivery Model Push Pull Pull Request Source Git Repository Image Registry Config Git Repository Kubernetes Deploy Monitor Detect drift CD Take action
39. @kevindubois V2 Scan the QR Code with your phone to play
40. @kevindubois By default: a big bang / all or nothing release
41. @kevindubois Progressive Delivery
42. @kevindubois What is Progressive Delivery? • No Big Bang • Deploy != Release • Metrics • Subset of Users
43. @kevindubois Why Progressive Delivery? • Decreases Downtime • Limits the Tragedy • Deploy & Release to Production faster • Less mocking or setting up unreliable ‘fake’ services
44. @kevindubois Delivery Techniques
45. @kevindubois Blue Green Deployment • All Or Nothing • Quick Rollback 45
46. @kevindubois Canary Releases • Small Percentage • Increase depending on metrics 46
47. @kevindubois Dark Launches • Mirroring Traffic • Dark Canaries • Feature Flags 47
48. @kevindubois 48
49. @kevindubois The New Pyramid? 49
50. @kevindubois How to accomplish Progressive Delivery 50
51. @kevindubois Accomplishing Progressive Delivery with 51
52. @kevindubois Blue - Green apiVersion: v1 kind: Service metadata: name: my-service labels: app: mystuff spec: ports: - name: http port: 8000 selector: inservice: mypods type: LoadBalancer apiVersion: apps/v1 kind: Deployment metadata: name: mynode-deployment spec: replicas: 1 selector: matchLabels: app: mynode template: metadata: labels: app: mynode spec: containers: - name: mynode image: quay.io/rhdevelopers/mynode:v1 ports: - containerPort : 8000 kubectl label pod -l app=mynode inservice=mypods 52
53. @kevindubois Canary Releases kubectl scale deployment myapp-v1 --replicas=3 kubectl scale deployment myapp-v2 --replicas=1 53
54. @kevindubois
55. @kevindubois Controlling Microservices with a Service Mesh Code Independent (Polyglot) • Intelligent Routing and Load-Balancing • Smarter Canary Releases • Dark Launch • Chaos: Fault Injection • Resilience: Circuit Breakers • Observability & Telemetry: Metrics and Tracing • Security: Encryption & Authorization • Fleet wide policy enforcement 55
56. @kevindubois Istio Architecture Control Plane The data plane is composed of a set of intelligent proxies (Envoy) deployed as sidecars. These proxies mediate and control all network communication between microservices. They also collect and report telemetry on all mesh traffic. The control plane manages and configures the proxies to route traffic. Data Plane
57. @kevindubois Pod Container JVM Service A Sidecar Container Pod Container JVM Service C Sidecar Container Pod Container JVM Service B Sidecar Container The sidecar intercepts all network traffic 57
58. @kevindubois Canary Release apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: recommendation spec: hosts: - recommendation http: - route: - destination: host: recommendation subset: version-v1 weight: 75 - destination: host: recommendation subset: version-v2 weight: 25 58
59. @kevindubois Shadowing Traffic apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: recommendation spec: hosts: - recommendation http: - route: - destination: host: recommendation subset: version-v1 mirror: host: recommendation subset: version-v2 59
60. @kevindubois Dark Canary apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: recommendation spec: hosts: - recommendation http: - match: - headers: end-user: exact: Alexandra route: - destination: host: recommendation subset: version-v2 - route: - destination: host: recommendation subset: version-v1 60
61. @kevindubois Automated Progressive Delivery 61
62. @kevindubois Argo Rollouts 62
63. @kevindubois Argo Rollouts ArgoCD detects change rollout Monitors Data 63
64. @kevindubois Rolling out automatically apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: name: rollouts-demo labels: app: rollouts-demo spec: strategy: canary: steps: - setWeight: 20 - pause: duration: "1m" - setWeight: 50 - pause: duration: "2m" canaryService: rollouts-demo-canary stableService: rollouts-demo-backend trafficRouting: istio: virtualService: name: rollout-vsvc routes: - primary … 64
65. @kevindubois “Smart” Progressive Delivery Based on Metrics 65
66. @kevindubois Metrics Based Rollouts strategy: canary: analysis: args: - name: service-name value: rollouts-demo-canary.canary.svc.cluster.local templates: - templateName: success-rate canaryService: rollouts-demo-canary stableService: rollouts-demo-stable trafficRouting: istio: virtualService: name: rollout-vsvc routes: - primary steps: - setWeight: 30 - pause: { duration: 20s } - setWeight: 40 - pause: { duration: 10s } - setWeight: 60 - pause: { duration: 10s } - setWeight: 80 - pause: { duration: 5s } - setWeight: 90 - pause: { duration: 5s } - setWeight: 100 - pause: { duration: 5s } 66
67. @kevindubois apiVersion: argoproj.io/v1alpha1 kind: AnalysisTemplate metadata: name: success-rate spec: args: - name: service-name metrics: - name: success-rate interval: 10s successCondition: len(result) == 0 || result[0] >= 0.95 failureLimit: 2 provider: prometheus: address: https://internal:demo@prometheus.istio-system.svc.cluster.local:9090 query: | sum(irate(istio_requests_total{ reporter="source", destination_service=~"{{args.service-name}}", response_code!~"5.*"}[30s]) ) Metrics Based Rollouts 67
68. @kevindubois Progressive Delivery Demo @kevindubois 68 🚀🚀🚀
69. @kevindubois Final Notes • State is always hard ◦ start with stateless; work with features; non-destructive schema changes; event-driven architectures (use eg. Debezium to work with ‘classic’ DBs). • Step by Step • Embrace GitOps • If you haven’t automatically destroyed something by mistake, you aren’t automating enough • Demos ◦ https://dn.dev/istio-tutorial ◦ https://github.com/kdubois/progressive-delivery ◦ https://github.com/redhat-developer-demos/bubbles-progressive-delivery ◦ github.com/redhat-developer-demos/quinoa-wind-turbine 69
70. @kevindubois Free Developer e-Books! https://developers.redhat.com/eventtutorials
71. @kevindubois Start exploring in the OpenShift Sandbox. Learn containers, Kubernetes, and OpenShift in your browser. developers.redhat.com/developer-sandbox Try Red Hat's products and technologies without setup or configuration.
72. @kevindubois Thank you! @kevindubois@mastodon.social youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois @kevindubois.com
73. @kevindubois Join Red Hat Developer. Build here. Go anywhere. facebook.com/RedHatDeveloper youtube.com/RedHatDevelopers twitter.com/rhdevelopers linkedin.com/showcase/red-hat-developer Thank you!