Next-gen CI/CD with Gitops and Progressive Delivery | Code Europe

Transcript

1. Next-gen CI/CD with Gitops and Progressive Delivery Kevin Dubois Principal Developer Advocate, Red Hat
2. @kevindubois Kevin Dubois • Principal Developer Advocate at Red Hat • 18+ years of experience writing and delivering software applications • Speak English, Dutch, French, Italian • Passionate about improving Dev Experience with Open Source @kevindubois@mastodon.social youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois
3. @kevindubois Developer Flow Outer loop Inner loop Pull/Merge Request Production Build / Package Code Push Debug Code Review Build Deploy Security Tests Compliance Inner loop Outer loop Developer Test
4. @kevindubois @kevindubois Today, let’s focus on the “Outer Loop” The outer loop consists of the larger team processes that your code flows through on its way to the cluster: code reviews, integration tests, security and compliance. It needs be transparent and fast for developers from all teams. Outer Loop Development
5. @kevindubois Love Thy Mono Every 4 months Every week/day/hour
6. @kevindubois CI / CD Build Test Security Checks Release Deploy Stage Deploy Prod Continuous Integration Continuous Delivery Continuous Deployment Manual Auto
7. @kevindubois Continuous Delivery of a game :)
8. @kevindubois The application Push to give energy windmill 1.Sends click Kafka Topic 2.Sends the interaction Dashboard: Green Energy Nickname Team Push/Tap to generate energy Cars that need energy Two teams competing (top 5 players) First wins
9. @kevindubois Architecture 3: Generate power (REST) Game Dashboard 1: Assign player Name & Team (REST) 6: Update dashboard (SSE) 2: Increment player cluster counter 4: Send power event 5: Receive power events 9
10. @kevindubois YOU PLAY! 10 red.ht/3Wubuxu
11. @kevindubois What if we added a new feature?
12. @kevindubois Dev Ops Friday | 4:45 PM Wall of confusion
13. @kevindubois
14. @kevindubois Developer Flow Outer loop Inner loop Pull/Merge Request Production Build / Package Code Push Debug Code Review Build Deploy Security Tests Compliance Inner loop Outer loop Developer Test
15. @kevindubois Live Coding
16. @kevindubois Tekton is a Graduated Continuous Delivery Foundation project and follows the OpenSSF best practices. Contributions from Google, Red Hat, Cloudbees, IBM, Elastic, Puppet, and many more An open-source project for providing a set of shared and standard components for building Kubernetes-style CI/CD systems https://tekton.dev
17. @kevindubois 17 Gitops
18. @kevindubois What is GitOps? Treat everything as code Git is the single source of truth Operations through Git workflows
19. @kevindubois GitOps Application Delivery Model Source Git Repository Image Registry CI
20. @kevindubois GitOps Application Delivery Model Source Git Repository Image Registry CI Config Git Repository Kubernetes CD Pull Request / Commit Push Pull
21. @kevindubois V2 red.ht/3Wubuxu
22. @kevindubois Progressive Delivery 22
23. @kevindubois What is Progressive Delivery? Build Test Security Checks Release Deploy Stage Deploy Prod Continuous Integration Continuous Delivery Continuous Deployment Manual Auto
24. @kevindubois What is Progressive Delivery? • No Big Bang • Deploy != Release • Metrics • Subset of Users
25. @kevindubois Why Progressive Delivery? • Decreases Downtime • Limits the Tragedy • Deploy & Release to Production faster • Less mocking or setting up unreliable ‘fake’ services
26. @kevindubois 26
27. @kevindubois Delivery Techniques
28. @kevindubois Blue Green Deployment • All Or Nothing • Quick Rollback 28
29. @kevindubois Canary Releases • Small Percentage • Increase depending on metrics 29
30. @kevindubois Dark Launches • Mirroring Traffic • Dark Canaries • Feature Flags 30
31. @kevindubois 31
32. @kevindubois The New Pyramid? 32
33. @kevindubois How to accomplish Progressive Delivery 33
34. @kevindubois Accomplishing Progressive Delivery with
35. @kevindubois Blue - Green 35 apiVersion: v1 kind: Service metadata: name: my-service labels: app: mystuff spec: ports: - name: http port: 8000 selector: inservice: mypods type: LoadBalancer apiVersion: apps/v1 kind: Deployment metadata: name: mynode-deployment spec: replicas: 1 selector: matchLabels: app: mynode template: metadata: labels: app: mynode spec: containers: - name: mynode image: quay.io/rhdevelopers/mynode:v1 ports: - containerPort: 8000 kubectl label pod -l app=mynode inservice=mypods
36. @kevindubois Canary Releases kubectl scale deployment myapp-v1 --replicas=3 kubectl scale deployment myapp-v2 --replicas=1 36
38. @kevindubois Controlling Microservices with a Service Mesh 38 Code Independent (Polyglot) • Intelligent Routing and Load-Balancing • Smarter Canary Releases • Dark Launch • Chaos: Fault Injection • Resilience: Circuit Breakers • Observability & Telemetry: Metrics and Tracing • Security: Encryption & Authorization • Fleet wide policy enforcement
39. @kevindubois 39 Istio Architecture Control Plane The data plane is composed of a set of intelligent proxies (Envoy) deployed as sidecars. These proxies mediate and control all network communication between microservices. They also collect and report telemetry on all mesh traffic. The control plane manages and configures the proxies to route traffic. Data Plane
40. @kevindubois Pod Container JVM Service A Sidecar Container Pod Container JVM Service C Sidecar Container Pod Container JVM Service B Sidecar Container The sidecar intercepts all network traffic
41. @kevindubois Canary Release 41 apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: recommendation spec: hosts: - recommendation http: - route: - destination: host: recommendation subset: version-v1 weight: 75 - destination: host: recommendation subset: version-v2 weight: 25
42. @kevindubois Shadowing Traffic 42 apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: recommendation spec: hosts: - recommendation http: - route: - destination: host: recommendation subset: version-v1 mirror: host: recommendation subset: version-v2
43. @kevindubois Dark Canary 43 apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: recommendation spec: hosts: - recommendation http: - match: - headers: end-user: exact: Alexandra route: - destination: host: recommendation subset: version-v2 - route: - destination: host: recommendation subset: version-v1
44. @kevindubois Automated Progressive Delivery 44
45. @kevindubois 45 Argo Rollouts
46. @kevindubois Argo Rollouts 46 ArgoCD detects change rollout Monitors Data
47. @kevindubois Rolling out automatically 47 apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: name: rollouts-demo labels: app: rollouts-demo spec: strategy: canary: steps: - setWeight: 20 - pause: duration: "1m" - setWeight: 50 - pause: duration: "2m" canaryService: rollouts-demo-canary stableService: rollouts-demo-backend trafficRouting: istio: virtualService: name: rollout-vsvc routes: - primary …
48. @kevindubois “Smart” Progressive Delivery Based on Metrics 48
49. @kevindubois Metrics Based Rollouts strategy: canary: analysis: args: - name: service-name value: rollouts-demo-canary.canary.svc.cluster.local templates: - templateName: success-rate canaryService : rollouts-demo-canary stableService : rollouts-demo-stable trafficRouting : istio: virtualService : name: rollout-vsvc routes: - primary steps: - setWeight: 30 - pause: { duration: 20s } - setWeight: 40 - pause: { duration: 10s } - setWeight: 60 - pause: { duration: 10s } - setWeight: 80 - pause: { duration: 5s } - setWeight: 90 - pause: { duration: 5s } - setWeight: 100 - pause: { duration: 5s } 49
50. @kevindubois apiVersion: argoproj.io/v1alpha1 kind: AnalysisTemplate metadata: name: success-rate spec: args: - name: service-name metrics: - name: success-rate interval: 10s successCondition : len(result) == 0 || result[0] >= 0.95 failureLimit: 2 provider: prometheus: address: https://internal:demo@prometheus.istio-system.svc.cluster.local:9090 query: | sum(irate(istio_requests_total{ reporter="source", destination_service=~"{{args.service-name}}", response_code!~"5.*"}[30s]) ) 50 Metrics Based Rollouts
51. @kevindubois Progressive Delivery Demo @kevindubois @kevindubois
52. @kevindubois Final Notes • State is always hard, start with stateless • Step by Step • Embrace GitOps • If you haven’t automatically destroyed something by mistake, you aren’t automating enough • Demos ◦ https://dn.dev/istio-tutorial ◦ https://github.com/kdubois/progressive-delivery ◦ https://github.com/redhat-developer-demos/bubbles-progressive-delivery ◦ github.com/redhat-developer-demos/quinoa-wind-turbine 52
53. @kevindubois Start exploring in the OpenShift Sandbox. Learn containers, Kubernetes, and OpenShift in your browser. developers.redhat.com/developer-sandbox Try Red Hat's products and technologies without setup or configuration.
54. @kevindubois https://red.ht/gitops-cookbook
55. @kevindubois 55 Learn OpenShift and all Cloud Native for free in our web based learning portal Resources Try OpenShift GitOps on our learning portal Red Hat Developers Deep Dive Join our workshops to explore Argo CD, Tekton, Kubernetes, Helm and more! Sign up at developers.redhat.com Find out more about Red Hat’s products and what it offers developers GitOps ebooks Find out more on GitOps with ebooks https://developers.redhat.com/e-books Get more about GitOps on Red Hat Developer Portal!
56. @kevindubois https://red.ht/modernize-enterprise-java
57. @kevindubois speakerdeck.com/kdubois/ cd-with-gitops-and-progressive-delivery-code-europe
58. @kevindubois #CodeEurope @kevindubois@mastodon.social youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois Thank you!