Progressive Delivery in the Kubernetes Era - Voxxed Days Bucharest

Production is the place where all applications should live. Even though you are using continuous integration and delivery, you might wonder every time you release a new version to production if it will work or there will be some breakage on the latest version, eventually making production unavailable to the customers.

Progressive delivery is the next step after Continuous Delivery to test your application in production before it becomes fully available to all your user bases.

Embrace progressive delivery with techniques like the blue-green, canary release, shadowing traffic, or dark launches to validate the application in production using Kubernetes and tools like Istio, Prometheus, ArgoCD, or Argo Rollouts.

Come to this session to learn progressive delivery and GitOps in action using Kubernetes.

Transcript

1. Progressive Delivery In the Kubernetes Era Kevin Dubois Principal Developer Advocate, Red Hat Natale Vinto, Head of Developer Advocacy, Red Hat
2. @kevindubois @natalevinto Natale Vinto $ whoami Developer Advocate for OpenShift, helping people within communities and customers having success with their Kubernetes and Cloud Native strategy. Love Free and Open Source software, Blues 🎸 and coffee ☕ @natalevinto OpenShift.TV Coffee Break every Wed 10 AM CEST
3. @kevindubois @natalevinto Kevin Dubois • Principal Developer Advocate at Red Hat • 18+ years of experience writing and delivering software applications • Speak English, Dutch, French, Italian • Passionate about improving Dev Experience with Open Source @kevindubois@mastodon.social youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois
4. @kevindubois @natalevinto Trial and Onboarding Inner Loop (for Code, Build, Debug and Push) Outer Loop (for Code review, Compliance, Security, Test and Deploy) Developer Sandbox Fastest and least friction way, at zero cost, for a developer to try our OpenShift platform, key developer tools and services, and Red Hat cloud services. https://developers.redhat. com/developer-sandbox OpenShift Local Formerly known as “Red Hat CodeReady Containers” Pre-built development environment, based on OpenShift v4, for quick container-based application development - on developer laptops. https://developers.redhat. com/products/openshift-lo cal/overview ◉ Podman Desktop This upstream project enables developers to easily build and share containerized applications, facilitating their transition to Kubernetes. https://podman.io ◉ Docker Desktop extension Enables deployment of local images to OpenShift directly from Docker Desktop. https://github.com/redhat-developer/openshift-dd-ext ◉ OpenShift Dev Spaces Formerly Red Hat CodeReady Workspaces Centralized, shareable, in-browser Kubernetes-based Integrated Development Environment, focused on OpenShift customers. https://developers.redhat.com/products/openshift-dev-spaces/overview ◉ IDE Extensions (for Visual Studio Code, IntelliJ, Eclipse) for Languages (Java, XML, YAML, Apache Camel) and Runtimes (Quarkus, EAP) Makes it easy for developers to build cloud-native and hybrid cloud applications with RH supported languages, and use RH preferred runtimes. ◉ CLI tools: odo OpenShift Developer CLI (command-line interface) that helps developers to build on Kubernetes/OpenShift, hiding Kubernetes complexity and supporting developers to get up and running Maven/Grade tooling (JKube) Adds the ability to bring Java applications to Kubernetes by extending native Maven/Gradle packaging tools. ◉ Dependency analytics Formerly known as “CodeReady Dependency Analytics” Hosted service, reachable by IDE extension and CLI/API for Software Composition Analysis and vulnerability remediation of Java, Javascript, Python, and Go applications. App Studio Hosted, fully managed experience to build full stack applications, easily connect to leading cloud services, adopt DevSecOps practices, and deploy to any hybrid-cloud platform. https://developers.redhat.com/app-studio/overview Plug-ins for CI/CD tools (GitHub Actions, Microsoft Azure DevOps, GitLab Runner, Jenkins) Provide templates for automated deployment of applications to OpenShift OpenShift Pipelines, Tekton Hub, Tekton IDE extension Is a cloud-native CI/CD solution to securely build containerized applications and deploy to OpenShift. Tekton Hub provides a catalog of Tekton task templates. Tekton IDE extension enables developers to create and manage Tekton pipelines from their IDE IDE Extensions (for Visual Studio Code, IntelliJ, Eclipse) for Deployment (OpenShift Connector, Serverless) Enable developers to push their code to OpenShift without needing to build containers, and to also configure serverless deployments. OpenShift GitOps Enables developers to declare and version application and cluster configuration in Git, automate application deployments, configuration drift detection and remediation. kam (Kubernetes application management) enables developers to easily seed GitOps declarative application models in Git. OpenShift Builds Provides a Kubernetes-native build tool on OpenShift. It supports multiple build strategies, and generates container images that are portable across Kubernetes distributions. OpenShift UI tools: OpenShift Dev Console Provides an end-to-end developer experience in OpenShift, across all cloud platforms, for building, deploying, and managing containerized applications and their related services. Web Terminal Runs within the OpenShift Dev Console. It provides a command line interface, pre-loaded with developer-focused CLIs. Helm is a package manager for Kubernetes. We provide support for that packaging in OpenShift, and a certified catalog for developers to quickly create applications based on curated Helm charts. Service Binding Enables developers to easily connect their applications to services running on the OpenShift cluster and Red Hat cloud services. Developer Tools Portfolio
5. @kevindubois @natalevinto Agenda Progressive Delivery Explained Delivery Techniques Progressive Delivery in Practice Continuous Delivery Metrics based Progressive Delivery
6. @kevindubois @natalevinto What we’ll talk about today • Inner/Outer Loop • Continuous Delivery • CD with Kubernetes • Gitops • Traffic control with Istio • Progressive Delivery • Argo Rollouts
7. @kevindubois @natalevinto Developer Flow Outer loop Inner loop Pull/Merge Request Production Build / Package Code Push Debug Code Review Build Deploy Security Tests Compliance Inner loop Outer loop Developer Test
8. @kevindubois @natalevinto @kevindubois @natalevinto Today, let’s focus on the “Outer Loop” The outer loop consists of the larger team processes that your code flows through on its way to the cluster: code reviews, integration tests, security and compliance. It needs be transparent and fast for developers from all teams. Outer Loop Development
9. @kevindubois @natalevinto Love Thy Mono Every 4 months Every week/day/hour
10. @kevindubois @natalevinto Before we go on, let’s play a game :)
11. @kevindubois @natalevinto The application Push to give energy windmill 1.Sends click Kafka Topic 2.Sends the interaction Dashboard: Green Energy Nickname Team Push/Tap to generate energy Cars that need energy Two teams competing (top 5 players) First wins
12. @kevindubois @natalevinto Architecture 3: Generate power (REST) Game Dashboard 1: Assign player Name & Team (REST) 6: Update dashboard (SSE) 2: Increment player cluster counter 4: Send power event 5: Receive power events 12
13. @kevindubois @natalevinto YOU PLAY! red.ht/voxxed23 13
14. @kevindubois @natalevinto What if we added a new feature?
15. @kevindubois @natalevinto
16. @kevindubois @natalevinto github.com/ redhat-developer-demos/ quinoa-wind-turbine github.com/ redhat-developer-demos/ quinoa-wind-turbine-manifests
17. @kevindubois @natalevinto Let’s deploy a new feature in a Gitops way!
18. @kevindubois @natalevinto Developer Flow Outer loop Inner loop Pull/Merge Request Production Build / Package Code Push Debug Code Review Build Deploy Security Tests Compliance Inner loop Outer loop Developer Test
19. @kevindubois @natalevinto 19 Gitops
20. @kevindubois @natalevinto What is GitOps? Treat everything as code Git is the single source of truth Operations through Git workflows
21. @kevindubois @natalevinto GitOps Application Delivery Model Source Git Repository Image Registry CI
22. @kevindubois @natalevinto GitOps Application Delivery Model Source Git Repository Image Registry CI Config Git Repository Kubernetes CD Pull Request / Commit Push Pull
23. @kevindubois @natalevinto V2 red.ht/voxxed23
24. @kevindubois @natalevinto Progressive Delivery 24
25. @kevindubois @natalevinto What is Progressive Delivery? Build Test Security Checks Release Deploy Stage Deploy Prod Continuous Integration Continuous Delivery
26. @kevindubois @natalevinto What is Progressive Delivery? • No Big Bang • Deploy != Release • Metrics • Subset of Users
27. @kevindubois @natalevinto Why Progressive Delivery? • Decreases Downtime • Limits the Tragedy • Deploy & Release to Production faster • Less mocking or setting up unreliable ‘fake’ services
28. @kevindubois @natalevinto 28
29. @kevindubois @natalevinto Delivery Techniques
30. @kevindubois @natalevinto Blue Green Deployment • All Or Nothing • Quick Rollback 30
31. @kevindubois @natalevinto Canary Releases • Small Percentage • Increase depending on metrics 31
32. @kevindubois @natalevinto Dark Launches • Mirroring Traffic • Dark Canaries • Feature Flags 32
33. @kevindubois @natalevinto 33
34. @kevindubois @natalevinto 34 Production is (again) not sacred anymore
35. @kevindubois @natalevinto The New Pyramid? 35
36. @kevindubois @natalevinto How to accomplish Progressive Delivery 36
37. @kevindubois @natalevinto Progressive Delivery with
38. @kevindubois @natalevinto Blue - Green 38 apiVersion: v1 kind: Service metadata: name: my-service labels: app: mystuff spec: ports: - name: http port: 8000 selector: inservice: mypods type: LoadBalancer apiVersion: apps/v1 kind: Deployment metadata: name: mynode-deployment spec: replicas: 1 selector: matchLabels: app: mynode template: metadata: labels: app: mynode spec: containers: - name: mynode image: quay.io/rhdevelopers/mynode:v1 ports: - containerPort: 8000 kubectl label pod -l app=mynode inservice=mypods
39. @kevindubois @natalevinto Canary Releases kubectl scale deployment myapp-v1 --replicas=3 kubectl scale deployment myapp-v2 --replicas=1 39
41. @kevindubois @natalevinto Controlling Microservices with a Service Mesh 41 Code Independent (Polyglot) • Intelligent Routing and Load-Balancing • Smarter Canary Releases • Dark Launch • Chaos: Fault Injection • Resilience: Circuit Breakers • Observability & Telemetry: Metrics and Tracing • Security: Encryption & Authorization • Fleet wide policy enforcement
42. @kevindubois @natalevinto Sidecar Container 42
43. @kevindubois @natalevinto Sidecar Container Pod Container JVM Service A Sidecar Container Pod Container JVM Service C Sidecar Container Pod Container JVM Service B Sidecar Container With Istio The sidecar intercepts all network traffic
44. @kevindubois @natalevinto Canary Release 44 apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: recommendation spec: hosts: - recommendation http: - route: - destination: host: recommendation subset: version-v1 weight: 75 - destination: host: recommendation subset: version-v2 weight: 25
45. @kevindubois @natalevinto Shadowing Traffic 45 apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: recommendation spec: hosts: - recommendation http: - route: - destination: host: recommendation subset: version-v1 mirror: host: recommendation subset: version-v2
46. @kevindubois @natalevinto Dark Canary 46 apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: recommendation spec: hosts: - recommendation http: - match: - headers: end-user: exact: Alexandra route: - destination: host: recommendation subset: version-v2 - route: - destination: host: recommendation subset: version-v1
47. @kevindubois @natalevinto Automated Progressive Delivery 47
48. @kevindubois @natalevinto 48 Argo Rollouts
49. @kevindubois @natalevinto Argo Rollouts 49 ArgoCD detects change rollout Monitors Data
50. @kevindubois @natalevinto Rolling out automatically 50 apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: name: rollouts-demo labels: app: rollouts-demo spec: strategy: canary: steps: - setWeight: 20 - pause: duration: "1m" - setWeight: 50 - pause: duration: "2m" canaryService: rollouts-demo-canary stableService: rollouts-demo-backend trafficRouting: istio: virtualService: name: rollout-vsvc routes: - primary …
51. @kevindubois @natalevinto “Smart” Progressive Delivery Based on Metrics 51
52. @kevindubois @natalevinto Metrics Based Rollouts strategy: canary: analysis: args: - name: service-name value: rollouts-demo-canary.canary.svc.cluster.local templates: - templateName: success-rate canaryService : rollouts-demo-canary stableService : rollouts-demo-stable trafficRouting : istio: virtualService : name: rollout-vsvc routes: - primary steps: - setWeight: 30 - pause: { duration: 20s } - setWeight: 40 - pause: { duration: 10s } - setWeight: 60 - pause: { duration: 10s } - setWeight: 80 - pause: { duration: 5s } - setWeight: 90 - pause: { duration: 5s } - setWeight: 100 - pause: { duration: 5s } 52
53. @kevindubois @natalevinto apiVersion: argoproj.io/v1alpha1 kind: AnalysisTemplate metadata: name: success-rate spec: args: - name: service-name metrics: - name: success-rate interval: 10s successCondition: len(result) == 0 || result[0] >= 0.95 failureLimit: 2 provider: prometheus: address: https://internal:demo@prometheus.istio-system.svc.cluster.local:9090 query: | sum(irate(istio_requests_total{ reporter="source", destination_service=~"{{args.service-name}}", response_code!~"5.*"}[30s]) ) 53 Metrics Based Rollouts
54. @kevindubois @natalevinto Live Demo @kevindubois @natalevinto @kevindubois @natalevinto
55. @kevindubois @natalevinto Final Notes • State is always hard, start with stateless • Step by Step • Embrace GitOps • If you haven’t automatically destroyed something by mistake, you aren’t automating enough • Demos ◦ https://dn.dev/istio-tutorial ◦ https://github.com/kdubois/ocp-sm-ar ◦ https://github.com/redhat-developer-demos/bubbles-progressive-delivery ◦ github.com/redhat-developer-demos/quinoa-wind-turbine 55
56. @kevindubois @natalevinto https://red.ht/gitops-cookbook
57. @kevindubois @natalevinto https://red.ht/modernize-enterprise-java
58. @kevindubois @natalevinto 58 Learn OpenShift and all Cloud Native for free in our web based learning portal Resources Try OpenShift GitOps on our learning portal Red Hat Developers Deep Dive Join our workshops to explore Argo CD, Tekton, Kubernetes, Helm and more! Sign up at developers.redhat.com Find out more about Red Hat’s products and what it offers developers GitOps ebooks Find out more on GitOps with ebooks https://developers.redhat.com/e-books Get more about GitOps on Red Hat Developer Portal!
59. @kevindubois @natalevinto Start exploring in the OpenShift Sandbox. Learn containers, Kubernetes, and OpenShift in your browser. developers.redhat.com/developer-sandbox Try Red Hat's products and technologies without setup or configuration.
60. @kevindubois @natalevinto Join Red Hat Developer. Build here. Go anywhere. facebook.com/RedHatDeveloper youtube.com/RedHatDevelopers twitter.com/rhdevelopers linkedin.com/showcase/red-hat-developer Thank you!