keptn - unbreakable continuous delivery

Transcript

1. keptn: Unbreakable Continuous Delivery Jürgen Etzlstorfer @jetzlstorfer
2. Confidential 2 Agenda • Why we have to build keptn • What problems keptn solves • What keptn is and how it works
3. 3 Confidential Why we have to build ! Because Cloud Native Continuous Delivery is a big challenge!
4. Confidential 4 Your App / Container Cloud Native: Deliver High Quality Faster
5. Confidential 5 < 1 day < 1 week 1 to 2 weeks 2 to 3 weeks 3 to 4 weeks > 4 weeks Small (11-100 employees) Medium (101-1000 employees) Large (1001-5000 employees) Extra large (over 5000 employees) 3% 20% 35% 42% 2 days 12.5 days 95th Percentile Median Commit Cycle Time: From Dev to Pro Goal: 1h to Production 1 hour
6. Confidential 6 3 out of 10 Business Impacting Deployments 3 hotfixes Per Production Deployment 4.8 days MTTR (Mean Time to Repair) Evaluate for yourself: https://dynatrace.ai/acsurvey 12.5 days Code to Production (Commit Cycle Time) 1 out of 10 0 hotfixes ~4 hours 2 days Median 95th Percentile Verdict: The Majority is not delivering high quality faster
7. Confidential 7 How do we elevate an organization from Median to 95th percentile! • Automated Quality (Shift-Left) • Goal: Stop Bad Code Changes Early & Automated • Automated Testing & Quality Gates • Automated Multi-Stage Deployments (Shift-Right) • Goal: Increase deployments into stable environments • Dark, Shadow, Blue/Green Deployments with Auto-Validation • Automated Operations (Self-Healing) • Goal: NoOps & Zero-Impact on End Users • Automated Remediation & (On-Demand) Scaling 97% deployment lead time Faster More Frequently 12 26 releases per year to 75% production incidents Better
8. Confidential 8 What problems solves! Let‘s developers build code vs maintaining pipelines & k8s clusters Let‘s operations automate run books vs manual firefighting
9. Confidential 9 What we have seen organizations actually do! • Building Custom Integrations with OpenSource & Commercial tools for • Deployment Pipelines • Testing Pipelines • Auto-Remediation • Notifications • Auditing
10. Confidential 10 What we have seen organizations struggle with Quote: „Pipelines seem to become our new future unmanagable legacy code!“ • Teams want to stick with existing tools to protect investement • Containing lots of custom code for tool integration, error handling, logging, ... • Getting harder to maintain the more tools get integrated • Pipelines becoming more complex requiring dedicated teams • Uncoordinated deployments between pipelines resulting in unstable Environments
11. Confidential 11 What is and how it works! Enterprise-grade framework for shipping and running cloud native apps
12. Confidential 12 from 10000ft GitOps-based collaboration Operator patterns for all logic components Monitoring and operations as code Built on and for Kubernetes Event-driven and serverless Pluggable tooling Automated multistage unbreakable delivery pipelines Self-healing blue/green deployments Event-driven runbook automation Design Principles Core capabilities
13. Confidential 13 keptn – conceptual architecture Autonomous Cloud Control Plane GitOps Container Registry Continuous Delivery AIOps Operations Automation Test Automation Environment Definition (shipyard file) ChatOps Dev Namespace Staging Namespace Production Namespace Core Services Platform Data Provider
14. Confidential 14 keptn – Shipping through Unbreakable Continuous Delivery Pipelines Autonomous Cloud Control Plane prod stage dev 1: push 2: deploy 3: test 4: evaluate 6: deploy 7: test 8: evaluate 10: deploy 11: evaluate 12: operate 5: promote 9: promote Dev
15. Confidential 15 Keptn: Phase 1 – Creating a Project $ keptn create project sockshop shipyard.yaml prod stage dev prod stage dev 1 1 Create GitOps Config Repos 2 2 Create k8s Namespaces shipyard.yaml stages: - name: "dev" deployment_strategy: "dark" test_strategy: "functional" - name: "stage" deployment_strategy: "shadow" test_strategy: "performance" - name: "prod" deployment_strategy: "blue_green"
16. Confidential 16 Keptn: Phase 2 – Onboarding a service $ keptn onboard service --project=sockshop --values=values_carts.yaml prod stage dev 1 1 Setup Registry Hooks 2 2 Apply Configuration + Istio 2 prod stage dev 2 2 values_carts.yaml replicaCount: 1 service: name: carts type: LoadBalancer externalPort: 80 container: name: carts resources: limits: cpu: 100m memory: 128Mi ...
17. Confidential 17 Keptn: Phase 3 – Pushing an artifact and deploy to dev $ docker push yourregistry/carts:0.2 prod stage dev 1 prod stage dev 1 1 New Artifact: carts:0.2 dev 2 2 Start Deployment: carts:0.2 (dev, dark) 3 3 Deployment Finished: Success or Failure 4 4 Start Tests: Functional, Dev, carts 5 5 Tests Finished: Test Result & State 6 6 Start Deployment Evaluation: Perfspec 7 7 Evaluation Finished: Deployment Score 8 "indicators": [ { "id" : “Prometheus:go_memstats_alloc", "metricScore": 20 },{ "id": “Dynatrace:Avg_ActionDuration", "metricScore": 20 } ], "objective" : { "pass" : 90, “warning” : 75 } 6 8 Send new Artifact Event: carts:0.2 stage
18. Confidential 18 Keptn: Phase 4 – Promoting an Artifact to Staging $ keptn send event newartifact carts:0.2 stage prod stage dev 1 prod stage dev 1 1 New Artifact: carts:0.2 stage 2 2 Start Deployment: carts:0.2 (stage, shadow) 3 3 Deployment Finished: Success or Failure 4 4 Start Tests: Performance, Stage, carts 5 5 Tests Finished: Test Result & State 6 6 Start Deployment Evaluation: Perfspec 7 7 Evaluation Finished: Deployment Score 8 6 8 Send new Artifact Event: carts:0.2 prod "indicators": [ { "id": “Dynatrace:RT_Real", "metricScore": 20 },{ "id": “Dynatrace:RT_Shadow", "metricScore": 20 } ], "objective" : { "pass" : 95 }
19. Confidential 19 Keptn: Phase 4 – Promoting an Artifact to Production: Blue/Green no issues! $ keptn send event newartifact carts:0.2 production prod stage dev 1 prod stage dev 1 1 New Artifact: carts:0.2 prod 2 2 Start Deployment: carts:0.2 (prod, bluegreen) 3 3 Deployment Finished: Success or Failure 4 4 Start Deployment Evaluation: Perfspec 5 5 Evaluation Finished: Deployment Score 4 6 6 Blue/Green Switch "indicators": [ { "id": “Dynatrace:ConversionRate", "metricScore": 100 }], "objective" : { "pass" : 95 }
20. Confidential 20 Keptn: Phase 4 – Promoting an Artifact to Production: Blue/Green with problem! $ keptn send event newartifact carts:0.2 production prod stage dev 1 prod stage dev 1 1 New Artifact: carts:0.2 2 2 Start Deployment: carts:0.2 (prod, bluegreen) 3 3 Deployment Finished: Success or Failure 4 4 Start Deployment Evaluation: Perfspec 5 5 Evaluation Result: Dynatrace Problem Notify 4 6 6 Remediation: Back to Blue 6 Remediation: Trigger ServiceNow Workflow 6 6 6 6 Remediation: Update Configuration: carts:0.1
21. Confidential 22 Want to learn more? • Learn how to use keptn: https://keptn.sh/docs/ • Clone, fork, star, contribute to keptn: https://github.com/keptn/ • Develop your own keptn-services: https://github.com/keptn/keptn-service-template • Join our community (slack invite, meetings): https://github.com/keptn/community • Follow us @keptnproject • Fill out our survey: https://dynatrace.ai/acsurvey • Reach out with any requests: • juergen.etzlstorfer@dynatrace.com • @jetzlstorfer
22. Confidential 23 dynatrace.com