CI Build Scoring: A Formula for Blocking SEV1s from Production

Finding the one anomaly that actually risks introducing a SEV1 issue into Production can be a seemingly impossible feat. OverOps' R&D team set out to devise a formula for assessing the reliability of a build across the release pipeline. Chen Harel (Co-founder & VP of Products at OverOps) will walk through an open source scoring system in Grafana and CI-build systems that takes into account new errors, increasing errors and slowdowns to provide DevOps teams with a formulaic approach to understanding the stability of their releases -– before they go into production.

Transcript

1. Presentation Name CI Build Scoring A Formula for Blocking SEV1s from Production
2. Welcome... @chenharel
3. What are we talking about today? quality customer impact quality gates
4. The Problem(s) $2.8T in 2018 alone
6. How are organizations addressing this today?
7. How are organizations addressing this today?
8. Measuring Software Quality is All About GOOD Data
9. Types of Data Conventional
10. Types of Data Conventional New Approach
11. The Reliability Score new errors existing errors higher rate slowdowns
12. The Scoring Process Detect Classify Prioritize Score Block Visualize
13. Detection Manual Automatic
14. Classification
15. Prioritizing: New Errors new severe 1. 2. 3.
16. Prioritizing: Increasing Errors increasing severe 1. 2. 3.
17. Seasonality
18. Prioritizing: Slowdowns
19. Slowdowns Example slowdown
20. The Scoring Formula
21. The Scoring Formula new_event_score> severe_new_event_score> regression_score> critical_regression_score> score_weight> key_score_weight> normalized - - * /
22. Mileage May Vary
23. Example: Scoring with Jenkins
24. Code Quality Gates Plugin
25. Score Calculation
26. Code Quality Report
27. Feeding Data into Grafana
28. Feeding Data into Grafana
29. Summary
30. Resources • • • • • •
31. Demo
32. Questions?
33. Presentation Name Thanks for joining!