Metrics, beyond “it depends”

If you ask people for good DevOps and agile metrics, they tell you “it depends.” Few people want to get pinned down on metrics it seems. Well, they might be right, of course, it does depend. But, you still need a framework for de-it-dependsing yourself: how do you figure which metrics to pay attention to? This talk will go over thinking through that model and suggest a few metrics for IT performance to more product-centric, business focused metrics. All based on what organizations are actually doing. Well, all except for the part where I just make things up because they sound good.

This is based on my work-in-progress book on DevOps-y, agile, etc organizational change. In particular, [the chapter on metrics](https://medium.com/@cote/tracking-your-improvement-metrics-f47eff38772d). See [the listing in PaperCall.io](https://www.papercall.io/speakers/cote/speaker_talks/89527-metrics-beyond-it-depends) as well.

Transcript

1. Metrics, beyond “it depends” @cote 24 Sep 2018
2. Why? • Optimize – speed, better design • Monitoring – find problems • Be useful – helping the business
3. LEAD TIME request created start work in production cycle time Source: modified from “Kanban: Definition of Lead Time and Cycle Time,” Stefan Roock, March, 2010.
4. Example 6 week lead time Lead Time Request Delivery Value Added Process Time Non Value Added Process Time f Initial Env Setup Network Config Prod deploy ! ! ! ! ! ! ! White = wait time Red = ops time Green = dev time Source: scenario based on Pivotal customers; "Crossing the Value Stream: Improving Development with Pivotal and Cloud Foundry," Matt Gunter, Pivotal, 2018.
5. Better value stream, 3-6 weeks lead time Apply ruthless automation Remove “bullshit” work ~85% reduction in ops time ~50% reduction in release timeline ! ! ! Source: scenario based on Pivotal customers; "Crossing the Value Stream: Improving Development with Pivotal and Cloud Foundry," Matt Gunter, Pivotal, 2018.
6. Uses • Drives focus, trackable • Dumb stuff discovery • Silo breaking
7. 7 Halve cycles for more feedback or Deliver more in each cycle or Don’t tell no one, work less
8. Velocity: how much, how often
9. Uses • Planning • Predictability
10. 10 Measure your backlog to keep it small Source: photo by Jeser Bröring.
11. Latency: response time, click-to-screen
12. “53% of mobile users abandon sites that take longer than 3 seconds to load” 12 Source: "Mobile Load Time and User Abandonment," Tammy Everts, Akamai, Sep, 2016.
13. Uses • Happy user, happy developer • Easily find obvious problems
14. All the great hipster metrics • Google SRE: Latency, Traffic, Errors, and Saturation • USE Method: Utilization, Saturation, and Errors • RED Method: Rate, Errors, and Duration Source: ”How to Monitor the SRE Golden Signals,” Steve Mushero, Nov, 2017.
15. “The Business” (mostly money)
17. Source: Pivotal Platform Architect organization. Focus on value for spend (and also less spend)
18. TIME TO MARKET MTTR Velocity Lead Time HA HA SMARTART (Get money faster, grab marketshare before competition)
19. Source: "Cost of Delay - How PCF Helped Demonstrate the DoD Can't Afford Business as Usual,” Capt. Bryon Kroger & Tory Galvin, USAF, CF Summit NA 2018, April, 2018. (Theoretic) opportunity cost: If we stay the same, how much money we will not make (or, “lose”)
20. More http://cote.coffee/bettersoftware/