Measuring the Success of Developer Experience

In this session, we will be exploring different frameworks for measuring Developer Experience, including DORA, SPACE, and DX itself.

We will discuss key metrics such as deployment frequency and change failure rate. Additionally, we will explore a comprehensive framework for measuring developer experience, covering aspects from satisfaction and performance to communication and efficiency.

We will also look into how to incorporate DX metrics into existing workflows to ensure they align with organizational goals and provide actionable insights for continuous improvement.

Transcript

1. MEASURING SUCCESS
2. TL;DR; “An improved developer experience has positive outcomes – and not just on developers; it also helps Improve team and organization outcomes.” Based on on study DevEx in Action, A study of its tangible impacts By - Nicole Forsgren, Microsoft Research - Eirini Kalliamvakou, Github Next - Abi Noda, DX - Michaela Greiler, DX - Brian Houck, Microsoft - Margaret-Anne Storey, University Of Victoria and DX
3. SOK S Group consists of 19 independent regional cooperatives. SOK is owned by the cooperatives and provides them with services S Group is a customer-owned Finnish network of companies* in the retail and service sectors, with more than 1,800 outlets in Finland
4. Niko Kivelä Head of DevOps @ SOK Previously § Finnair § Verkkokauppa.com Numbers Facts https://linkedin.com/in/nikokivela § 41 years § 20 years § 4 companies § 16.6 % 🤖 § 3 🐈 § DevOps Finland Meetup Organizer § AWS Community Builder § Tech Nordic Advocates Startup Mentor § Public Speaker & Published Author
5. Agenda § DevOps != Dev + Ops § DX += Developer Experience § SPACE ~= Developer Productivity § DORA = velocity and stability of software
6. 🌍 What is DevOps
8. 3. DevOps § Extends Agile § Emphasizes end-to-end responsibility § Automate and eliminate waste § Focus on cross- functional autonomous teams Monitor Release Verify Create Plan Configure Package Agile
9. DevOps is a human problem 3. The Theory of Constraints 1. The Three Ways 2. Value Stream 4. Systems Thinking Velocity
11. Agile Continuous Delivery Continuous Monitoring Site Reliability Engineering Continuous Deployment Continuous Integration Delivery Code Build Test Deploy Plan Configure Monitor
12. DevOps is a culture, based on a service mindset and a role, . that promotes developer productivity.. DevOps is not a job, but it’s an organizational strategy. . Ultimately, DevOps is all about communication and in one doing.
13. Why This Matters Now
15. Buy vs Make Software Buyer Sales preservative Communication, Processes, Trust, Leadership, Well- being, Recruitment, Capabilities, Cognitive-load, …
16. What to Measure
17. Goal Metric Developer Value Monitor Observe
18. 19 Developer Experience, Value Creation Developer Observe Monitor Value Produce Value Perceive Quality Observability § Granular insights § Temporary § Analyzed and aggregated Your teams § Developer Experience § Tells you WHY Monitoring § Single data points § Mostly static § Universally understood Your customers § NPS § Tells you WHAT Data Measure leading indicators lagging indicators
19. DORA: Lagging Indicators DORA metrics, also known as the DORA (DevOps Research and Assessment) metrics, are a way to measure how well an organization is doing in terms of software development and delivery. 20 Key Metrics § Lead Time For Changes (LTC) § Deployment Frequency (DF) § Change Failure Rate (CFR) § Mean Time to Restore Service (MTTR) Velocity Stability
20. Developer Experience Developer Experience is the satisfaction and usability of tools, technologies, and processes that developers use in their work. - Dr. Nicole Forsgren, Partner at Microsoft Research Developers - Increased productivity - Boost creativity Teams - Better quality - Less technical depth Organizations - Drive innovation - Better retentions Flow State Feedback Loops Cognitive Load
21. 14.11.2024 22 The SPACE framework is a research- based approach to measuring, understanding, and improving software development productivity. It encourages engineering leaders to have a holistic approach to productivity.
22. https://cacm.acm.org/magazines/2021/6/252825-the-space-of-developer-productivity/fulltext
23. Quality Deployment Frequency Change Failure Rate Continuous Integration Lead Time for Changes How code is tested? How to improve quality? Smaller Batch Size Easier Code Reviews Faster Feedback Less Cognitive Load Smaller Impact Radius Less Rework Smaller Outages Developer Happy Developer Developer Effectiveness Less Waste Observability Monitor Signal Observe
24. D2M: Development Maturity Model SMA: Service Maturity Assessment DMA: State of Development Maturity Adoption Do you have a documented test strategy or plan relevant to your current product lifecycle? VS
25. How Do I Even?
26. Developer Experience Platform Engineering Developer Relations What How Why Value Chain DevOps DevOps originated from Agile practices. ... and applies those principles beyond software teams. DevOps produces better Developer Experience Platform Engineering is a logical extension of DevOps. … and is the enabler of value delivery. It produces Internal Developer Portals. You need DevOps skills... You need Product Management skills... You need Marketing skills.. ... to build Platform Engineering teams. Product Self-service Discovery Advocacy Enablement Community
27. Efficiency vs Productivity Internal Communication P: people * (people - 1) / 2 P(7) = 21 Adding people 1 + 1 = 1 Efficiency = optimize resources Productivity = maximum output Efficiency is doing things right; effectiveness is doing the right things. - Peter Drucker Productivity is doing the right things right in the right order. - Niko Kivelä Handoffs Math 0.9 x 0.9 = 0.81 0.9^7 = 0.48
28. The Rise & Fall of DevOps DevOps is NOT dead, it’s all about People DevOps != Dev + Ops
29. Your success depends on the uniqueness of your organization’s culture, history, context, and values, and what you do with it
30. ALAOTSIKKO üAn individual developer's experience is highly personal and contextually dependent. üMeasure Developer Experience by assessing developers' perceptions and workflows within the sociotechnical they work in. üBetter Developer Experience positively affects organization outcomes – retention, innovation, profitability, and ability to meet goals. Key takeaways
31. Niko Kivelä https://www.linkedin.com/in/nikokivela https://medium.com/@niko.kivela S Group https://github.com/s-group-dev https://www.s-group.dev https://medium.com/s-group-dev
32. References 1. DevEx in Action – A study of its tangible impacts https://queue.acm.org/detail.cfm?id=3639443 (January 14, 2024) 2. DevEx: What Actually Drives Productivity – The developer-centric approach to measuring and improving productivity. https://queue.acm.org/detail.cfm?id=3595878(May 3, 2023) 3. The SPACE of Developer Productivity – There's more to it than you think. https://queue.acm.org/detail.cfm?id=3454124 (March 6, 2021)