Mastering Developer Experience: A Roadmap for Success Nicole Forsgren, PhD

Hi! I’m Nicole

DevEx What it’s like to build software – all the way from start to finish Developer Experience Improving developer experience – so we can improve outcomes for our stakeholders (e.g., organization and developers) The JOB of Developer Experience

Developer Experience trends - The New Stack found 10% resource allocation - Netflix developer productivity team allocates 15% - A report by DX found 11-19% of headcount dedicated to DevEx (11% at companies over 10,000 employees, 19% at companies under 1,000 employees) Companies are dedicating resources to improving DevEx McKinsey’s “2024 and Beyond” report: “There is no better time than today to invest in a prosperous and sustainable future… Business leaders who pursue this productivity imperative… will not only position their organizations for outperformance but help make 2024 the advent of future abundance. Some say now is the time to invest in productivity

Improving DevEx matters… High- performing teams twice as likely to exceed goals Impact Devs with easy to use tools feel 50% more innovative 50% Innovation Devs with faster feedback report 50% less tech debt 50% Tech debt 2x more less

? What does that look like? How do we get there?

Getting Started Status Check & Next Moves Create Plan Execute Plan First contact Tackle obvious problem Collect data Determine strategy and priority Technical details Cultural details Comms plan Implementation Reassess

Getting Started Status Check & Next Moves Create Plan Execute Plan First contact Tackle obvious problem Collect data Determine strategy and priority Technical details Cultural details Comms plan Implementation Reassess ⭐

Approaches to strategy and priority Aka What is the blocker? VSM, algorithmic, etc Constraints-based How do stakeholders view the problem – and is it a priority? Diff. lenses to impact Can we take advantage of momentum? Existing investments Who can do this, and does it affect the work? Global vs. local action When improving one improves another Improvement Synergies How fast could you make progress – and feel good? Time to complete

An example evaluation Constraints- based Documentation Build Performance On-call Different lenses to impact Existing investments TIme to complete [Can use LLMs to help close gap] Improvement synergies Glocal vs local action Test Performance M-H [Enable remote work] M? Hi? L-H L-H Hi M? Lo Hi ? Hi L M ? ? ? ? L-H L-H L/G L/G L/G M L-M L-M * Based on some typical challenges and patterns I see Hi Lo? M-H M-H Hi L ?

An example evaluation Constraints- based Documentation Build Performance On-call Different lenses to impact Existing investments TIme to complete [Can use LLMs to help close gap] Improvement synergies Glocal vs local action Test Performance M-H [Enable remote work] M? Hi? L-H L-H Hi M? Lo Hi ? Hi L M ? ? ? ? L-H L-H L/G L/G L/G M L-M L-M * Based on some typical challenges and patterns I see Hi Lo? M-H M-H Hi L ?

@nicolefv Why documentation can be so powerful 2023 Accelerate State of DevOps Report found that quality documentation improves team performance, productivity, and job satisfaction They also found that documentation can also amplify the impacts of our technical practices productivity boost with easy-to-source documentation 2021 State of the Octoverse Report found that across open source and enterprise projects, developers saw: 50%

Why testing can be so powerful… "A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable.” - Leslie Lamport https://github.com/CaitieM20/TheVerificationOfDistributedSystem

one of my favorite takeaways @caitie

@nicolefv Simple testing can prevent most critical failures ● three nodes or less can reproduce 98% of failures ● testing error handling code could have prevented 58% of catastrophic failures ● 35% of catastrophic failures: ○ error handling code is simply empty or contains a log statement ○ error handler aborts cluster on an overly general exception ○ error handler contains comments like FIXME or TODO

Create Plan Technical details How are we gonna do the thing? Technical details & specs: - Technical approaches - Breakdown & dependencies - Implementation schedule Rollout - Tech stack, architecture, etc - Teams & team structure, etc - Pilot then rollout, or other? Approaches - Central/ dedicated effort - 20% time - Hack week Create Plan

Tech influences culture

Tiny wins Improving productivity doesn’t have to come from large projects. Sometimes the most rewarding things we do are small things with big impact. Joel talks about “tiny” projects with big impact https://joelcalifa.com/blog/tiny-wins/

Create Plan Cultural details Tech is easy. People are hard. Revise your plan - Double your time estimates - Explicitly allow for adjustment – change is hard and scary - Identify places of biggest/ scariest change Create processes and patterns to help reinforce plan - Organization: Engineering reviews - Teams/ICs: Town halls/ AMAs - Individually: Incentive structures - Leaders: Personal BOD, mentors, AMAs and shareouts Create Plan

Culture influences tech

Create Plan Comms plan We gotta get everyone on the same page Use transparency to motivate your plan Clearly explain the problem and how your plan solves it Create collateral to explain your change to major stakeholders: ICs, eng managers, product, leaders - Consider workshops for leaders and supporters like HR - Remember that different organizational levels have different information needs & expectations - Consider automated information delivery for long-lived initiatives (e.g., dashboards, shared queries) Create Plan

Create Plan Implementation Do the thing Some additional notes: ● Have regular check points to see how things are going ● Executing all three plans – technical, cultural, and comms – will be very important Execute Plan

Support ongoing improvement Keep this on your radar Deal with technical debt Information flow is critical Share knowledge Successful DevEx is global and local Think about resourcing Avoid a one-and-done Reinforce continuous improvement Reduce friction Consider standardization How to balance this with standardization Support innovation

@nicolefv How can we measure DevEx and our progress? Each environment is different, and each strategy is different, so the metrics will need to be tailored. The most commonly used frameworks are: ● DORA ● SPACE ● McKinsey ● ACM DevEx metrics … but what’s good and bad about each?

How can we measure DevEx and our progress? Framework Strengths Weaknesses DORA metrics Prescribed (identified metrics) Specifics are flexible to suit context Only capture the outer loop Flexibility can create confusion SPACE framework Framework leads to holistic measures Can be adapted to measure overall concepts (SDLC) or specific areas (PRs) Specific metrics not identified (tho examples are given) McKinsey DevEx McKinsey is a respected brand among executives Beyond DORA and SPACE, they rely on coding activities ACM DevEx metrics Sociotechnical framework for DevEx: cognitive load, feedback loops, flow Specific metrics not identified (tho examples are given)

An example with SPACE Dimension Definition S: Satisfaction & Well-being How fulfilled, happy, and healthy one is, or how satisfied with a tool P: Performance An outcome of a process A: Activity The count of actions or outputs C: Communication & Collaboration How people talk and work together; how systems talk and work together E: Efficiency & Flow Doing work with minimal delays or interruptions; time aspects of a process A quick intro

Applying SPACE to Testing (1 of 2) Dimension Example metrics S: Satisfaction & Well-being how satisfied testers and/or developers are with: the tools available to them for testing, the test suite, test times, manual testing support and visibility P: Performance these measures focus on testing performance and reliability; test suite times/ flakiness A: Activity number of issues caught by the tests, number of tests run, number of tests automated, proportion of automated tests run on the delivery pipeline, number of tests skipped, number of bugs hit/missed by the test suite – overall and by severity

Applying SPACE to Testing (2 of 2) Dimension Example metrics C: Communication & Collaboration test documentation describes the steps to run automated tests. manual test documentation describes the defect and the steps involved to recreate. this can also include system calls and APIs for tests. E: Efficiency & Flow number of times a test suite is run before completed successfully; time spent fixing acceptance test failures; interruption statistics for testers doing work

💡 DevEx ● What is DevEx ● Why DevEx matters ● The DevEx journey ● Measuring DevEx DevEx is growing and becoming more important – you’re working in an exciting and important field! I’m working on a book with Abi Noda. For more: www.developerexperiencebook.com TL;DR:

Thank you! I’d love to hear from you! Q & A