Upgrade to Pro — share decks privately, control downloads, hide ads and more …

開発生産性Conference Keynote Dr.Nicole Forsgren : From Metrics to Mastery: Improving Performance with DORA and the SPACE Framework

Findy
July 24, 2023

開発生産性Conference Keynote Dr.Nicole Forsgren : From Metrics to Mastery: Improving Performance with DORA and the SPACE Framework

開発生産性ConferenceのKeynoteにて登壇いただいたDr.Nicole Forsgrenの資料になります。
ご本人に許可をいただき公開させていただきます。

◆タイトル◆
From Metrics to Mastery: Improving Performance with DORA and the SPACE Framework

◆アブストラクト◆
Join Dr. Nicole Forsgren for an enlightening talk on utilizing DORA metrics, the SPACE productivity framework, and the latest productivity research in software development. Discover how to use DORA metrics to measure DevOps performance and drive actionable insights. Learn how DORA and the SPACE productivity framework are similar – and different. Explore the impact of culture on software delivery and productivity. And learn about some of the latest productivity research, covering hybrid work models and tools like GitHub Copilot. In this talk, you’ll gain practical insights to enhance software delivery, improve productivity, and navigate the evolving landscape of development. Don't miss this opportunity to learn from Nicole Forsgren and optimize you and your team's work.

(キーノートでは、DORAメトリクス、SPACEの開発生産性フレームワーク、そしてソフトウェア開発における最新の開発生産性の研究結果についてお話しします。
まずは、DevOpsのパフォーマンス計測のためにDORAメトリクスをどのように活用し、どのように実践的なインサイトを得るのかについて探ります。そして、DORAメトリクスとSPACEフレームワークの共通点、相違点についてもお伝えします。
最後に、リモート・出社のハイブリットワークやGitHub Copilotのような生産性を高めるツールを含めて、最新の開発生産性の研究結果についてお話しします。
セッションを通して、ソフトウェアデリバリーを強化しながら開発生産性を上げ、進化し続ける開発環境を同ナビゲートしていくのかについての実践的なインサイトを得ることができるでしょう。
ぜひ本セッションから学びを得て、自分自身とチームの仕事でより生産性を高めてください。)

Findy

July 24, 2023
Tweet

More Decks by Findy

Other Decks in Technology

Transcript

  1. FROM METRICS
    TO MASTERY
    Improving Performance with DORA
    and the SPACE Framework
    Nicole Forsgren, PhD

    View full-size slide

  2. TODAY’S AGENDA
    01
    DORA
    02
    SIGNAL &
    ACTION
    03
    SPACE
    05
    CULTURE
    06
    RECENT
    RESEARCH

    View full-size slide

  3. Hello!
    I’m Nicole!
    I’ve been a software engineer, sysadmin, and
    researcher.

    View full-size slide

  4. DevOps was the original
    hipster for making work more
    productive and sustainable.

    View full-size slide

  5. DevOps was the original
    hipster for making work more
    productive and sustainable.
    DORA and SPACE help us
    understand, measure, and
    improve our productivity and
    our well-being.

    View full-size slide

  6. DORA – quick review
    DORA 4 metrics measure speed and stability:
    ● Lead time for changes
    ● Number of deploys
    ● Mean time to recover
    ● Change fail rate
    DORA is also a research program that helps teams and
    organizations improve their performance on those four
    metrics

    View full-size slide

  7. What is DORA for?
    benchmarking

    View full-size slide

  8. What is DORA for?
    benchmarking

    View full-size slide

  9. What is DORA for?
    DORA is a framework that helps us:
    ● Improve outcomes
    ● Measure performance
    ● Identify capabilities for improvement – and
    provide evaluative criteria
    Check out dora.dev for the Quick Check and more!

    View full-size slide

  10. SIGNAL & ACTION
    02

    View full-size slide

  11. SIGNAL AND ACTION
    SIGNAL: How am I
    doing?
    ACTION: What can –
    and should – I do to
    improve?

    View full-size slide

  12. DORA as Signal & Action
    SIGNAL: DORA Metrics
    → How well can my
    team develop and
    deliver software?
    ACTION: What capabilities
    should I focus on to improve
    my software delivery?
    → Can identify using DevEx
    methods, value stream, or
    others.

    View full-size slide

  13. Dora.dev Quick Check

    View full-size slide

  14. What is SPACE for?
    one set of metrics for everything

    View full-size slide

  15. What is SPACE for?
    one set of metrics for everything

    View full-size slide

  16. What is SPACE for?
    SPACE is a framework that helps us think about how to
    measure productivity – or any complex, creative work

    View full-size slide

  17. SPACE and DORA are
    complementary
    Once you’ve identified the capabilities you want to improve (using
    DORA), you can use SPACE to identify how you want to measure it

    View full-size slide

  18. More on SPACE
    It IS
    - Complementary to DORA
    - a framework used to identify
    metrics appropriate for a situation
    or goal
    - more broad than DORA (e.g., can
    be used for sales teams or
    copywriters)
    It is NOT
    - An evolution of DORA
    - a set of prescribed
    metrics

    View full-size slide

  19. SPACE FRAMEWORK
    DIMENSION DEFINITION
    S: Satisfaction and well-being How fulfilled, happy, and healthy one is
    P: Performance An outcome of a process
    A: Activity The count of actions or outputs
    C: Communication and
    collaboration
    How people talk and work together; how systems
    talk and work together
    E: Efficiency and flow Doing work with minimal delays or interruptions
    A holistic way to measure productivity
    https://queue.acm.org/detail.cfm?id=3454124

    View full-size slide

  20. APPLYING SPACE TO INCIDENT
    MANAGEMENT(1 OF 2)
    DIMENSION EXAMPLE METRICS
    S: Satisfaction and well-being
    how satisfied SREs are with the IM process, escalation
    and routing, and on-call rotations are key metrics to
    capture, especially since burnout is a significant issue
    among SREs.
    P: Performance
    these measures focus on system reliability;
    monitoring systems' ability to detect and flag issues
    faster, before they hit the customer and become an
    incident. MTTR (mean time to repair) overall, and by
    severity.
    A: Activity
    number of issues caught by the monitoring systems,
    number of incidents created, number of incidents
    resolved—and their severity distribution.

    View full-size slide

  21. APPLYING SPACE TO INCIDENT
    MANAGEMENT(2 OF 2)
    DIMENSION EXAMPLE METRICS
    C: Communication and
    collaboration
    people included in resolving the incident, how many
    teams those people came from, and how they
    communicate during an incident. Incident resolution
    documentation outlines the steps involved in
    resolving incidents; this can be measured by
    completeness (to check if any resolution data was
    entered) or quick quality scores (e.g., thumbs
    up/down). Teams may also include a metric that
    measures the percentage of incidents resolved that
    reference these guides and documentation.
    E: Efficiency and flow
    incident handoffs, incident assignment/re-
    assignment, number of hops an incident has to take
    before it is assigned to the right individual or team.

    View full-size slide

  22. DORA AS SPACE
    DORA METRIC SPACE DIMENSION
    LEAD TIME EFFICIENCY AND FLOW
    DEPLOY FREQUENCY ACTIVITY
    MTTR EFFICIENCY AND FLOW
    CHANGE FAIL RATE PERFORMANCE
    AVAILABILITY PERFORMANCE
    Criteria: Have metrics across at least 3 of the 5 dimensions for a given focus area; this
    covers dimensions for software delivery.

    View full-size slide

  23. GOOD DAY PROJECT
    We did a study to help developers get quick and easy signals
    and patterns to help them have better days, more consistently.
    (“What makes a good day, and how can I have one more often?”)
    The deets:
    ● Based on holistic concept of productivity using the
    SPACE framework
    ● Focus on individual: measures for us, not for managers
    ● To help us measure our energy, not our time
    ● Goal is quick and easy measures with actionable signal

    View full-size slide

  24. GOOD DAY PROJECT
    To capture SPACE quickly, we asked questions like:
    ● How was your work day?
    ● I worked with other people
    ● My work was interrupted
    ● How many meetings did you have today?
    ● Today, I felt most productive… (and least productive)
    https://github.blog/2021-05-25-octoverse-spotlight-good-day-project/

    View full-size slide

  25. FINDING FLOW IS KEY AND
    INTERRUPTIONS ARE A DRAG
    Minimal or no interruptions give developers
    Chance of having a good day
    https://github.blog/2021-05-25-octoverse-spotlight-good-day-project/
    Interruptions throughout the day
    Chance of having a good day
    Minimal or no interruptions give developers
    Chance of having a good day
    82% 7%

    View full-size slide

  26. Metrics can give us insight
    and direction to improve
    productivity and well-
    being, but they aren’t
    enough to create change

    View full-size slide

  27. Technology and
    culture influence
    each other

    View full-size slide

  28. Tech
    influences
    culture

    View full-size slide

  29. NUMMI
    When we change the way we work, we influence our culture

    View full-size slide

  30. Culture
    influences
    tech

    View full-size slide

  31. —GREG LINDEN
    "I think building this culture is the key to innovation.
    Creativity must flow from everywhere. Whether
    you are a summer intern or the CTO, any good idea
    must be able to seek an objective test, preferably a
    test that exposes the idea to real customers.
    Everyone must be able to experiment, learn, and
    iterate. Position, obedience, and tradition should
    hold no power. For innovation to flourish,
    measurement must rule.”
    http://glinden.blogspot.com/2006/04/early-amazon-shopping-cart.html

    View full-size slide

  32. RECENT
    RESEARCH
    04

    View full-size slide

  33. A COUPLE HIGHLIGHTS
    Patterns in behavior change when using AI-
    assisted programming. For example, we see
    (and maybe expect) that users will spend
    more time reviewing than writing, but we
    now also see that users spend much more
    time not only prompt crafting (and waiting
    for a suggestion) but then returning to verify
    suggestions. This upends traditional models
    and suggests new user interfaces when AI-
    assisted models are present.
    https://arxiv.org/pdf/2210.14306.pdf
    This work study investigated hybrid work among
    3400+ people, 2,300 of them developers+, across
    28 companies. It found that the highest
    productivity is found among those who work
    entirely from home, even when controlling for
    several factors. This may be because hybrid work is
    difficult to optimize. Key challenges include
    missing social interactions, too many meetings,
    inefficient work practices, lack of clear business
    objectives, and difficulty collaborating. Several
    additional findings and implications are discussed.
    aka.ms/hybridworkpaper
    Reading Between the Lines: Modeling
    User Behavior and Costs in AI-Assisted
    Programming. 2022 Mozannar et al.
    The Best of Both Worlds: Unlocking the
    Potential of Hybrid Work for Software
    Engineers. 2023 Houck et al.

    View full-size slide

  34. THANK YOU!
    I’M EXCITED TO JOIN YOU FOR Q&A

    View full-size slide