The Only Good Quality Metric is Morale

Every metric for a QA team has pitfalls.

Some are combative and drive a wedge between teams. Some are useless or easily gamed. Some don’t make any sense at all. So what’s an agile team to do if they want to measure the quality of the software they are working so hard to produce?

Jenny Bramble suggests looking inward, past all the obvious metrics and into the heart of the team. The team’s morale can accurately predict the quality of the software they produce. In this talk, she will discuss some common metrics and their pit falls before making the case for morale as the top QA metric as well as how to measure changes in morale. She’ll show how to measure changes in morale over time and what you can do to help increase the morale–and thus quality!–of your team.

Transcript

1. THE ONLY GOOD QUALITY
   METRIC IS MORALE
   @jennydoesthings
   

   View Slide

2. @JENNYDOESTHINGS
   HI I’M JENNY!
   ▸ Test Lead at Willowtree
   ▸ Yes, we’re hiring!
   ▸ Human interfacing is my favorite thing
   ▸ Testing is also my favorite thing
   ▸ Two cats—Dante and Dax
   ▸ Ask me for a sticker later!
   ▸ Yes, they’re also my favorite things
   ▸ My pronouns are she/her
   

   View Slide

3. IT’S STORY TIME!
   Me, probably
   @JENNYDOESTHINGS
   

   View Slide

4. THIS QUESTION HAS PLAGUED
   ME FOR MY ENTIRE CAREER
   SERIOUSLY
   @jennydoesthings
   

   View Slide

5. @JENNYDOESTHINGS
   SO, LET’S TALK ABOUT IT
   ▸Less effective metrics
   ▸What do you actually
   want to measure?
   ▸The case for morale
   as a metric
   ▸Measuring morale
   

   View Slide

6. LESS EFFECTIVE
   METRICS
   BABY DON’T HURT ME
   DON’T HURT ME NO MORE
   @jennydoesthings
   

   View Slide

7. WHO DO YOUR METRICS HURT?
   ASK YOURSELF THIS:
   @jennydoesthings
   

   View Slide

8. WHAT IS THE CONTEXT FOR
   YOUR METRICS?
   ASK YOURSELF THIS:
   @jennydoesthings
   

   View Slide

9. @JENNYDOESTHINGS
   LESS EFFECTIVE METRICS - WHAT ARE THEY
   ▸ Test cases executed
   ▸ Bugs found
   ▸ Bugs resolved
   ▸ Deferred defects
   ▸ X of Y completed
   

   View Slide

10. ANYTHING THAT IS A STRICT
    COUNT CAN BE LESS EFFECTIVE
    THE SHORT STORY IS…
    @jennydoesthings
    

    View Slide

11. @JENNYDOESTHINGS
    LESS EFFECTIVE METRICS - WHY ARE THEY LESS EFFECTIVE
    ▸ Very game-able
    ▸ Encourages dishonest reporting
    ▸ Pit parts of your team against each other
    ▸ Enhances toxic patterns in teams
    ▸ Most of them measure the process not the product
    ▸ They are inaccurate…depending on what you want to
    measure
    

    View Slide

12. @JENNYDOESTHINGS
    LESS EFFECTIVE METRICS - WHAT DRAWS US TO THEM
    ▸ Easy to count
    ▸ Easy to measure
    ▸ Easy to compare
    ▸ Easy to explain
    ▸ Graphs!! Everyone loves graphs!!
    

    View Slide

13. WE ARE CONDITIONED TO ACCEPTS THESE AS
    VALID MEASURES OF OUR EFFECTIVENESS
    IN THE END…
    @jennydoesthings
    

    View Slide

14. @JENNYDOESTHINGS
    LESS EFFECTIVE METRICS - CAVEAT
    ▸ This assumes you’re primarily wanting to measure the
    product and the people
    ▸ If you are looking to measure your process and the flow of
    defects, these metrics may be exactly what you need
    ▸ So…
    

    View Slide

15. WHAT DO YOU WANT
    TO MEASURE?
    YOU MEAN I HAVE TO
    CHOOSE??
    @jennydoesthings
    

    View Slide

16. @JENNYDOESTHINGS
    WHAT DO YOU WANT TO MEASURE
    ▸ Numbers?
    ▸ Work done?
    ▸ Quality of…?
    ▸ Satisfaction?
    ▸ Compliance
    ▸ Performance
    ▸ Your process?
    ▸ Just want some pretty graphs
    

    View Slide

17. @JENNYDOESTHINGS
    WHAT DO YOU WANT TO MEASURE
    ▸ Take time to think about what matters to you
    ▸ What supports that?
    ▸ How can you quantify it?
    ▸ Metrics are hard
    

    View Slide

18. METRICS ARE HARD
    REMEMBER:
    @jennydoesthings
    

    View Slide

19. YOU GET WHAT YOU MEASURE
    REMEMBER:
    @jennydoesthings
    

    View Slide

20. THE CASE FOR
    MORALE
    NOT MORELS
    OR MORALS
    @jennydoesthings
    

    View Slide

21. @JENNYDOESTHINGS
    THE CASE FOR MORALE - WHAT IS IT
    ▸ Psychological safety
    ▸ Emotional health
    ▸ Contentedness
    ▸ Delight
    ▸ Pride
    ▸ Core Values
    

    View Slide

22. @JENNYDOESTHINGS
    THE CASE FOR MORALE - CORE VALUES
    ▸ Core Values are a set of values that govern a team or
    company
    ▸ Some companies have these codified
    ▸ Sometimes they actually follow them
    ▸ Sometimes they don’t
    ▸ Each team should have values they adhere to
    

    View Slide

23. @JENNYDOESTHINGS
    THE CASE FOR MORALE - WHY
    ▸ Not as easy to game without straight up lying
    ▸ One person is not signaled out
    ▸ Describes the health of the team
    ▸ Happy, safe teams makes better products
    ▸ Retention is much higher on high morale teams
    

    View Slide

24. MORALE SUPPORTS ALL YOUR
    MAJOR GOALS
    MOST OF ALL
    @jennydoesthings
    

    View Slide

25. @JENNYDOESTHINGS
    THE CASE FOR MORALE - WHY
    ▸ Happy, safe teams consistently out perform unhappy, unsafe
    teams.
    ▸ Health of the team correlates to performance
    ▸ Google study shows psychological safety as the top driver
    of performance
    ▸ Safe teams will work together to make the team as a whole
    better
    

    View Slide

26. LET’S TALK
    MEASUREMENTS
    WAIT, DIDN’T WE JUST
    DO THIS?
    @jennydoesthings
    

    View Slide

27. HOW DO YOU MEASURE
    FEELINGS?
    ASK YOURSELF THIS:
    @jennydoesthings
    

    View Slide

28. HOW DO YOU PUT THEM INTO
    CONTEXT?
    ASK YOURSELF THIS:
    @jennydoesthings
    

    View Slide

29. @JENNYDOESTHINGS
    THE CASE FOR MORALE - CONTEXT
    ▸ Pay attention
    ▸ One-on-ones
    ▸ That One Person
    ▸ Company wide initiatives
    ▸ Glassdoor
    

    View Slide

30. CURATE SAFE SPACES
    TASK:
    @jennydoesthings
    

    View Slide

31. FORM A SURVEY
    TASK:
    @jennydoesthings
    

    View Slide

32. @JENNYDOESTHINGS
    LET’S TALK MEASUREMENTS - SURVEYS
    ▸ Your team needs to be psychologically safe to participate
    ▸ You need a team of large enough size to be anonymous
    ▸ Or agree with the team that it is okay if it’s clear who is
    answering
    ▸ Surveys need to be done regularly
    ▸ Actions must be taken on the results
    ▸ Do not focus on individuals, focus on the team
    

    View Slide

33. @JENNYDOESTHINGS
    LET’S TALK MEASUREMENTS - SURVEYS
    ▸ Run the surveys regularly - ie: every sprint
    ▸ Score questions consistently
    ▸ 1-5, 1-7
    ▸ Strongly agree to strongly disagree
    ▸ Keep an eye on the trends of the team, but don’t place too
    much stock until you have enough data
    

    View Slide

34. WHAT DO YOU ASK?
    OKAY, SO…
    @jennydoesthings
    

    View Slide

35. @JENNYDOESTHINGS
    SURVEYS - THE GOOGLE QUESTIONS
    ▸ If you make a mistake on this team, it is often held against you.
    ▸ Members of this team are able to bring up problems and tough issues.
    ▸ People on this team sometimes reject others for being different
    ▸ It is safe to take a risk on this team.
    ▸ It is difficult to ask other members of this team for help.
    ▸ No one on this ream would deliberately act in a way that undermines my
    efforts.
    ▸ Working with members of this team, my unique talents are valued and utilized.
    

    View Slide

36. @JENNYDOESTHINGS
    SURVEYS - OTHER QUESTIONS/PHRASINGS
    ▸ I am comfortable asking for help within my team.
    ▸ I feel I can fail openly without being shamed by my team.
    ▸ I can express ideas that might be contrary to the rest of the
    team.
    ▸ I like my team.
    ▸ I am not comfortable giving or asking for constructive
    feedback with my team
    

    View Slide

37. @JENNYDOESTHINGS
    SURVEYS - OTHER QUESTIONS/PHRASINGS
    ▸ We can brainstorm together or in front of each other.
    ▸ My team has a shared purpose or goal.
    ▸ We understand how we work together as a team.
    ▸ The team collaborates well together.
    ▸ My team has strong communication.
    ▸ No one is left out of the team.
    

    View Slide

38. RELATE TO QUALITY?
    SO HOW DOES THAT
    @jennydoesthings
    

    View Slide

39. ANY QUESTIONS?
    DERP
    @jennydoesthings
    

    View Slide

40. @JENNYDOESTHINGS
    FIND ME ONLINE!
    ▸ jenny.bramble@gmail.com
    ▸ http://twitter.com/jennydoesthings
    ▸ https://linkedin.com/in/
    jennybramble
    ▸ http://jennydoesthings.com
    ▸ Willowtree!
    

    View Slide