Coaching in a data-driven world (#ScanAgile18)

Slides from my talk at Scan-Agile 2018 on using metrics as a coaching aid for better conversations with your team(s).

Transcript

1. Coaching in a data-driven world Nicolas Brown @nbrown02 #ScanAgile18 1
2. TLDR; Stop using metrics as targets. Use metrics as a coaching aid to inform, to have safe conversations, open questions and model curiosity.
3. Google Ngram
4. The growth in metric obsession
5. The growth in metric obsession
6. Which of the following best describes decision-making in your organisation? 8% Rarely data-driven Somewhat data-driven Highly data-driven 53% 39% PwC's Global Data and Analytics Survey: Big Decisions TM. Base: 2,106 senior executives 6
7. “When a measure becomes a target, it ceases to be a good measure” - Charles Goodhart
10. Commissioner Ervin Burrell Deputy Commissioner Bill Rawls
11. Police Majors & Lieutenants
13. No more murders. You will reduce the felony rate by 5% or more or…. let no man come back alive
14. Anyone of you who can’t bring in the numbers we need will be replaced by someone who can
15. Major Bunny Colvin
17. “Hamsterdam”
18. We’ve had a 14% drop in felony rate over the last 4 weeks
19. Ah yes, now I am happy. You won’t mind if I check your numbers will you?
24. The tyranny of metrics • Penalizing hospitals based on the percentage of patients who fail to survive for thirty days after surgery Patients sometimes kept alive for thirty-one days, so that their mortality is not reflected in the hospitals metrics • The National Health Service (NHS) in the UK declared that hospitals would be evaluated based on to what extent patients were admitted within four hours Some hospitals responded by keeping incoming patients in queues of ambulances outside the hospital doors, until staff were confident that the patient could be seen within the allotted four hours of being admitted • No Child Left Behind (NCLB) act measured students on certain subjects, aiming to bring all students to academic proficiency whilst having penalties and sanctions for schools when students did not progress Teachers diverted class time to teaching the subjects tested, neglecting other subjects as well as reclassifying weaker students as disabled (thus removing from the assessment pool), altering results, reducing test difficulty or lowering the grades needed to pass them
25. The tyranny of velocity?
26. Police Majors & Lieutenants Agile Project Managers/ Scrum Masters
27. You asked to improve velocity and I have delivered. We are now agile.
28. We are so friggin agile...
30. So long as I have increased velocity, none of this will matter Hamsterdam Technical Debt
32. Two-thirds of the population are visual-spatial learners
33. Agile experience in relation to hair Date 2017 2018 2016 2015 2014 ? Volume (hair) ? 2013
34. Our Agile roots... Transparency Inspection Adaptation Scrum
35. Data-based coaching uses data visualisation as a basis for safe conversations and open questions, as well as facilitating more informed, transparent decision making whilst staying true to our Agile roots...
36. Coaching in a data-driven world Descriptive reporting
37. WIP Items Per Week What happened? What is happening?
38. Scatter Plot for Retrospectives Let’s take anything above 10 days - what happened?
39. Scatter Plot Retrospectives Let’s take anything above 10 days - what happened? Data not available (improve DoR check) - Better BDD needed - Work as a team when it’s not working - Too long for PO feedback (20 days!) - Better DoR check needed as was not met PO Feedback Data not available (improve DoR check) Rework due to tech debt - Better DoR check needed - Flowed quickly once sat with the SME Delayed due to lack of clarity on deployment
40. Scatter Plot Retrospectives What is happening? Process Policies Process Policies Process Policies Continuous attention to technical excellence Process Policies Process Policies Feedback loops Feedback loops Feedback loops Feedback loops Process Policies
41. Net Flow per Week Flow? Stop starting, start finishing?
42. Coaching in a data-driven world Descriptive reporting Questions What happened? What is happening? Visuals Diagnostic discover & explore - WIP Items per Week - Cycle Time Scatter Plot - Net flow per week - Epic/Feature Progress
43. Stuck Work Where is the problem? What are the trends? Why is it happening?
44. Estimate Correlation What happened? #NoEstimates? Why is it happening?
45. Coaching in a data-driven world Descriptive reporting Questions Where is the problem? Why is it happening? What are the trends? What happened? What is happening? Visuals Diagnostic discover & explore Predictive forecast - WIP Items per Week - Cycle Time Scatter Plot - Net flow per week - Epic/Feature Progress - Stuck Work - Flow Efficiency - Estimate Correlation - Cumulative Flow Diagram
46. Landing Zone (Burn-up) What is likely to happen? What options are there?
47. Potentially Deliverable Scope (PDS) What options are there? What is likely to happen?
48. Throughput Forecaster
49. Throughput Forecaster 1 story out!
50. Coaching in a data-driven world Descriptive reporting Questions Where is the problem? Why is it happening? What are the trends? What happened? What is happening? Visuals Diagnostic discover & explore Predictive forecast Prescriptive anticipative - WIP Items per Week - Cycle Time Scatter Plot - Net flow per week - Epic/Feature Progress - Stuck Work - Flow Efficiency - Estimate Correlation - Cumulative Flow Diagram - PDS - Landing Zone (Burn-up) - Throughput Forecaster - Monte Carlo: How Many? What is likely to happen? What options are there?
51. WIP by Person What should I do?
52. Stale Work What is the next best action?
54. “When a measure becomes a target, it ceases to be a good measure” - Charles Goodhart
55. Goodhart’s law... “I’m working on that card but didn’t want to move it forward on the board because it would mean we go over our WIP limit”
56. “The ostrich problem includes situations in which people receive relevant information but intentionally fail to evaluate the implications for their goal progress - in other words, they reject the information.” - Dr Thomas Webb
57. The Ostrich Problem... “Hmm….let’s see how we get on for the next 4 weeks before we show the data of what we can deliver to the business”
58. Coaching in a data-driven world Descriptive reporting Questions Where is the problem? Why is it happening? What are the trends? What happened? What is happening? Visuals What should I do? What is the next best action? Diagnostic discover & explore Predictive forecast Prescriptive anticipative - WIP Items per Week - Scatter Plot - Net flow per week - Throughput History Trend - Stuck Work - Flow Efficiency - Estimate Correlation - Cumulative Flow (CFD) - Stale Work - WIP by Person - Aging Work In Progress - Requirements Readiness - PDS - Landing Zone (Burn-up) - Throughput Forecaster - Monte Carlo: How Many? What is likely to happen? What options are there?
59. Coaching in a data-driven world Descriptive reporting Questions What happened? What is happening? Visuals Diagnostic discover & explore Predictive forecast Prescriptive anticipative - WIP Items per Week - Scatter Plot - Net flow per week - Throughput History Trend Where can you get it? - Focused Objective
60. Coaching in a data-driven world Descriptive reporting Questions What happened? What is happening? Visuals Diagnostic discover & explore Predictive forecast Prescriptive anticipative - WIP Items per Week - Scatter Plot - Net flow per week - Throughput History Trend Where can you get it? - SenseAdapt
61. Coaching in a data-driven world Descriptive reporting Questions What happened? What is happening? Visuals Diagnostic discover & explore Predictive forecast Prescriptive anticipative - WIP Items per Week - Scatter Plot - Net flow per week - Throughput History Trend Where can you get it? - Focused Objective
62. Coaching in a data-driven world Descriptive reporting Questions What happened? What is happening? Visuals Diagnostic discover & explore Predictive forecast Prescriptive anticipative - WIP Items per Week - Scatter Plot - Net flow per week - Throughput History Trend Where can you get it? - Focused Objective
63. Coaching in a data-driven world Questions Where is the problem? Why is it happening? What are the trends? Visuals Diagnostic discover & explore Predictive forecast Prescriptive anticipative - Stuck Work - Flow Efficiency - Estimate Correlation - Cumulative Flow (CFD) Descriptive reporting Where can you get it? - SenseAdapt
64. Coaching in a data-driven world Questions Where is the problem? Why is it happening? What are the trends? Visuals Diagnostic discover & explore Predictive forecast Prescriptive anticipative - Stuck Work - Flow Efficiency - Estimate Correlation - Cumulative Flow (CFD) Descriptive reporting Where can you get it? - Actionable Agile
65. Coaching in a data-driven world Questions Where is the problem? Why is it happening? What are the trends? Visuals Diagnostic discover & explore Predictive forecast Prescriptive anticipative - Stuck Work - Flow Efficiency - Estimate Correlation - Cumulative Flow (CFD) Descriptive reporting Where can you get it? - SenseAdapt
66. Coaching in a data-driven world Questions Where is the problem? Why is it happening? What are the trends? Visuals Diagnostic discover & explore Predictive forecast Prescriptive anticipative - Stuck Work - Flow Efficiency - Estimate Correlation - Cumulative Flow (CFD) Descriptive reporting Where can you get it? - Actionable Agile
67. Coaching in a data-driven world Questions Visuals Predictive forecast Prescriptive anticipative Descriptive reporting Diagnostic discover & explore - PDS - Landing Zone (Burn-up) - Throughput Forecaster - Monte Carlo: How Many? What is likely to happen? What options are there? Where can you get it? - SenseAdapt
68. Coaching in a data-driven world Questions Visuals Predictive forecast Prescriptive anticipative Descriptive reporting Diagnostic discover & explore - PDS - Landing Zone (Burn-up) - Throughput Forecaster - Monte Carlo: How Many? What is likely to happen? What options are there? Where can you get it? - SenseAdapt
69. Coaching in a data-driven world Questions Visuals Predictive forecast Prescriptive anticipative Descriptive reporting Diagnostic discover & explore - PDS - Landing Zone (Burn-up) - Throughput Forecaster - Monte Carlo: How Many? What is likely to happen? What options are there? Where can you get it? - Focused Objective
70. Coaching in a data-driven world Questions Visuals Predictive forecast Prescriptive anticipative Descriptive reporting Diagnostic discover & explore - PDS - Landing Zone (Burn-up) - Throughput Forecaster - Monte Carlo: How Many? What is likely to happen? What options are there? Where can you get it? - Actionable Agile
71. Coaching in a data-driven world Descriptive reporting Questions Visuals Diagnostic discover & explore Predictive forecast Prescriptive anticipative What should I do? What is the next best action? - Stale Work - WIP by Person - Aging Work In Progress - Requirements Readiness Where can you get it? - SenseAdapt
72. Coaching in a data-driven world Descriptive reporting Questions Visuals Diagnostic discover & explore Predictive forecast Prescriptive anticipative What should I do? What is the next best action? - Stale Work - WIP by Person - Aging Work In Progress - Requirements Readiness Where can you get it? - SenseAdapt
73. Coaching in a data-driven world Descriptive reporting Questions Visuals Diagnostic discover & explore Predictive forecast Prescriptive anticipative What should I do? What is the next best action? - Stale Work - WIP by Person - Aging Work In Progress - Requirements Readiness Where can you get it? - Actionable Agile
74. Coaching in a data-driven world Descriptive reporting Questions Visuals Diagnostic discover & explore Predictive forecast Prescriptive anticipative What should I do? What is the next best action? - Stale Work - WIP by Person - Aging Work In Progress - Requirements Readiness Where can you get it? - SenseAdapt
75. Coaching in a data-driven world Descriptive reporting Questions Where is the problem? Why is it happening? What are the trends? What happened? What is happening? Visuals What should I do? What is the next best action? Diagnostic discover & explore Predictive forecast Prescriptive anticipative - WIP Items per Week - Scatter Plot - Net flow per week - Throughput History Trend - Stuck Work - Flow Efficiency - Estimate Correlation - Cumulative Flow (CFD) - Stale Work - WIP by Person - Aging Work In Progress - Requirements Readiness - PDS - Landing Zone (Burn-up) - Throughput Forecaster - Monte Carlo: How Many? What is likely to happen? What options are there?
76. Further reading • Troy Magennis - Focused Objective - bit.ly/SimResources • Dan Vacanti - Actionable Agile Metrics for Predictability - tiny.cc/danvac • Bazil Arden - Try data-based coaching to counteract cognitive biases - tiny.cc/bazil1 • Bazil Arden - How data-based coaching enhances psychological safety - tiny.cc/bazil2 • Dominica Degrandis - Making Work Visible - http://tiny.cc/workvisible • Jerry Z. Muller - The Tyranny of Metrics - tiny.cc/tyranny • Goodhart, Charles (1981). "Problems of Monetary Management: The U.K. Experience" • Webb, T., Chang, B. and Benn, Y. (2013). ‘The Ostrich Problem’: Motivated Avoidance or Rejection of Information About Goal Progress.
77. © 2017 PwC. All rights reserved. PwC refers to the PwC network and/or one or more of its member firms, each of which is a separate legal entity. Please see www.pwc.com/structure for further details. Nicolas Brown, Agile Lead nicolas.brown@pwc.com @nbrown02 Thank you!