Estimation made easy (ACE)

Instead of relying on vulnerable human memory we use the actual historical data and run statistical simulations to produce a forecast. Not only is the outcome significantly better than what expert guess provides but also it requires less work from development teams. And, most importantly, it changes the nature of the conversation with customers.

Transcript

3. LET’S FACE IT.
4. WE SUCK AT ESTIMATING.
5. 5 ME
6. Tomek Rusiłko @rusilko
7. ESTIMATION MADE EASY STATISTICAL FORECASTING
8. 1. WHY? EXPERT GUESS, PLANNING FALLACY, MYTH OF SIZING 2. HOW? MONTE CARLO, STATISTICAL FORECAST, PROJECTR 3. WHAAAAT? REAL WORLD, REAL PROJECTS, REAL DATA
11. 11 EXPERT GUESS
12. IF A PROJECT HAS NO RISKS, DON'T DO IT. Tom DeMarco & Timothy Lister TIMES OF UNCERTAINTY T. DEMARCO, T. LISTER: WALTZING WITH BEARS
13. ACCURACY IN ESTIMATING DID NOT IMPROVE AS INFORMATION ACCUMULATED, WHILE CONFIDENCE INCREASED CONSISTENTLY. Claire Tsai, Joshua Klayman, Reid Hastie ACCURACY ↘︎, CONFIDENCE⇧ SOURCE: TSAI, KLAYMAN, HASTIE: EFFECTS OF AMOUNT OF INFORMATION ON JUDGMENT ACCURACY AND CONFIDENCE
14. SCIENTISTS AND WRITERS ARE NOTORIOUSLY PRONE TO UNDERESTIMATE THE TIME REQUIRED TO COMPLETE A PROJECT, EVEN WHEN THEY HAVE CONSIDERABLE EXPERIENCE OF PAST FAILURES TO LIVE UP TO PLANNED SCHEDULES. A SIMILAR BIAS HAS BEEN DOCUMENTED IN ENGINEERS' ESTIMATES. Daniel Kahneman, Amos Tversky PLANNING FALLACY
15. WE SUCK AT ESTIMATING.
16. STORY POINTS
20. 20 LEAD TIME WORK TIME LEA WAIT TIME
21. 21 WORK TIME LEAD TIME FLOW EFFICIENCY= LEAD TIME WORK TIME LEA WAIT TIME
22. 22 LEA WAIT TIME LEA WAIT TIME
23. COUNTING THE NUMBER OF STORIES METRIC DOESN'T TAKE THE SIZE INTO ACCOUNT. IT TURNS OUT IT DOESN'T MATTER. THE SIZE OF STORIES IS GELLED TO A VERY COMMON SIZE. WE COULD USE THROUGHPUT VERY SUCCESSFULLY WITH THE RESEARCH. Larry Maccherone THROUGHPUT VS STORY POINTS
25. 25 estimation.lunarlogic.io
26. EXPERT GUESS DOESN’T WORK
27. YOUR PLANS UNIVERSE’S PLANS
28. YOUR PLANS UNIVERSE’S PLANS source: DOGHOUSEDIARIES
29. ALTERNATIVES?
30. STATISTICAL FORECASTING BASED ON THE WORK OF TROY MAGENNIS, LARRY MACCHERONE AND DOUGLAS HUBBARD
31. PREDICTING THE FUTURE FROM (HISTORICAL) DATA
32. 32 TROY MAGENNIS LARRY MACCHERONE source: FOCUSEDOBJECTIVE.COM
34. MONTE CARLO SIMULATION
35. 0 10 20 30 40 50 Iteration 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 MONTE CARLO EXAMPLE Today Burn-up Scope 2 2 5 8 4 Throughput Values
37. HISTORICAL DATA: COMPLETED STORIES start date end date 2016-02-01 2016-02-04 2016-02-01 2016-02-03 2016-02-05 2016-02-11 2016-02-04 2016-02-10 2016-02-08 2016-02-08 2016-02-09 2016-02-12 2016-02-09 2016-02-11 2016-02-12 2016-02-16 2016-02-11 2016-02-16 2016-02-19 2016-02-22 2016-02-18 2016-02-22 start date end date 2016-02-19 2016-02-24 2016-02-17 2016-02-24 2016-02-23 2016-02-24 2016-02-23 2016-02-25 2016-02-25 2016-02-25 2016-02-25 2016-02-26 2016-02-26 2016-03-03 2016-02-26 2016-03-03 2016-02-29 2016-03-02 2016-03-01 2016-03-03
38. HISTORICAL DATA: TIMELINE 1 2 3 4 5 8 9 10 11 12 15 16 17 18 19 22 23 24 25 26 29 1 2 3
39. LEAD TIME STATISTICAL MODEL: INPUT #1
40. WHY LEAD TIMES? Mon Tue Wed Thur Fri WHY LEAD TIMES? 5 DAYS
41. WHY LEAD TIMES? Mon Tue Wed Thur Fri BUG FIX SICK LEAD TIME 5 DAYS WT
42. HISTORICAL DATA: LEAD TIMES 1 2 3 4 5 8 9 10 11 12 15 16 4 3 5 5 4 3 1 3 4 17 18 19 22 23 24 25 26 29 1 2 3 3 2 2 4 6 1 2 3 5 5 3 3
43. LEAD TIME DISTRIBUTION 4 3 5 5 4 3 1 3 4 3 2 2 4 6 1 2 3 5 5 3 3
44. LEAD TIME DISTRIBUTION 6 days 5 days 4 days 3 days 2 days 1 day
45. WORK IN PROGRESS STATISTICAL MODEL: INPUT #2
46. HISTORICAL DATA: WORK IN PROGRESS 1 2 3 4 5 8 9 10 11 12 15 16 17 18 19 22 23 24 25 26 29 1 2 3 2 2 2 2 2 3 4 4 4 3 1 2 4 4 4 4 3 3 3 4 2 2 4 3
47. WORK IN PROGRESS DISTRIBUTION 4 tasks 3 tasks 2 tasks 1 task
48. READY… 4 tasks 3 tasks 2 tasks 1 task 6 days 5 days 4 days 3 days 2 days 1 day 0 10 20 30 40 50 Iteration 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 } 24 tasks to go
49. 6 days 5 days 4 days 3 days 2 days 1 day 3 5 5 4 3 1 3 4 3 2 2 6 1 3 5 5 3 3 3 3 5 5 5 5 SAMPLING LEAD TIME VALUES
50. 13 14 15 16 17 18 19 20 21 22 23 24 1 2 3 4 5 6 7 8 9 10 11 12 SAMPLING LEAD TIME VALUES 3 5 5 4 3 1 3 4 3 2 2 6 1 3 5 5 3 3 3 3 5 5 5 5
51. SUM(LEAD TIME VALUES) = UNITS OF WORK 3 5 5 4 3 1 3 4 3 2 2 6 1 3 5 5 3 3 3 3 5 5 5 5 87
52. 1 2 3 4 5 6 7 8 9 10 11 12 SAMPLING WORK IN PROGRESS 4 tasks 3 tasks 2 tasks 1 task 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 87
53. 1 2 3 4 5 6 7 8 9 10 11 12 SAMPLING WORK IN PROGRESS 4 tasks 3 tasks 2 tasks 1 task 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 87
54. 54 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
55. DAYS 29
56. 28 29 27 30 29 29 27 31 32 31
58. 0 400 800 1200 1600 Workdays 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 0 400 800 1200 1600 Workdays 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
59. 0 400 800 1200 1600 Workdays 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 0 400 800 1200 1600 Workdays 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 50%
60. 0 400 800 1200 1600 Workdays 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 0 400 800 1200 1600 Workdays 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 90%
62. 0 400 800 1200 1600 Workdays 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 0 400 800 1200 1600 Workdays 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 99%
63. GETPROJECTR.COM
64. A NEW PROJECT VERSUS AN ONGOING PROJECT
66. 90% CONFIDENCE LEVEL MEANS THAT OUR ESTIMATE IS CORRECT 9 TIMES OUT OF 10. KINDA. Pawel Brodzinski 90% CONFIDENCE INTERVAL?
67. UNCERTAINTY SOURCES OF UNCERTAINTY
68. PROJECT ALFA
69. PROJECT ALFA
70. PROJECT ALFA
71. PROJECT ALFA
72. PROJECT BRAVO
73. PROJECT BRAVO
74. PROJECT BRAVO
75. PROJECT BRAVO
76. PROJECT BRAVO
77. PROJECT BRAVO
78. PROJECT BRAVO
79. 79 Garbage in Garbage out
80. ANY PROPOSED FORECASTING METHOD JUST HAS TO BE BETTER THAN WHAT YOU DO NOW, OR AT LEAST LESS EXPENSIVE WITH A SIMILAR RESULT. Troy Magennis BE BETTER
81. 81 Thank You! Tomek Rusiłko @rusilko
82. SOURCES & RESOURCES ▸ Troy Magennis original work: http://www.lkce13.com/videos/magennis/ ▸ http://focusedobjective.com/wp-content/uploads/2013/05/Modeling-and-Simulating-Software- Projects-Troy-Magennis.pdf ▸ http://blog.lunarlogic.io/2016/how-we-estimate-at-lunar-logic/ ▸ https://www.chicagobooth.edu/research/workshops/marketing/archive/workshoppapers/s06/tsai.pdf ▸ Planning Fallacy: https://books.google.pl/books?id=R- syxO7M67AC&pg=PA9&q=&redir_esc=y#v=onepage&q&f=false ▸ https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555 ▸ Flow efficiency: https://hakanforss.wordpress.com/2014/06/17/flow-thinking-aceconf/ ▸ http://zsoltfabok.com/blog/2013/12/flow-efficiency/ ▸ https://www.infoq.com/presentations/agile-quantify ▸ http://brodzinski.com/2015/02/story-points-velocity-the-good-bits.html ▸ https://estimation.lunarlogic.io/ ▸ https://www.agilealliance.org/estimation-and-forecasting/ ▸ Projectr: http://getprojectr.com/