Estimation confidence calibration using equivalent bet test

Estimation confidence calibration using equivalent bet test

Transcript

1. Calibrating estimation confidence using Confidence intervals
2. Confidence Test • You get $2 if your range is correct! • You must pay $2 if you are incorrect.
3. Confidence Test • You get $2 if your range is correct! • You must pay $2 if you are incorrect. … who’s down?
4. 90% Confidence • How many books did Thomas Jefferson own? Give your answer as a range, e.g. 10-20 books
5. Now for a twist—
6. Proceed with your range If the correct answer is within your range, you collect $2.00. (if you’re wrong, you pay $-2) OR Pivot to wheel-based bet Spin the wheel to receive $2.00. (if it lands in red, you pay $-2)
7. 9,500 Number of books in Thomas Jefferson’s library
8. Q: Which was the better choice?
9. A: They are EQUIVALENT!
10. Equivalent Bet Test • Preference shows where you do not have a calibrated estimate • The “best choice” is ambivalence
11. Applied to Story Estimates
12. Mike Cohn Mountain Goat Software “ each team member is making an implicit assumption about how accurate the estimate needs to be. That is, should I provide an estimate where I’m 50% confident, or one I’m 90% confident in?”
13. < Epic & Story Examples >
14. Mike Cohn Mountain Goat Software “ Once your team has a discussion & agrees on a common confidence level for estimates, they’ll find it much faster and easier to estimate together”
15. For more information, see: • Doug Hubbard, How to Measure Anything, 2007 • Original research: Daniel Kahneman