Introducing Thinking in Bets principles in product development

Transcript

1. blog: simonecasciaroli.com twitter: @simonecasc Introducing Thinking in Bets principles in product development how we created a process that help the team embrace uncertainty and increased outcomes achieved
2. Simone Casciaroli Engineering Manager at Babylon Health
3. SecureSafe (2008) • Lean Startup: • Problem team/Solution team • Hypothesis and experiments • Assumption testing
4. Spotter (2017) • Async communication • How do I make more visible the process that bring us from ideation to validation? • How do you prioritise what to build next?
5. How problem and solution teams interact
6. Ready In Progress Done Standard implementation flow (simplified)
7. Implementation flow Ready In Progress Done Solution definition flow
8. Implementation flow Solution definition flow Ready In Progress Done Problem definition flow (simplified)
9. Implementation flow Solution flow Problem flow Sequentially
10. Implementation flow Solution flow Problem flow Incrementally
11. Implementation Solution Problem Done if: Users can interact with it in production and feedback can be collected Done if: Spikes or implementation cards are ready Done if: A Solution card with a detailed description of the problem and a draft solution is ready
12. Implementation Solution Problem If you have all the details for defining the scenarios for the cards and you don’t need someone else feedback or a technical spike write this If you know the problem very well OR it’s fine to assume you know it well but you only have a draft idea of the scenarios and/or require input from other write this Always think about the problem BUT only write this if you need more information about the problem itself or you are not sure about the problem
13. In Summary • Everyone was more aware of what was going on • It helped question more our understanding of the problem • It works with small teams or with teams that keep WIP low (via pairing/mob)
14. Guider (2019) • Struggle to convey the uncertainty of our decision making to other stakeholders (commercial side of the business) and to the team • How do we know when to run further research? • How do you help the team thinking about multiple solution when solving a problem?
15. How to express uncertainty
16. Annie Duke • Decisions are probabilities • Dangers of “resulting” • Being comfortable with uncertainty • Interview with Shane Parrish Thinking in Bets
17. Bet Execution Bet Definition Problem Bet vs Experiment
18. What problems are we solving • Problems exist at different levels • Business level: increase revenue MoM 10% in Q2 • Product level: how does the user currently takes notes during meetings? • UX level: does the user understand how to save a new note? Problem card
19. Our first draft for a bet or initial hypothesis • Outcome • Our Bet • Confidence level • Chips (how much it will cost to play the bet) • How to verify if we won Bet Definition Card
20. The bet is on the table Bet Execution Card • After we have done: spikes, design, analysis • We are ready to roll the dices and put the bet in front of our users/customers
21. In Summary • It increased awareness on the uncertainty of the product team job • Increased engagement during discovery • The process was different so for the first few iterations I had to play a much more active role
22. How to increase your options
23. When we discuss a problem card What happen in Planning • Draft a number of Bets cards • Define what we are betting on and how to measure it • Discuss our confidence level • How much it cost • We would then either: • Pick cheap bets even if confidence level was low • Pick bets where confidence was high • Play a card to increase confidence
24. An example of Planning Session The Problem • Context: Dating Website • Problem: Increase 30% conversion for the Signup -> First Date Show Matches Send Request Request Accepted Book your First date 60% 50% 70% 90% Meet
25. Lets define some bets • An important part of funnel is finding your best match so we should improve our matching algorithm • We could remove accepting the request and allow to book the first date upfront 60% £20’000 70% £20’000 Confidence level Chips • Interview customers who haven’t find a match (1 week) • Interview customers on how important is to keep control over matches (2 weeks) • Keep the option to accept request but allow to book in advance the first date 75% £5’000
26. How do I increase outcomes achieved? • Increase Your Options • Involve The All Team in finding options for your problems • Pick higher certainty bets or cheap bets first • Make sure you have plenty of diversity in the sessions
27. Did you know we’re hiring? We’re looking for exceptional talent globally across the organization. Check out our Career Site to find your next opportunity with Babylon. babylonhealth.com/careers/technology
28. Thank you What questions do you have for me? blog: simonecasciaroli.com twitter: @simonecasc
29. Bonus Slides
30. Validate Columns • If you are serious about being outcome driven add a validation column in your board and review with teams if you won or lost your bets • It’s very hard to win with your first bet, it’s more likely to incrementally getting there • If you are winning all the time either you play too safe or you are gaming the numbers :) Allow to book first date while selecting your match
31. How to use uncertainty to prioritise