blog: simonecasciaroli.com twitter: @simonecasc Introducing Thinking in Bets principles in product development how we created a process that help the team embrace uncertainty and increased outcomes achieved

Simone Casciaroli Engineering Manager at Babylon Health

SecureSafe (2008) • Lean Startup: • Problem team/Solution team • Hypothesis and experiments • Assumption testing

Spotter (2017) • Async communication • How do I make more visible the process that bring us from ideation to validation? • How do you prioritise what to build next?

How problem and solution teams interact

Ready In Progress Done Standard implementation flow (simplified)

Implementation flow Ready In Progress Done Solution definition flow

Implementation flow Solution definition flow Ready In Progress Done Problem definition flow (simplified)

Implementation flow Solution flow Problem flow Sequentially

Implementation flow Solution flow Problem flow Incrementally

Implementation Solution Problem Done if: Users can interact with it in production and feedback can be collected Done if: Spikes or implementation cards are ready Done if: A Solution card with a detailed description of the problem and a draft solution is ready

Implementation Solution Problem If you have all the details for defining the scenarios for the cards and you don’t need someone else feedback or a technical spike write this If you know the problem very well OR it’s fine to assume you know it well but you only have a draft idea of the scenarios and/or require input from other write this Always think about the problem BUT only write this if you need more information about the problem itself or you are not sure about the problem

In Summary ● Everyone was more aware of what was going on ● It helped question more our understanding of the problem ● It works with small teams or with teams that keep WIP low (via pairing/mob)

Guider (2019) • Struggle to convey the uncertainty of our decision making to other stakeholders (commercial side of the business) and to the team • How do we know when to run further research? • How do you help the team thinking about multiple solution when solving a problem?

How to express uncertainty

Annie Duke • Decisions are probabilities • Dangers of “resulting” • Being comfortable with uncertainty • Interview with Shane Parrish Thinking in Bets

Bet Execution Bet Definition Problem Bet vs Experiment

What problems are we solving • Problems exist at different levels • Business level: increase revenue MoM 10% in Q2 • Product level: how does the user currently takes notes during meetings? • UX level: does the user understand how to save a new note? Problem card

Our first draft for a bet or initial hypothesis • Outcome • Our Bet • Confidence level • Chips (how much it will cost to play the bet) • How to verify if we won Bet Definition Card

The bet is on the table Bet Execution Card • After we have done: spikes, design, analysis • We are ready to roll the dices and put the bet in front of our users/customers

In Summary ● It increased awareness on the uncertainty of the product team job ● Increased engagement during discovery ● The process was different so for the first few iterations I had to play a much more active role

How to increase your options

When we discuss a problem card What happen in Planning • Draft a number of Bets cards • Define what we are betting on and how to measure it • Discuss our confidence level • How much it cost • We would then either: • Pick cheap bets even if confidence level was low • Pick bets where confidence was high • Play a card to increase confidence

An example of Planning Session The Problem • Context: Dating Website • Problem: Increase 30% conversion for the Signup -> First Date Show Matches Send Request Request Accepted Book your First date 60% 50% 70% 90% Meet

Lets define some bets • An important part of funnel is finding your best match so we should improve our matching algorithm • We could remove accepting the request and allow to book the first date upfront 60% £20’000 70% £20’000 Confidence level Chips • Interview customers who haven’t find a match (1 week) • Interview customers on how important is to keep control over matches (2 weeks) • Keep the option to accept request but allow to book in advance the first date 75% £5’000

How do I increase outcomes achieved? ● Increase Your Options ● Involve The All Team in finding options for your problems ● Pick higher certainty bets or cheap bets first ● Make sure you have plenty of diversity in the sessions

Did you know we’re hiring? We’re looking for exceptional talent globally across the organization. Check out our Career Site to find your next opportunity with Babylon. babylonhealth.com/careers/technology

Thank you What questions do you have for me? blog: simonecasciaroli.com twitter: @simonecasc

Bonus Slides

Validate Columns • If you are serious about being outcome driven add a validation column in your board and review with teams if you won or lost your bets • It’s very hard to win with your first bet, it’s more likely to incrementally getting there • If you are winning all the time either you play too safe or you are gaming the numbers :) Allow to book first date while selecting your match

How to use uncertainty to prioritise