Your assumptions are probably wrong

My August 2018 talk to The Product Mentor group

Transcript

1. Your assumptions are probably wrong. by nis frome
2. Nis Frome Enabling product teams to experiment faster Co-founder at Alpha product content 2 This Is Product Management Product Management Insider About Me
3. 3 Objective Why, when, and how effective product managers de-risk assumptions & inform decisions.
4. 4 Challenge No matter how smart or empathic you are, your assumptions are (often) still going to be wrong.
5. 5 Breakdown When running experiments, product teams that are certain their assumptions are right quickly learn: Customers don’t understand insider jargon Assumptions used to be right but are now wrong A wrong conclusion was derived from data or anecdotes Assumption is right but about the wrong audience
6. 6 Amazon’s Approach Customers are divinely discontent. “One thing I love about customers is that they are divinely discontent. Their expectations are never static — they go up. It’s human nature…You cannot rest on your laurels in this world. Customers won’t have it.”
7. 7 Stratechery Analysis “What is amazing today is table stakes tomorrow.” How most companies perceive reality Reality
8. 8 It’s Not Just Product Teams Assumptions are everywhere! Facebook could charge a similar rate to what it currently earns from users via ads…Charging $1.65 per month, or even $7 per month to remove ads from Facebook could feel very reasonable to some users. Tulin Erdem, a professor of marketing at NYU's Stern business school, said she likes the Facebook [apology] ad more than the others. There's "more of an emotional appeal there," she said. "I think it's more effective." Taxi apps may be a cheap and convenient way to travel but they've created their fair share of controversy…there have been a disturbing number of Lyft and Uber drivers accused of sexual assault over the years.
9. 9 Assumptions are Testable Fight assumptions with experiments! While respondents trusted Amazon more than the other companies, they hesitated when asked how willing they would be to engage the tech giant on certain healthcare initiatives…For each of nine healthcare-related options, the majority of respondents said they were not at all interested in receiving those services from Amazon. If you look at how much kids' content factors into the choice of whether to subscribe to Netflix for the respondents to this survey, 38% said it was extremely important.
10. 10 Example Assumption: Uber’s new safety features are brilliant and the solution to safety concerns.
11. 11 Pre-launch Results Consumers have safety concerns!
12. 12 Post-Launch Results Reality is often more nuanced...
13. 13 Substantiating Assumptions When and how should you test assumptions? Cost of being wrong Decision irreversibility Ship and iterate Test alternatives Test alternatives extensively Test extensively
14. 14 Intuition So what is the value of developing empathy and intuition? Prioritize experiments and methodologies Foresee potential customer expectations Smarter iterations and pattern recognition
15. 15 Getting Started What assumptions are you holding onto that haven’t been challenged recently? It’s time to consider the risks of being wrong.