Unit 3 - An Experimental Approach to Product Development
This class will present hypothesis-driven development, the modern paradigm for evolving validated products. We’ll dive into how to frame hypotheses, design experiments, and use A/B testing to gather data to prove or disprove our ideas.
Design” http://bit.ly/TylT6A hypothesis-driven delivery We believe that [building this feature] [for these people] will achieve [this outcome]. We will know we are successful when we see [this signal from the market].
design an experiment to test your hypothesis • what do you expect the results to be? • what result will conﬁrm your hypothesis? • what result will disprove your hypothesis? • how soon can we get the result?
Creativity must ﬂow from everywhere. Whether you are a summer intern or the CTO, any good idea must be able to seek an objective test, preferably a test that exposes the idea to real customers. Everyone must be able to experiment, learn, and iterate.” http://glinden.blogspot.com/2006/04/early-amazon-shopping-cart.html
with a working system faster feedback (assuming people pay attention) higher motivation quickly release high priority features / bugﬁxes working in small batches Don Reinertsen, Principles of Product Development Flow, ch5.
• Negotiable: a conversation not a contract • Valuable: delivers beneﬁt to a stakeholder • Estimable: to a good degree of precision • Small: less than a week to build • Testable: clearly deﬁned acceptance criteria http://bit.ly/small-batches-invest
For example: “in order to make our business model work, 5% of people who sign up for my service that meet my criteria must pay $100 for the service.” • Write down hypothesis. • Design an experiment to test this with small sample size and acceptable level of precision, including testable success criteria. • Gather data. Analyze. State results. • State if hypothesis was validated or not. • What did you learn? What changes will you make?