Navigating the tool jungle - How to pick the right tools for your hypothesis experiments

Transcript

1. Navigating the tool jungle - How to pick the right

   tools for your experiments Thomas Leitermann Product Manager MTP Engage Hamburg 24.05.2019 @thomasleiterman

2. 2

3. Prerequisites 3

4. © Business Model Foundry AG Set target & success criteria

   4 Hypothesis Experiment Metric Criteria Users appreciate our value proposition so much, they are willing to pay for it. Insert a paywall after a few example questions. the conversion rate to purchasing the whole course. We are … … super happy with 3% … okay with 1,3% … disappointed with 0,5%. Assumption Measurement Success „Read your plants“-App

5. › Evaluative or generative? › Strength of evidence vs. time-to-insight

   Type of experiment 5

6. Inspired by „The Real Startup Book“, www.realstartupbook.com Evaluative or Generative

   Experiment? 6 Market Product Question Method Question Method Generative Customer  Discovery interview  Ethnography  Surveys, focus groups Problem  Solution interview  Ethnography  Concierge-test  Demo pitch Evaluiative Revenue  Landing Page-Test  Sales pitch & Pre-sales  Fake door  5-second test  Flyer Solution  Prototypes (paper, clickdummy)  Wizard of Oz-test  Analytics  Usability test  Finished product

7. Derived from A.Osterwalder, Strategyzer (https://blog.strategyzer.com/posts/2017/3/21/how-strong-is-your-innovation-evidence) Strength of evidence vs. time-to-insight

   7 Strong evidence Weak evidence Short time to set up and conduct experiment Long time to set up and conduct experiment Landing page-test Wizard of Oz Concierge Pre-sales Validation interview Discovery interview Crowdfunding

8. Fulfilled all prerequisites?  Test card: hypothesis, metric, success criteria

    Matching type of experiment  Budget (team, time, money) 8

9. Learnings 9

10. You are not the user! 10 Slider Feature w/ augmented

    reality

11. Automate user research recruitment 11 Find Set up Schedule Acquire

12. The simpler, the better – and faster 12

13. The simpler, the better – and faster 13

14. The simpler, the better – and faster 14

15. User-test your user testing tools 15

16. Iterate within the experiment 16 18 – 9 o'clock Advertisment

    live 9 – 11 o'clock Analysis of replays & data 11 - 12 o'clock Review & planning 12 - 17 o'clock Write, design and implement changes & extensions 17 - 18 o'clock QA & bug fixing

17. 17 v1 v2 v3

18. Learnings to navigate the tool jungle  Phrase revisable hypothesis

     Find type of experiment  Acquire budget › You are not the user › Automatate user research recruitment › The simpler, the better – and faster › User-test your user testing tools › Iterate within the experiment 18

19. 19 Payment A/B-Testing Prototyping Analytics Page Builder Social Media Ads

    Forms Automation Scheduling Organizing

20. Thank you! Thomas Leitermann Product Manager @thomasleiterman inovex GmbH blog.inovex.de

    www.inovex.de