Finance@Google2015 Simply Business - Living the Test Culture

Transcript

1. Case: Living the Test Culture Lukas Oberhuber | CTO -

   Simply Business

2. 55,000,000 € Revenues 300 Employees 300,000 Policies

3. None

4. flickr photo by Pink Sherbet Photography http://flickr.com/photos/pinksherbet/3370498053 shared under a

   Creative Commons (BY) license

5. The Principles of Living a Test Culture Build, Measure, Learn

   Test What’s Important How You Should Roll Be Bold It’s Hard

6. Foundation to Test Build Learn Measure

7. What we test Everything Everything Everything Everything Everything Everything Everything

8. How we set up Tests • Decide what to learn

   (Hypothesis) • Write experiment (Measures) • Build and run experiment • Results • Pivot or persevere

9. How to Ensure Accurate Data • Not easy • Biases

   and missing skills get in the way • Avoid post hoc analysis

10. Results “You will walk through the valley of the shadow

    of death” 26% YoY growth in conversion through incremental change

11. Comparison Pages Multi-month build Driven by customer requests Good usability

    results A/B Test showed 3 point uplift

12. How should you Roll? Sell policies via brokers • Small

    investment • Failed • New hypothesis • None worked • Pivoted Test what’s important, not what’s easy

13. Be Bold Source https://www.123-reg.co.uk/blog/top-6-lists/six-ab-tests-to-skyrocket-your-sales/

14. Be Bold Small tests, small results People win the lottery

    all the time! Could it be? Yes it could. Something’s coming, something good. Gonna be great. Source https://www.123-reg.co.uk/blog/top-6-lists/six-ab-tests-to-skyrocket-your-sales/

15. How should you Roll? Look beyond the Comfortable And A/B

    testing is only one of the methods Source: http://www.nngroup.com/articles/which-ux-research-methods/

16. flickr photo by stephen w morris http://flickr.com/photos/nonlin/4013035510 shared under a

    Creative Commons (BY) license Culture

17. A Word about HiPPOs

18. THANKS! Lukas Oberhuber lukas.oberhuber@simplybusiness.co.uk www.simplybusiness.co.uk @lukasco