Finance@Google2015 Simply Business - Living the Test Culture

A talk I gave on how and when we test at Simply Business, and some of the challenges.

Transcript

1. Case: Living the Test Culture Lukas Oberhuber | CTO - Simply Business
2. 55,000,000 € Revenues 300 Employees 300,000 Policies
4. flickr photo by Pink Sherbet Photography http://flickr.com/photos/pinksherbet/3370498053 shared under a Creative Commons (BY) license
5. The Principles of Living a Test Culture Build, Measure, Learn Test What’s Important How You Should Roll Be Bold It’s Hard
6. Foundation to Test Build Learn Measure
7. What we test Everything Everything Everything Everything Everything Everything Everything
8. How we set up Tests • Decide what to learn (Hypothesis) • Write experiment (Measures) • Build and run experiment • Results • Pivot or persevere
9. How to Ensure Accurate Data • Not easy • Biases and missing skills get in the way • Avoid post hoc analysis
10. Results “You will walk through the valley of the shadow of death” 26% YoY growth in conversion through incremental change
11. Comparison Pages Multi-month build Driven by customer requests Good usability results A/B Test showed 3 point uplift
12. How should you Roll? Sell policies via brokers • Small investment • Failed • New hypothesis • None worked • Pivoted Test what’s important, not what’s easy
13. Be Bold Source https://www.123-reg.co.uk/blog/top-6-lists/six-ab-tests-to-skyrocket-your-sales/
14. Be Bold Small tests, small results People win the lottery all the time! Could it be? Yes it could. Something’s coming, something good. Gonna be great. Source https://www.123-reg.co.uk/blog/top-6-lists/six-ab-tests-to-skyrocket-your-sales/
15. How should you Roll? Look beyond the Comfortable And A/B testing is only one of the methods Source: http://www.nngroup.com/articles/which-ux-research-methods/
16. flickr photo by stephen w morris http://flickr.com/photos/nonlin/4013035510 shared under a Creative Commons (BY) license Culture
17. A Word about HiPPOs
18. THANKS! Lukas Oberhuber lukas.oberhuber@simplybusiness.co.uk www.simplybusiness.co.uk @lukasco