Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Finance@Google2015 Simply Business - Living the Test Culture
Lukas Oberhuber
September 16, 2015
Technology
0
26
Finance@Google2015 Simply Business - Living the Test Culture
A talk I gave on how and when we test at Simply Business, and some of the challenges.
Lukas Oberhuber
September 16, 2015
Tweet
Share
More Decks by Lukas Oberhuber
See All by Lukas Oberhuber
lukaso
0
150
lukaso
0
340
Other Decks in Technology
See All in Technology
ocise
1
1.8k
askul
1
200
sat
40
29k
andysumi
0
160
azara
1
890
iwashi
1
190
binarymist
0
1.3k
hmatsu47
3
110
swoon
1
590
lmi
3
1.1k
koukyo1994
3
470
tosh2230
3
270
Featured
See All Featured
iamctodd
19
2k
hatefulcrawdad
257
17k
thoeni
3
610
davidbonilla
70
3.6k
jlugia
217
16k
chriscoyier
498
130k
andyhume
63
3.7k
mthomps
38
2.3k
jnunemaker
PRO
40
4.6k
kastner
54
1.9k
wjessup
339
16k
reverentgeek
167
7.3k
Transcript
Case: Living the Test Culture Lukas Oberhuber | CTO -
Simply Business
55,000,000 € Revenues 300 Employees 300,000 Policies
None
flickr photo by Pink Sherbet Photography http://flickr.com/photos/pinksherbet/3370498053 shared under a
Creative Commons (BY) license
The Principles of Living a Test Culture Build, Measure, Learn
Test What’s Important How You Should Roll Be Bold It’s Hard
Foundation to Test Build Learn Measure
What we test Everything Everything Everything Everything Everything Everything Everything
How we set up Tests • Decide what to learn
(Hypothesis) • Write experiment (Measures) • Build and run experiment • Results • Pivot or persevere
How to Ensure Accurate Data • Not easy • Biases
and missing skills get in the way • Avoid post hoc analysis
Results “You will walk through the valley of the shadow
of death” 26% YoY growth in conversion through incremental change
Comparison Pages Multi-month build Driven by customer requests Good usability
results A/B Test showed 3 point uplift
How should you Roll? Sell policies via brokers • Small
investment • Failed • New hypothesis • None worked • Pivoted Test what’s important, not what’s easy
Be Bold Source https://www.123-reg.co.uk/blog/top-6-lists/six-ab-tests-to-skyrocket-your-sales/
Be Bold Small tests, small results People win the lottery
all the time! Could it be? Yes it could. Something’s coming, something good. Gonna be great. Source https://www.123-reg.co.uk/blog/top-6-lists/six-ab-tests-to-skyrocket-your-sales/
How should you Roll? Look beyond the Comfortable And A/B
testing is only one of the methods Source: http://www.nngroup.com/articles/which-ux-research-methods/
flickr photo by stephen w morris http://flickr.com/photos/nonlin/4013035510 shared under a
Creative Commons (BY) license Culture
A Word about HiPPOs
THANKS! Lukas Oberhuber lukas.oberhuber@simplybusiness.co.uk www.simplybusiness.co.uk @lukasco