Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Finance@Google2015 Simply Business - Living the...
Search
Lukas Oberhuber
September 16, 2015
Technology
0
57
Finance@Google2015 Simply Business - Living the Test Culture
A talk I gave on how and when we test at Simply Business, and some of the challenges.
Lukas Oberhuber
September 16, 2015
Tweet
Share
More Decks by Lukas Oberhuber
See All by Lukas Oberhuber
Lean Startup Conference 2019: 8 Maladies of Rapid Experimentation
lukaso
0
340
Technical Bankruptcy
lukaso
0
610
Other Decks in Technology
See All in Technology
Autonomous Database - Dedicated 技術詳細 / adb-d_technical_detail_jp
oracle4engineer
PRO
4
10k
複数サービスを支えるマルチテナント型Batch MLプラットフォーム
lycorptech_jp
PRO
0
280
大「個人開発サービス」時代に僕たちはどう生きるか
sotarok
20
9.6k
Flutterでキャッチしないエラーはどこに行く
taiju59
0
220
落ちる 落ちるよ サーバーは落ちる
suehiromasatoshi
0
150
DevIO2025_継続的なサービス開発のための技術的意思決定のポイント / how-to-tech-decision-makaing-devio2025
nologyance
1
370
S3アクセス制御の設計ポイント
tommy0124
2
190
企業の生成AIガバナンスにおけるエージェントとセキュリティ
lycorptech_jp
PRO
2
150
おやつは300円まで!の最適化を模索してみた
techtekt
PRO
0
290
[ JAWS-UG 東京 CommunityBuilders Night #2 ]SlackとAmazon Q Developerで 運用効率化を模索する
sh_fk2
3
370
Webブラウザ向け動画配信プレイヤーの 大規模リプレイスから得た知見と学び
yud0uhu
0
230
Django's GeneratedField by example - DjangoCon US 2025
pauloxnet
0
110
Featured
See All Featured
The Art of Programming - Codeland 2020
erikaheidi
55
13k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Embracing the Ebb and Flow
colly
87
4.8k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.9k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Intergalactic Javascript Robots from Outer Space
tanoku
272
27k
VelocityConf: Rendering Performance Case Studies
addyosmani
332
24k
Code Reviewing Like a Champion
maltzj
525
40k
Raft: Consensus for Rubyists
vanstee
140
7.1k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.6k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.4k
Documentation Writing (for coders)
carmenintech
74
5k
Transcript
Case: Living the Test Culture Lukas Oberhuber | CTO -
Simply Business
55,000,000 € Revenues 300 Employees 300,000 Policies
None
flickr photo by Pink Sherbet Photography http://flickr.com/photos/pinksherbet/3370498053 shared under a
Creative Commons (BY) license
The Principles of Living a Test Culture Build, Measure, Learn
Test What’s Important How You Should Roll Be Bold It’s Hard
Foundation to Test Build Learn Measure
What we test Everything Everything Everything Everything Everything Everything Everything
How we set up Tests • Decide what to learn
(Hypothesis) • Write experiment (Measures) • Build and run experiment • Results • Pivot or persevere
How to Ensure Accurate Data • Not easy • Biases
and missing skills get in the way • Avoid post hoc analysis
Results “You will walk through the valley of the shadow
of death” 26% YoY growth in conversion through incremental change
Comparison Pages Multi-month build Driven by customer requests Good usability
results A/B Test showed 3 point uplift
How should you Roll? Sell policies via brokers • Small
investment • Failed • New hypothesis • None worked • Pivoted Test what’s important, not what’s easy
Be Bold Source https://www.123-reg.co.uk/blog/top-6-lists/six-ab-tests-to-skyrocket-your-sales/
Be Bold Small tests, small results People win the lottery
all the time! Could it be? Yes it could. Something’s coming, something good. Gonna be great. Source https://www.123-reg.co.uk/blog/top-6-lists/six-ab-tests-to-skyrocket-your-sales/
How should you Roll? Look beyond the Comfortable And A/B
testing is only one of the methods Source: http://www.nngroup.com/articles/which-ux-research-methods/
flickr photo by stephen w morris http://flickr.com/photos/nonlin/4013035510 shared under a
Creative Commons (BY) license Culture
A Word about HiPPOs
THANKS! Lukas Oberhuber
[email protected]
www.simplybusiness.co.uk @lukasco