Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
An Introduction to A/B Testing
Search
timminor
October 03, 2011
2
510
An Introduction to A/B Testing
Barcamp talk given at UX Camp Brighton 2011.
timminor
October 03, 2011
Tweet
Share
More Decks by timminor
See All by timminor
UX Quiz, UX Camp Brighton 2014
timminor
0
120
A UX Quiz, v2
timminor
0
160
A UX Quiz
timminor
1
510
Thinking fast, thinking slow: designing for Cognitive ease
timminor
5
1.7k
Featured
See All Featured
Building Applications with DynamoDB
mza
91
6.1k
What's in a price? How to price your products and services
michaelherold
244
12k
Reflections from 52 weeks, 52 projects
jeffersonlam
347
20k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
666
120k
YesSQL, Process and Tooling at Scale
rocio
169
14k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
8
1.2k
Imperfection Machines: The Place of Print at Facebook
scottboms
266
13k
GraphQLの誤解/rethinking-graphql
sonatard
67
10k
Visualization
eitanlees
146
15k
Fireside Chat
paigeccino
34
3.1k
Into the Great Unknown - MozCon
thekraken
33
1.5k
A designer walks into a library…
pauljervisheath
205
24k
Transcript
@"mminor An Introduction to A/B Testing Photo credit: @lucyjspence
and h7p://www.flickr.com/photos/futureshape/4530140521/
What we’ll be looking at • What is A/B Tes"ng?
• What to test? • What is A/B Tes"ng good for? • A/B tes"ng should not replace • Do’s and Don’ts • Tools
Source: h7p://www.smashingmagazine.com/2010/06/24/the-‐ul"mate-‐guide-‐to-‐a-‐b-‐tes"ng/ What is A/B Testing?
What to test? • Bu7ons: colours, sizes, loca"ons •
Headlines • Product descrip"ons • Form fields (adding and removing) • PPC landing pages • Email content • Pricing and promo"ons • Images
What is A/B Testing good for? • Data-‐driven design
• Ge\ng stakeholders on-‐board • Avoiding personal preferences
A/B testing should not replace: • Talking to users
• Usability tests • Acceptance tests
Do’s and Don’ts • Test simultaneously • Test o^en
• Include only new visitors • Show repeat visitors the same version • Use results to inform future development • Require sta"s"cal significance DO:
• Test too many ideas at once • No
clear idea • Don’t let your preconcep"ons override results • Peek at your results and stop as soon as a significant result has been achieved. See: Evan Miller. Do’s and Don’ts DON’T:
The tools Other tools are available: A/Bingo, Vanity, Unbounce,
Performable, Vertster, SiteSpect, Webtrends Op"mize and Omniture’s Test&Target
The Quiz Show
The Corkscrew A
The Corkscrew B
The Corkscrew A/B
A
Great American Days A
Great American Days B
Great American Days A/B
B
Real Gap
Real Gap A/B
B
CityCliq A
CityCliq B
CityCliq A/B
A
Vendio A
Vendio B
Vendio B
A
Quick Fire Round • What were the two main tools
men"oned? • What percentage increase did the winning RealGap test achieve? • Name one of the three things A/B tes"ng doesn’t replace.
What we looked at • What is A/B Tes"ng?
• What to test? • What is A/B Tes"ng good for? • A/B tes"ng should not replace • Do’s and Don’ts • Tools
References • First slide photograph: h7p://www.flickr.com/photos/futureshape/4530140521/ • www.google.com/websiteop"mizer
• h7p://visualwebsiteop"mizer.com/ • h7p://www.abtests.com/ • h7p://whichtestwon.com/ • h7p://www.smashingmagazine.com/2010/06/24/the-‐ul"mate-‐guide-‐to-‐a-‐b-‐tes"ng/ • h7p://www.conversion-‐rate-‐experts.com/ar"cles/101-‐google-‐website-‐op"mizer-‐"ps/ • h7p://www.evanmiller.org/how-‐not-‐to-‐run-‐an-‐ab-‐test.html • h7p://elem.com/~b"lly/effec"ve-‐ab-‐tes"ng/ • h7p://remoteresear.ch/