Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
An Introduction to A/B Testing
Search
timminor
October 03, 2011
2
500
An Introduction to A/B Testing
Barcamp talk given at UX Camp Brighton 2011.
timminor
October 03, 2011
Tweet
Share
More Decks by timminor
See All by timminor
UX Quiz, UX Camp Brighton 2014
timminor
0
110
A UX Quiz, v2
timminor
0
160
A UX Quiz
timminor
1
510
Thinking fast, thinking slow: designing for Cognitive ease
timminor
5
1.7k
Featured
See All Featured
Product Roadmaps are Hard
iamctodd
43
9.6k
GraphQLとの向き合い方2022年版
quramy
28
12k
From Idea to $5000 a Month in 5 Months
shpigford
376
45k
What the flash - Photography Introduction
edds
64
11k
How to Ace a Technical Interview
jacobian
272
22k
[RailsConf 2023] Rails as a piece of cake
palkan
21
3.8k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
111
35k
YesSQL, Process and Tooling at Scale
rocio
160
13k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
272
12k
Infographics Made Easy
chrislema
237
18k
What’s in a name? Adding method to the madness
productmarketing
PRO
14
2.6k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
319
20k
Transcript
@"mminor An Introduction to A/B Testing Photo credit: @lucyjspence
and h7p://www.flickr.com/photos/futureshape/4530140521/
What we’ll be looking at • What is A/B Tes"ng?
• What to test? • What is A/B Tes"ng good for? • A/B tes"ng should not replace • Do’s and Don’ts • Tools
Source: h7p://www.smashingmagazine.com/2010/06/24/the-‐ul"mate-‐guide-‐to-‐a-‐b-‐tes"ng/ What is A/B Testing?
What to test? • Bu7ons: colours, sizes, loca"ons •
Headlines • Product descrip"ons • Form fields (adding and removing) • PPC landing pages • Email content • Pricing and promo"ons • Images
What is A/B Testing good for? • Data-‐driven design
• Ge\ng stakeholders on-‐board • Avoiding personal preferences
A/B testing should not replace: • Talking to users
• Usability tests • Acceptance tests
Do’s and Don’ts • Test simultaneously • Test o^en
• Include only new visitors • Show repeat visitors the same version • Use results to inform future development • Require sta"s"cal significance DO:
• Test too many ideas at once • No
clear idea • Don’t let your preconcep"ons override results • Peek at your results and stop as soon as a significant result has been achieved. See: Evan Miller. Do’s and Don’ts DON’T:
The tools Other tools are available: A/Bingo, Vanity, Unbounce,
Performable, Vertster, SiteSpect, Webtrends Op"mize and Omniture’s Test&Target
The Quiz Show
The Corkscrew A
The Corkscrew B
The Corkscrew A/B
A
Great American Days A
Great American Days B
Great American Days A/B
B
Real Gap
Real Gap A/B
B
CityCliq A
CityCliq B
CityCliq A/B
A
Vendio A
Vendio B
Vendio B
A
Quick Fire Round • What were the two main tools
men"oned? • What percentage increase did the winning RealGap test achieve? • Name one of the three things A/B tes"ng doesn’t replace.
What we looked at • What is A/B Tes"ng?
• What to test? • What is A/B Tes"ng good for? • A/B tes"ng should not replace • Do’s and Don’ts • Tools
References • First slide photograph: h7p://www.flickr.com/photos/futureshape/4530140521/ • www.google.com/websiteop"mizer
• h7p://visualwebsiteop"mizer.com/ • h7p://www.abtests.com/ • h7p://whichtestwon.com/ • h7p://www.smashingmagazine.com/2010/06/24/the-‐ul"mate-‐guide-‐to-‐a-‐b-‐tes"ng/ • h7p://www.conversion-‐rate-‐experts.com/ar"cles/101-‐google-‐website-‐op"mizer-‐"ps/ • h7p://www.evanmiller.org/how-‐not-‐to-‐run-‐an-‐ab-‐test.html • h7p://elem.com/~b"lly/effec"ve-‐ab-‐tes"ng/ • h7p://remoteresear.ch/