Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
An Introduction to A/B Testing
Search
timminor
October 03, 2011
2
500
An Introduction to A/B Testing
Barcamp talk given at UX Camp Brighton 2011.
timminor
October 03, 2011
Tweet
Share
More Decks by timminor
See All by timminor
UX Quiz, UX Camp Brighton 2014
timminor
0
120
A UX Quiz, v2
timminor
0
160
A UX Quiz
timminor
1
510
Thinking fast, thinking slow: designing for Cognitive ease
timminor
5
1.7k
Featured
See All Featured
Into the Great Unknown - MozCon
thekraken
10
980
Rebuilding a faster, lazier Slack
samanthasiow
72
8.2k
The Illustrated Children's Guide to Kubernetes
chrisshort
29
46k
Web development in the modern age
philhawksworth
202
10k
A designer walks into a library…
pauljervisheath
199
23k
Typedesign – Prime Four
hannesfritz
36
2.1k
Mobile First: as difficult as doing things right
swwweet
216
8.6k
Happy Clients
brianwarren
91
6.4k
Why Our Code Smells
bkeepers
PRO
331
56k
Robots, Beer and Maslow
schacon
PRO
155
7.9k
Done Done
chrislema
178
15k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
39
4.4k
Transcript
@"mminor An Introduction to A/B Testing Photo credit: @lucyjspence
and h7p://www.flickr.com/photos/futureshape/4530140521/
What we’ll be looking at • What is A/B Tes"ng?
• What to test? • What is A/B Tes"ng good for? • A/B tes"ng should not replace • Do’s and Don’ts • Tools
Source: h7p://www.smashingmagazine.com/2010/06/24/the-‐ul"mate-‐guide-‐to-‐a-‐b-‐tes"ng/ What is A/B Testing?
What to test? • Bu7ons: colours, sizes, loca"ons •
Headlines • Product descrip"ons • Form fields (adding and removing) • PPC landing pages • Email content • Pricing and promo"ons • Images
What is A/B Testing good for? • Data-‐driven design
• Ge\ng stakeholders on-‐board • Avoiding personal preferences
A/B testing should not replace: • Talking to users
• Usability tests • Acceptance tests
Do’s and Don’ts • Test simultaneously • Test o^en
• Include only new visitors • Show repeat visitors the same version • Use results to inform future development • Require sta"s"cal significance DO:
• Test too many ideas at once • No
clear idea • Don’t let your preconcep"ons override results • Peek at your results and stop as soon as a significant result has been achieved. See: Evan Miller. Do’s and Don’ts DON’T:
The tools Other tools are available: A/Bingo, Vanity, Unbounce,
Performable, Vertster, SiteSpect, Webtrends Op"mize and Omniture’s Test&Target
The Quiz Show
The Corkscrew A
The Corkscrew B
The Corkscrew A/B
A
Great American Days A
Great American Days B
Great American Days A/B
B
Real Gap
Real Gap A/B
B
CityCliq A
CityCliq B
CityCliq A/B
A
Vendio A
Vendio B
Vendio B
A
Quick Fire Round • What were the two main tools
men"oned? • What percentage increase did the winning RealGap test achieve? • Name one of the three things A/B tes"ng doesn’t replace.
What we looked at • What is A/B Tes"ng?
• What to test? • What is A/B Tes"ng good for? • A/B tes"ng should not replace • Do’s and Don’ts • Tools
References • First slide photograph: h7p://www.flickr.com/photos/futureshape/4530140521/ • www.google.com/websiteop"mizer
• h7p://visualwebsiteop"mizer.com/ • h7p://www.abtests.com/ • h7p://whichtestwon.com/ • h7p://www.smashingmagazine.com/2010/06/24/the-‐ul"mate-‐guide-‐to-‐a-‐b-‐tes"ng/ • h7p://www.conversion-‐rate-‐experts.com/ar"cles/101-‐google-‐website-‐op"mizer-‐"ps/ • h7p://www.evanmiller.org/how-‐not-‐to-‐run-‐an-‐ab-‐test.html • h7p://elem.com/~b"lly/effec"ve-‐ab-‐tes"ng/ • h7p://remoteresear.ch/