Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
An Introduction to A/B Testing
Search
timminor
October 03, 2011
2
500
An Introduction to A/B Testing
Barcamp talk given at UX Camp Brighton 2011.
timminor
October 03, 2011
Tweet
Share
More Decks by timminor
See All by timminor
UX Quiz, UX Camp Brighton 2014
timminor
0
120
A UX Quiz, v2
timminor
0
160
A UX Quiz
timminor
1
510
Thinking fast, thinking slow: designing for Cognitive ease
timminor
5
1.7k
Featured
See All Featured
Art, The Web, and Tiny UX
lynnandtonic
291
20k
It's Worth the Effort
3n
181
27k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
26
1.8k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
16
1.6k
How to Think Like a Performance Engineer
csswizardry
4
590
For a Future-Friendly Web
brad_frost
173
9.2k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
277
13k
A Philosophy of Restraint
colly
200
16k
Building Your Own Lightsaber
phodgson
101
5.9k
Web Components: a chance to create the future
zenorocha
307
41k
Bootstrapping a Software Product
garrettdimon
PRO
304
110k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
149
45k
Transcript
@"mminor An Introduction to A/B Testing Photo credit: @lucyjspence
and h7p://www.flickr.com/photos/futureshape/4530140521/
What we’ll be looking at • What is A/B Tes"ng?
• What to test? • What is A/B Tes"ng good for? • A/B tes"ng should not replace • Do’s and Don’ts • Tools
Source: h7p://www.smashingmagazine.com/2010/06/24/the-‐ul"mate-‐guide-‐to-‐a-‐b-‐tes"ng/ What is A/B Testing?
What to test? • Bu7ons: colours, sizes, loca"ons •
Headlines • Product descrip"ons • Form fields (adding and removing) • PPC landing pages • Email content • Pricing and promo"ons • Images
What is A/B Testing good for? • Data-‐driven design
• Ge\ng stakeholders on-‐board • Avoiding personal preferences
A/B testing should not replace: • Talking to users
• Usability tests • Acceptance tests
Do’s and Don’ts • Test simultaneously • Test o^en
• Include only new visitors • Show repeat visitors the same version • Use results to inform future development • Require sta"s"cal significance DO:
• Test too many ideas at once • No
clear idea • Don’t let your preconcep"ons override results • Peek at your results and stop as soon as a significant result has been achieved. See: Evan Miller. Do’s and Don’ts DON’T:
The tools Other tools are available: A/Bingo, Vanity, Unbounce,
Performable, Vertster, SiteSpect, Webtrends Op"mize and Omniture’s Test&Target
The Quiz Show
The Corkscrew A
The Corkscrew B
The Corkscrew A/B
A
Great American Days A
Great American Days B
Great American Days A/B
B
Real Gap
Real Gap A/B
B
CityCliq A
CityCliq B
CityCliq A/B
A
Vendio A
Vendio B
Vendio B
A
Quick Fire Round • What were the two main tools
men"oned? • What percentage increase did the winning RealGap test achieve? • Name one of the three things A/B tes"ng doesn’t replace.
What we looked at • What is A/B Tes"ng?
• What to test? • What is A/B Tes"ng good for? • A/B tes"ng should not replace • Do’s and Don’ts • Tools
References • First slide photograph: h7p://www.flickr.com/photos/futureshape/4530140521/ • www.google.com/websiteop"mizer
• h7p://visualwebsiteop"mizer.com/ • h7p://www.abtests.com/ • h7p://whichtestwon.com/ • h7p://www.smashingmagazine.com/2010/06/24/the-‐ul"mate-‐guide-‐to-‐a-‐b-‐tes"ng/ • h7p://www.conversion-‐rate-‐experts.com/ar"cles/101-‐google-‐website-‐op"mizer-‐"ps/ • h7p://www.evanmiller.org/how-‐not-‐to-‐run-‐an-‐ab-‐test.html • h7p://elem.com/~b"lly/effec"ve-‐ab-‐tes"ng/ • h7p://remoteresear.ch/