Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A/B Testing to Create Happy Users

Lara Hogan
September 23, 2012

A/B Testing to Create Happy Users

We barely know our users. We know what technology they use, where they live, what they look at on our site - but we don't know their personality or decision-making process. However, A/B testing can give you insight into what works best for your audience on your site. It can make your stakeholders happy, it'll make your users happy, and it'll definitely make you happy.

Lara Hogan

September 23, 2012
Tweet

More Decks by Lara Hogan

Other Decks in Design

Transcript

  1. @laraswanson Test screams (not whispers) call to action text form

    length subject lines 41 shades of blue landing page design
  2. @laraswanson Test screams (not whispers) call to action text form

    length subject lines 41 shades of blue landing page design
  3. @laraswanson http://www.whichmvt.com/ A/B testing technology initial implementation di ficulty ease

    of editing by non-techies page load time integration with analytics
  4. @laraswanson http://www.whichmvt.com/visual-website-optimizer/ Example: Visual Website Optimizer WYSIWYG view of tests

    integration with GA calculates significance page load time-optimized can do multiple tests on one page
  5. @laraswanson http://www.whichmvt.com/visual-website-optimizer/ Example: Visual Website Optimizer WYSIWYG view of tests

    integration with GA calculates significance page load time-optimized can do multiple tests on one page pricing starts at $49/month
  6. @laraswanson Decoder ring: Online donations are down, someone complained about

    the online experience, I'm lost when it comes to the web so I need your help.
  7. @laraswanson You: I'm ready to help! We can actually test

    a few di ferent versions of the form to see which gets more donations. Decoder ring: Online donations are down, someone complained about the online experience, I'm lost when it comes to the web so I need your help.
  8. @laraswanson You: By running a test we can statistically determine

    which version of the form is better. Decoder ring: I don't understand why we'd wait any longer.
  9. @laraswanson You: It avoids putting up a fix now that

    may actually make things worse, because we won't have any data about how users respond to it. Decoder ring: I don't understand why we'd wait any longer.
  10. @laraswanson You: Half of visitors will see a new version

    that I think will make our users happier and more likely to donate. Decoder ring: This is sounding very technical. Blurgh. JUST FIX IT!
  11. @laraswanson You: The other half will see the current form.

    I'll measure donations from each version so we can see which performs better. Decoder ring: This is sounding very technical. Blurgh. JUST FIX IT!
  12. @laraswanson You: You bet. And I can continue to run

    tests on it to improve it more and more over time. Decoder ring: I just want to hear that you're going to fix this.
  13. @laraswanson You: We can make the form better than it's

    ever been and because we'll have data, we'll be confident in its results. Decoder ring: I just want to hear that you're going to fix this.
  14. @laraswanson Alternatives to A/B testing Multivariate testing: http://bit.ly/smashing-mvt Multi-armed bandit:

    http://bit.ly/multi-armed-bandit Tons of tools: http://whichtestwon.com/techguide