Using Analytics to Improve UX

Dd24adb5a3a430fed83a33ed552fe1b5?s=47 Paul McMahon
May 01, 2014
3.3k

Using Analytics to Improve UX

Dd24adb5a3a430fed83a33ed552fe1b5?s=128

Paul McMahon

May 01, 2014
Tweet

Transcript

  1. 2.

    New Relic • Tells us if our site is up

    • Identifies slow response times
  2. 3.

    Sentry • Monitors application errors such as crashes • Less

    than 1% of users will report errors • Record user id if possible so you can follow up with the user after
  3. 4.

    Crazy Egg • Creates a heatmap of where users click

    on your page • Can be useful in identifying things that should be clickable, but aren't
  4. 5.

    Kiss Metrics • Record arbitrary events like “View Event Page”,

    “Start Registration”, and “Complete Registration” • Set arbitrary properties like “Subscription Plan” on events/users
  5. 6.

    Funnel Reports 
 (Kiss Metrics) • Track how users move

    through a series of steps towards a goal • Identify where people drop out, which gives you of ideas where you can focus UX effort • Track your changes over time
  6. 7.

    Measuring Improvements • Using data, we can come up with

    hypothesizes for UX improvements • Let’s imagine we are trying to improve Doorkeeper’s event page Existing Experiment
  7. 8.

    A/B Testing • Put each visitor into a control or

    experimental group randomly • Show existing version to control group,
  8. 9.

    The larger your sample size, the smaller the improvement you

    can measure • If we have 2000 people visit our event page, one version needs to outperform the other by 25% to be statistically significant • With 20000 people, it drops to 8% • Hypothesize appropriate sample size before running experiment
  9. 10.

    Measure conversion rate to the goal • Make sure you

    measure the conversion rate of what is actually your goal. • If you have a multi-step funnel, optimizations can increase the first step, but still lead to a lower overall conversion.
  10. 11.

    Read More • A/B testing is powerful but easy to

    abuse if you don’t understand the statistics behind it • See Jason Cohen’s Business of Software Conference talk “Why data will deceive you”.