Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using Analytics to Improve UX

A91e97b5278c384a1d452297df90e83b?s=47 Paul McMahon
May 01, 2014

Using Analytics to Improve UX


Paul McMahon

May 01, 2014


  1. Using Analytics to Improve UX Paul McMahon @pwim

  2. New Relic • Tells us if our site is up

    • Identifies slow response times
  3. Sentry • Monitors application errors such as crashes • Less

    than 1% of users will report errors • Record user id if possible so you can follow up with the user after
  4. Crazy Egg • Creates a heatmap of where users click

    on your page • Can be useful in identifying things that should be clickable, but aren't
  5. Kiss Metrics • Record arbitrary events like “View Event Page”,

    “Start Registration”, and “Complete Registration” • Set arbitrary properties like “Subscription Plan” on events/users
  6. Funnel Reports 
 (Kiss Metrics) • Track how users move

    through a series of steps towards a goal • Identify where people drop out, which gives you of ideas where you can focus UX effort • Track your changes over time
  7. Measuring Improvements • Using data, we can come up with

    hypothesizes for UX improvements • Let’s imagine we are trying to improve Doorkeeper’s event page Existing Experiment
  8. A/B Testing • Put each visitor into a control or

    experimental group randomly • Show existing version to control group,
  9. The larger your sample size, the smaller the improvement you

    can measure • If we have 2000 people visit our event page, one version needs to outperform the other by 25% to be statistically significant • With 20000 people, it drops to 8% • Hypothesize appropriate sample size before running experiment
  10. Measure conversion rate to the goal • Make sure you

    measure the conversion rate of what is actually your goal. • If you have a multi-step funnel, optimizations can increase the first step, but still lead to a lower overall conversion.
  11. Read More • A/B testing is powerful but easy to

    abuse if you don’t understand the statistics behind it • See Jason Cohen’s Business of Software Conference talk “Why data will deceive you”.