Using Analytics to Improve UX

Transcript

1. Using Analytics to Improve UX Paul McMahon @pwim
2. New Relic • Tells us if our site is up • Identifies slow response times
3. Sentry • Monitors application errors such as crashes • Less than 1% of users will report errors • Record user id if possible so you can follow up with the user after
4. Crazy Egg • Creates a heatmap of where users click on your page • Can be useful in identifying things that should be clickable, but aren't
5. Kiss Metrics • Record arbitrary events like “View Event Page”, “Start Registration”, and “Complete Registration” • Set arbitrary properties like “Subscription Plan” on events/users
6. Funnel Reports 
 (Kiss Metrics) • Track how users move through a series of steps towards a goal • Identify where people drop out, which gives you of ideas where you can focus UX effort • Track your changes over time
7. Measuring Improvements • Using data, we can come up with hypothesizes for UX improvements • Let’s imagine we are trying to improve Doorkeeper’s event page Existing Experiment
8. A/B Testing • Put each visitor into a control or experimental group randomly • Show existing version to control group,
9. The larger your sample size, the smaller the improvement you can measure • If we have 2000 people visit our event page, one version needs to outperform the other by 25% to be statistically significant • With 20000 people, it drops to 8% • Hypothesize appropriate sample size before running experiment
10. Measure conversion rate to the goal • Make sure you measure the conversion rate of what is actually your goal. • If you have a multi-step funnel, optimizations can increase the first step, but still lead to a lower overall conversion.
11. Read More • A/B testing is powerful but easy to abuse if you don’t understand the statistics behind it • See Jason Cohen’s Business of Software Conference talk “Why data will deceive you”.