Predict visitor frustration with website navigation data

We show a method to analyse visitor navigation data (mouse movements, scrolls and clicks).
We analyse how to build complex metrics form simple mouse and click positions, associate them with frustration and build a model to predict frustration (or engagement) on any website.

Transcript

1. Predict visitor Frustration 
 (and Engagement) using website navigation data Data Beers Milan, 14 November 2018 Antonio Tropiano
 @tropianhs
 atropiano@decibelinsight.com
2. A bit of context 
 Web analytics tools track website visitor’s behaviour in high detail
 Mouse moves, clicks, scrolls… 
 
 
 Aim is to reconstruct the journey for offline analysis 
 
 Session Replay 
 Funnel Analysis …
3. The problem 
 Session replay is very useful to highlight frustration points in the website navigation But can you watch through O(1000) session replays every day?
4. The solution 
 Build an algorithm to automatically detect frustration issues (multiclicks, fast moves, …)
 How? 
 
 Feature engineering Train Test Evaluate
5. Features Mouse movements 
 =
 2D Kinematics
 
 Speed
 Acceleration
 Change of direction
 …
 
 Mouse clicks
 =
 Discrete Events
 
 Number of clicks
 Click rate
 Distance between consecutive clicks
 …
6. Train & Test Challenge
 Huge unlabelled dataset
 
 Strategy
 Visually Explore Features Multi clicks! Regular clicks
7. Train & Test Challenge
 Huge unlabelled dataset
 
 Strategy
 
 Run simple algorithm 
 Select and label some journeys Number of clicks Click distance Click speed Click Rate … Label 4 1.4 4.4 3.1 … 1 5 3.4 8.2 5.5 … 1 55 4.4 8.8 3.9 … 0 43 0.6 2.9 3.3 … 0 0 5.5 6.6 4.8 … 1 7 3.4 5.6 7.7 … 0 … … … … … …
8. Train & Test Reduced an unsupervised problem 
 to a supervised one Number of clicks Click distance Click speed Click Rate … Label 4 1.4 4.4 3.1 … 1 5 3.4 8.2 5.5 … 1 55 4.4 8.8 3.9 … 0 43 0.6 2.9 3.3 … 0 0 5.5 6.6 4.8 … 1 7 3.4 5.6 7.7 … 0 … … … … … …
9. Algorithm Evaluation 
 Classification problem Positive Negative Positive TP FP Negative FN TN Actual Value Predicted 
 Value Which one to optimise? 
 
 Precision? 
 Efficiency?
10. Algorithm Evaluation 
 Classification problem Positive Negative Positive TP FP Negative FN TN Actual Value Predicted 
 Value Which one to optimise? 
 
 Business problem
11. Engagement 
 Many ways to measure 
 Engagement Behaviours (reading, scrolling, text selection…)
 Standard metrics (focus, scroll reach…)
 
 Efficiency is as important as precision
12. Antonio Tropiano
 @tropianhs
 atropiano@decibelinsight.com
14. http://app.decibelinsight.com/Player? id=79cc517a60501d6f41e819418648ce15&propertyId=273&websiteId=2 73&accountId=18&sessionId=414676&api=api.decibelinsight.net