Pro Yearly is on sale from $80 to $50! »

UX & Performance: Metrics that Matter

3de01a0c1a9c0e55efc6ecfa72b4eab3?s=47 Philip Tellis
October 09, 2018

UX & Performance: Metrics that Matter

In this talk, we’ll look at some of the new user experience metrics that the boomerang team has been collecting. We’ll find out how to measure page responsiveness, smoothness, jank, and usability. We’ll learn about things like Rage Clicks, Missed Clicks, and Dead Clicks. We’ll also look at real user data that we’ve collected showing how these aspects of the page affect user behaviour.

JavaScript provides us with many hooks into measuring performance and user experience. Let’s learn to collect them and understand what to expect when we optimise for them.

3de01a0c1a9c0e55efc6ecfa72b4eab3?s=128

Philip Tellis

October 09, 2018
Tweet

Transcript

  1. UX & Performance Metrics that Matter

  2. Philip Tellis Principal RUM Distiller @ Akamai Author of the

    OpenSource boomerang RUM library twitter:@bluesmoon * github:@bluesmoon
  3. The Art & Science of Real User Measurement

  4. What should we measure?

  5. How should we measure it?

  6. A user’s emotional response to latency and tolerance of errors

    appears to correlate with their emotional state. Measuring Emotion... https://speakerdeck.com/bluesmoon/sciencing-data
  7. Methods of measuring Emotion • Ask the user • Bounce

    / Conversion Rate / LD 50 • Rage Clicks / Cursor Thrashing • Facial analysis • Wireless Brain Interface
  8. Rage Clicks occur when users rapid-fire click (or tap) on

    your site or app. Rage clicking is the digital equivalent of cursing to release frustration. https://www.fullstory.com/resources/guide-to-understanding-frustrating-user-experiences-online/ https://www.psychologytoday.com/us/blog/hide-and-seek/201205/hell-yes-the-7-best-reasons-swearing Rage Clicks
  9. People who are angry are more likely to use the

    mouse in a jerky and sudden, but surprisingly slow fashion. People who feel frustrated, confused or sad are less precise in their mouse movements and move it at different speeds. https://www.telegraph.co.uk/technology/news/12050481/Websites-could-read-emotions-by-seeing-how-fast-you-move-your-mouse.html Cursor Thrashing
  10. https://webgazer.cs.brown.edu/ Facial Analysis Mind Reading https://www.emotiv.com/ We do NOT do

    either of these
  11. Ask the User We do NOT do this either •

    Selection Bias • Hawthorne Effect • Intrusive action (boomerang is passive)
  12. Rage Clicks are measured in JavaScript by looking for more

    than 2 clicks within the same 10 pixels in a short period of time. onMouse*, event.timestamp, and an array The code is in boomerang’s Continuity Plugin. Measuring Rage Clicks https://developer.akamai.com/tools/boomerang/docs/BOOMR.plugins.Continuity.html
  13. Rage Click likelihood depends on Latency of Page Usability •

    Interacting with a page before onload OR interactive is the most common cause of Rage Clicks • Some Rage Clicks happen after the page is usable, possibly due to JavaScript errors or CPU intensive tasks; they could also be false positives. • In over 30% of cases, a page is interactive after onload fires, and in 15% of cases, users try interacting between onload and interactive. Data collected and analysed with boomerang and Akamai mPulse
  14. Data collected and analysed with boomerang and Akamai mPulse Rage

    Clicking is fairly consistent if first Interaction is before the page becomes Interactive Steady drop off if first Interaction is after the page becomes Interactive Rage Clicks are most likely if first Interaction is just before onload, possibly because DOM Ready event handlers are hogging CPU. (This is true even if the page becomes Interactive before onload)
  15. Data collected and analysed with boomerang and Akamai mPulse

  16. • We found that the majority of users who Rage

    Click attempt to interact with a page between 1.25 and 1.5x the visually ready time. • We don’t know why, but users expect a page to be interactive at 1.3x the time when it is visual. • So, make sure your page is interactive and loaded before 1.3x the visually ready time. What is the optimum time to reduce Rage Clicking? Data collected and analysed with boomerang and Akamai mPulse Ratio of First Interaction to Visually Ready Rage Clicks
  17. • There is a clear linear correlation • The slope

    varies between 1.2 and 1.5 Rage Clicks by First Interaction & Visually Ready Data collected and analysed with boomerang and Akamai mPulse
  18. Cursor Thrashing is measured in JavaScript by looking at mouse

    movements over time, both in absolute terms and as a fraction of screen size. onMouseMove, setTimeout, event.clientX, event.clientY, ... The code is in boomerang’s Continuity Plugin. Measuring Cursor Thrashing https://developer.akamai.com/tools/boomerang/docs/BOOMR.plugins.Continuity.html
  19. Rage Clicks are • A user’s response to lack of

    responsiveness on your site. • Indicative of design or performance issues with your site. Key differences between Rage Clicks & Cursor Thrashing... Cursor Thrashing is • Indicative of the user’s emotional state before they got to your site • A signal you can use to change how you present your site.
  20. Definitions of our Metrics that Matter https://developer.akamai.com/tools/boomerang/docs/BOOMR.plugins.Continuity.html

  21. Timers • Visually Ready: When did the user feel like

    they could interact with the site? • Interactive: After the page was Visually Ready, when was the first time the user could have interacted with the site, and had a good experience? • First Interaction: When was the first time the user tried to interact with the site? • First Input Delay: For the first interaction on the page, how responsive was it?
  22. • Delayed Interactions: How often was the user's interaction delayed

    more than 50ms? • Rage Clicks: Did the user repeatedly click on the same element/region? • Mouse Movements: How did the user move the mouse around the screen? Interaction Metrics
  23. Page Performance Metrics • Frame Rate data: FPS during page

    load, minimum FPS, number of long frames • Long Task data: Number of Long Tasks, how much time they took, attribution to what caused them • Page Busy: Measurement of the page's “busyness”
  24. How do we measure what matters? • Use boomerang! •

    Read the docs of boomerang’s Continuity Plugin • Watch the talk by Nic Jansma and Philip Tellis at Velocity . . .
  25. • First Paint, First Contentful Paint, DOMContentLoaded • Hero Images

    (eg: class="hero-image") • Framework Ready Event Pick the last event from the above, that is when the user thinks the page is visually ready Visually Ready
  26. • Measure Frame Rate using requestAnimationFrame. • Measure Long Tasks

    using PerformanceObserver (if available). • Measure Page Busy using setInterval (if Long Tasks not available). • Measure Interaction Latency First period of 500ms after Visually Ready with No Long Tasks, FPS > 20, and Page Busy < 10% Time to Interactive
  27. • Instrument Mouse, Keyboard, and Scroll Events in passive mode

    • Measure latency between event.timestamp, and event handler being called performance.now(). Delayed Interactions
  28. • Use requestAnimationFrame • Measure time between consecutive calls to

    rAF • 16ms for the best experience This measurement is expensive on the CPU and cause excessive battery usage when the page is idle, so best to use it only during the page load process. Frame Rate
  29. var observer = new PerformanceObserver(function(list) { var perfEntries = list.getEntries();

    for (var i = 0; i < perfEntries.length; i++) { // Process long task notifications: } }); // register observer for long task notifications observer.observe({entryTypes: ["longtask"]}); Long Tasks: Use PerformanceObserver
  30. Summary • Measure what affects your user’s emotional state. •

    Reduce the amount of work you do in onload and readyState handlers. • Measure Responsiveness and Smoothness of your UI. Pacific Islander Navigation Map, Museum of Fine Arts, Boston https://www.flickr.com/photos/bluesmoon/1266590108/
  31. Thank You Check out boomerang on github for much more

    detail
  32. References • Boomerang’s Continuity Plugin https://developer.akamai.com/tools/boomerang/docs/BOOMR.plugins.Continuity.html • Fullstory’s guide to

    User Frustration https://www.fullstory.com/resources/guide-to-understanding-frustrating-user-experiences-online/ • Reading emotion through measuring mouse movements https://www.telegraph.co.uk/technology/news/12050481/Websites-could-read-emotions-by-seeing-how... • Inferring Emotion through HCI Devices https://www.researchgate.net/publication/283422711_How_Is_Your_User_Feeling_Inferring_Emotion... • Correlation between emotional state and browsing behaviour https://speakerdeck.com/bluesmoon/sciencing-data • Scroll behaviour across the web http://blog.chartbeat.com/2013/08/12/scroll-behavior-across-the-web/ • Debouncing & Throttling Events https://css-tricks.com/debouncing-throttling-explained-examples/ • Measuring Web Page Jank https://webperf.ninja/2015/jank-meter/ • Scroll Performance with Passive Event Listeners https://developers.google.com/web/updates/2016/06/passive-event-listeners • What Every Browser knows about You http://webkay.robinlinus.com/ • An Audit of Boomerang’s Performance https://nicj.net/an-audit-of-boomerangs-performance/ • The Long Tasks API https://w3c.github.io/longtasks/ • The Paint Timing API https://www.w3.org/TR/paint-timing/ • Resource Timing Level 2 https://www.w3.org/TR/resource-timing-2/ • Intersection Observer API https://developer.mozilla.org/en-US/docs/Web/API/Intersection_Observer_API • requestIdleCallback https://developers.google.com/web/updates/2015/08/using-requestidlecallback • requestAnimationFrame https://developer.mozilla.org/en-US/docs/Web/API/window/requestAnimationFrame • Last Painted Hero https://speedcurve.com/blog/last-painted-hero/ • The Runtime Performance Checklist https://calendar.perfplanet.com/2013/the-runtime-performance-checklist/