$30 off During Our Annual Pro Sale. View Details »

UX & Performance: Metrics that Matter

Philip Tellis
October 09, 2018

UX & Performance: Metrics that Matter

In this talk, we’ll look at some of the new user experience metrics that the boomerang team has been collecting. We’ll find out how to measure page responsiveness, smoothness, jank, and usability. We’ll learn about things like Rage Clicks, Missed Clicks, and Dead Clicks. We’ll also look at real user data that we’ve collected showing how these aspects of the page affect user behaviour.

JavaScript provides us with many hooks into measuring performance and user experience. Let’s learn to collect them and understand what to expect when we optimise for them.

Philip Tellis

October 09, 2018
Tweet

More Decks by Philip Tellis

Other Decks in Technology

Transcript

  1. UX & Performance
    Metrics that Matter

    View Slide

  2. Philip Tellis
    Principal RUM Distiller @ Akamai
    Author of the OpenSource boomerang RUM
    library
    twitter:@bluesmoon * github:@bluesmoon

    View Slide

  3. The Art & Science of Real User Measurement

    View Slide

  4. What should we
    measure?

    View Slide

  5. How should we
    measure it?

    View Slide

  6. A user’s emotional response to latency and tolerance of
    errors appears to correlate with their emotional state.
    Measuring Emotion...
    https://speakerdeck.com/bluesmoon/sciencing-data

    View Slide

  7. Methods of measuring Emotion
    ● Ask the user
    ● Bounce / Conversion Rate / LD
    50
    ● Rage Clicks / Cursor Thrashing
    ● Facial analysis
    ● Wireless Brain Interface

    View Slide

  8. Rage Clicks occur when users rapid-fire click
    (or tap) on your site or app.
    Rage clicking is the digital equivalent of
    cursing to release frustration.
    https://www.fullstory.com/resources/guide-to-understanding-frustrating-user-experiences-online/
    https://www.psychologytoday.com/us/blog/hide-and-seek/201205/hell-yes-the-7-best-reasons-swearing
    Rage Clicks

    View Slide

  9. People who are angry are more likely to use the mouse
    in a jerky and sudden, but surprisingly slow fashion.
    People who feel frustrated, confused or sad are less
    precise in their mouse movements and move it at
    different speeds.
    https://www.telegraph.co.uk/technology/news/12050481/Websites-could-read-emotions-by-seeing-how-fast-you-move-your-mouse.html
    Cursor Thrashing

    View Slide

  10. https://webgazer.cs.brown.edu/
    Facial Analysis Mind Reading
    https://www.emotiv.com/
    We do NOT do either of these

    View Slide

  11. Ask the User
    We do NOT do this either
    ● Selection Bias
    ● Hawthorne Effect
    ● Intrusive action (boomerang is passive)

    View Slide

  12. Rage Clicks are measured in JavaScript by looking for more than 2 clicks
    within the same 10 pixels in a short period of time.
    onMouse*, event.timestamp, and an array
    The code is in boomerang’s Continuity Plugin.
    Measuring Rage Clicks
    https://developer.akamai.com/tools/boomerang/docs/BOOMR.plugins.Continuity.html

    View Slide

  13. Rage Click likelihood depends on Latency of Page Usability
    ● Interacting with a page before onload OR interactive is
    the most common cause of Rage Clicks
    ● Some Rage Clicks happen after the page is usable,
    possibly due to JavaScript errors or CPU intensive
    tasks; they could also be false positives.
    ● In over 30% of cases, a page is interactive after onload
    fires, and in 15% of cases, users try interacting between
    onload and interactive.
    Data collected and analysed with boomerang and Akamai mPulse

    View Slide

  14. Data collected and analysed with boomerang and Akamai mPulse
    Rage Clicking is fairly consistent if first Interaction is before the page
    becomes Interactive
    Steady drop off if first Interaction is after the page becomes
    Interactive
    Rage Clicks are most likely if first Interaction is just before onload,
    possibly because DOM Ready event handlers are hogging CPU.
    (This is true even if the page becomes Interactive before onload)

    View Slide

  15. Data collected and analysed with boomerang and Akamai mPulse

    View Slide

  16. ● We found that the majority of users who Rage Click
    attempt to interact with a page between 1.25 and 1.5x
    the visually ready time.
    ● We don’t know why, but users expect a page to be
    interactive at 1.3x the time when it is visual.
    ● So, make sure your page is interactive and loaded
    before 1.3x the visually ready time.
    What is the optimum time to reduce Rage Clicking?
    Data collected and analysed with boomerang and Akamai mPulse
    Ratio of First Interaction to Visually Ready
    Rage Clicks

    View Slide

  17. ● There is a clear linear correlation
    ● The slope varies between 1.2 and 1.5
    Rage Clicks by First Interaction & Visually Ready
    Data collected and analysed with boomerang and Akamai mPulse

    View Slide

  18. Cursor Thrashing is measured in JavaScript by looking at mouse movements
    over time, both in absolute terms and as a fraction of screen size.
    onMouseMove, setTimeout, event.clientX, event.clientY, ...
    The code is in boomerang’s Continuity Plugin.
    Measuring Cursor Thrashing
    https://developer.akamai.com/tools/boomerang/docs/BOOMR.plugins.Continuity.html

    View Slide

  19. Rage Clicks are
    ● A user’s response to lack of
    responsiveness on your site.
    ● Indicative of design or performance
    issues with your site.
    Key differences between Rage Clicks & Cursor Thrashing...
    Cursor Thrashing is
    ● Indicative of the user’s emotional state
    before they got to your site
    ● A signal you can use to change how
    you present your site.

    View Slide

  20. Definitions of our
    Metrics that Matter
    https://developer.akamai.com/tools/boomerang/docs/BOOMR.plugins.Continuity.html

    View Slide

  21. Timers

    Visually Ready: When did the user feel like they could interact with the site?

    Interactive: After the page was Visually Ready, when was the first time the user could
    have interacted with the site, and had a good experience?

    First Interaction: When was the first time the user tried to interact with the site?

    First Input Delay: For the first interaction on the page, how responsive was it?

    View Slide


  22. Delayed Interactions: How often was the user's interaction delayed more than 50ms?

    Rage Clicks: Did the user repeatedly click on the same element/region?

    Mouse Movements: How did the user move the mouse around the screen?
    Interaction Metrics

    View Slide

  23. Page Performance Metrics

    Frame Rate data: FPS during page load, minimum FPS, number of long frames

    Long Task data: Number of Long Tasks, how much time they took, attribution to what
    caused them

    Page Busy: Measurement of the page's “busyness”

    View Slide

  24. How do we measure what matters?

    Use boomerang!

    Read the docs of boomerang’s Continuity Plugin

    Watch the talk by Nic Jansma and Philip Tellis at Velocity
    . . .

    View Slide

  25. ● First Paint, First Contentful Paint, DOMContentLoaded
    ● Hero Images (eg: class="hero-image")
    ● Framework Ready Event
    Pick the last event from the above, that is when the user thinks the page is visually ready
    Visually Ready

    View Slide

  26. ● Measure Frame Rate using requestAnimationFrame.
    ● Measure Long Tasks using PerformanceObserver (if available).
    ● Measure Page Busy using setInterval (if Long Tasks not available).
    ● Measure Interaction Latency
    First period of 500ms after Visually Ready with No Long Tasks, FPS > 20, and Page Busy < 10%
    Time to Interactive

    View Slide

  27. ● Instrument Mouse, Keyboard, and Scroll Events in passive mode
    ● Measure latency between event.timestamp, and event handler being called
    performance.now().
    Delayed Interactions

    View Slide

  28. ● Use requestAnimationFrame
    ● Measure time between consecutive calls to rAF
    ● 16ms for the best experience
    This measurement is expensive on the CPU and cause excessive battery usage when the page is
    idle, so best to use it only during the page load process.
    Frame Rate

    View Slide

  29. var observer = new PerformanceObserver(function(list) {
    var perfEntries = list.getEntries();
    for (var i = 0; i < perfEntries.length; i++) {
    // Process long task notifications:
    }
    });
    // register observer for long task notifications
    observer.observe({entryTypes: ["longtask"]});
    Long Tasks: Use PerformanceObserver

    View Slide

  30. Summary
    ● Measure what affects your user’s
    emotional state.
    ● Reduce the amount of work you do in
    onload and readyState handlers.
    ● Measure Responsiveness and
    Smoothness of your UI.
    Pacific Islander Navigation Map, Museum of Fine Arts, Boston
    https://www.flickr.com/photos/bluesmoon/1266590108/

    View Slide

  31. Thank You
    Check out boomerang on github for much more detail

    View Slide

  32. References
    ● Boomerang’s Continuity Plugin
    https://developer.akamai.com/tools/boomerang/docs/BOOMR.plugins.Continuity.html
    ● Fullstory’s guide to User Frustration
    https://www.fullstory.com/resources/guide-to-understanding-frustrating-user-experiences-online/
    ● Reading emotion through measuring mouse movements
    https://www.telegraph.co.uk/technology/news/12050481/Websites-could-read-emotions-by-seeing-how...
    ● Inferring Emotion through HCI Devices
    https://www.researchgate.net/publication/283422711_How_Is_Your_User_Feeling_Inferring_Emotion...
    ● Correlation between emotional state and browsing behaviour
    https://speakerdeck.com/bluesmoon/sciencing-data
    ● Scroll behaviour across the web
    http://blog.chartbeat.com/2013/08/12/scroll-behavior-across-the-web/
    ● Debouncing & Throttling Events
    https://css-tricks.com/debouncing-throttling-explained-examples/
    ● Measuring Web Page Jank
    https://webperf.ninja/2015/jank-meter/
    ● Scroll Performance with Passive Event Listeners
    https://developers.google.com/web/updates/2016/06/passive-event-listeners
    ● What Every Browser knows about You
    http://webkay.robinlinus.com/
    ● An Audit of Boomerang’s Performance
    https://nicj.net/an-audit-of-boomerangs-performance/
    ● The Long Tasks API
    https://w3c.github.io/longtasks/
    ● The Paint Timing API
    https://www.w3.org/TR/paint-timing/
    ● Resource Timing Level 2
    https://www.w3.org/TR/resource-timing-2/
    ● Intersection Observer API
    https://developer.mozilla.org/en-US/docs/Web/API/Intersection_Observer_API
    ● requestIdleCallback
    https://developers.google.com/web/updates/2015/08/using-requestidlecallback
    ● requestAnimationFrame
    https://developer.mozilla.org/en-US/docs/Web/API/window/requestAnimationFrame
    ● Last Painted Hero
    https://speedcurve.com/blog/last-painted-hero/
    ● The Runtime Performance Checklist
    https://calendar.perfplanet.com/2013/the-runtime-performance-checklist/

    View Slide