Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How I Learned to Stop Worrying and Love UX Metrics [PerfNow 2018]

How I Learned to Stop Worrying and Love UX Metrics [PerfNow 2018]

This talk at the 2018 performance.now() conference (Amsterdam) walks through a brief history of UX and web performance research, highlighting landmark studies that helped connect the dots between performance and user experience. I also demystify the current state of performance metrics and help you understand what you need to focus on for your site and your users.

Tammy Everts

November 09, 2018
Tweet

More Decks by Tammy Everts

Other Decks in Technology

Transcript

  1. @tameverts
    #PerfNow
    How I learned
    to stop worrying
    and love UX metrics

    View full-size slide

  2. @tameverts
    WPOstats.com

    View full-size slide

  3. What is the best UX metric?
    How fast should it be?
    How can we stay on track?

    View full-size slide

  4. https://www.cnet.com/news/appliance-science-the-well-done-physics-chemistry-of-the-toaster/

    View full-size slide

  5. TTFB DNS TCP Start render
    DOM loading DOM ready Page load Fully loaded
    TTI Resource timing Number of requests Bytes in
    Speed Index PageSpeed Navigation timing DOM elements
    DOM size Visually complete TTFP TTFMP

    View full-size slide

  6. “The real thing we are after is
    to create a user experience that
    people love and they feel is fast…
    and so we might be front-end
    engineers, we might be dev,
    we might be ops,
    but what we really are is
    perception brokers.”
    Steve Souders

    View full-size slide

  7. How do we measure
    perception… at scale?

    View full-size slide

  8. Is it happening?
    Is it useful?
    Is it usable?
    Is it delightful?
    https://developers.google.com/web/fundamentals/
    performance/user-centric-performance-metrics

    View full-size slide

  9. TTFB DNS TCP
    TTI FCP FMP
    FID OMG WTF

    View full-size slide

  10. ❑ Correlates to what users actually see in the browser
    ❑ Is easy to use and accessible right out of the box
    ❑ Recognizes that not all pixels and page elements
    are equal
    ❑ Allows us to customize what we measure on
    specific pages
    The best UX metric…
    #PerfNow
    @tameverts

    View full-size slide

  11. A brief history
    of performance metrics

    View full-size slide

  12. Load Time
    the time from the start of the initial
    navigation until the beginning of the
    window load event

    View full-size slide

  13. BBC loses 10% of users for every additional
    second it takes their site to load
    Ancestry.com saw a 7% conversion rate
    increase after improving load time by 64%
    FT.com reduced desktop load times to
    1.5s and mobile to 2.1s and increased
    user engagement by 30%
    @tameverts #PerfNow

    View full-size slide

  14. Start Render
    the time from the start of the initial
    navigation until the first non-white
    content is painted

    View full-size slide

  15. start render repeat visits

    View full-size slide

  16. @tameverts #PerfNow

    View full-size slide

  17. Speed Index
    average time at which visible parts of
    the page are in the viewport

    View full-size slide

  18. When good metrics go bad…

    View full-size slide

  19. First Paint
    First Contentful Paint
    First Meaningful Paint
    First CPU Idle
    First Interactive

    View full-size slide

  20. First Paint (FP)
    Pixels first start to render

    View full-size slide

  21. First Contentful Paint (FCP)
    Text and graphics start to render…
    BUT often catches non-meaningful
    paints (e.g. headers, nav bars)
    https://tinyurl.com/paint-metrics

    View full-size slide

  22. First Meaningful Paint
    (FMP)
    The paint after which the biggest
    ATF layout change has happened
    and web fonts have loaded
    https://tinyurl.com/paint-metrics

    View full-size slide

  23. https://speedcurve.com/blog/an-analysis-of-chromiums-paint-timing-metrics/

    View full-size slide

  24. “The rendering pipeline is very complex,
    and the timestamp should be the latest
    timestamp the browser is able to note in
    the pipeline (best effort).
    Typically the time at which the frame is
    submitted to the OS for display is
    recommended for this API.”
    https://w3c.github.io/paint-timing/#sec-terminology

    View full-size slide

  25. Analysis of 40 top Alexa-ranked sites
    95% of FP events occur before Start Render
    85% of FCP events occur before Start Render
    50% of FMP events occur before Start Render
    https://speedcurve.com/blog/
    an-analysis-of-chromiums-paint-timing-metrics/

    View full-size slide

  26. ❑ Correlates to what users actually see in the browser
    ❑ Is easy to use and accessible right out of the box
    ❑ Recognizes that not all pixels and page elements are
    equal
    ❑ Allows us to customize what we measure on specific
    pages
    The best UX metric…

    View full-size slide

  27. First CPU Idle
    (formerly known as First Interactive)
    Page is minimally interactive
    Most visible UI elements are interactive
    Responds to user input reasonably
    quickly

    View full-size slide

  28. Time to Interactive (TTI)
    (formerly known as Time to Consistently Interactive)
    Displays useful content
    Event handlers are registered for most visible
    elements
    Page responds to user interaction within 50ms

    View full-size slide

  29. ❑ Correlates to what users actually see in the browser
    ❑ Is easy to use and accessible right out of the box
    ❑ Recognizes that not all pixels and page elements are
    equal
    ❑ Allows us to customize what we measure on specific
    pages
    The best UX metric…

    View full-size slide

  30. Okay… so then what?

    View full-size slide

  31. Hero Rendering Times
    H1
    Largest Image
    Largest Background Image
    First Painted Hero
    Last Painted Hero
    https://speedcurve.com/blog/web-performance-monitoring-hero-times/

    View full-size slide

  32. One-size-fits-most (not all)
    Dependent on how page is built
    Don’t work for image carousels and
    popups
    Hero times gotchas

    View full-size slide

  33. So which metric is best?

    View full-size slide

  34. http://lab.speedcurve.com/rendering/picker.php

    View full-size slide

  35. And the winner is…

    View full-size slide

  36. It depends!
    (You probably saw that coming.)

    View full-size slide

  37. Deliver any content?
    Start Render
    Deliver significant
    amount of content?
    Speed Index, FMP
    Deliver critical content?
    Hero Rendering Times
    https://speedcurve.com/blog/rendering-metrics/

    View full-size slide

  38. ❑ Correlates to what users actually see in the browser
    ❑ Is easy to use and accessible right out of the box
    ❑ Recognizes that not all pixels and page elements are
    equal
    ❑ Allows us to customize what we measure on specific
    pages
    The best UX metric…

    View full-size slide

  39. Custom metrics
    Measure performance with high-precision timestamps
    Available in both synthetic and RUM (yay!)
    https://www.w3.org/TR/user-timing/
    https://speedcurve.com/blog/user-timing-and-custom-metrics/

    View full-size slide

  40. how long does it
    take to display the
    main product image
    on my site?

    View full-size slide

  41. Time to First Tweet
    The time from clicking the link to
    viewing the first tweet on each page’s
    timeline
    Pinner Wait Time (PWT)
    The time from initiating an action
    (e.g., tapping a pin) until the action is
    complete (pin close-up view is loaded)
    Time to Interact (TTI)
    @tameverts #PerfNow

    View full-size slide

  42. @tameverts #PerfNow

    View full-size slide

  43. ❑ Correlates to what users actually see in the browser
    ❑ Is easy to use and accessible right out of the box
    ❑ Recognizes that not all pixels and page elements are
    equal
    ❑ Allows us to customize what we measure on specific
    pages
    The best UX metric…

    View full-size slide

  44. What else can we track?

    View full-size slide

  45. First Click
    First Scroll
    First Key
    First Input Delay
    (only available with real user monitoring)

    View full-size slide

  46. Great!
    We have metrics.
    Now… what to do with them?

    View full-size slide

  47. Performance budgets FTW!

    View full-size slide

  48. Thresholds YOU create for metrics
    that are meaningful for YOUR site
    https://addyosmani.com/blog/performance-budgets/
    Milestone timings (e.g. start render)
    Quantity-based (e.g. image weight)
    Rules-based (e.g. Lighthouse scores)

    View full-size slide

  49. A good performance budget
    should show you…
    What your budget is
    When you go out of bounds
    How long you’re out of bounds
    When you’re back within budget

    View full-size slide

  50. Recommended UX perf budgets*
    Metric How to track it Budget
    Start render Synthetic & RUM 2s
    First Painted Hero Synthetic 2s
    Speed Index Synthetic 4s
    First Interactive Synthetic 4s
    First Input Delay RUM 10ms
    @tameverts *See super important stuff on the next slide

    View full-size slide

  51. Super important!
    Look at your own data
    Monitor your competitors
    No sandbagging allowed
    Take a step-by-step approach if necessary
    Use synthetic and RUM (numbers may will vary)

    View full-size slide

  52. Pro tips
    Create budgets for your popular and regularly
    changing pages
    Review violations early and always
    Compare before and after releases
    Update budgets accordingly
    https://www.zillow.com/engineering/bigger-faster-more-engaging-budget/

    View full-size slide

  53. 1. There is no unicorn metric
    2. Use your eyes
    3. Don’t just look at averages and medians
    4. Validate your metrics by correlating them with
    business and engagement KPIs
    5. Set targets (e.g., performance budgets)
    6. Make yourself accountable (e.g., alerts)
    7. Don’t be afraid to investigate new metrics!
    @tameverts #PerfNow

    View full-size slide

  54. thanks!
    @tameverts
    speedcurve.com/blog

    View full-size slide