Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

How to be a Performance Detective - Performance...

How to be a Performance Detective - Performance.now() Conference 2022

When starting on a web performance journey, many companies’ first step is to start monitoring performance metrics over time. But the second step is often much harder - now that you have all of this data, what do you do when you see a regression? How do you sift through the clues to connect a change in a graph to the code that engineers ship to production? How do you become a performance detective?

In this session, Katie will crack open her casefiles to share some real-world examples where performance metrics changed for the worse, and walk through how to find the culprit. You’ll learn what clues to look for to understand a regression in your Core Web Vitals, how different performance metrics influence each other, and when to use data gathered from real users vs. synthetic tests in the lab.

Katie Sylor-Miller

October 31, 2022
Tweet

Other Decks in Programming

Transcript

  1. DETECTIVE DETECTIVE H O W T O B E A

    E P o FRM Nc R e a Katie Sylor-Miller Frontend Architect, Etsy Fairy Gitmother, OhShitGit.com @ksylor Detective Pikachu © 2019 Warner Bros., Pokémon © 2022 Pokémon. © 1995–2022 Nintendo/Creatures Inc./GAME FREAK inc.
  2. PERFORMANCE METRIC TYPES Icons By Andreuvv - Own work, CC

    BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=90073970 @ksylor
  3. PERFORMANCE METRIC TYPES - TIMELINE-BASED VS. INDEPENDENT TIMELINE-BASED METRICS Start

    a timer at 0, log a timeline of events as the page loads Navigation Start TTFB FCP LCP DCL TTI Page Load 0ms 200ms 300ms 400ms 500ms 600ms 700ms @ksylor
  4. PERFORMANCE METRIC TYPES - TIMELINE-BASED VS. INDEPENDENT TIMELINE-BASED METRICS Start

    a timer at 0, log a timeline of events as the page loads Early slowdowns impact later metrics TTFB FCP TTI Page Load 0ms 200ms 300ms 400ms 500ms 600ms 700ms TTFB FCP TTI Page Load 0ms 200ms 300ms 400ms 500ms 600ms 700ms 800m @ksylor LCP DCL LCP DCL Navigation Start Navigation Start
  5. PERFORMANCE METRIC TYPES - TIMELINE-BASED VS. INDEPENDENT INDEPENDENT METRICS Events

    that are measured independent of the loading timeline 0ms 200ms 300ms 400ms 500ms 600ms 700ms 150ms FID / INP / TBT @ksylor
  6. PERFORMANCE METRIC TYPES - TIMELINE-BASED VS. INDEPENDENT INDEPENDENT METRICS Events

    that are measured independent of the loading timeline 150ms FID / INP / TBT 0ms 200ms 300ms 400ms 500ms 600ms 700ms 0.05 Layout Shift 0.125 Layout Shift 0ms 200ms 300ms 400ms 500ms 600ms 700ms 0.175 CLS @ksylor
  7. PERFORMANCE METRIC TYPES - FIELD/REAL USER VS. LAB/SYNTHETIC THE METRIC

    VENN DIAGRAM Field/Real User Lab/Synthetic Icons By Andreuvv - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=90073970 @ksylor FID TBT TTFB TTI Speed Index FCP LCP CLS DCL INP Page Load CPU Usage Start Render Resource Timings User Timings
  8. PERFORMANCE METRIC TYPES - FIELD/REAL USER VS. LAB/SYNTHETIC REAL USER

    MONITORING (RUM) ✦ Collect data from Real User sessions via browser-exposed APIs ✦ Available via: ✦ Third-party monitoring services ✦ Public datasets Chrome User Experience (CrUX), Akamai RumArchive ✦ Open source JS libraries ✦ or Do-It-Yourself @ksylor
  9. PERFORMANCE METRIC TYPES - FIELD/REAL USER VS. LAB/SYNTHETIC REAL USER

    MONITORING (RUM) ✦ Cool stuff: ✦ Captures the full story of your real users' experiences ✦ Interaction-based metrics like FID and INP are only available in RUM @ksylor
  10. PERFORMANCE METRIC TYPES - FIELD/REAL USER VS. LAB/SYNTHETIC REAL USER

    MONITORING (RUM) ✦ Cool stuff: ✦ Captures the full story of your real users' experiences ✦ Interaction-based metrics like FID and INP are only available in RUM ✦ Meh stuff: ✦ Browser support lacking for some metrics ✦ Every data point is a tradeoff ✦ Good for trends, not in-depth analysis @ksylor
  11. PERFORMANCE METRIC TYPES - FIELD/REAL USER VS. LAB/SYNTHETIC RUM POPULATIONS

    ✦ Performance data is variable and follows a log-normal distribution ✦ Cumulative Distribution Function (CDF) ✦ How do you compare a continuum of populations? Industry standard of p75 @ksylor
  12. PERFORMANCE METRIC TYPES - FIELD/REAL USER VS. LAB/SYNTHETIC SYNTHETIC ✦

    Collect data from a page load in controlled conditions ✦ Available via: ✦ Webpagetest.org or Lighthouse ✦ In the browser or programmatically ✦ Continuous monitoring or in-the-moment @ksylor
  13. PERFORMANCE METRIC TYPES - FIELD/REAL USER VS. LAB/SYNTHETIC SYNTHETIC ✦

    Cool stuff: ✦ Apples-to-apples comparison for diagnosis ✦ In-depth metrics not easily available from RUM ✦ Available in pre-prod @ksylor
  14. PERFORMANCE METRIC TYPES - FIELD/REAL USER VS. LAB/SYNTHETIC SYNTHETIC ✦

    Cool stuff: ✦ Apples-to-apples comparison for diagnosis ✦ In-depth metrics not easily available from RUM ✦ Available in pre-prod ✦ Meh stuff: ✦ Not every metric is available, but there are correlated metrics (e.g. TBT or TTI instead of FID or INP) ✦ Doesn't capture interactions @ksylor
  15. PERFORMANCE METRIC TYPES - PRIMARY VS. DIAGNOSTIC PRIMARY METRICS -

    WHAT IS MOST IMPORTANT TO USERS? Icons By Andreuvv - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=90073970 @ksylor LCP Largest Contentful Paint FID First Input Delay CLS Cumulative Layout Shift
  16. PERFORMANCE METRIC TYPES - PRIMARY VS. DIAGNOSTIC PRIMARY METRICS -

    WHAT IS MOST IMPORTANT TO YOUR USERS? LCP Largest Contentful Paint FID First Input Delay CLS Cumulative Layout Shift FCP First Contentful Paint ??? Your Important Metric TTFB Time to First Byte @ksylor Icons By Andreuvv - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=90073970
  17. PERFORMANCE METRIC TYPES - PRIMARY VS. DIAGNOSTIC DIAGNOSTIC METRICS -

    WHAT CAUSED A CHANGE? Navigation Timing Resource Timing LCP/CLS Attribution Synthetic-only Metrics Long Task Timing Custom Timings Icons By Andreuvv - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=90073970 @ksylor
  18. PERFORMANCE METRIC TYPES - PRIMARY VS. DIAGNOSTIC DIAGNOSTIC METRICS -

    NAVIGATION TIMING ✦ E.g. the old school metrics ✦ Before Fetch ✦ Connection Time ✦ Time to First Byte ✦ DOM Content Loaded ✦ Page Load Image via MDN https://developer.mozilla.org/en-US/docs/Learn/Performance/Measuring_performance @ksylor
  19. PERFORMANCE METRIC TYPES - PRIMARY VS. DIAGNOSTIC DIAGNOSTIC METRICS -

    RESOURCE TIMING IN SYNTHETIC ✦ What resources loaded? ✦ In what order? ✦ How big were they? @ksylor
  20. PERFORMANCE METRIC TYPES - PRIMARY VS. DIAGNOSTIC DIAGNOSTIC METRICS -

    RESOURCE TIMING IN RUM Track the count and sizes of all resources: @ksylor ✦ HTML ✦ CSS ✦ JS ✦ Images ✦ Videos ✦ Fonts ✦ XHR requests
  21. PERFORMANCE METRIC TYPES - PRIMARY VS. DIAGNOSTIC DIAGNOSTIC METRICS -

    LCP/CLS ATTRIBUTION IN SYNTHETIC @ksylor ✦ Which elements shifted? ✦ What was the LCP element?
  22. PERFORMANCE METRIC TYPES - PRIMARY VS. DIAGNOSTIC DIAGNOSTIC METRICS -

    LONG TASKS IN SYNTHETIC ✦ How many Long Tasks? ✦ When did they start? @ksylor
  23. PERFORMANCE METRIC TYPES - PRIMARY VS. DIAGNOSTIC DIAGNOSTIC METRICS -

    LONG TASKS IN RUM ✦ Log count and duration ✦ Sum of duration = proxy for TBT ✦ Attribution is per container so useful only for 1st vs. 3rd party tracking but not much else @ksylor
  24. PERFORMANCE METRIC TYPES - PRIMARY VS. DIAGNOSTIC DIAGNOSTIC METRICS -

    CUSTOM TIMINGS ✦ Make your own timings! ✦ Webpagetest will report your custom timings in the UI ✦ Full cross-browser support @ksylor
  25. PERFORMANCE METRIC TYPES - PRIMARY VS. DIAGNOSTIC DIAGNOSTIC METRICS -

    SYNTHETIC-ONLY METRICS ✦ JS Execution details ✦ Total blocking time (TBT) ✦ Time to Interactive (TTI) ✦ CPU usage ✦ Processing breakdown (script parse/eval/render/layout/paint etc.) @ksylor
  26. PERFORMANCE METRIC TYPES - PRIMARY VS. DIAGNOSTIC DIAGNOSTIC METRICS -

    SYNTHETIC-ONLY METRICS ✦ JS Execution details ✦ Total blocking time (TBT) ✦ Time to Interactive (TTI) ✦ CPU usage ✦ Processing breakdown (script parse/eval/render/layout/paint etc.) ✦ Rendering details ✦ Filmstrip screenshots ✦ Start Render ✦ Speed Index ✦ Visually Complete @ksylor
  27. Emcee and tall person Tim Kadlec TRAINER Emcee and running

    enthusiast Henri Helvetica TRAINER HTTPS://REPLICATE.COM/LAMBDAL/TEXT-TO-POKEMON @ksylor
  28. CASE FILES: LARGEST CONTENTFUL PAINT (LCP) BEFORE (OFF) AFTER (ON)

    Aha! The design of the likely LCP element changed! @ksylor
  29. TRAINER WEBPAGETEST TRAINER CARD Use to run synthetic tests on

    a real browser and device Webpagetest.org @ksylor
  30. CASE FILES: LARGEST CONTENTFUL PAINT (LCP) GOTCHA! This is not

    the LCP element you are looking for @ksylor
  31. TAKE 2 The LCP Element did not change CASE FILES:

    LARGEST CONTENTFUL PAINT (LCP) @ksylor
  32. DIAGNOSING CHANGES TO LCP 1. Did the LCP element change?

    NO 2. Did some characteristic of the LCP element change? - If image or video, did the asset size get bigger? - (If text, did the font or text change?) CASE FILES: LARGEST CONTENTFUL PAINT (LCP) @ksylor
  33. BEFORE (OFF) AFTER (ON) CASE FILES: LARGEST CONTENTFUL PAINT (LCP)

    Smaller Image size @ksylor Hint to the real culprit!
  34. DIAGNOSING CHANGES TO LCP 1. Did the LCP element change?

    NO 2. Did some characteristic of the LCP element change? NO - If image or video, did the asset size get bigger? - (If text, did the font or text change?) 3. Did metrics before LCP in the timeline change? - e.g. TTFB, FCP, DCL? CASE FILES: LARGEST CONTENTFUL PAINT (LCP) @ksylor
  35. DIAGNOSING CHANGES TO LCP 1. Did the LCP element change?

    NO 2. Did some characteristic of the LCP element change? NO - If image or video, did the asset size change? - (If text, did the font or text change?) 3. Did metrics before LCP in the timeline change? NO - e.g. TTFB, FCP, DCL? 4. Did the loading order change? CASE FILES: LARGEST CONTENTFUL PAINT (LCP) @ksylor
  36. BEFORE (OFF) AFTER (ON) LCP Element #12 LCP Element #13

    A wild CSS file appears! CASE FILES: LARGEST CONTENTFUL PAINT (LCP) @ksylor
  37. DIAGNOSING CHANGES TO LCP 1. Did the LCP element change?

    NO 2. Did some characteristic of the LCP element change? NO - If image or video, did the asset size change? - (If text, did the font or text change?) 3. Did metrics before LCP in the timeline change? NO - e.g. TTFB, FCP, DCL? 4. Did the loading order change? YES! 5. Did render-blocking assets grow in size or number? CASE FILES: LARGEST CONTENTFUL PAINT (LCP) @ksylor
  38. DIAGNOSING CHANGES TO LCP 1. Did the LCP element change?

    NO 2. Did some characteristic of the LCP element change? NO - If image or video, did the asset size change? - (If text, did the font or text change?) 3. Did metrics before LCP in the timeline change? NO - e.g. TTFB, FCP, DCL? 4. Did the loading order change? YES! 5. Did render-blocking assets grow in size or number? YES! CASE FILES: LARGEST CONTENTFUL PAINT (LCP) @ksylor
  39. BEFORE (OFF) AFTER (ON) Aha! A new module above the

    fold CASE FILES: CUMULATIVE LAYOUT SHIFT (CLS) @ksylor
  40. DIAGNOSING CLS 1. What elements are shifting? A. Font loading

    FOUT/FOIT? B. Images without dimensions? C. Lazy-loaded content without a correct height placeholder? D. Injected banners, ads or iframes? CASE FILES: CUMULATIVE LAYOUT SHIFT (CLS) @ksylor
  41. ZOOMING IN ON AFTER (ON) CASE FILES: CUMULATIVE LAYOUT SHIFT

    (CLS) Everything under the new module @ksylor
  42. DIAGNOSING CLS 1. What elements are shifting? A. Font loading

    FOUT/FOIT? NO B. Images without dimensions? NO C. Lazy-loaded content without a correct height placeholder? YES! D. Injected banners, ads or iframes? NO CASE FILES: CUMULATIVE LAYOUT SHIFT (CLS) @ksylor
  43. DIAGNOSING CHANGES TO FID 1. Is there more Javascript execution,

    e.g. Long Tasks? CASE FILES: FIRST INPUT DELAY (FID) @ksylor
  44. DIAGNOSING CHANGES TO FID 1. Is there more Javascript execution,

    e.g. Long Tasks? YES! 2. Is there more Javascript downloading? - Check both first- and third-party CASE FILES: FIRST INPUT DELAY (FID) @ksylor
  45. DIAGNOSING CHANGES TO FID 1. Is there more Javascript execution,

    e.g. Long Tasks? YES! 2. Is there more Javascript downloading? NO? I think? - Check both first- and third-party 3. Did TBT or TTI get worse? CASE FILES: FIRST INPUT DELAY (FID) @ksylor
  46. TBT OR TTI? ✦ ~30 Synthetic tests were all inconclusive

    :( CASE FILES: FIRST INPUT DELAY (FID) @ksylor
  47. DIAGNOSING CHANGES TO FID 1. Is there more Javascript execution,

    e.g. Long Tasks? YES! 2. Is there more Javascript downloading? NO? I think? - Check both first- and third-party 3. Did TBT or TTI get worse? NO 4. Did user behavior change? IDK CASE FILES: FIRST INPUT DELAY (FID) @ksylor
  48. DIAGNOSING CHANGES TO FID 1. Is there more Javascript execution,

    e.g. Long Tasks? YES! 2. Is there more Javascript downloading? NO? I think? - Check both first- and third-party 3. Did TBT or TTI get worse? NO 4. Did user behavior change? IDK 5. Can we get FID attribution please Yoav FFS CASE FILES: FIRST INPUT DELAY (FID) @ksylor