Real-World Performance Budgets [PerfNow 2022]

Slide 1

Slide 1 text

Real-World Performance Budgets @tameverts #perfnow

Slide 2

Slide 2 text

@tameverts

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

@tameverts

Slide 5

Slide 5 text

speedcurve.com/benchmarks/ @tameverts

Slide 6

Slide 6 text

How fast should my pages be to keep my visitors happy? How can I make sure they stay fast?

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

2009 Improved average load time from 6s à 1.2s 7-12% increase in conversion rate + 25% increase in PVs Average load time degraded to 5s User feedback: “I will not come back to this site again.” Re-focused on performance 0.4% increase in conversion rate 2010 2011 2009 @tameverts

Slide 9

Slide 9 text

1. Constant feature development 2. Badly implemented third-parties 3. Waited too long to tackle problems 4. Relied on performance sprints 5. Stopped doing front-end performance measurement 6. No way to track regressions @tameverts

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

What is a performance budget? Which metrics should I focus on? What should my budget thresholds be? How can I stay on top of my budgets? @tameverts

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

MVB

Slide 14

Slide 14 text

MVB Minimum Viable Budget

Slide 15

Slide 15 text

What is a performance budget?

Slide 16

Slide 16 text

Thresholds YOU create for metrics that are meaningful for YOUR site Milestone timings • e.g. Start Render, Largest Contentful Paint Quantity-based • e.g. Page size, image weight, Long Tasks Score-based • e.g. Cumulative Layout Shift, Lighthouse scores @tameverts

Slide 17

Slide 17 text

A good performance budget should show you… q What your budget is q When you go out of bounds q How long you’re out of bounds q When you’re back within budget @tameverts

Slide 18

Slide 18 text

budget exceeded fixed @tameverts

Slide 19

Slide 19 text

Performance budget tools Synthetic (lab) Mimics defined network & browser conditions No installation required Limited URLs Limited test locations Compare any sites Detailed analysis & visuals Real user monitoring (field) Real network & browser conditions Requires JavaScript installation Large sample size (up to 100%) Geographic spread Only measure your own site Correlation with other metrics (e.g., bounce rate) @tameverts

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

Which metrics should I focus on?

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

TTFB DNS FCP FMP LCP CLS FID TBT TTI INP OMG WTF

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

50+ synthetic metrics 25 additional metrics for EACH third party 30+ RUM metrics ??? custom metrics

Slide 26

Slide 26 text

Photo by Brett Cagle on Unsplash

Slide 27

Slide 27 text

Is the page loading? Can I use it? How does it feel?

Slide 28

Slide 28 text

❑ Tracks the most important content ❑ Is accessible out of the box ❑ Is available in synthetic and RUM ❑ Can be correlated to UX and business metrics ❑ Has broad browser support The ideal UX metric… @tameverts

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

Is the page loading?

Slide 32

Slide 32 text

Backend time AKA Time to First Byte (TTFB) Synthetic & RUM

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

❑ Tracks the most important content ❑ Is accessible out of the box ❑ Has broad browser support ❑ Is available in synthetic and RUM ❑ Can be correlated to UX and business metrics √ √ √ Backend time √

Slide 35

Slide 35 text

Start Render The time from the start of navigation until the first non-white content is painted Synthetic & RUM

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

As Start Render improves, so does bounce rate (typically) @tameverts

Slide 38

Slide 38 text

So does conversion rate (again, typically) @tameverts

Slide 39

Slide 39 text

❑ Tracks the most important content ❑ Is accessible out of the box ❑ Has broad browser support ❑ Is available in synthetic and RUM ❑ Can be correlated to UX and business metrics √ √ √ Start Render √ √ @tameverts

Slide 40

Slide 40 text

Can I use the page?

Slide 41

Slide 41 text

41 When do users start interacting with a page? @tameverts

Slide 42

Slide 42 text

Largest Contentful Paint (LCP) Amount of time it takes for the largest visual element (image or video) to render Synthetic & RUM

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

45 LCP can be tricky on mobile

Slide 46

Slide 46 text

46 LCP can vary for first-view vs repeat visitors speedcurve.com/blog/element-timing-one-true-metric

Slide 47

Slide 47 text

47 Largest LCP element can vary by viewport size speedcurve.com/blog/element-timing-one-true-metric

Slide 48

Slide 48 text

Largest Contentful Paint (often) correlates to bounce rate @tameverts

Slide 49

Slide 49 text

❑ Tracks the most important content ❑ Is accessible out of the box ❑ Has broad browser support ❑ Is available in synthetic and RUM ❑ Can be correlated to UX and business metrics √ √ Largest Contentful Paint √ √ @tameverts

Slide 50

Slide 50 text

@tameverts

Slide 51

Slide 51 text

How does the page feel?

Slide 52

Slide 52 text

Cumulative Layout Shift (CLS) Aggregate score that reflects how much page elements shift during rendering Synthetic & RUM

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

Bounce rate gets worse as CLS degrades Bounce rate improves as CLS degrades Bounce rate stays the same as CLS degrades speedcurve.com/blog/google-cumulative-layout-shift/

Slide 55

Slide 55 text

❑ Tracks the most important content ❑ Is accessible out of the box ❑ Has broad browser support ❑ Is available in synthetic and RUM ❑ Can be correlated to UX and business metrics √ √ Cumulative Layout Shift @tameverts √

Slide 56

Slide 56 text

First Input Delay (FID) Amount of time it takes for the page to respond to user input (e.g., click, tap, key) RUM

Slide 57

Slide 57 text

No correlation when looking at all sessions speedcurve.com/blog/first-input-delay-google-core-web-vitals/ @tameverts

Slide 58

Slide 58 text

Stronger correlation at 75th percentile speedcurve.com/blog/first-input-delay-google-core-web-vitals/ @tameverts

Slide 59

Slide 59 text

❑ Tracks the most important content ❑ Is accessible out of the box ❑ Has broad browser support ❑ Is available in synthetic and RUM ❑ Can be correlated to UX and business metrics √ First Input Delay @tameverts

Slide 60

Slide 60 text

Long Tasks Any JavaScript function that takes >50ms to execute Long tasks don’t always block page rendering, but they can cause page jank Synthetic & RUM

Slide 61

Slide 61 text

Slide 62

Slide 62 text

speedcurve.com/blog/first-input-delay-google-core-web-vitals/ Long Tasks (more often than FID) correlate to conversions @tameverts

Slide 63

Slide 63 text

63 Long Tasks can reveal issues with third parties @tameverts

Slide 64

Slide 64 text

Interaction to Next Paint (INP) Measures a page's responsiveness to individual user interactions Observes latency and reports a single value which all (or nearly all) interactions were below RUM

Slide 65

Slide 65 text

? Impact of INP on business and UX

Slide 66

Slide 66 text

Element Timing Measures when specified DOM elements are rendered (and works in SPAs) You decide which element(s) you want to measure RUM & Synthetic

Slide 67

Slide 67 text

speedcurve.com/blog/element-timing-one-true-metric/

Slide 68

Slide 68 text

Meaningful content Usable out of the box Broad browser support Synthetic RUM Correlates to business/UX Time to First Byte ★ ★ ★ ★ ☆ Start Render ☆ ★ ★ ★ ★ ★ Largest Contentful Paint ☆ ★ ★ ★ ★ Cumulative Layout Shift ☆ ☆ ★ ★ First Input Delay ★ ★ Long Tasks ★ ★ ★ ★ ☆ Interaction to Next Paint ☆ ★ Element Timing ★ ★ ★ ★ ★ @tameverts

Slide 69

Slide 69 text

Slide 70

Slide 70 text

YMMV

Slide 71

Slide 71 text

What should my budget thresholds be?

Slide 72

Slide 72 text

performance budgets ≠ performance goals

Slide 73

Slide 73 text

Goals are aspirational How fast do I want to be eventually? Budgets are pragmatic How can I keep my site from getting slower while I work toward my goals? @tameverts

Slide 74

Slide 74 text

Priority 1 Create budgets to fight regression

Slide 75

Slide 75 text

75 Find the worst result in your last 2-4 weeks of data @tameverts

Slide 76

Slide 76 text

76 Make that number your performance budget @tameverts

Slide 77

Slide 77 text

As you optimize your pages (and hopefully get faster!) update your budgets

Slide 78

Slide 78 text

Priority 2 Set long-term goals

Slide 79

Slide 79 text

Slide 80

Slide 80 text

How fast are my competitors? speedcurve.com/benchmarks/ @tameverts

Slide 81

Slide 81 text

@tameverts

Slide 82

Slide 82 text

How can I improve business metrics? @tameverts

Slide 83

Slide 83 text

@tameverts

Slide 84

Slide 84 text

How do I manage my budgets?

Slide 85

Slide 85 text

Should I set up budgets in RUM, synthetic, or both?

Slide 86

Slide 86 text

Scenario 1 Create budgets in RUM, and drill down via synthetic

Slide 87

Slide 87 text

@tameverts

Slide 88

Slide 88 text

Scenario 2 Create budgets in synthetic, and integrate with CI/CD

Slide 89

Slide 89 text

@tameverts

Slide 90

Slide 90 text

Should I create performance budgets for every metric I track?

Slide 91

Slide 91 text

No content

Slide 92

Slide 92 text

No content

Slide 93

Slide 93 text

No content

Slide 94

Slide 94 text

Do I need performance budgets for all my pages?

Slide 95

Slide 95 text

1. Product 2. Category 3. Home 4. Cart 5. Search @tameverts

Slide 96

Slide 96 text

Who should be responsible for managing budgets?

Slide 97

Slide 97 text

Everyone* who touches a page should understand the performance impact of their choices *Yes, this includes marketing people

Slide 98

Slide 98 text

“One of the original directives of the performance team was we weren’t going to set ourselves up to be performance cops.” Dan Chilton, Vox Media responsivewebdesign.com/podcast/vox-media-performance/

Slide 99

Slide 99 text

Our goals • Set up best practices • Make recommendations • Be a resource within the company that people can turn to when they have to make performance-related decisions Dan Chilton, Vox Media responsivewebdesign.com/podcast/vox-media-performance/

Slide 100

Slide 100 text

No content

Slide 101

Slide 101 text

1. Start small (even a single metric will do!) 2. Visually validate your metrics (filmstrips, videos) 3. Validate your metrics some more (UX, business) 4. Get buy-in from different stakeholders 5. Focus on the pages that matter most 6. Revisit your budgets regularly (2-4 weeks) 7. Remember that metrics are always evolving 8. Never stop measuring

Slide 102

Slide 102 text

Thanks! @tameverts

Slide 103

Slide 103 text

Performance Budgets 101 support.speedcurve.com/docs/performance-budgets-101 Setting a Performance Budget timkadlec.com/2013/01/setting-a-performance-budget/ Performance Budgets, Pragmatically csswizardry.com/2020/01/performance-budgets-pragmatically/ Web Vitals web.dev/vitals/ Element Timing: One true metric to rule them all? speedcurve.com/blog/element-timing-one-true-metric/ First Input Delay: How vital is it? speedcurve.com/blog/first-input-delay-google-core-web-vitals/ Cumulative Layout Shift: What it measures (and what it doesn’t) speedcurve.com/blog/google-cumulative-layout-shift/