Tracking Page Speed

Over the years, multitude of metrics were developed, from Time To First Byte and Page Load Time all the way to more modern and sophisticated ones like Time to First Paint, ATF, Speed Index and Frames per Second. One challenge with current performance metrics is that they all are proxies for user experience and do not translate directly into user satisfaction and business gains, but with each new metric we are getting closer to measuring what matters. Sergey will talk about pros and cons of technical metrics historically used to monitor performance, describe two core methods or measurement/monitoring (synthetic and RUM) and introduce latest developments in tracking user-centric metrics.

Transcript

1. T R A C K I N G PA G E S P E E D S E R G E Y C H E R N Y S H E V
2. S C A L A B I L I T Y & P E R F O R M A N C E number of users time to load
3. number of users time to load S C A L A B I L I T Y & P E R F O R M A N C E 0
4. S C A L A B I L I T Y & P E R F O R M A N C E One User time to load 0 S1
5. W H Y D O W E M E A S U R E S P E E D ? • Monitor for degradations (Ops) • Analyze code for perf issues (Devs) • Verify improvements (Devs) • Prioritize improvements (Business) • Budget for WPO initiatives (Business)
6. S TAT S • DoubleClick: 5s vs 19s on mobile => 2x ad revenue (2016) • Trainline: -300ms latency => +$11.5M / year revenue (2016) • Etsy: +160Kb => +12% bounce rate (2014) • DoubleClick: -1 redirect => +12% CTR (2011) • Edmunds: -77% load time => +20% page views (2011) • Mozilla: -2.2s => +15.4% Downloads (2010) • Google: +400ms => -0.21% searches after experiment! (2009) • Shopzilla: -5s => +12% Conversion rate (2009) • Netflix: +GZip => -43% Traffic cost (2008) • Amazon: +100ms => -1% revenue (2008) • Google: +500ms => -25% searches (2006) WPOStats.com
7. S Y N T H E T I C M O N I T O R I N G , T E S T I N G & A N A LY S I S • From particular location • Tester controls instrumentation • One metric value • Data can have lots of details for analysis
8. R E A L U S E R M E A S U R E M E N T • Real users (a lot of them) • A lot of data (need to store it) • All noise you can get, requires filtering • Metrics are distributions • Can correlate to business KPIs
9. T O D AY ' S M E T R I C S • DNS, SSL/TLS, Time To First Byte (TTFB) • Page Load, Document Complete, Fully Loaded • Size of DOM, Frames per Second • First Paint, First “Contentful” or Meaningful Paint • Above the Fold Time (AFT) • SpeedIndex
10. T I M E T O F I R S T PA I N T • Shows when completely useless part is over TTFP 3.5s
11. A F T ( A B O V E T H E F O L D T I M E ) • When everything is finally visible AFT 15.3s
12. S P E E D I N D E X • Unifies rendering progress in one number Speed Index 8618
13. M E A S U R E U S E R E X P E R I E N C E • Great experience for users • "Fast" is just a component • Correlate what you measure to business KPIs • Do not measure what's easy, measure what matters N O T T E C H N O L O G Y
14. I N T E R A C T I O N O N T H E W E B 1. Verify destination 2. Provide primary content 3. Allow interaction 4. Show secondary content 5. Invisible tasks ✦ Acknowledge action
15. D E S I G N C O M P O N E N T S 1. Verify destination 2. Primary content …
16. E X T E N D E D M E T R I C S • Time To First Paint (TTFP) - 3.5s • Verify Destination - 8s • Primary content - 11s • ... • Above The Fold (AFT) - 15.3s
17. U S E R T I M I N G A P I • Records custom JS timings on the page • Recorded by both RUM and Synthetic tools • Can be hard to match with user's reality • Uses JavaScript instrumentation https://github.com/sergeychernyshev/ux-capture
18. T H A N K Y O U ! sergey.chernyshev@gmail.com @SergeyChe