Tracking Page Speed

Over the years, multitude of metrics were developed, from Time To First Byte and Page Load Time all the way to more modern and sophisticated ones like Time to First Paint, ATF, Speed Index and Frames per Second. One challenge with current performance metrics is that they all are proxies for user experience and do not translate directly into user satisfaction and business gains, but with each new metric we are getting closer to measuring what matters. Sergey will talk about pros and cons of technical metrics historically used to monitor performance, describe two core methods or measurement/monitoring (synthetic and RUM) and introduce latest developments in tracking user-centric metrics.

Transcript

1. T R A C K I N G
   PA G E
   S P E E D
   S E R G E Y C H E R N Y S H E V
   

   View Slide

2. S C A L A B I L I T Y & P E R F O R M A N C E
   number of users
   time to load
   

   View Slide

3. number of users
   time to load
   S C A L A B I L I T Y & P E R F O R M A N C E
   0
   

   View Slide

4. S C A L A B I L I T Y & P E R F O R M A N C E
   One User
   time to load
   0
   S1
   

   View Slide

5. W H Y D O W E M E A S U R E S P E E D ?
   • Monitor for degradations (Ops)
   • Analyze code for perf issues (Devs)
   • Verify improvements (Devs)
   • Prioritize improvements (Business)
   • Budget for WPO initiatives (Business)
   

   View Slide

6. S TAT S
   • DoubleClick: 5s vs 19s on mobile => 2x ad revenue (2016)
   • Trainline: -300ms latency => +$11.5M / year revenue (2016)
   • Etsy: +160Kb => +12% bounce rate (2014)
   • DoubleClick: -1 redirect => +12% CTR (2011)
   • Edmunds: -77% load time => +20% page views (2011)
   • Mozilla: -2.2s => +15.4% Downloads (2010)
   • Google: +400ms => -0.21% searches after experiment! (2009)
   • Shopzilla: -5s => +12% Conversion rate (2009)
   • Netflix: +GZip => -43% Traffic cost (2008)
   • Amazon: +100ms => -1% revenue (2008)
   • Google: +500ms => -25% searches (2006)
   WPOStats.com
   

   View Slide

7. S Y N T H E T I C M O N I T O R I N G ,
   T E S T I N G & A N A LY S I S
   • From particular location
   • Tester controls instrumentation
   • One metric value
   • Data can have lots of details for analysis
   

   View Slide

8. R E A L U S E R M E A S U R E M E N T
   • Real users (a lot of them)
   • A lot of data (need to store it)
   • All noise you can get, requires filtering
   • Metrics are distributions
   • Can correlate to business KPIs
   

   View Slide

9. T O D AY ' S M E T R I C S
   • DNS, SSL/TLS, Time To First Byte (TTFB)
   • Page Load, Document Complete, Fully Loaded
   • Size of DOM, Frames per Second
   • First Paint, First “Contentful” or Meaningful Paint
   • Above the Fold Time (AFT)
   • SpeedIndex
   

   View Slide

10. T I M E T O F I R S T PA I N T
    • Shows when completely useless part is over
    TTFP
    3.5s
    

    View Slide

11. A F T ( A B O V E T H E F O L D T I M E )
    • When everything is finally visible
    AFT
    15.3s
    

    View Slide

12. S P E E D I N D E X
    • Unifies rendering progress in one number
    Speed Index
    8618
    

    View Slide

13. M E A S U R E U S E R E X P E R I E N C E
    • Great experience for users
    • "Fast" is just a component
    • Correlate what you measure
    to business KPIs
    • Do not measure what's easy,
    measure what matters
    N O T T E C H N O L O G Y
    

    View Slide

14. I N T E R A C T I O N O N T H E W E B
    1. Verify destination
    2. Provide primary content
    3. Allow interaction
    4. Show secondary content
    5. Invisible tasks
    ✦ Acknowledge action
    

    View Slide

15. D E S I G N C O M P O N E N T S
    1. Verify destination
    2. Primary content
    …
    

    View Slide

16. E X T E N D E D M E T R I C S
    • Time To First Paint (TTFP) - 3.5s
    • Verify Destination - 8s
    • Primary content - 11s
    • ...
    • Above The Fold (AFT) - 15.3s
    

    View Slide

17. U S E R T I M I N G A P I
    • Records custom JS timings on the page
    • Recorded by both RUM and Synthetic tools
    • Can be hard to match with user's reality
    • Uses JavaScript instrumentation
    https://github.com/sergeychernyshev/ux-capture
    

    View Slide

18. T H A N K Y O U !
    sergey.chernyshev@gmail.com
    @SergeyChe
    

    View Slide