The Good, The Bad, and The Ugly of the HTTP Archive

C620790ae5bf5b50c245b2e0ef95f338?s=47 Dean Hume
September 17, 2014

The Good, The Bad, and The Ugly of the HTTP Archive

Presented at Velocity Conference, September 2014.

The web is a fascinating place. Every day millions of websites around the world are constantly evolving and growing. Fortunately, great resources such as the HTTP archive capture statistics from over 300K URLS, and provide us with useful insights into the world of web performance.

http://deanhume.com
http://robinosborne.co.uk

C620790ae5bf5b50c245b2e0ef95f338?s=128

Dean Hume

September 17, 2014
Tweet

Transcript

  1. The Good, the Bad, and the Ugly ....of the HTTP

    Archive
  2. Robin Osborne

  3. Dean Hume

  4. The Talk

  5. HTTP Archive

  6. HTTP Archive • Started in November 2011 • Contains over

    300K URLs • Both Mobile and Desktop sites
  7. HTTP Archive • Download as CSV or MySql archive •

    Page Data • Request Data
  8. Great on its own.. HTTP Archive

  9. + Google BigQuery HTTP Archive

  10. = Awesome!

  11. SELECT AVG(PageSpeed) as AvgPageSpeed FROM [httparchive:runs.2014_07_15_pages]

  12. http://bigqueri.es

  13. Good BAD UGLY

  14. The Criteria

  15. The Criteria • The Alexa top 100 sites have been

    excluded from this list
  16. The Criteria • The Alexa top 100 sites have been

    excluded from this list • The site needs to be usable and modern
  17. The Criteria • The Alexa top 100 sites have been

    excluded from this list • The site needs to be usable and modernmodern(ish)
  18. The Criteria • The Alexa top 100 sites have been

    excluded from this list • The site needs to be usable and modern • Follows best practices modern(ish)
  19. The Criteria • The Alexa top 100 sites have been

    excluded from this list • The site needs to be usable and modern • Follows best practices • Not a porn site modern(ish)
  20. The Criteria • The Alexa top 100 sites have been

    excluded from this list • The site needs to be usable and modern • Follows best practices • Not a porn site (sorry) modern(ish)
  21. The Criteria • The Alexa top 100 sites have been

    excluded from this list • The site needs to be usable and modern • Follows best practices • Not a porn site (sorry) • Bonus points for a responsive site modern(ish)
  22. easy?

  23. None
  24. None
  25. BEWARE! Here be dragons!

  26. Key takeaway • Super fast holding pages Http Archive +

    Big Query is a great way to find:
  27. Key takeaway • Super fast holding pages • Super fast

    porn sites Http Archive + Big Query is a great way to find:
  28. 80% 20%

  29. How we measured

  30. How we measured Fully Loaded

  31. Fully loaded A time measurement that includes any activity that

    is triggered by JavaScript after the main page loads.
  32. How we measured Page Speed Fully Loaded

  33. Page Speed The PageSpeed Score (0- 100) is calculated on

    a number of factors. A high score is good. A low score is not so good.
  34. How we measured Speed Index Page Speed Fully Loaded

  35. Speed Index Speed Index measures how complete a page is

    during various points in page load A low score is good. A high score is not so good.
  36. The Good

  37. SELECT url, pagespeed, SpeedIndex, fullyLoaded, rank FROM [httparchive:runs.latest_pages] WHERE (rank

    is not null) and (speedindex is not null) and pagespeed > 80 ORDER BY rank asc, pagespeed desc, speedindex asc limit 1000 ;
  38. Filament Group

  39. None
  40. filament group • 8-person development studio based all around the

    US • Develop a wide range of sites and apps • Responsive website
  41. 100 Page Speed A Grade YSlow

  42. What is your SECRET?

  43. “How we load assets matters just as much as how

    many assets we’re loading. We focus on perceived performance”
  44. Reduce the Critical Path

  45. tools

  46. tools Grunt-CriticalCSS

  47. tools LoadCss Grunt-CriticalCSS

  48. LoadJS LoadCss Grunt-CriticalCSS tools

  49. KEY TAKEAWAYS

  50. KEY TAKEAWAYS REDUCE THE CRITICAL PATH

  51. KEY TAKEAWAYS REDUCE THE CRITICAL PATH FOCUS ON PERCEIVED PERFORMANCE

  52. nature.com

  53. None
  54. nature.com • Scientific journal for research across a wide range

    of fields • Part of the Macmillan publishing group • Responsive website • Based in London, UK
  55. 86 Page Speed A Grade YSlow

  56. Explain

  57. “ We try to stick to the 14 rules as

    outlined in High Performance Web Sites ”
  58. How ?

  59. “Monitoring is key for us”

  60. Showslow.com “Monitoring is key for us”

  61. StatsD & Graphite Showslow.com “Monitoring is key for us”

  62. KEY TAKEAWAYS

  63. KEY TAKEAWAYS STICK TO BASICS

  64. KEY TAKEAWAYS STICK TO BASICS MONITOR

  65. Zomato.com

  66. None
  67. Online restaurant guide - Based in New Delhi, India -

    13 countries in 5 languages - Responsive site - zomato.xxx -
  68. 88 Page Speed C Grade YSlow

  69. What is your SECRET?

  70. “A blank page is always going to be the fastest,

    start from there. Add only what is needed.”
  71. Heavy use of caching

  72. KEY TAKEAWAYS

  73. KEY TAKEAWAYS START SMALL

  74. KEY TAKEAWAYS START SMALL CACHE BIG

  75. Envato Marketplaces

  76. None
  77. envato marketplaces ➔ Specializes in online marketplaces ➔ Based in

    Melbourne, Australia ➔ Themeforest, CodeCanyon
  78. 85 Page Speed B Grade YSlow

  79. What is your SECRET?

  80. “Stick to basics and aim for low hanging fruit.”

  81. “ We think about performance from the earliest design stage.

  82. If users add their own images you need a performance

    strategy
  83. KEY TAKEAWAYS

  84. KEY TAKEAWAYS CONSIDER USER GENERATED CONTENT

  85. KEY TAKEAWAYS CONSIDER USER GENERATED CONTENT STICK TO BASICS

  86. The Bad

  87. SELECT url, visualComplete, fullyLoaded, reqTotal, bytesTotal,numdomelements FROM [httparchive:runs.latest_pages] as pages

    ORDER BY visualcomplete DESC, fullyloaded DESC;
  88. welovefashion.it

  89. welovefashion.it ➔ Visually complete in 20 secs

  90. welovefashion.it ➔ Visually complete in 20 secs ➔ Fully loaded

    in 30 secs
  91. welovefashion.it ➔ Visually complete in 20 secs ➔ Fully loaded

    in 30 secs ➔ 3.5 MB of resources are downloaded
  92. One simple change

  93. Enable Compression

  94. 3.5 MB

  95. 2.5 MB

  96. gu-japan.com

  97. gu-japan.com ➔ 15.5 MB of images are loaded

  98. gu-japan.com ➔ 15.5 MB of images are loaded ➔ Visually

    complete in 16 secs
  99. gu-japan.com ➔ 15.5 MB of images are loaded ➔ Visually

    complete in 16 secs ➔ Fully loaded in 35 secs
  100. gu-japan.com ➔ 15.5 MB of images are loaded ➔ Visually

    complete in 16 secs ➔ Fully loaded in 35 secs ➔ 717 HTTP requests
  101. One simple change

  102. Lose the carousels

  103. 1%

  104. shouldiuseacarousel.com

  105. 15 MB

  106. 1.5 MB

  107. gamepedia.com

  108. 10MB of images gamepedia.com

  109. 10MB of images 130 Resources gamepedia.com

  110. 10MB of images 130 Resources Fully loaded in 15 secs

    gamepedia.com
  111. One change?

  112. One change? Serve scaled images

  113. 10 MB

  114. 6 MB

  115. The Ugly

  116. SELECT url, visualComplete, fullyLoaded, reqTotal, bytesTotal,numdomelements FROM [httparchive:runs.latest_pages] as pages

    ORDER BY visualcomplete DESC, fullyloaded DESC;
  117. None
  118. 44

  119. sailboatlistings.com • Fully loaded in 18 secs

  120. sailboatlistings.com • Fully loaded in 18 secs • Visually complete

    in 20 secs
  121. sailboatlistings.com • Fully loaded in 18 secs • Visually complete

    in 20 secs • 3 MB total
  122. sailboatlistings.com • Fully loaded in 18 secs • Visually complete

    in 20 secs • 3 MB total • 580 requests in total
  123. sailboatlistings.com • Fully loaded in 18 secs • Visually complete

    in 20 secs • 3 MB total • 580 requests in total • 535 image requests
  124. sailboatlistings.com • Fully loaded in 18 secs • Visually complete

    in 20 secs • 3 MB total • 580 requests in total • 535 image requests • 14872 dom elements
  125. colorsbycherry.com

  126. 14

  127. colorsbycherry.com Visually complete in 30 secs

  128. colorsbycherry.com Fully loaded in 1 min Visually complete in 30

    secs
  129. colorsbycherry.com 34 MB of resources Fully loaded in 1 min

    Visually complete in 30 secs
  130. colorsbycherry.com 34 MB of resources Fully loaded in 1 min

    Visually complete in 30 secs say what!?
  131. Just. Plain. Rude.

  132. None
  133. 11

  134. callofduty.com

  135. callofduty.com 15.2 MB of resources callofduty.com

  136. callofduty.com 15.2 MB of resources Visually complete in 1.3 secs

    callofduty.com
  137. callofduty.com 15.2 MB of resources Visually complete in 1.3 secs

    Fully loaded in 13 secs callofduty.com
  138. callofduty.com 15.2 MB of resources Visually complete in 1.3 secs

    Fully loaded in 13 secs Page Speed score of 11 callofduty.com
  139. Round Up

  140. DO THIS Stick to the basics Monitor Start with the

    minimum
  141. don’t do this Serve unscaled images Use bloated carousels Forget

    compression
  142. definitely don’t do this Serve humungous pages Use non-streaming videos

    Serve unoptimized images
  143. Tools

  144. Tools psi

  145. psi • Page Speed Insights • Get PSI reporting from

    the command line • On GitHub, by Addy Osmani • Can be hooked into a build process via the Gulp or Grunt plugins
  146. None
  147. Tools psi ngrok

  148. ngrok • Creates a tunnel to localhost • Request inspection

  149. None
  150. @rposbo @deanohume

  151. Thank you