The Good, The Bad, and The Ugly of the HTTP Archive

Presented at Velocity Conference, September 2014.

The web is a fascinating place. Every day millions of websites around the world are constantly evolving and growing. Fortunately, great resources such as the HTTP archive capture statistics from over 300K URLS, and provide us with useful insights into the world of web performance.

http://deanhume.com
http://robinosborne.co.uk

Transcript

1. The Good, the Bad, and the Ugly ....of the HTTP Archive
2. Robin Osborne
3. Dean Hume
4. The Talk
5. HTTP Archive
6. HTTP Archive • Started in November 2011 • Contains over 300K URLs • Both Mobile and Desktop sites
7. HTTP Archive • Download as CSV or MySql archive • Page Data • Request Data
8. Great on its own.. HTTP Archive
9. + Google BigQuery HTTP Archive
10. = Awesome!
11. SELECT AVG(PageSpeed) as AvgPageSpeed FROM [httparchive:runs.2014_07_15_pages]
12. http://bigqueri.es
13. Good BAD UGLY
14. The Criteria
15. The Criteria • The Alexa top 100 sites have been excluded from this list
16. The Criteria • The Alexa top 100 sites have been excluded from this list • The site needs to be usable and modern
17. The Criteria • The Alexa top 100 sites have been excluded from this list • The site needs to be usable and modernmodern(ish)
18. The Criteria • The Alexa top 100 sites have been excluded from this list • The site needs to be usable and modern • Follows best practices modern(ish)
19. The Criteria • The Alexa top 100 sites have been excluded from this list • The site needs to be usable and modern • Follows best practices • Not a porn site modern(ish)
20. The Criteria • The Alexa top 100 sites have been excluded from this list • The site needs to be usable and modern • Follows best practices • Not a porn site (sorry) modern(ish)
21. The Criteria • The Alexa top 100 sites have been excluded from this list • The site needs to be usable and modern • Follows best practices • Not a porn site (sorry) • Bonus points for a responsive site modern(ish)
22. easy?
25. BEWARE! Here be dragons!
26. Key takeaway • Super fast holding pages Http Archive + Big Query is a great way to find:
27. Key takeaway • Super fast holding pages • Super fast porn sites Http Archive + Big Query is a great way to find:
28. 80% 20%
29. How we measured
30. How we measured Fully Loaded
31. Fully loaded A time measurement that includes any activity that is triggered by JavaScript after the main page loads.
32. How we measured Page Speed Fully Loaded
33. Page Speed The PageSpeed Score (0- 100) is calculated on a number of factors. A high score is good. A low score is not so good.
34. How we measured Speed Index Page Speed Fully Loaded
35. Speed Index Speed Index measures how complete a page is during various points in page load A low score is good. A high score is not so good.
36. The Good
37. SELECT url, pagespeed, SpeedIndex, fullyLoaded, rank FROM [httparchive:runs.latest_pages] WHERE (rank is not null) and (speedindex is not null) and pagespeed > 80 ORDER BY rank asc, pagespeed desc, speedindex asc limit 1000 ;
38. Filament Group
40. filament group • 8-person development studio based all around the US • Develop a wide range of sites and apps • Responsive website
41. 100 Page Speed A Grade YSlow
42. What is your SECRET?
43. “How we load assets matters just as much as how many assets we’re loading. We focus on perceived performance”
44. Reduce the Critical Path
45. tools
46. tools Grunt-CriticalCSS
47. tools LoadCss Grunt-CriticalCSS
48. LoadJS LoadCss Grunt-CriticalCSS tools
49. KEY TAKEAWAYS
50. KEY TAKEAWAYS REDUCE THE CRITICAL PATH
51. KEY TAKEAWAYS REDUCE THE CRITICAL PATH FOCUS ON PERCEIVED PERFORMANCE
52. nature.com
54. nature.com • Scientific journal for research across a wide range of fields • Part of the Macmillan publishing group • Responsive website • Based in London, UK
55. 86 Page Speed A Grade YSlow
56. Explain
57. “ We try to stick to the 14 rules as outlined in High Performance Web Sites ”
58. How ?
59. “Monitoring is key for us”
60. Showslow.com “Monitoring is key for us”
61. StatsD & Graphite Showslow.com “Monitoring is key for us”
62. KEY TAKEAWAYS
63. KEY TAKEAWAYS STICK TO BASICS
64. KEY TAKEAWAYS STICK TO BASICS MONITOR
65. Zomato.com
67. Online restaurant guide - Based in New Delhi, India - 13 countries in 5 languages - Responsive site - zomato.xxx -
68. 88 Page Speed C Grade YSlow
69. What is your SECRET?
70. “A blank page is always going to be the fastest, start from there. Add only what is needed.”
71. Heavy use of caching
72. KEY TAKEAWAYS
73. KEY TAKEAWAYS START SMALL
74. KEY TAKEAWAYS START SMALL CACHE BIG
75. Envato Marketplaces
77. envato marketplaces ➔ Specializes in online marketplaces ➔ Based in Melbourne, Australia ➔ Themeforest, CodeCanyon
78. 85 Page Speed B Grade YSlow
79. What is your SECRET?
80. “Stick to basics and aim for low hanging fruit.”
81. “ We think about performance from the earliest design stage. ”
82. If users add their own images you need a performance strategy
83. KEY TAKEAWAYS
84. KEY TAKEAWAYS CONSIDER USER GENERATED CONTENT
85. KEY TAKEAWAYS CONSIDER USER GENERATED CONTENT STICK TO BASICS
86. The Bad
87. SELECT url, visualComplete, fullyLoaded, reqTotal, bytesTotal,numdomelements FROM [httparchive:runs.latest_pages] as pages ORDER BY visualcomplete DESC, fullyloaded DESC;
88. welovefashion.it
89. welovefashion.it ➔ Visually complete in 20 secs
90. welovefashion.it ➔ Visually complete in 20 secs ➔ Fully loaded in 30 secs
91. welovefashion.it ➔ Visually complete in 20 secs ➔ Fully loaded in 30 secs ➔ 3.5 MB of resources are downloaded
92. One simple change
93. Enable Compression
94. 3.5 MB
95. 2.5 MB
96. gu-japan.com
97. gu-japan.com ➔ 15.5 MB of images are loaded
98. gu-japan.com ➔ 15.5 MB of images are loaded ➔ Visually complete in 16 secs
99. gu-japan.com ➔ 15.5 MB of images are loaded ➔ Visually complete in 16 secs ➔ Fully loaded in 35 secs
100. gu-japan.com ➔ 15.5 MB of images are loaded ➔ Visually complete in 16 secs ➔ Fully loaded in 35 secs ➔ 717 HTTP requests
101. One simple change
102. Lose the carousels
103. 1%
104. shouldiuseacarousel.com
105. 15 MB
106. 1.5 MB
107. gamepedia.com
108. 10MB of images gamepedia.com
109. 10MB of images 130 Resources gamepedia.com
110. 10MB of images 130 Resources Fully loaded in 15 secs gamepedia.com
111. One change?
112. One change? Serve scaled images
113. 10 MB
114. 6 MB
115. The Ugly
116. SELECT url, visualComplete, fullyLoaded, reqTotal, bytesTotal,numdomelements FROM [httparchive:runs.latest_pages] as pages ORDER BY visualcomplete DESC, fullyloaded DESC;
119. sailboatlistings.com • Fully loaded in 18 secs
120. sailboatlistings.com • Fully loaded in 18 secs • Visually complete in 20 secs
121. sailboatlistings.com • Fully loaded in 18 secs • Visually complete in 20 secs • 3 MB total
122. sailboatlistings.com • Fully loaded in 18 secs • Visually complete in 20 secs • 3 MB total • 580 requests in total
123. sailboatlistings.com • Fully loaded in 18 secs • Visually complete in 20 secs • 3 MB total • 580 requests in total • 535 image requests
124. sailboatlistings.com • Fully loaded in 18 secs • Visually complete in 20 secs • 3 MB total • 580 requests in total • 535 image requests • 14872 dom elements
125. colorsbycherry.com
127. colorsbycherry.com Visually complete in 30 secs
128. colorsbycherry.com Fully loaded in 1 min Visually complete in 30 secs
129. colorsbycherry.com 34 MB of resources Fully loaded in 1 min Visually complete in 30 secs
130. colorsbycherry.com 34 MB of resources Fully loaded in 1 min Visually complete in 30 secs say what!?
131. Just. Plain. Rude.
134. callofduty.com
135. callofduty.com 15.2 MB of resources callofduty.com
136. callofduty.com 15.2 MB of resources Visually complete in 1.3 secs callofduty.com
137. callofduty.com 15.2 MB of resources Visually complete in 1.3 secs Fully loaded in 13 secs callofduty.com
138. callofduty.com 15.2 MB of resources Visually complete in 1.3 secs Fully loaded in 13 secs Page Speed score of 11 callofduty.com
139. Round Up
140. DO THIS Stick to the basics Monitor Start with the minimum
141. don’t do this Serve unscaled images Use bloated carousels Forget compression
142. definitely don’t do this Serve humungous pages Use non-streaming videos Serve unoptimized images
143. Tools
144. Tools psi
145. psi • Page Speed Insights • Get PSI reporting from the command line • On GitHub, by Addy Osmani • Can be hooked into a build process via the Gulp or Grunt plugins
147. Tools psi ngrok
148. ngrok • Creates a tunnel to localhost • Request inspection
150. @rposbo @deanohume
151. Thank you