The Good, The Bad, and The Ugly of the HTTP Archive

Presented at Velocity Conference, September 2014.

The web is a fascinating place. Every day millions of websites around the world are constantly evolving and growing. Fortunately, great resources such as the HTTP archive capture statistics from over 300K URLS, and provide us with useful insights into the world of web performance.

http://deanhume.com
http://robinosborne.co.uk

Transcript

1. The Good, the
   Bad, and the
   Ugly
   ....of the HTTP Archive
   

   View Slide

2. Robin
   Osborne
   

   View Slide

3. Dean
   Hume
   

   View Slide

4. The Talk
   

   View Slide

5. HTTP
   Archive
   

   View Slide

6. HTTP Archive
   ● Started in November 2011
   ● Contains over 300K URLs
   ● Both Mobile and Desktop sites
   

   View Slide

7. HTTP Archive
   ● Download as CSV or MySql archive
   ● Page Data
   ● Request Data
   

   View Slide

8. Great on its own..
   HTTP Archive
   

   View Slide

9. +
   Google
   BigQuery
   HTTP Archive
   

   View Slide

10. = Awesome!
    

    View Slide

11. SELECT AVG(PageSpeed) as AvgPageSpeed
    FROM [httparchive:runs.2014_07_15_pages]
    

    View Slide

12. http://bigqueri.es
    

    View Slide

13. Good BAD
    UGLY
    

    View Slide

14. The Criteria
    

    View Slide

15. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    

    View Slide

16. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern
    

    View Slide

17. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modernmodern(ish)
    

    View Slide

18. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern
    ● Follows best practices
    modern(ish)
    

    View Slide

19. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern
    ● Follows best practices
    ● Not a porn site
    modern(ish)
    

    View Slide

20. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern
    ● Follows best practices
    ● Not a porn site (sorry)
    modern(ish)
    

    View Slide

21. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern
    ● Follows best practices
    ● Not a porn site (sorry)
    ● Bonus points for a responsive site
    modern(ish)
    

    View Slide

22. easy?
    

    View Slide

23. View Slide

24. View Slide

25. BEWARE!
    Here be dragons!
    

    View Slide

26. Key takeaway
    ● Super fast holding pages
    Http Archive + Big Query is a great
    way to find:
    

    View Slide

27. Key takeaway
    ● Super fast holding pages
    ● Super fast porn sites
    Http Archive + Big Query is a great
    way to find:
    

    View Slide

28. 80%
    20%
    

    View Slide

29. How we measured
    

    View Slide

30. How we measured
    Fully Loaded
    

    View Slide

31. Fully
    loaded
    A time measurement that
    includes any activity that is
    triggered by JavaScript after
    the main page loads.
    

    View Slide

32. How we measured
    Page Speed
    Fully Loaded
    

    View Slide

33. Page
    Speed
    The PageSpeed Score (0-
    100) is calculated on a
    number of factors.
    A high score is good.
    A low score is not so good.
    

    View Slide

34. How we measured
    Speed Index
    Page Speed
    Fully Loaded
    

    View Slide

35. Speed
    Index
    Speed Index measures how
    complete a page is during
    various points in page load
    A low score is good.
    A high score is not so good.
    

    View Slide

36. The Good
    

    View Slide

37. SELECT url, pagespeed, SpeedIndex,
    fullyLoaded, rank
    FROM [httparchive:runs.latest_pages]
    WHERE (rank is not null) and (speedindex is
    not null) and pagespeed > 80
    ORDER BY rank asc, pagespeed desc,
    speedindex asc limit 1000 ;
    

    View Slide

38. Filament Group
    

    View Slide

39. View Slide

40. filament group
    ● 8-person development studio based all
    around the US
    ● Develop a wide range of sites and apps
    ● Responsive website
    

    View Slide

41. 100 Page Speed
    A Grade YSlow
    

    View Slide

42. What is your SECRET?
    

    View Slide

43. “How we load assets matters
    just as much as how many
    assets we’re loading. We
    focus on perceived
    performance”
    

    View Slide

44. Reduce the
    Critical Path
    

    View Slide

45. tools
    

    View Slide

46. tools
    Grunt-CriticalCSS
    

    View Slide

47. tools
    LoadCss
    Grunt-CriticalCSS
    

    View Slide

48. LoadJS
    LoadCss
    Grunt-CriticalCSS
    tools
    

    View Slide

49. KEY TAKEAWAYS
    

    View Slide

50. KEY TAKEAWAYS
    REDUCE THE CRITICAL PATH
    

    View Slide

51. KEY TAKEAWAYS
    REDUCE THE CRITICAL PATH
    FOCUS ON PERCEIVED
    PERFORMANCE
    

    View Slide

52. nature.com
    

    View Slide

53. View Slide

54. nature.com
    ● Scientific journal for research across
    a wide range of fields
    ● Part of the Macmillan publishing
    group
    ● Responsive website
    ● Based in London, UK
    

    View Slide

55. 86 Page Speed
    A Grade YSlow
    

    View Slide

56. Explain
    

    View Slide

57. “ We try to stick to the 14
    rules as outlined in
    High Performance
    Web Sites ”
    

    View Slide

58. How ?
    

    View Slide

59. “Monitoring is key for us”
    

    View Slide

60. Showslow.com
    “Monitoring is key for us”
    

    View Slide

61. StatsD & Graphite
    Showslow.com
    “Monitoring is key for us”
    

    View Slide

62. KEY TAKEAWAYS
    

    View Slide

63. KEY TAKEAWAYS
    STICK TO BASICS
    

    View Slide

64. KEY TAKEAWAYS
    STICK TO BASICS
    MONITOR
    

    View Slide

65. Zomato.com
    

    View Slide

66. View Slide

67. Online restaurant guide -
    Based in New Delhi, India -
    13 countries in 5 languages -
    Responsive site -
    zomato.xxx -
    

    View Slide

68. 88 Page Speed
    C Grade YSlow
    

    View Slide

69. What is your SECRET?
    

    View Slide

70. “A blank page is always
    going to be the fastest,
    start from there. Add only
    what is needed.”
    

    View Slide

71. Heavy use of caching
    

    View Slide

72. KEY TAKEAWAYS
    

    View Slide

73. KEY TAKEAWAYS
    START SMALL
    

    View Slide

74. KEY TAKEAWAYS
    START SMALL
    CACHE BIG
    

    View Slide

75. Envato
    Marketplaces
    

    View Slide

76. View Slide

77. envato marketplaces
    ➔ Specializes in online
    marketplaces
    ➔ Based in Melbourne, Australia
    ➔ Themeforest, CodeCanyon
    

    View Slide

78. 85 Page Speed
    B Grade YSlow
    

    View Slide

79. What is your SECRET?
    

    View Slide

80. “Stick to basics and aim
    for low hanging fruit.”
    

    View Slide

81. “ We think about
    performance from the
    earliest design stage. ”
    

    View Slide

82. If users add their own images
    you
    need a performance strategy
    

    View Slide

83. KEY TAKEAWAYS
    

    View Slide

84. KEY TAKEAWAYS
    CONSIDER USER
    GENERATED CONTENT
    

    View Slide

85. KEY TAKEAWAYS
    CONSIDER USER
    GENERATED CONTENT
    STICK TO BASICS
    

    View Slide

86. The Bad
    

    View Slide

87. SELECT url, visualComplete, fullyLoaded,
    reqTotal, bytesTotal,numdomelements
    FROM [httparchive:runs.latest_pages] as
    pages
    ORDER BY visualcomplete DESC, fullyloaded
    DESC;
    

    View Slide

88. welovefashion.it
    

    View Slide

89. welovefashion.it
    ➔ Visually complete in 20 secs
    

    View Slide

90. welovefashion.it
    ➔ Visually complete in 20 secs
    ➔ Fully loaded in 30 secs
    

    View Slide

91. welovefashion.it
    ➔ Visually complete in 20 secs
    ➔ Fully loaded in 30 secs
    ➔ 3.5 MB of resources are
    downloaded
    

    View Slide

92. One simple
    change
    

    View Slide

93. Enable
    Compression
    

    View Slide

94. 3.5 MB
    

    View Slide

95. 2.5 MB
    

    View Slide

96. gu-japan.com
    

    View Slide

97. gu-japan.com
    ➔ 15.5 MB of images are loaded
    

    View Slide

98. gu-japan.com
    ➔ 15.5 MB of images are loaded
    ➔ Visually complete in 16 secs
    

    View Slide

99. gu-japan.com
    ➔ 15.5 MB of images are loaded
    ➔ Visually complete in 16 secs
    ➔ Fully loaded in 35 secs
    

    View Slide

100. gu-japan.com
     ➔ 15.5 MB of images are loaded
     ➔ Visually complete in 16 secs
     ➔ Fully loaded in 35 secs
     ➔ 717 HTTP requests
     

     View Slide

101. One simple
     change
     

     View Slide

102. Lose the
     carousels
     

     View Slide

103. 1%
     

     View Slide

104. shouldiuseacarousel.com
     

     View Slide

105. 15 MB
     

     View Slide

106. 1.5 MB
     

     View Slide

107. gamepedia.com
     

     View Slide

108. 10MB of images
     gamepedia.com
     

     View Slide

109. 10MB of images
     130 Resources
     gamepedia.com
     

     View Slide

110. 10MB of images
     130 Resources
     Fully loaded in 15 secs
     gamepedia.com
     

     View Slide

111. One change?
     

     View Slide

112. One change?
     Serve scaled
     images
     

     View Slide

113. 10 MB
     

     View Slide

114. 6 MB
     

     View Slide

115. The Ugly
     

     View Slide

116. SELECT url, visualComplete, fullyLoaded,
     reqTotal, bytesTotal,numdomelements
     FROM [httparchive:runs.latest_pages] as
     pages
     ORDER BY visualcomplete DESC, fullyloaded
     DESC;
     

     View Slide

117. View Slide

118. View Slide

119. sailboatlistings.com
     ● Fully loaded in 18 secs
     

     View Slide

120. sailboatlistings.com
     ● Fully loaded in 18 secs
     ● Visually complete in 20 secs
     

     View Slide

121. sailboatlistings.com
     ● Fully loaded in 18 secs
     ● Visually complete in 20 secs
     ● 3 MB total
     

     View Slide

122. sailboatlistings.com
     ● Fully loaded in 18 secs
     ● Visually complete in 20 secs
     ● 3 MB total
     ● 580 requests in total
     

     View Slide

123. sailboatlistings.com
     ● Fully loaded in 18 secs
     ● Visually complete in 20 secs
     ● 3 MB total
     ● 580 requests in total
     ● 535 image requests
     

     View Slide

124. sailboatlistings.com
     ● Fully loaded in 18 secs
     ● Visually complete in 20 secs
     ● 3 MB total
     ● 580 requests in total
     ● 535 image requests
     ● 14872 dom elements
     

     View Slide

125. colorsbycherry.com
     

     View Slide

126. View Slide

127. colorsbycherry.com
     Visually complete in 30 secs
     

     View Slide

128. colorsbycherry.com
     Fully loaded in 1 min
     Visually complete in 30 secs
     

     View Slide

129. colorsbycherry.com
     34 MB of resources
     Fully loaded in 1 min
     Visually complete in 30 secs
     

     View Slide

130. colorsbycherry.com
     34 MB of resources
     Fully loaded in 1 min
     Visually complete in 30 secs
     say what!?
     

     View Slide

131. Just.
     Plain.
     Rude.
     

     View Slide

132. View Slide

133. View Slide

134. callofduty.com
     

     View Slide

135. callofduty.com
     15.2 MB of resources
     callofduty.com
     

     View Slide

136. callofduty.com
     15.2 MB of resources
     Visually complete in 1.3 secs
     callofduty.com
     

     View Slide

137. callofduty.com
     15.2 MB of resources
     Visually complete in 1.3 secs
     Fully loaded in 13 secs
     callofduty.com
     

     View Slide

138. callofduty.com
     15.2 MB of resources
     Visually complete in 1.3 secs
     Fully loaded in 13 secs
     Page Speed score of 11
     callofduty.com
     

     View Slide

139. Round Up
     

     View Slide

140. DO THIS
     Stick to the basics
     Monitor
     Start with the minimum
     

     View Slide

141. don’t do this
     Serve unscaled images
     Use bloated carousels
     Forget compression
     

     View Slide

142. definitely don’t do this
     Serve humungous pages
     Use non-streaming videos
     Serve unoptimized images
     

     View Slide

143. Tools
     

     View Slide

144. Tools
     psi
     

     View Slide

145. psi
     ● Page Speed Insights
     ● Get PSI reporting from the
     command line
     ● On GitHub, by Addy Osmani
     ● Can be hooked into a build
     process via the Gulp or Grunt
     plugins
     

     View Slide

146. View Slide

147. Tools
     psi
     ngrok
     

     View Slide

148. ngrok
     ● Creates a tunnel to localhost
     ● Request inspection
     

     View Slide

149. View Slide

150. @rposbo
     @deanohume
     

     View Slide

151. Thank you
     

     View Slide