Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Good, The Bad, and The Ugly of the HTTP Archive

Dean Hume
September 17, 2014

The Good, The Bad, and The Ugly of the HTTP Archive

Presented at Velocity Conference, September 2014.

The web is a fascinating place. Every day millions of websites around the world are constantly evolving and growing. Fortunately, great resources such as the HTTP archive capture statistics from over 300K URLS, and provide us with useful insights into the world of web performance.

http://deanhume.com
http://robinosborne.co.uk

Dean Hume

September 17, 2014
Tweet

More Decks by Dean Hume

Other Decks in Technology

Transcript

  1. The Good, the
    Bad, and the
    Ugly
    ....of the HTTP Archive

    View Slide

  2. Robin
    Osborne

    View Slide

  3. Dean
    Hume

    View Slide

  4. The Talk

    View Slide

  5. HTTP
    Archive

    View Slide

  6. HTTP Archive
    ● Started in November 2011
    ● Contains over 300K URLs
    ● Both Mobile and Desktop sites

    View Slide

  7. HTTP Archive
    ● Download as CSV or MySql archive
    ● Page Data
    ● Request Data

    View Slide

  8. Great on its own..
    HTTP Archive

    View Slide

  9. +
    Google
    BigQuery
    HTTP Archive

    View Slide

  10. = Awesome!

    View Slide

  11. SELECT AVG(PageSpeed) as AvgPageSpeed
    FROM [httparchive:runs.2014_07_15_pages]

    View Slide

  12. http://bigqueri.es

    View Slide

  13. Good BAD
    UGLY

    View Slide

  14. The Criteria

    View Slide

  15. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list

    View Slide

  16. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern

    View Slide

  17. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modernmodern(ish)

    View Slide

  18. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern
    ● Follows best practices
    modern(ish)

    View Slide

  19. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern
    ● Follows best practices
    ● Not a porn site
    modern(ish)

    View Slide

  20. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern
    ● Follows best practices
    ● Not a porn site (sorry)
    modern(ish)

    View Slide

  21. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern
    ● Follows best practices
    ● Not a porn site (sorry)
    ● Bonus points for a responsive site
    modern(ish)

    View Slide

  22. easy?

    View Slide

  23. View Slide

  24. View Slide

  25. BEWARE!
    Here be dragons!

    View Slide

  26. Key takeaway
    ● Super fast holding pages
    Http Archive + Big Query is a great
    way to find:

    View Slide

  27. Key takeaway
    ● Super fast holding pages
    ● Super fast porn sites
    Http Archive + Big Query is a great
    way to find:

    View Slide

  28. 80%
    20%

    View Slide

  29. How we measured

    View Slide

  30. How we measured
    Fully Loaded

    View Slide

  31. Fully
    loaded
    A time measurement that
    includes any activity that is
    triggered by JavaScript after
    the main page loads.

    View Slide

  32. How we measured
    Page Speed
    Fully Loaded

    View Slide

  33. Page
    Speed
    The PageSpeed Score (0-
    100) is calculated on a
    number of factors.
    A high score is good.
    A low score is not so good.

    View Slide

  34. How we measured
    Speed Index
    Page Speed
    Fully Loaded

    View Slide

  35. Speed
    Index
    Speed Index measures how
    complete a page is during
    various points in page load
    A low score is good.
    A high score is not so good.

    View Slide

  36. The Good

    View Slide

  37. SELECT url, pagespeed, SpeedIndex,
    fullyLoaded, rank
    FROM [httparchive:runs.latest_pages]
    WHERE (rank is not null) and (speedindex is
    not null) and pagespeed > 80
    ORDER BY rank asc, pagespeed desc,
    speedindex asc limit 1000 ;

    View Slide

  38. Filament Group

    View Slide

  39. View Slide

  40. filament group
    ● 8-person development studio based all
    around the US
    ● Develop a wide range of sites and apps
    ● Responsive website

    View Slide

  41. 100 Page Speed
    A Grade YSlow

    View Slide

  42. What is your SECRET?

    View Slide

  43. “How we load assets matters
    just as much as how many
    assets we’re loading. We
    focus on perceived
    performance”

    View Slide

  44. Reduce the
    Critical Path

    View Slide

  45. tools

    View Slide

  46. tools
    Grunt-CriticalCSS

    View Slide

  47. tools
    LoadCss
    Grunt-CriticalCSS

    View Slide

  48. LoadJS
    LoadCss
    Grunt-CriticalCSS
    tools

    View Slide

  49. KEY TAKEAWAYS

    View Slide

  50. KEY TAKEAWAYS
    REDUCE THE CRITICAL PATH

    View Slide

  51. KEY TAKEAWAYS
    REDUCE THE CRITICAL PATH
    FOCUS ON PERCEIVED
    PERFORMANCE

    View Slide

  52. nature.com

    View Slide

  53. View Slide

  54. nature.com
    ● Scientific journal for research across
    a wide range of fields
    ● Part of the Macmillan publishing
    group
    ● Responsive website
    ● Based in London, UK

    View Slide

  55. 86 Page Speed
    A Grade YSlow

    View Slide

  56. Explain

    View Slide

  57. “ We try to stick to the 14
    rules as outlined in
    High Performance
    Web Sites ”

    View Slide

  58. How ?

    View Slide

  59. “Monitoring is key for us”

    View Slide

  60. Showslow.com
    “Monitoring is key for us”

    View Slide

  61. StatsD & Graphite
    Showslow.com
    “Monitoring is key for us”

    View Slide

  62. KEY TAKEAWAYS

    View Slide

  63. KEY TAKEAWAYS
    STICK TO BASICS

    View Slide

  64. KEY TAKEAWAYS
    STICK TO BASICS
    MONITOR

    View Slide

  65. Zomato.com

    View Slide

  66. View Slide

  67. Online restaurant guide -
    Based in New Delhi, India -
    13 countries in 5 languages -
    Responsive site -
    zomato.xxx -

    View Slide

  68. 88 Page Speed
    C Grade YSlow

    View Slide

  69. What is your SECRET?

    View Slide

  70. “A blank page is always
    going to be the fastest,
    start from there. Add only
    what is needed.”

    View Slide

  71. Heavy use of caching

    View Slide

  72. KEY TAKEAWAYS

    View Slide

  73. KEY TAKEAWAYS
    START SMALL

    View Slide

  74. KEY TAKEAWAYS
    START SMALL
    CACHE BIG

    View Slide

  75. Envato
    Marketplaces

    View Slide

  76. View Slide

  77. envato marketplaces
    ➔ Specializes in online
    marketplaces
    ➔ Based in Melbourne, Australia
    ➔ Themeforest, CodeCanyon

    View Slide

  78. 85 Page Speed
    B Grade YSlow

    View Slide

  79. What is your SECRET?

    View Slide

  80. “Stick to basics and aim
    for low hanging fruit.”

    View Slide

  81. “ We think about
    performance from the
    earliest design stage. ”

    View Slide

  82. If users add their own images
    you
    need a performance strategy

    View Slide

  83. KEY TAKEAWAYS

    View Slide

  84. KEY TAKEAWAYS
    CONSIDER USER
    GENERATED CONTENT

    View Slide

  85. KEY TAKEAWAYS
    CONSIDER USER
    GENERATED CONTENT
    STICK TO BASICS

    View Slide

  86. The Bad

    View Slide

  87. SELECT url, visualComplete, fullyLoaded,
    reqTotal, bytesTotal,numdomelements
    FROM [httparchive:runs.latest_pages] as
    pages
    ORDER BY visualcomplete DESC, fullyloaded
    DESC;

    View Slide

  88. welovefashion.it

    View Slide

  89. welovefashion.it
    ➔ Visually complete in 20 secs

    View Slide

  90. welovefashion.it
    ➔ Visually complete in 20 secs
    ➔ Fully loaded in 30 secs

    View Slide

  91. welovefashion.it
    ➔ Visually complete in 20 secs
    ➔ Fully loaded in 30 secs
    ➔ 3.5 MB of resources are
    downloaded

    View Slide

  92. One simple
    change

    View Slide

  93. Enable
    Compression

    View Slide

  94. 3.5 MB

    View Slide

  95. 2.5 MB

    View Slide

  96. gu-japan.com

    View Slide

  97. gu-japan.com
    ➔ 15.5 MB of images are loaded

    View Slide

  98. gu-japan.com
    ➔ 15.5 MB of images are loaded
    ➔ Visually complete in 16 secs

    View Slide

  99. gu-japan.com
    ➔ 15.5 MB of images are loaded
    ➔ Visually complete in 16 secs
    ➔ Fully loaded in 35 secs

    View Slide

  100. gu-japan.com
    ➔ 15.5 MB of images are loaded
    ➔ Visually complete in 16 secs
    ➔ Fully loaded in 35 secs
    ➔ 717 HTTP requests

    View Slide

  101. One simple
    change

    View Slide

  102. Lose the
    carousels

    View Slide

  103. 1%

    View Slide

  104. shouldiuseacarousel.com

    View Slide

  105. 15 MB

    View Slide

  106. 1.5 MB

    View Slide

  107. gamepedia.com

    View Slide

  108. 10MB of images
    gamepedia.com

    View Slide

  109. 10MB of images
    130 Resources
    gamepedia.com

    View Slide

  110. 10MB of images
    130 Resources
    Fully loaded in 15 secs
    gamepedia.com

    View Slide

  111. One change?

    View Slide

  112. One change?
    Serve scaled
    images

    View Slide

  113. 10 MB

    View Slide

  114. 6 MB

    View Slide

  115. The Ugly

    View Slide

  116. SELECT url, visualComplete, fullyLoaded,
    reqTotal, bytesTotal,numdomelements
    FROM [httparchive:runs.latest_pages] as
    pages
    ORDER BY visualcomplete DESC, fullyloaded
    DESC;

    View Slide

  117. View Slide

  118. 44

    View Slide

  119. sailboatlistings.com
    ● Fully loaded in 18 secs

    View Slide

  120. sailboatlistings.com
    ● Fully loaded in 18 secs
    ● Visually complete in 20 secs

    View Slide

  121. sailboatlistings.com
    ● Fully loaded in 18 secs
    ● Visually complete in 20 secs
    ● 3 MB total

    View Slide

  122. sailboatlistings.com
    ● Fully loaded in 18 secs
    ● Visually complete in 20 secs
    ● 3 MB total
    ● 580 requests in total

    View Slide

  123. sailboatlistings.com
    ● Fully loaded in 18 secs
    ● Visually complete in 20 secs
    ● 3 MB total
    ● 580 requests in total
    ● 535 image requests

    View Slide

  124. sailboatlistings.com
    ● Fully loaded in 18 secs
    ● Visually complete in 20 secs
    ● 3 MB total
    ● 580 requests in total
    ● 535 image requests
    ● 14872 dom elements

    View Slide

  125. colorsbycherry.com

    View Slide

  126. 14

    View Slide

  127. colorsbycherry.com
    Visually complete in 30 secs

    View Slide

  128. colorsbycherry.com
    Fully loaded in 1 min
    Visually complete in 30 secs

    View Slide

  129. colorsbycherry.com
    34 MB of resources
    Fully loaded in 1 min
    Visually complete in 30 secs

    View Slide

  130. colorsbycherry.com
    34 MB of resources
    Fully loaded in 1 min
    Visually complete in 30 secs
    say what!?

    View Slide

  131. Just.
    Plain.
    Rude.

    View Slide

  132. View Slide

  133. 11

    View Slide

  134. callofduty.com

    View Slide

  135. callofduty.com
    15.2 MB of resources
    callofduty.com

    View Slide

  136. callofduty.com
    15.2 MB of resources
    Visually complete in 1.3 secs
    callofduty.com

    View Slide

  137. callofduty.com
    15.2 MB of resources
    Visually complete in 1.3 secs
    Fully loaded in 13 secs
    callofduty.com

    View Slide

  138. callofduty.com
    15.2 MB of resources
    Visually complete in 1.3 secs
    Fully loaded in 13 secs
    Page Speed score of 11
    callofduty.com

    View Slide

  139. Round Up

    View Slide

  140. DO THIS
    Stick to the basics
    Monitor
    Start with the minimum

    View Slide

  141. don’t do this
    Serve unscaled images
    Use bloated carousels
    Forget compression

    View Slide

  142. definitely don’t do this
    Serve humungous pages
    Use non-streaming videos
    Serve unoptimized images

    View Slide

  143. Tools

    View Slide

  144. Tools
    psi

    View Slide

  145. psi
    ● Page Speed Insights
    ● Get PSI reporting from the
    command line
    ● On GitHub, by Addy Osmani
    ● Can be hooked into a build
    process via the Gulp or Grunt
    plugins

    View Slide

  146. View Slide

  147. Tools
    psi
    ngrok

    View Slide

  148. ngrok
    ● Creates a tunnel to localhost
    ● Request inspection

    View Slide

  149. View Slide

  150. @rposbo
    @deanohume

    View Slide

  151. Thank you

    View Slide