Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Good, The Bad, and The Ugly of the HTTP Archive

Dean Hume
September 17, 2014

The Good, The Bad, and The Ugly of the HTTP Archive

Presented at Velocity Conference, September 2014.

The web is a fascinating place. Every day millions of websites around the world are constantly evolving and growing. Fortunately, great resources such as the HTTP archive capture statistics from over 300K URLS, and provide us with useful insights into the world of web performance.

http://deanhume.com
http://robinosborne.co.uk

Dean Hume

September 17, 2014
Tweet

More Decks by Dean Hume

Other Decks in Technology

Transcript

  1. The Good, the
    Bad, and the
    Ugly
    ....of the HTTP Archive

    View full-size slide

  2. Robin
    Osborne

    View full-size slide

  3. HTTP Archive
    ● Started in November 2011
    ● Contains over 300K URLs
    ● Both Mobile and Desktop sites

    View full-size slide

  4. HTTP Archive
    ● Download as CSV or MySql archive
    ● Page Data
    ● Request Data

    View full-size slide

  5. Great on its own..
    HTTP Archive

    View full-size slide

  6. +
    Google
    BigQuery
    HTTP Archive

    View full-size slide

  7. SELECT AVG(PageSpeed) as AvgPageSpeed
    FROM [httparchive:runs.2014_07_15_pages]

    View full-size slide

  8. http://bigqueri.es

    View full-size slide

  9. Good BAD
    UGLY

    View full-size slide

  10. The Criteria

    View full-size slide

  11. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list

    View full-size slide

  12. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern

    View full-size slide

  13. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modernmodern(ish)

    View full-size slide

  14. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern
    ● Follows best practices
    modern(ish)

    View full-size slide

  15. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern
    ● Follows best practices
    ● Not a porn site
    modern(ish)

    View full-size slide

  16. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern
    ● Follows best practices
    ● Not a porn site (sorry)
    modern(ish)

    View full-size slide

  17. The Criteria
    ● The Alexa top 100 sites have been
    excluded from this list
    ● The site needs to be usable and
    modern
    ● Follows best practices
    ● Not a porn site (sorry)
    ● Bonus points for a responsive site
    modern(ish)

    View full-size slide

  18. BEWARE!
    Here be dragons!

    View full-size slide

  19. Key takeaway
    ● Super fast holding pages
    Http Archive + Big Query is a great
    way to find:

    View full-size slide

  20. Key takeaway
    ● Super fast holding pages
    ● Super fast porn sites
    Http Archive + Big Query is a great
    way to find:

    View full-size slide

  21. How we measured

    View full-size slide

  22. How we measured
    Fully Loaded

    View full-size slide

  23. Fully
    loaded
    A time measurement that
    includes any activity that is
    triggered by JavaScript after
    the main page loads.

    View full-size slide

  24. How we measured
    Page Speed
    Fully Loaded

    View full-size slide

  25. Page
    Speed
    The PageSpeed Score (0-
    100) is calculated on a
    number of factors.
    A high score is good.
    A low score is not so good.

    View full-size slide

  26. How we measured
    Speed Index
    Page Speed
    Fully Loaded

    View full-size slide

  27. Speed
    Index
    Speed Index measures how
    complete a page is during
    various points in page load
    A low score is good.
    A high score is not so good.

    View full-size slide

  28. SELECT url, pagespeed, SpeedIndex,
    fullyLoaded, rank
    FROM [httparchive:runs.latest_pages]
    WHERE (rank is not null) and (speedindex is
    not null) and pagespeed > 80
    ORDER BY rank asc, pagespeed desc,
    speedindex asc limit 1000 ;

    View full-size slide

  29. Filament Group

    View full-size slide

  30. filament group
    ● 8-person development studio based all
    around the US
    ● Develop a wide range of sites and apps
    ● Responsive website

    View full-size slide

  31. 100 Page Speed
    A Grade YSlow

    View full-size slide

  32. What is your SECRET?

    View full-size slide

  33. “How we load assets matters
    just as much as how many
    assets we’re loading. We
    focus on perceived
    performance”

    View full-size slide

  34. Reduce the
    Critical Path

    View full-size slide

  35. tools
    Grunt-CriticalCSS

    View full-size slide

  36. tools
    LoadCss
    Grunt-CriticalCSS

    View full-size slide

  37. LoadJS
    LoadCss
    Grunt-CriticalCSS
    tools

    View full-size slide

  38. KEY TAKEAWAYS

    View full-size slide

  39. KEY TAKEAWAYS
    REDUCE THE CRITICAL PATH

    View full-size slide

  40. KEY TAKEAWAYS
    REDUCE THE CRITICAL PATH
    FOCUS ON PERCEIVED
    PERFORMANCE

    View full-size slide

  41. nature.com
    ● Scientific journal for research across
    a wide range of fields
    ● Part of the Macmillan publishing
    group
    ● Responsive website
    ● Based in London, UK

    View full-size slide

  42. 86 Page Speed
    A Grade YSlow

    View full-size slide

  43. “ We try to stick to the 14
    rules as outlined in
    High Performance
    Web Sites ”

    View full-size slide

  44. “Monitoring is key for us”

    View full-size slide

  45. Showslow.com
    “Monitoring is key for us”

    View full-size slide

  46. StatsD & Graphite
    Showslow.com
    “Monitoring is key for us”

    View full-size slide

  47. KEY TAKEAWAYS

    View full-size slide

  48. KEY TAKEAWAYS
    STICK TO BASICS

    View full-size slide

  49. KEY TAKEAWAYS
    STICK TO BASICS
    MONITOR

    View full-size slide

  50. Online restaurant guide -
    Based in New Delhi, India -
    13 countries in 5 languages -
    Responsive site -
    zomato.xxx -

    View full-size slide

  51. 88 Page Speed
    C Grade YSlow

    View full-size slide

  52. What is your SECRET?

    View full-size slide

  53. “A blank page is always
    going to be the fastest,
    start from there. Add only
    what is needed.”

    View full-size slide

  54. Heavy use of caching

    View full-size slide

  55. KEY TAKEAWAYS

    View full-size slide

  56. KEY TAKEAWAYS
    START SMALL

    View full-size slide

  57. KEY TAKEAWAYS
    START SMALL
    CACHE BIG

    View full-size slide

  58. Envato
    Marketplaces

    View full-size slide

  59. envato marketplaces
    ➔ Specializes in online
    marketplaces
    ➔ Based in Melbourne, Australia
    ➔ Themeforest, CodeCanyon

    View full-size slide

  60. 85 Page Speed
    B Grade YSlow

    View full-size slide

  61. What is your SECRET?

    View full-size slide

  62. “Stick to basics and aim
    for low hanging fruit.”

    View full-size slide

  63. “ We think about
    performance from the
    earliest design stage. ”

    View full-size slide

  64. If users add their own images
    you
    need a performance strategy

    View full-size slide

  65. KEY TAKEAWAYS

    View full-size slide

  66. KEY TAKEAWAYS
    CONSIDER USER
    GENERATED CONTENT

    View full-size slide

  67. KEY TAKEAWAYS
    CONSIDER USER
    GENERATED CONTENT
    STICK TO BASICS

    View full-size slide

  68. SELECT url, visualComplete, fullyLoaded,
    reqTotal, bytesTotal,numdomelements
    FROM [httparchive:runs.latest_pages] as
    pages
    ORDER BY visualcomplete DESC, fullyloaded
    DESC;

    View full-size slide

  69. welovefashion.it

    View full-size slide

  70. welovefashion.it
    ➔ Visually complete in 20 secs

    View full-size slide

  71. welovefashion.it
    ➔ Visually complete in 20 secs
    ➔ Fully loaded in 30 secs

    View full-size slide

  72. welovefashion.it
    ➔ Visually complete in 20 secs
    ➔ Fully loaded in 30 secs
    ➔ 3.5 MB of resources are
    downloaded

    View full-size slide

  73. One simple
    change

    View full-size slide

  74. Enable
    Compression

    View full-size slide

  75. gu-japan.com

    View full-size slide

  76. gu-japan.com
    ➔ 15.5 MB of images are loaded

    View full-size slide

  77. gu-japan.com
    ➔ 15.5 MB of images are loaded
    ➔ Visually complete in 16 secs

    View full-size slide

  78. gu-japan.com
    ➔ 15.5 MB of images are loaded
    ➔ Visually complete in 16 secs
    ➔ Fully loaded in 35 secs

    View full-size slide

  79. gu-japan.com
    ➔ 15.5 MB of images are loaded
    ➔ Visually complete in 16 secs
    ➔ Fully loaded in 35 secs
    ➔ 717 HTTP requests

    View full-size slide

  80. One simple
    change

    View full-size slide

  81. Lose the
    carousels

    View full-size slide

  82. shouldiuseacarousel.com

    View full-size slide

  83. gamepedia.com

    View full-size slide

  84. 10MB of images
    gamepedia.com

    View full-size slide

  85. 10MB of images
    130 Resources
    gamepedia.com

    View full-size slide

  86. 10MB of images
    130 Resources
    Fully loaded in 15 secs
    gamepedia.com

    View full-size slide

  87. One change?
    Serve scaled
    images

    View full-size slide

  88. SELECT url, visualComplete, fullyLoaded,
    reqTotal, bytesTotal,numdomelements
    FROM [httparchive:runs.latest_pages] as
    pages
    ORDER BY visualcomplete DESC, fullyloaded
    DESC;

    View full-size slide

  89. sailboatlistings.com
    ● Fully loaded in 18 secs

    View full-size slide

  90. sailboatlistings.com
    ● Fully loaded in 18 secs
    ● Visually complete in 20 secs

    View full-size slide

  91. sailboatlistings.com
    ● Fully loaded in 18 secs
    ● Visually complete in 20 secs
    ● 3 MB total

    View full-size slide

  92. sailboatlistings.com
    ● Fully loaded in 18 secs
    ● Visually complete in 20 secs
    ● 3 MB total
    ● 580 requests in total

    View full-size slide

  93. sailboatlistings.com
    ● Fully loaded in 18 secs
    ● Visually complete in 20 secs
    ● 3 MB total
    ● 580 requests in total
    ● 535 image requests

    View full-size slide

  94. sailboatlistings.com
    ● Fully loaded in 18 secs
    ● Visually complete in 20 secs
    ● 3 MB total
    ● 580 requests in total
    ● 535 image requests
    ● 14872 dom elements

    View full-size slide

  95. colorsbycherry.com

    View full-size slide

  96. colorsbycherry.com
    Visually complete in 30 secs

    View full-size slide

  97. colorsbycherry.com
    Fully loaded in 1 min
    Visually complete in 30 secs

    View full-size slide

  98. colorsbycherry.com
    34 MB of resources
    Fully loaded in 1 min
    Visually complete in 30 secs

    View full-size slide

  99. colorsbycherry.com
    34 MB of resources
    Fully loaded in 1 min
    Visually complete in 30 secs
    say what!?

    View full-size slide

  100. Just.
    Plain.
    Rude.

    View full-size slide

  101. callofduty.com

    View full-size slide

  102. callofduty.com
    15.2 MB of resources
    callofduty.com

    View full-size slide

  103. callofduty.com
    15.2 MB of resources
    Visually complete in 1.3 secs
    callofduty.com

    View full-size slide

  104. callofduty.com
    15.2 MB of resources
    Visually complete in 1.3 secs
    Fully loaded in 13 secs
    callofduty.com

    View full-size slide

  105. callofduty.com
    15.2 MB of resources
    Visually complete in 1.3 secs
    Fully loaded in 13 secs
    Page Speed score of 11
    callofduty.com

    View full-size slide

  106. DO THIS
    Stick to the basics
    Monitor
    Start with the minimum

    View full-size slide

  107. don’t do this
    Serve unscaled images
    Use bloated carousels
    Forget compression

    View full-size slide

  108. definitely don’t do this
    Serve humungous pages
    Use non-streaming videos
    Serve unoptimized images

    View full-size slide

  109. psi
    ● Page Speed Insights
    ● Get PSI reporting from the
    command line
    ● On GitHub, by Addy Osmani
    ● Can be hooked into a build
    process via the Gulp or Grunt
    plugins

    View full-size slide

  110. Tools
    psi
    ngrok

    View full-size slide

  111. ngrok
    ● Creates a tunnel to localhost
    ● Request inspection

    View full-size slide

  112. @rposbo
    @deanohume

    View full-size slide