Web Performance at The New York Times

Front end software architect Eitan Konigsburg talked about the history of NYTimes.com and The Times’ own efforts to improve performance in light of a new opportunity to overhaul the website.

This talk was presented at TimesOpen in October 2013. Watch video of the talk.

Transcript

1. WEB PERFORMANCE AT THE NEW YORK TIMES Eitan Konigsburg Front End Software Architect @eitanmk
2. Is performance of "the Web" getting better or worse? My past week was painfully slow marred by top sites w/ bad . Is it too hard? — Steve Souders (@Souders) #WebPerf September 30, 2013
3. IN A WORLD (OF STATIC PAGES)...
4. HTML: START THE PRESSES. Pages are published, just like a newspaper Every page on the site was run through templates and written to disk Mixed with a proprietary compiled template language for dynamicism And it was fast!
5. HTML: PUBLISH ALL THE THINGS! There are over 1M static pages since 2004 Re-publishing is not an option Extremely hard to change the markup once published Every markup change is present from deploy date onward Unknown number of permutations of markup variations increasing after each deploy JS and CSS files grow over time to accommodate older and newer markup Low cache TTL on JS and CSS in order to be able to update the same path Don't know which pages call which assets
6. CSS: TRY THIS ON. There was no CSS framework Extremely repetitive styles Files @import'ed haphazardly as needed Some stylesheets approached the @import limits in old IE Inconsistent CSS load style - <link> vs <style>@import url(...) Styles input into CMS and dropped anywhere on the page
7. JS: GIVE THIS A WHIRL. Ad hoc JS development Most dynamicism done server side in compiled templates JS used as needed, for fixing cross-browser issues, old markup, or widgets that handle browser events (tabbed modules etc) Most variables and functions in global namespace <script> tags dropped in where needed, no deferred execution No DOM framework to help navigate cross-browser functionality
8. 1000 WORDS ON IMAGES UI images uploaded individually to production - don't forget them when you deploy! Content photographs extremely hard to optimize Pride in our photographs, resistance to modifying size and quality Experiments with lossless compression caused noticeable color changes Manual process to crop images to avoid clipping important objects in the picture
9. BUSINESS: KEEP THE LIGHTS ON. Other challenges Editorial constraints Producers have a need to adjust styles and modify behavior as news happens "Free-form" modules used to drop adjustments to the page that can modify anything Ads, of course Top-level objects in the DOM and block loading Bring their own copies of DOM frameworks Animating ads start as soon as they load - layouts, reflows, and repaints oh my. On the homepage, those ads at the top are before our own logo in the DOM!
10. OK, IT WASN'T ALL BAD... We use a CDN! We enabled gzip in 2007
11. A NEW WIND WAS ABOUT TO BLOW...
12. 2009: TOOLS AND LIBRARIES New proprietary build system for front-end files Our first foray into automated builds The end of manually un-tar'ing files Pulled changed files from version control Semi-stateful rollbacks Hooks for build scripts
13. 2009: TOOLS AND LIBRARIES Impoved CSS build system First system to use the new build system hooks Wrote a proprietary script to unravel @import's and concatenate files Wrote a proprietary script to minify CSS Downstream from development work
14. 2009: TOOLS AND LIBRARIES Created an organized CSS framework Standardized design concepts into base files Split the code into reusable modules Aimed to have a single CSS include at the top of the page
15. 2009: NEW TOOLS AND LIBRARIES Started to lock down JS development Introduction of the NYTD namespace and the module pattern Introduced PrototypeJS as a DOM library Attempted to write a proprietary async JS loader
16. 2009: TOOLS AND LIBRARIES Started with the Homepage It is published frequently, so legacy markup isn't an issue Gets the most traffic, so most user savings over time Isolated implementation, so unlikely to affect other pages Viewed very heavily internally - more eyes to find any issues
17. 2009: TOOLS AND LIBRARIES Nov. 2009 CSS optimizations launched to the Homepage Saved 25 HTTP connections to small CSS files First paint occurred 1s earlier, Homepage felt faster Woot!
18. 2010: INCREMENTAL IMPROVEMENT CSS framework rollout continued to sections and articles Single entry point Files never optimized - concerns with the proprietary optimization scripts 2012 audit revealed we were right to be concerned References to missing files @import cycles: A -> B -> C -> A Syntax errors
19. 2010: INCREMENTAL IMPROVEMENT New JS build system Attempted to fork the codebase to create a clean build system Concatenation achieved using a CSS @import-like syntax inside JS comments Minification using YUI Compressor Focus on two build files - top.js and bottom.js Executed manually Didn't see wide-spread usage Now we have two codebases and no automatic build process
20. 2010: INCREMENTAL IMPROVEMENT Results were pretty drastic on the homepage 50% speedup in load event - 6 seconds down from 12 Woot!
21. 2011: A SLOW YEAR. Digital Subscriptions Resourcing for WPO work was hard to come by Aborted efforts to try an optimize analytics packages - no existing regression tests Code reviews and style guides to keep the status quo
22. 2011: A SLOW YEAR. Fall 2011: The Decision s/PrototypeJS/jQuery/g REALLY HARD Both would coexist during attempted transition
23. 2012: SERVICE ENGINE SOON A group of 5 developers set out to excise PrototypeJS from article code We were making excellent progress until... Inline scripts that used PrototypeJS functions Remember static pages? Remember deployment to same URLs? Unable to remove PrototypeJS fully without breaking things Now we have two frameworks, both still in use This causes weird bugs - PrototypeJS shims missing DOM methods that jQuery will attempt to use if present...
24. 2012: SERVICE ENGINE SOON End of Life for proprietary build system Migrated to full repository mirroring via Hudson Wrote new scripts to concatenate CSS imports Handles import cycles Can skip (and warn us of) missing files Can handle relative import paths Runs CSSLint to find syntax and other errors Uses YUI Compressor to minify Has to be run manually
25. PAYBACK. THIS TIME, IT'S FOR REAL.
26. A RARE OPPORTUNITY. March 2013, NYT unveils article redesign More than just a new UI and UX Technological reboot A chance to fix some legacy issues
27. DYNAMIC PAGES! No more unknown permutations of pages Can now change asset URLs to instantly bypass cache when needed Pushed most user customization client-side to improve cacheability HTML5 Boilerplate Modernizr
28. MODERN BUILD SYSTEM. New unified build system - Grunt Can be run in developer sandboxes and in Hudson Most WPO best-practices available as Grunt tasks Creates RPM artifacts for deployments
29. LESS IS MORE. Switched to using LESS for style development Handles concatenation and minification Variables and mixins allow us to do more things we couldn't do before Ex: arrows, buttons, data URIs etc. And sprites...
30. SPRITE ME. Wrote a Grunt task to automate the creation of spritesheets Task takes in a list of images and exports the spritesheet and LESS mixins with CSS background coordinates Easy to make combined sprite-sheets Easy for developers to get them in the code If the coordinates change, there is no dev work needed to update the LESS Currently combining over 100 small images into one spritesheet
31. DELETE WINDOW.NYTD Switched to using RequireJS for JS development Handles concatenation and minification - and source maps No longer a need for a global namespace Async loading of JS Focused on larger build files to avoid latency on multiple connections
32. FAST TEMPLATES Underscore JS Templates used instead of DOM building or HTML strings Way faster to precompile the templates than to leave them as strings Grunt automates the compilation Compiled templates are turned into a RequireJS module for inclusion in the build file
33. MODULES Use Backbone to organize our frontend modules Reusable and shared modules Inherited functionality - touch events, analytics, common element references Event delegation PubSub communication system between modules
34. NON-BLOCKING ADS Took ads out of the critical path Write ads into new iframe documents on DOM ready Don't lose the document if an ad calls document.write Don't care if they bust out of the iframe once the page is loaded "Trusted" ads are not framed "Trusted" means they don't do anything bad for page speed Also that we can easily change implementation if there's a problem This has been really effective for us
35. THE RESULTS: Requests before start render: ~60 → ~10 Requests before fully loaded: 200+ → ~70 Start render is under 1s Document complete is under 3s
36. SO, IS #WEBPERF TOO HARD?
37. NO. AND YES. No. The optimization practices aren't that hard Excellent tools exist now And yes. Big websites may have unique challenges to deal with Not to mention legacy decisions that are hard to change Negative feedback cycle: bad setup -> proprietary fix -> worse setup
38. WHAT WE LEARNED Fight to solve the big, difficult problems Don't have to rely on proprietary solutions as much anymore Big, easy WPO wins first Automate everything
39. MISCHIEF MANAGED. THANKS!