Building Upworthy on Rails

Transcript

1. Building Upworthy on Rails Luigi Ray-Montanez, Founding Engineer Connect JS October 2015
2. Luigi Ray-Montanez Founding Engineer @1uigi Civic Hack Night: Oct. 27 meetup.com/codeforatlanta
3. Mission: To drive massive amounts of attention to the most important stories in the world.
5. “massive amounts of attention” really means “massive amounts of traffic”
6. Monthly Uniques
7. 24 Hour Traffic Graph
9. Today’s topic: Managing the growth of a startup’s web app in the face of very high traffic* * Yes, it can happen to you!
10. In the beginning…
11. February 2012 June 2014 Initial commit April 2013 Launch! October 2015
12. “The elegant Ruby web framework”
13. What is Padrino? • Sinatra++ • Stole great ideas from Django • First-class mountable apps • Middleware-centric • Built-in Admin area
14. Why Padrino?!
15. All web frameworks fundamentally exist to: Emit HTML, CSS, JSON, & JavaScript
16. Get to know HTTP
17. Padrino: The Good Parts • It’s just Ruby. It’s just Rack. • Less magic. • Unopinionated. • Writing middleware is fun! • It’s light and performant.
18. Padrino: Architecture Main (Public) Admin (CMS) RedisCache Content Request Publisher
19. Initial commit 2nd Rails engineer February 2012 June 2014 April 2013 October 2015
20. Padrino Pain Points • Ecosystem for libraries not as strong as Rails. • Infrequently maintained.* • Admin system was ugly.* • No real community. • Easier to hire Rails developers than Padrino developers.
21. The Big Move
22. Initial commit Create Rails app, mount Padrino inside February 2012 June 2014 April 2013 October 2015
23. Padrino Rails 1. Generate Rails app. Stick Padrino app in lib/upworthy & mount it in routes.rb. 2. Migrate models and utilities. 3. Migrate assets. 4. Migrate front-end views and controllers. 5. Migrate CMS views to Bootstrap and use Rails controllers.
24. Initial commit Create Rails
 app Delete final Padrino code February 2012 June 2014 April 2013 October 2015
25. Rails: Architecture
26. Scaling The Monorail, Phase 1 • Action caching for content pages, backed by memcached. • Assets on S3 & Cloudfront. • Lots of Heroku dynos (AJAX requests). Up to 40 during traffic spikes.
27. Scaling The Monorail, Phase 2 • Move all HTML, CSS, JS, & images to Fastly CDN. • Turn off Rails action caching. Manually set cache headers. • Dial down Heroku dynos. Only needed for AJAX requests.
29. Monorail Pain Points • One Rails app that dealt with both front-end and back-end concerns. • Traffic spikes on public site could render CMS unusable. • Codebase becomes very large. God objects starting to appear.
30. The Breakup
31. Start on Padrino Create Rails
 app Delete final Padrino code February 2012 June 2014 April 2013 October 2015 Split up Rails monolith
32. Monorail Two Apps 1. Decide on how many apps to split into. We chose two: www and cms. 2. Clone monorail into separate git repos to maintain history. 3. Split up controllers, views, assets and concerns.
33. Monorail Two Apps 4. Deploy each app to separate Heroku applications. 5. Switch Fastly to point at new www app. Resulted in zero downtime. 6. De-duplicate code between the two apps, creating a core gem (mostly models) to keep code DRY.
34. Multi-App Architecture Fastly www core cms Public Visitor CMS User
35. Multi-App Benefits • Instability in one app doesn’t affect the other app. • Apps have different scaling needs. Automated scaling based on CPU usage. • Divides teamwork more naturally. Less blocking on features.
36. Multi-App Pain Points • Must run N apps in development to exercise the full system. • Commits across three different codebases (two Rails apps, one gem). • Specs run on three different codebases. • Coordinate versioning on the shared core gem. • Coordinate deploys amongst N apps. • Dependencies would get out of sync.
37. What We Wanted • Separately deployed apps • The ease of development of a single codebase • Clean separation of concerns
38. Rails Engines! • A single container codebase • Mount engines for each app that runs • Shared models and other classes in the container app • github.com/taskrabbit/rails_engines_example
39. The Big Merge
40. Merging Back To One App • Create a container Rails app: “umbrella” • Convert each existing app (www and cms) and the gem (core) into its own engine. First in its own codebase, then copy it over to umbrella with git subtrees. • Combine asset building and test suites
41. Engines Architecture engines/www/ engines/cms/ app/ Audience Staff
42. Benefits of Engines • Single codebase. No shared gem to manage. • Each engine runs as its own app in production, but can both can be run together in one app for development. • When a small feature that involves both apps is built, that can be expressed in a single commit instead of three commits in three different codebases. • Specs can all be run at once.
43. Start on Padrino Create Rails
 app February 2012 June 2014 April 2013 October 2015 Split up Rails monolith Merge Engines into Rails container
44. About JavaScript… • 2012: Start as jQuery Spaghetti • 2013: Organize as CommonJS modules • 2014: More functional with Lo-Dash and Bacon • 2015: Reactify everything!
45. Lessons Learned • New teammates can identify pain points without emotional attachment. • Wait to make changes to your technical architecture until it really hurts. • It’s okay to take a really long time to migrate to the better solution. • Serve everything you possibly can from a CDN. • Remember: it’s just HTML, CSS, JavaScript, and JSON. Learn HTTP.
46. Questions? Luigi Ray-Montanez // @1uigi Slides & writeup: upworthy.github.io