You'll Never Believe Which Web Framework Powers Upworthy

Given at RailsConf 2014.

Transcript

1. You’ll Never Believe Which Web Framework Powers Upworthy Luigi Montanez, Founding Engineer Ryan Resella, Senior Engineer ! RailsConf 2014
3. Luigi Montanez Founding Engineer @1uigi
4. Ryan Resella Senior Engineer @ryanresella
5. Mission: To drive massive amounts of attention to the topics that matter most.
6. “topics that matter most”
7. “massive amounts of attention” really means “massive amounts of traffic”
8. Monthly Uniques
9. Quantcast Ranking
10. 24 Hour Traffic Graph
15. Our topic: Managing the growth of a startup’s web app in the face of very high traffic* * Yes, it can happen to you!
16. In the beginning…
17. February 2012 April 2014 Initial commit March 2013 Launch!
18. “The elegant web framework”
19. What is Padrino? • Sinatra++ • Stole great ideas from Django • First-class mountable apps • Middleware-centric • Built-in Admin area
20. Why Padrino?!
21. All web frameworks fundamentally exist to: Emit HTML, CSS, JSON, & JavaScript
22. Padrino: The Good Parts • It’s just Ruby. It’s just Rack. • Less magic. • Unopinionated. • Writing middleware is fun! • It’s light and performant.
23. Padrino: Architecture Main (Public) Admin (CMS) RedisCache Content Request Publisher
24. February 2012 April 2014 Initial commit March 2013 2nd Rails engineer
25. Padrino Pain Points • Ecosystem for libraries not as strong as Rails. • Infrequently maintained.* • Admin system was ugly.* • No real community. • Easier to hire Rails developers than Padrino developers.
26. The Big Move
27. February 2012 April 2014 Initial commit March 2013 Create Rails app, mount Padrino inside
28. Padrino Rails 1. Generate Rails app. Stick Padrino app in lib/upworthy & mount it in routes.rb. 2. Migrate models and utilities. 3. Migrate assets. 4. Migrate front-end views and controllers. 5. Migrate CMS views to Bootstrap and use Rails controllers.
29. February 2012 April 2014 Initial commit March 2013 Create Rails app, mount Padrino inside Delete final Padrino code
30. Rails: Architecture
31. Scaling The Monorail, Phase 1 • Action caching for content pages, backed by memcached. • Assets on S3 & Cloudfront. • Lots of Heroku dynos (AJAX requests). Up to 40 during traffic spikes.
32. Scaling The Monorail, Phase 2 • Move all HTML, CSS, JS, & images to Fastly CDN. • Turn off Rails action caching. Manually set cache headers. • Dial down Heroku dynos. Only needed for AJAX requests. (We currently use 2X dynos.)
34. Monorail Pain Points • One Rails app that dealt with both front-end and back-end concerns. • Traffic spikes on public site could render CMS unusable. • Codebase becomes very large. God objects starting to appear.
35. The Breakup
36. February 2012 April 2014 Initial commit March 2013 Create Rails app, mount Padrino inside Delete final Padrino code Split up Rails monolith
37. Monorail Services 1. Decide on how many services to split into. We chose two: www and cms. 2. Clone monorail into separate git repos to maintain history. 3. Split up controllers, views, assets and concerns.
38. Monorail Services ! ! ! 4. Deploy each app to separate Heroku applications. 5. Switch Fastly to point at new www app. Resulted in zero downtime. 6. De-duplicate code between the two apps, creating a core gem (mostly models) to keep code DRY.
39. Service-Oriented Architecture Fastly www core cms Public Visitor ! CMS User
40. SOA Benefits • Instability in one app doesn’t affect the other app. • Apps have different scaling needs. • Divides teamwork more naturally. Less blocking on features.
41. SOA Drawbacks • Must run N apps in development to exercise the full system. • Coordinate deploys amongst N apps. • Migrating to a fully DRY set of codebases is tedious. • “What goes where?”
42. Future Work • Continue adding code to core gem and removing duplication from the apps. • Consider breaking up apps even more. For example, www can be split into a static site generator and an AJAX API service. • Apps communicate by sharing data stores. Should they communicate via RESTful APIs instead?
43. Lessons Learned • Wait to make changes to your technical architecture until it really hurts. • It’s okay to take a really long time to migrate to the better technology. • Serve everything you possibly can from a CDN. • Remember: it’s just HTML, CSS, JavaScript, and JSON. • It can happen to you!
44. Questions? Luigi Montanez // @1uigi Ryan Resella // @ryanresella ! Slides & writeup: upworthy.github.io The answers will surprise you.