Growing X20 without spending an extra penny on hosting

A year ago we started a big change in FTBpro. We completely changed the visual design, moved to a single page architecture and started exploring new ways to minimise load on our servers - both when serving our actual website, and mobile API responses.We'll focus on how scaling considerations are now an integral part of our architecture, which enabled us to serve 20x more traffic than we did 1 year ago, with the same setup and with no additional costs.

Transcript

1. Growing X20 without spending an extra penny on hosting Dor Kalev, CTO @ FTBpro
2. Largest fan-generated media platform in global football 130m pageviews / month
3. a Football fan #musthave 1M+ mobile installs
4. Content is king
5. Single Page Application Phantom.js renders pages on the server-side for SEO, crawlers and mobile web performance Load Balancer Memcached Redis Web 1 Web 2 Web 3 MySQL MongoDB ElasticSearch Amazon Web Services NginX + Ruby on Rails PhantomJS Web 4
6. How do we scale post pages?
7. Each Render Server can return 1 client request at a time 1:1 Client Server
 Render
8. Linear correlation between clients and servers 1:1 Client Server
 Render Client Server
 Render
9. A cache server can handle over x10 more traffic 10:1:1 Cache Server Render Client Client Client Client Client Client Client Client Client Client Client Client
10. Cache server still consumes CPU and with more traffic, more servers are needed 20:2:1 Cache Server Render Cache Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client
11. Accelerator like NGINX+MemCached or Varnish can be x100 more efficient than plain App Server Cache 100:1:1 Server Render Accelerator Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client https://www.igvita.com/2008/02/11/nginx-and-memcached-a-400-boost/ https://www.varnish-cache.org
12. 200:2:1 Server Render Accelerator Accelerator Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Accelerator also runs on limited hardware and needs more CPU in order to scale
13. Enter CDN ∞:1 Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client CDN Server Render
14. Client Client My Tiny Server PoP CDN SERVER CDN SERVER CDN SERVER PoP CDN SERVER CDN SERVER CDN SERVER PoP CDN SERVER CDN SERVER CDN SERVER Client Client Client Client Client Client Client What’s a CDN?
15. What’s a CDN? • Content Delivery Network • A set of autonomic distributed cache servers • Serve cache from a location geographically close to the client • e.g: Akamai, Cloudflare, Fastly, Highwinds, Edgecast, Cloudfront
16. Enter Trouble ∞:1 Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client Client CDN Server Render
17. Fixing minor Typo?
 Need to Refresh Cache??
18. TTL vs. ON CHANGE Cache Policy
19. Purge by URL can take over an hour on classic CDNs Z Z Z
20. Purge by TTL - isn’t realtime & causes redundant server hits CPU LOAD 0 7.5 15 22.5 30 TIME 1 2 3 4 5 (TTL) 6 7 8 9 (TTL) 10 11 12
21. multiply by all articles CPU LOAD 0 7.5 15 22.5 30 TIME 1 2 3 4 5 (TTL) 6 7 8 9 (TTL) 10 11 12
22. TTL = Constant waste of CPU CPU LOAD 0 7.5 15 22.5 30 TIME 1 2 3 4 5 6 7 8 9 10 11 12
23. Better Solution?
24. abuse new CDN technologies Super Fast Purge + Smart API! ! ! ! ! ! e.g. Fastly, EdgeCast
25. cache forever +
 purge on demand 0 7.5 15 22.5 30 1 (release) 2 3 4 5 6 7 8 9 (purge) 10 11 12
26. fast purge at ~10-45 seconds
27. Advantages • ∞ scaling • cheap • fast • keep the stack simple
28. Caveats • Purging everything can lead to downtime, e.g. when deploying • User Specific content can’t be cached with the entire page • Downtime on massive Push Notifications
29. Massive Push Notification + cold CDN Cache = traffic goes to app servers CPU LOAD 0 250 500 750 1000 TIME 1 (PN) 2 3 4 5 6 7 8 9 10 11 12
30. Client Client My Tiny Server PoP CDN SERVER CDN SERVER CDN SERVER PoP CDN SERVER CDN SERVER CDN SERVER PoP CDN SERVER CDN SERVER CDN SERVER Client Client Client Client Client Client Client Render (or cache & serve) X number of calls
 in parallel = self-DDoS
31. Client Client CACHE PoP CDN SERVER CDN SERVER CDN SERVER PoP CDN SERVER CDN SERVER CDN SERVER PoP CDN SERVER CDN SERVER CDN SERVER Client Client Client Client Client Client Client Solution: Another local cache layer & Preload CDN
 (few CDNs have this feature, e.g. EdgeCast) My Tiny Server
32. CPU vs TRAFFIC • 13:53 preloaded post to CDN • 13:55 PN sent • 13:57+ traffic stops @ CDN servers, no CPU throughput • 14:00 …
33. 14:00 Laugh a maniacal laugh of satisfaction
34. Summary • CDN allows massive scale • Smart CDN usage allows even more scale • Scale + fresh content + low hosting prices is a do-able mission ! Questions?
35. Read More… • Growing X20 without spending an extra penny on hosting (The original blog post)
 http://tech.ftbpro.com/post/78969626647/growing-x20-without-spending-an-extra-penny-on-hosting • Single Page Applications Done Right
 http://tech.ftbpro.com/post/84720180206/single-page-applications-done-right • Count von Count - Real time counting DB
 http://tech.ftbpro.com/post/73212710968/count-von-count-a-real-time-counting-database • The Dark Side of Single Page Applications
 http://tech.ftbpro.com/post/66182501420/the-dark-side-of-single-page-applications-rails OSS: http://github.com/ftbpro
 BLOG: http://tech.ftbpro.com