The fundamentals of scaling rails applications

B7d890bed68fa564c18ff00dfd8207cd?s=47 Avi Tzurel
December 18, 2013

The fundamentals of scaling rails applications

8 hours workshop I taught at the Google Campus in Israel.

B7d890bed68fa564c18ff00dfd8207cd?s=128

Avi Tzurel

December 18, 2013
Tweet

Transcript

  1. 1.

    FullStack Developers Israel Hosted by: !1 Fundamentals of scaling rails

    application ! With real world examples 15.12.2013! Google
  2. 5.

    text • Scaling a monolithic rails application • Caching •

    Key Value stores (Focusing on Redis) • Search indexing with NoSql enhancements (Focusing on SOLR) Agenda
  3. 6.

    text • Slides -> talk -> questions • Code together

    (moving at a normal pace) • Difficulties -> join someone else • More slides, more code, same process goes on How is this going to work?
  4. 7.

    text • Computer • Ruby • Vagrant • Virtual Box

    • Don’t have something? Please join someone else Prerequisites
  5. 8.

    text • Chef • Vagrant • Chef server • Redis

    • Memcached • SOLR (3.6) • Offloading from user thread • Micro services • Ruby & Rails (Obviously!) Tools & Concepts
  6. 9.

    Configuration management like a boss! Chef • Chef is a

    part of the “scaler” daily life • Easily (well, that’s a big fucking fat lie) manage and configure server • Chef-server with chef-client are a devops dream • Your code is your server documentation • Ruby!!!!!
  7. 10.

    up, hack, damn it!, destroy Vagrant • Cost effective way

    to “play” around with your environment configurations • Working with the same exact images as you would on your production env • Having a production env on your own machine is great for bug catching (princess env)
  8. 11.

    workshop-awesome Redis • Super fast key-value store on steroids •

    Has sets support, sorted sets, arrays and more • Not full ACID • If you don’t turn on journaling, redis may lose data on writes • Love to hate and lean the fsync() process
  9. 12.

    Date=12.12.2012 Memcached / Membase • Super fast memory (with disk

    backup if it’s membase) store • Storing values in binary • Simple yet effective API for what you need • Still running tons of caching solutions at all sizes of companies from your average 2-3 person startup to Facebook • Absolutely great for the average web app that has 80% reads with 20% writes
  10. 13.

    go look if i’m there Solr • Lucene based search

    index • Fully blown NoSql solution • The most mature solution out there • Can run on almost any Java server, Jetty, Tomcat (we’ll be running jetty) • The most misunderstood solution out there, people under-use it, we will learn how NOT to do it
  11. 15.
  12. 17.
  13. 18.

    text • Genuine full stack development • Chef for configuration

    (chef server) • Caching layers (CDN, object cache, page cache) • Key Value storage • Messaging layers • Micro Services written in other languages • Multiple monitoring systems, multiple logs The modern scaled rails application
  14. 20.

    “ “ Scaling - Replacing all components of a car

    while driving it at 100mph ! Mike Krieger - Instagram
  15. 22.

    text Setting out to scale an app What do you

    want to achieve? • Highly available application • Great performance both for api users and app users • Easily manageable and configureable • Cost effective
  16. 23.

    text Mixing up concepts • Scaling is NOT optimization /

    hardware / storate • Not about less lines of code or specific piece of software • We are going to talk about DESIGNING for scale, not actually scaling • If you don’t design to scale, you won’t scale • If you design to scale, scaling will be painful • There is NO pain free scaling
  17. 25.
  18. 32.

    We are going to add & scale features EXACTLY like

    we would do in real like ! Well, not “we” but regular developers :)
  19. 38.
  20. 39.

    Basics • Base class is ActiveSupport::Cache::Store • Has multiple adapters

    • Redis • Memcached • File (don’t) • Memory (don’t) Caching
  21. 40.

    Types • Full page caching • Action caching • Object

    caching • Partial caching Caching
  22. 41.

    API • Read • Write • Fetch (decorator for read/write)

    • Exist? ! NO Get all keys Search for keys Append, Prepend ! Nothing, learn to love it Caching
  23. 42.

    “ “ There are only two hard things in Computer

    Science: cache invalidation and naming things. ! ! -- Phil Karlton
  24. 43.

    I DO NOT recommend using anything other than partial caching

    in your rails application ! Caching solution explained
  25. 49.
  26. 50.

    • Magic strings • Knowledge of keys are in multiple

    places • Typos break cache invalidation Problems
  27. 55.

    Exercise ! Cache (partial) all the products on the page,

    invalidate the cache when a new product is created or when an existing product is updated ! 30 minutes
  28. 57.

    API • http://redis.io • get/set • sets • hashes •

    sorted sets • Best practice for keys “object_name:#{id}:#{key}” • product:2:likes • product:2:like_count • product:2:like Redis