Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scaling, Expending and Extending your app through messaging

Avi Tzurel
September 30, 2013

Scaling, Expending and Extending your app through messaging

This is a talk I gave at the monthly meetup of Ruby Underground in Israel.

I go through the concept of SOA using Ruby/Rails, the reasons to go down this path and the logic behind it.

Avi Tzurel

September 30, 2013

More Decks by Avi Tzurel

Other Decks in Technology


  1. What are we talking about? • Current way of building

    95% of rails apps • Why is that broken? • Why should you care? • New and improved way of building connected apps Tuesday, October 1, 13
  2. Writing a review • Set the review language • Set

    the review rank (based on some rules) • Update the user score • Was he first to write a full review? (better score) • Update the place score • Update the place score in a travel style • Update the personal place score for user graph Tuesday, October 1, 13
  3. The Rails Way • How would you say rails is

    designed to do this? Tuesday, October 1, 13
  4. Yes, I am asking a question What would you say

    is the rails way? Tuesday, October 1, 13
  5. Don’t get smart with me! Rewriting this to use after_commit

    does not help, this code is shit! Tuesday, October 1, 13
  6. Problems • Overloading the Lib Directory • Lib directory gets

    to be the junk of rails apps Tuesday, October 1, 13
  7. Problems • Application is bloating beyond your control • Gemfile

    • Initializers • Models • Super slow startup times • Super slow testing times Tuesday, October 1, 13
  8. Again... • Working with workers will make life a bit

    better • Still lots of bloat in the lib dir • Lots of workers in the application code to load • Where the hell is my logic? • Some engineers put it in the worker, some do classes Tuesday, October 1, 13
  9. • Going into the better way I will try to

    be... • Framework Agnostic • Queue system agnostic • I will give you the concept, you make it great for you. • It’s not a one-size-fits-all (things rarely are this way) There’s a better way Tuesday, October 1, 13
  10. NOT • Not a single application • Not a single

    logic location • Not a single test suite • Not a single Gemfile • Not a magic solution for all apps • Not something to start with right away (for a MVP) Tuesday, October 1, 13
  11. Publishing Events • Working with Events, not workers or classes

    • Every model/class can publish any event • There’s no ACK on the other side, it’s just publishing the message to your broker (I said I will be agnostic) • The model/observer responsibility ends here Tuesday, October 1, 13
  12. Subscribing • On the other side, your “mini-apps” can subscribe

    to those events • Events can be unique strings or topics • For example “recommendation_change” is a topic • Any app that cares about this, will register on the event Tuesday, October 1, 13
  13. Our Mini apps • LanguageDetectionApp • RecommendationRankingApp • PlaceScoringApp •

    TravelStyleScoringApp • UserScoringSystem • GraphScoringSystem Tuesday, October 1, 13
  14. • Apps can be written in • Full rails apps

    • Ruby • Sinatra apps • Go • Java more on apps Tuesday, October 1, 13
  15. more on apps • Different Gems for different apps •

    Tests only run on those apps on change • Now, when you change your main app, it runs only those relevant specs • Different API’s, can even be different languages Tuesday, October 1, 13
  16. Full Flow • Main app sends recommendation_changed • LanguageDetectionApp registers

    on this event and checks that the change includes “description” • If change does not include, exits • Change includes, detects the language, sets the column and sends the event again Tuesday, October 1, 13
  17. Gotchas • Configuration of apps • Redis connection • DB

    connection • Mongo Connection • Memcached connection • Deployments Tuesday, October 1, 13
  18. • Application config can be stored in... • Chef Data

    Bags (recommended) • ENV Variables (requires a bit more hassle) we work this way right now and making a move to the previous • Thanks to Avner @ Fiverr for the Chef idea Gotchas Tuesday, October 1, 13
  19. Advantages • Full decoupling • Throw complete chunks of code

    to the trash • No more feature XXX, no problem • Adding another feature dependent on an event, No problem, Main app does not change (doesn’t even have to be redeployed) AMAZING RIGHT? Tuesday, October 1, 13
  20. Advantages #2 • Improved response time (any async will give

    you that though) • Reduced complexity by decoupling • Build apps that use the language best suited instead of one gigantic monster app in Ruby/Rails • Deploy separately, scale separately Tuesday, October 1, 13
  21. Read more • Bunny for RabbitMQ • RabbitMQ docs •

    http://www.rubyinside.com/why-rubyists-should-care- about-messaging-a-high-level-intro-5017.html • http://en.wikipedia.org/wiki/ Fallacies_of_Distributed_Computing • Messaging middleware used at Wikipedia Tuesday, October 1, 13