Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scaling, Expending and Extending your app through messaging

Avi Tzurel
September 30, 2013

Scaling, Expending and Extending your app through messaging

This is a talk I gave at the monthly meetup of Ruby Underground in Israel.

I go through the concept of SOA using Ruby/Rails, the reasons to go down this path and the logic behind it.

Avi Tzurel

September 30, 2013
Tweet

More Decks by Avi Tzurel

Other Decks in Technology

Transcript

  1. Scaling Extending & Expanding Your
    Application Through Messaging
    Avi Tzurel
    Tuesday, October 1, 13

    View Slide

  2. ME
    • Avi Tzurel
    [email protected]
    • @KensoDev (Twitter, Github, Facebook)
    Tuesday, October 1, 13

    View Slide

  3. What are we talking about?
    • Current way of building 95% of rails apps
    • Why is that broken?
    • Why should you care?
    • New and improved way of building connected apps
    Tuesday, October 1, 13

    View Slide

  4. Our Use Case
    Tuesday, October 1, 13

    View Slide

  5. Tuesday, October 1, 13

    View Slide

  6. Writing a review
    • Set the review language
    • Set the review rank (based on some rules)
    • Update the user score
    • Was he first to write a full review? (better score)
    • Update the place score
    • Update the place score in a travel style
    • Update the personal place score for user graph
    Tuesday, October 1, 13

    View Slide

  7. The Rails Way
    • How would you say rails is designed to do this?
    Tuesday, October 1, 13

    View Slide

  8. Yes, I am asking a question
    What would you say is the rails
    way?
    Tuesday, October 1, 13

    View Slide

  9. The Rails Way
    Tuesday, October 1, 13

    View Slide

  10. Tuesday, October 1, 13

    View Slide

  11. Don’t get smart with me!
    Rewriting this to use
    after_commit does not help, this
    code is shit!
    Tuesday, October 1, 13

    View Slide

  12. Problems
    Tuesday, October 1, 13

    View Slide

  13. Problems
    • Overloading the Lib Directory
    • Lib directory gets to be the junk of rails apps
    Tuesday, October 1, 13

    View Slide

  14. It goes on and on....
    Tuesday, October 1, 13

    View Slide

  15. Problems
    • Application is bloating beyond your control
    • Gemfile
    • Initializers
    • Models
    • Super slow startup times
    • Super slow testing times
    Tuesday, October 1, 13

    View Slide

  16. The slightly better way
    Tuesday, October 1, 13

    View Slide

  17. Tuesday, October 1, 13

    View Slide

  18. Again...
    • Working with workers will make life a bit better
    • Still lots of bloat in the lib dir
    • Lots of workers in the application code to load
    • Where the hell is my logic?
    • Some engineers put it in the worker, some do classes
    Tuesday, October 1, 13

    View Slide

  19. Tuesday, October 1, 13

    View Slide

  20. It Still Sucks!
    Tuesday, October 1, 13

    View Slide

  21. • Going into the better way I will try to be...
    • Framework Agnostic
    • Queue system agnostic
    • I will give you the concept, you make it great for you.
    • It’s not a one-size-fits-all (things rarely are this way)
    There’s a better way
    Tuesday, October 1, 13

    View Slide

  22. NOT
    • Not a single application
    • Not a single logic location
    • Not a single test suite
    • Not a single Gemfile
    • Not a magic solution for all apps
    • Not something to start with right away (for a MVP)
    Tuesday, October 1, 13

    View Slide

  23. Tuesday, October 1, 13

    View Slide

  24. Tuesday, October 1, 13

    View Slide

  25. Publishing Events
    • Working with Events, not workers or classes
    • Every model/class can publish any event
    • There’s no ACK on the other side, it’s just publishing the
    message to your broker (I said I will be agnostic)
    • The model/observer responsibility ends here
    Tuesday, October 1, 13

    View Slide

  26. Subscribing
    • On the other side, your “mini-apps” can subscribe to those
    events
    • Events can be unique strings or topics
    • For example “recommendation_change” is a topic
    • Any app that cares about this, will register on the event
    Tuesday, October 1, 13

    View Slide

  27. Our Mini apps
    • LanguageDetectionApp
    • RecommendationRankingApp
    • PlaceScoringApp
    • TravelStyleScoringApp
    • UserScoringSystem
    • GraphScoringSystem
    Tuesday, October 1, 13

    View Slide

  28. • Apps can be written in
    • Full rails apps
    • Ruby
    • Sinatra apps
    • Go
    • Java
    more on apps
    Tuesday, October 1, 13

    View Slide

  29. more on apps
    • Different Gems for different apps
    • Tests only run on those apps on change
    • Now, when you change your main app, it runs only those
    relevant specs
    • Different API’s, can even be different languages
    Tuesday, October 1, 13

    View Slide

  30. Full Architecture
    Tuesday, October 1, 13

    View Slide

  31. Full Flow
    • Main app sends recommendation_changed
    • LanguageDetectionApp registers on this event and checks
    that the change includes “description”
    • If change does not include, exits
    • Change includes, detects the language, sets the column
    and sends the event again
    Tuesday, October 1, 13

    View Slide

  32. Tuesday, October 1, 13

    View Slide

  33. Gotchas
    • Configuration of apps
    • Redis connection
    • DB connection
    • Mongo Connection
    • Memcached connection
    • Deployments
    Tuesday, October 1, 13

    View Slide

  34. • Application config can be stored in...
    • Chef Data Bags (recommended)
    • ENV Variables (requires a bit more hassle) we work this
    way right now and making a move to the previous
    • Thanks to Avner @ Fiverr for the Chef idea
    Gotchas
    Tuesday, October 1, 13

    View Slide

  35. Advantages
    • Full decoupling
    • Throw complete chunks of code to the trash
    • No more feature XXX, no problem
    • Adding another feature dependent on an event, No
    problem, Main app does not change (doesn’t even have to
    be redeployed) AMAZING RIGHT?
    Tuesday, October 1, 13

    View Slide

  36. Advantages #2
    • Improved response time (any async will give you that
    though)
    • Reduced complexity by decoupling
    • Build apps that use the language best suited instead of
    one gigantic monster app in Ruby/Rails
    • Deploy separately, scale separately
    Tuesday, October 1, 13

    View Slide

  37. Read more
    • Bunny for RabbitMQ
    • RabbitMQ docs
    • http://www.rubyinside.com/why-rubyists-should-care-
    about-messaging-a-high-level-intro-5017.html
    • http://en.wikipedia.org/wiki/
    Fallacies_of_Distributed_Computing
    • Messaging middleware used at Wikipedia
    Tuesday, October 1, 13

    View Slide

  38. Thank you!
    Questions?
    Tuesday, October 1, 13

    View Slide