Scaling Rails with RabbitMQ and Memcached

Some of concept scaling web app using RabbitMQ and Memcached. Using RabbitMQ to distribute the work of some process and store the result in Memcached

Transcript

1. Scaling Rails with RabbitMQ and Memcached by Evan Purnama (qiscus) Wednesday, December 17, 14
2. About this Talk I created this keynote to guide me at Jogja Ruby Meetup #4 This talk will be about 25-35 mins long Wednesday, December 17, 14
3. Ruby on Rails Why do I learn Ruby on Rails (RoR)? How do I learn RoR? Do you have to learn RoR? Wednesday, December 17, 14
4. why because you cannot not heard about Ruby on Rails lower hang fruit cool but not cool know some friends who really good at it Wednesday, December 17, 14
5. How I didnt succeeded learn Rails at the first attempt Just start build real project (in my case, it is Gitspan. Qiscus is my 2nd project) Join community Wednesday, December 17, 14
6. Do you have to? Wednesday, December 17, 14
7. No Wednesday, December 17, 14
8. Keep in mind technology is always evolving we are paid for solving problem complex software is always difficult Wednesday, December 17, 14
9. Wednesday, December 17, 14
10. Problem I want to create app that can handle my request but I need response as fast as possible I want to process my data further I want to access the results of my data that already being process Wednesday, December 17, 14
11. user Rails request response Task A Task B Task C post content task B: At another machine, produce some result task A: Store in Database task A: etc (send email, ...) Wednesday, December 17, 14
12. user Rails request response get data from result task B Wednesday, December 17, 14
13. RabbitMQ Wednesday, December 17, 14
14. RabbitMQ is open source message broker software (sometimes called message-oriented middleware) that implements the Advanced Message Queuing Protocol (AMQP). The RabbitMQ server is written in the Erlang programming language and is built on the Open Telecom Platform framework for clustering and failover. Wednesday, December 17, 14
15. What can RabbitMQ do for you? Messaging enables software applications to connect and scale. Applications can connect to each other, as components of a larger application, or to user devices and data. Messaging is asynchronous, decoupling applications by separating sending and receiving data. You may be thinking of data delivery, non-blocking operations or push notifications. Or you want to use publish / subscribe, asynchronous processing, or work queues. All these are patterns, and they form part of messaging. RabbitMQ is a messaging broker - an intermediary for messaging. It gives your applications a common platform to send and receive messages, and your messages a safe place to live until received. Wednesday, December 17, 14
16. Memcached Wednesday, December 17, 14
17. Memcached is a general-purpose distributed memory caching system. It is often used to speed up dynamic database-driven websites by caching data and objects in RAM to reduce the number of times an external data source (such as a database or API) must be read. Wednesday, December 17, 14
18. Demo Wednesday, December 17, 14
19. Things that we need Heroku account Ruby on Rails 4.1.4 Heroku add ons: BigWig, Memcachier, Posgtresql Ruby gems: Bunny, Dalli Python pip packages: nltk, pika, bmemchaed Wednesday, December 17, 14
20. Architecture RoR on Heroku Localhost : Ruby listener Localhost : Python listener Heroku : Ruby listener Not doing anything NLTK, tokenize put on Memcached Insert to Postgre DB BigWig Memcachier Wednesday, December 17, 14
21. Resources http://rubybunny.info/articles/getting_started.html https://github.com/mperham/dalli http://www.rabbitmq.com/documentation.html http://www.nltk.org/ http://www.amazon.com/RabbitMQ-Action-Distributed- Messaging-Everyone/dp/1935182978 Wednesday, December 17, 14
22. Qiscus current Arch Wednesday, December 17, 14
23. Wednesday, December 17, 14
24. Q & A We are Hiring! email me at e@qiscus.com Wednesday, December 17, 14