Which messaging layer to use in a loosely coupled distributed app by Narahari Allamraju

Transcript

1. Which messaging layer to use in a loosely coupled distributed app? ! Narahari (Hari) Allamraju ! java developer; python enthusiast working with messaging systems for 9.5 years ! http://github.com/supercoderz anarahari@gmail.com 1
2. Mess everyone talks to everyone 2
3. Messaging everyone talks to everyone through a single medium 3
4. Work Queue everyone queues their tasks at the same place and waits for the results 4
5. So you want to build a loosely coupled app ! why? do you know what side effects you will have? ! ok you know … ! what do you need to do? 5
6. The basics • Will the application benefit by splitting up into components? • What are the components? How are they dependent on each other? • Do we need more than one instance of each to cope up with the load? • Do we need newer components to handle certain cases that arise from the new messaging layer? 6
7. Identify the patterns • What sort of interface do we need - request-reply, publish subscribe or queuing? • Do we need to separate out messages into various types? • Do we need schemas for the messages or free form? • Are there any components that have a higher or lower throughput than others? 7
8. Latency, routing, priority • Which components need the message yesterday; and which can wait till end of day? • Do we need to expire messages when they are too late? • How easily can we route messages? • Can we make some messages take highest priority? 8
9. Build, Manage and Support • Do we have a suitable API? • Easy to provision and manage • Good monitoring • Guaranteed delivery of messages • Should be persist messages to disk for crash recovery? • Cleanup in case of issues 9
10. A few options • RabbitMQ • ZeroMQ • Redis • RDBMS or NOSQL database • RPC 10
11. Considering future growth • What if the loads increased by 15%? • What if we had a consistent heavy run for 24 hours or more? • Storage costs etc • If we have to switch to a different underlying - how hard is it? Tokyo Metro 11
12. Questions! • This presentation can be found on slideshare at - http://goo.gl/KVJxoE ! • The version with speaker notes will be available soon ! • you can also email me at anarahari@gmail.com 12