Share
Embed
RabbitMQ Talk by Trong Dao Le
Presented at Singapore Spring User Group on January 8th
SingaSUG
January 08, 2015
Transcript
21 slides
-
Why would you use a Message System? ¤ Loose Coupling
¤ Automatic performance scaling ¤ Scaling through configuration changes in the exchange bindings (splitting queues in topic exchanges (later in AMQP part)) ¤ Various message patterns: ¤ Request/Reply (RPC) ¤ Fire and forget ¤ Publish/Subscribe ¤ Auto-retry behavior (timeouts and transaction) -
Loose coupling ¤ Sender is not responsible for issues such
as: ¤ Is the receiver up? ¤ Could the receiver consume the message? ¤ Failure of delivery ¤ Instead of thinking about interface of function calls, think about the data and the flow of data ¤ Handling failure of response gracefully (timeouts) which doesn’t overload a system -
Automatic performance scaling ¤ Deploy more servers listening to the
same set of queues to handle more requests ¤ Multiple requesters won’t bring down your server (they have to bring down your broker, which is unlikely) ¤ No configuration files on the requesters side (Zookeeper can solve this, but it’s another system to maintain) -
Our use case ¤ Accept a template and send text
message to all the numbers with best effort ¤ Original approach: ¤ Insert into DB ¤ Cron job to read from DB and send message ¤ All using JTA!! (dreadful, but required as the accounts and the messages are on different DBs) ¤ Doesn’t scale well (max 500 messages, JTA timeout at 60s) -
New simplified approach Catchs: 1/ DB contention as multiple requests
to deduct the same account come at the same time. 2/ Multiple similar DB insertions are not batched. Solved with scheduled tasks by Dead-letter exchanges -
Advantages of the new approach ¤ Distribute the work load
of workers evenly ¤ Automatic retry if the transaction fail (by requeuing the message) ¤ Send message concurrently and faster -
Migration from ActiveMQ to RabbitMQ ¤ Our chat system was
using ActiveMQ (JMS) with scheduled delay to check if a message was delivered ¤ The scheduled delay was a fixed amount ¤ Migration to ActiveMQ is easy with dead-letter exchanges ¤ Using Spring, both ActiveMQ and RabbitMQ listeners are POJOs (specify the handling method in the configuration) ¤ Mostly configuration changes for listeners and type changes for senders. -
AMQP vs JMS AMQP JMS 4 (5) models: direct, fanout,
topic, header, (system) 2 models: 1-to-1 and pub/sub Binary data 5 different data types Clients send to exchanges, exchanges forward to queues Clients send directly to queues or topics Language agnostic Only support JVM languages Define a wire-level protocol standard Define a set of APIs -
AMQP Ex Ex P P P C C C Producers
Consumers Channels Channels Exchanges Queues Bindings -
Topic Exchange Ex D.E.A N.S.A C.I.A key="#.A" key=“#.I.#" F.B.I key="#"
C.I.A N.S.A D.E.A F.B.I C.I.A D.E.A N.S.A C.I.A F.B.I -
Other use cases ¤ Schedule task using dead-letter exchanges ¤
Distributed logging using topic exchanges ¤ Intermittent failure resistant workers (unsubscribe and message reject) ¤ Complicated event triggering (award the 1 million transaction) -
Why use Spring-AMQP? ¤ The RabbitMQ Java Client has some
drawbacks (just like JDBC) ¤ No channel pooling ¤ No retry ¤ Ugly listener code using while-loop ¤ Boiler plate codes -
RabbitMQ extension features ¤ Publisher confirms ¤ Consumer priorities ¤
Fast reply-to for RPC ¤ Exchange to exchange bindings ¤ Per-queue message TTL, per- message TTL ¤ Queue TTL ¤ Dead-letter exchanges ¤ Length limit -
RabbitMQ vs ActiveMQ RabbitMQ ActiveMQ AMQP (0-8 and 0-9-1) AMQP
(1.0) STOMP STOMP REST RSS/ATOM -
HA: RabbitMQ vs ActiveMQ RabbitMQ ActiveMQ Master + Slaves Master
+ Slaves Dynamic clustering: Nodes can join/leave None that I know of Full replication (mirrored queues) or full visibility (clustering) Networks of brokers -
Looking for… ¤ Website Operations Engineer ¤ Data Operations Engineer
Ops works + Develop tools to manage the data we have. Scala + Java developers, who has a knack for Ops.