Middleware demystified - messaging queues

Transcript

1. MIDDLEWARE DEMYSTIFIED WHAT IS THIS ENTERPRISEY SOUNDING THING? Vineet Reynolds JBoss Developer Experience, Red Hat
3. AGENDA Introduction to middleware An overview of Message Queues Impact on storage
4. HEADS UP Middleware is software that wires up discrete systems capable of independent operation. We'll discuss Message-oriented-middleware here.
5. SCENARIO #1 Source: SPECjms 2007
6. Order/shipment handling between Supermarkets and Distribution Centers. Purchase order/shipment handling between Distribution Centers and Suppliers. Price updates sent by Headquarters to the Supermarkets. Supermarket inventory management. Sales statistics collection. New product announcements. Distribution of credit card hot lists.
7. There are multiple systems at work here. Not just ONE.
8. WHY IS THIS ARCHITECTED THIS WAY? Each constituent system/module has a different responsibility. Building capabilities to assume new responsibilities in an existing system: cost time, money etc. with the risk of budget overruns encourages monolithic architectures and the disadvantages that come with them increases coupling increases complexity
9. SPEAKING OF INTEGRATION You could integrate systems together using RPC, shared memory, files etc. But...
10. MESSAGING ORIENTED MIDDLEWARE MoM involves the use of Message passing that in turn: reduce coupling allows systems to evolve in isolation ensure compatibility via APIs or message formats allows messages to be stored until they're processed typically offers transactional guarantees allows systems to communicate asynchronously
11. MESSAGE CHANNELS - THE PATTERN Generally, one application writes to the channel, while another reads from it. Note - cases involving multiple producers and consumers are not covered (yet) Reference: Enterprise Integration Patterns, Gregor Hohpe and Bobby Woolf
12. MESSAGE QUEUES Offer multiple point-to-point and/or publish-subscribe channels. They form the staple of most system integrations. Not to be confused with POSIX queues (and other OS provided queues). Provides primitives for clients to enqueue and dequeue messages often in a durable manner with transactional guarantees. Usually offer delivery guarantees (a message is delivered only once).
13. POINT-TO-POINT MESSAGING PATTERN Ordering guarantees - FIFO Delivery guarantees - Only once
14. HOW DOES P2P MESSAGING AFFECT STORAGE? Messages once sent by a producer should not be lost. Often backed by filestores, or even OLTP databases. The choice is dictated by reliability concerns. Messages can be stored in memory, but often as an optimization.
15. PERFORMANCE CONCERNS ... Apps may expect low latency reads and writes with high reliability. Reads and writes are usually synchronous and transacted. In reality, this depends on the type of sessions established by the clients. Messages may be fetched from disk for consumption.
16. AVAILABILITY CONCERNS ... Message stores will be backed up. Backups can be either online or offline. Queues may operate in a HA mode where messages are replicated (split-brain scenarios are possible).
17. OTHER CONCERNS ... Producer flow controls may exist for overfilled queues. Disk quotas may be established for queues, usually for flow control. Message sizes. Large messages (order of MBs) are possible. Message sequencing will be employed to partition messages as sequences.
18. PUBLISH-SUBSCRIBE MESSAGING PATTERN Delivery guarantees - Only once for every consumer Subscribers receive messages after subscribing to the channel
19. STORAGE CONCERNS ... The same as for P2P, but also : Transacted consumptions are relatively more disk-intensive. Depends on the number of subscribers. Messages can be persisted for a long duration for durable subscribers to consume, after periods of disconnectedness.
20. QUESTIONS ?