Queue Based Architectures

Modern applications are no longer a single server doing single requests. Today applications need to collaborate with multiple internal and external systems. Your application might need some information from a third party service or start a process in an internal one.

Despite the difficulty of creating and orchestrating distributed systems, the application as a whole needs to be reliable and scalable.

In this talk I'd like to present the benefits (and pitfalls) of queue based architectures, and how they can help build (most) modern applications.

Transcript

1. Queue Based Architectures with RabbitMQ ! Queue Based Architectures - @nhocki
2. Nicolás Hock-Isaza @nhocki Queue Based Architectures - @nhocki
3. Queue Based Architectures with RabbitMQ ! Queue Based Architectures - @nhocki
4. Software Engineer @ Ride Queue Based Architectures - @nhocki
5. Modern Applications Queue Based Architectures - @nhocki
6. Modern Applications • Collaborate with other internal or external systems. • Creating and orchestrating services is hard. It requires maintainence. • Application must be reliable, robust and easy to scale. Queue Based Architectures - @nhocki
7. Queue Based Architectures - @nhocki
8. THIS IS HARD Queue Based Architectures - @nhocki
9. What is QBA? (Queue Based Architectures) A way of designing systems where the communitation between them is made with a reliable queue . Queue Based Architectures - @nhocki
10. Main Benefits Queue Based Architectures - @nhocki
11. Buffering Queue Based Architectures - @nhocki
12. Scalability Queue Based Architectures - @nhocki
13. Decoupling Queue Based Architectures - @nhocki
14. Resilience Queue Based Architectures - @nhocki
15. When is a good idea? • No need for real time (sub-second) responses. • Remove tasks from the request cycle. • Slow tasks or batched processes. Queue Based Architectures - @nhocki
16. Rabbit MQ ! (that's not the logo, but it's cute) Queue Based Architectures - @nhocki
17. Rabbit MQ Erlang ❤ AMQP (Advanced Message Queuing Protocol) Queue Based Architectures - @nhocki
18. Rabbit MQ - Simplified Queue Based Architectures - @nhocki
19. Exchanges Where messages are sent. Route messages to one or more queues. Queue Based Architectures - @nhocki
20. Queue Based Architectures - @nhocki
21. Queues Store messages for them to be consumed. If there are multiple consumers, round- robin is used. Queue Based Architectures - @nhocki
22. Queue Based Architectures - @nhocki
23. Bindings Ties between exchanges and queues. Define the routing rules for messages. Queue Based Architectures - @nhocki
24. Queue Based Architectures - @nhocki
25. Exchange Types Queue Based Architectures - @nhocki
26. Fanout: 1 — to —> ALL Sends the message to all queues. Queue Based Architectures - @nhocki
27. Fanout: 1 — to —> ALL Sends the message to all queues. Queue Based Architectures - @nhocki
28. Direct: 1 — to —> N Simple filter on routing key. Queue Based Architectures - @nhocki
29. Direct: 1 — to —> N Simple filter on routing key. Queue Based Architectures - @nhocki
30. Topic: 1 — to —> N Advanced filters for routing messages. Queue Based Architectures - @nhocki
31. Topic: 1 — to —> N Advanced filters for routing messages. Queue Based Architectures - @nhocki
32. Queue Based Architectures - @nhocki
33. Bunny Gem - Producer Queue Based Architectures - @nhocki
34. Bunny Gem - Producer Queue Based Architectures - @nhocki
35. Bunny Gem - Producer Queue Based Architectures - @nhocki
36. Bunny Gem - Producer Queue Based Architectures - @nhocki
37. Bunny Gem - Consumer Queue Based Architectures - @nhocki
38. Bunny Gem - Consumer Queue Based Architectures - @nhocki
39. Bunny Gem - Consumer Queue Based Architectures - @nhocki
40. Bunny Gem - Consumer Queue Based Architectures - @nhocki
41. Example Queue Based Architectures - @nhocki
42. Dashboard (Twitter) Queue Based Architectures - @nhocki
43. Dashboard (Twitter) • Way more reads than writes. • What about duplicated tweets? (retweets) • What happens when there's a conversation going? Querying for my dashboard gets harder & more complex. Queue Based Architectures - @nhocki
44. Queue Based Architectures - @nhocki
45. Queue Based Architectures - @nhocki
46. Queue Based Architectures - @nhocki
47. Queue Based Architectures - @nhocki
48. Queue Based Architectures - @nhocki
49. Queue Based Architectures - @nhocki
50. Queue Based Architectures - @nhocki
51. Queue Based Architectures - @nhocki
52. Queue Based Architectures - @nhocki
53. Queue Based Architectures - @nhocki
54. Queue Based Architectures - @nhocki
55. Performance vs Scalability Queue Based Architectures - @nhocki
56. Challenges Queue Based Architectures - @nhocki
57. Arquitecture Patterns Queue Based Architectures - @nhocki
58. Thanks! ✌ Queue Based Architectures - @nhocki