Reduciendo el acoplamiento entre aplicaciones con RabbitMQ

Charla que muestra como utilizar RabbitMQ para desacoplar nuestras aplicaciones a la vez que hacer más fácil la escalabilidad de las mismas.

Transcript

1. Monday, July 4, 2011
2. Monday, July 4, 2011
3. About Me • Desarrollador en Liip AG • Blog: http://videlalvaro.github.com/ • Twitter: @old_sound Monday, July 4, 2011
4. About Me Escribiendo RabbitMQ in Action http://bit.ly/rabbitmq Monday, July 4, 2011
5. ¿Por qué necesito usar Mensajería? Monday, July 4, 2011
6. Veamos un ejemplo Monday, July 4, 2011
7. Implementar una Galería de Imágenes Monday, July 4, 2011
8. Dos Partes: Monday, July 4, 2011
9. ¿Bastante fácil no? Monday, July 4, 2011
10. Hasta que nuevos requerimientos comienzan a llegar Monday, July 4, 2011
11. El Propietario del Producto Monday, July 4, 2011
12. ¿Podemos notificar a los amigos del usuario sobre nuevas imágenes? Monday, July 4, 2011
13. ¿Podemos notificar a los amigos del usuario sobre nuevas imágenes? Me olvidé de decirles que lo necesito para mañana Monday, July 4, 2011
14. El “Social Media Guru” Monday, July 4, 2011
15. Necesitamos premiar a los usuarios por cada foto que suben Monday, July 4, 2011
16. Necesitamos premiar a los usuarios por cada foto que suben y enviar notificaciones a Twitter Monday, July 4, 2011
17. El Administrador de Sistemas Monday, July 4, 2011
18. ¡Dolobu! Estamos sirviendo imágenes sin achicar. ¡La cuenta de ancho de banda a triplicado! Monday, July 4, 2011
19. ¡Necesitamos arreglar esto para mañana! ¡Dolobu! Estamos sirviendo imágenes sin achicar. ¡La cuenta de ancho de banda a triplicado! Monday, July 4, 2011
20. El Desarrollador en el otro equipo Monday, July 4, 2011
21. Necesito llamar tus sistemas PHP pero desde Python Monday, July 4, 2011
22. Necesito llamar tus sistemas PHP pero desde Python Y la semana que viene Java también Monday, July 4, 2011
23. El Usuario Monday, July 4, 2011
24. No quiero tener que esperar que tu aplicación procese la imagen Monday, July 4, 2011
25. Tu Monday, July 4, 2011
26. FML! Monday, July 4, 2011
27. Veamos la evolución del código Monday, July 4, 2011
28. %% image_controller handle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok. Primera implementación: Monday, July 4, 2011
29. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image), ok. Segunda implementación: Monday, July 4, 2011
30. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image), notify_friends(ReqData:get_user()), ok. Tercera implementación: Monday, July 4, 2011
31. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image), notify_friends(ReqData:get_user()), add_points_to_user(ReqData:get_user()), ok. Cuarta implementación: Monday, July 4, 2011
32. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), resize_image(Image), notify_friends(ReqData:get_user()), add_points_to_user(ReqData:get_user()), tweet_new_image(User, Image), ok. Implementación final: Monday, July 4, 2011
33. ¿Escala nuestro código a nuevos requerimientos? Monday, July 4, 2011
34. Qué pasaría si… Monday, July 4, 2011
35. Qué pasaría si… • Necesitamos acelerar la conversión de imágenes Monday, July 4, 2011
36. Qué pasaría si… • Necesitamos acelerar la conversión de imágenes • Las notificaciones a los usuarios tienen que ser enviadas por email Monday, July 4, 2011
37. Qué pasaría si… • Necesitamos acelerar la conversión de imágenes • Las notificaciones a los usuarios tienen que ser enviadas por email • Tenemos que dejar de twittear sobre nuevas imágenes Monday, July 4, 2011
38. Qué pasaría si… • Necesitamos acelerar la conversión de imágenes • Las notificaciones a los usuarios tienen que ser enviadas por email • Tenemos que dejar de twittear sobre nuevas imágenes • Convertir imágenes a diferentes formatos Monday, July 4, 2011
39. ¿Podemos hacerlo mejor? Monday, July 4, 2011
40. Por supuesto. Usando Mensajería. Monday, July 4, 2011
41. Diseño Publish / Subscribe Pattern Monday, July 4, 2011
42. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message('new_image', Msg). Primera Implementación: Monday, July 4, 2011
43. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message('new_image', Msg). Primera Implementación: %% friends notifier on('new_image', Msg) -> notify_friends(Msg.user, Msg.image). Monday, July 4, 2011
44. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message('new_image', Msg). Primera Implementación: %% friends notifier on('new_image', Msg) -> notify_friends(Msg.user, Msg.image). %% points manager on('new_image', Msg) -> add_points(Msg.user, 'new_image'). Monday, July 4, 2011
45. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()), Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message('new_image', Msg). Primera Implementación: %% friends notifier on('new_image', Msg) -> notify_friends(Msg.user, Msg.image). %% points manager on('new_image', Msg) -> add_points(Msg.user, 'new_image'). %% resizer on('new_image', Msg) -> resize_image(Msg.image). Monday, July 4, 2011
46. Segunda Implementación: Monday, July 4, 2011
47. Segunda Implementación: THIS PAGE INTENTIONALLY LEFT BLANK Monday, July 4, 2011
48. Mensajería Monday, July 4, 2011
49. Mensajería • Compartir datos entre procesos Monday, July 4, 2011
50. Mensajería • Compartir datos entre procesos • Procesos pueden ser parte de diferentes aplicaciones Monday, July 4, 2011
51. Mensajería • Compartir datos entre procesos • Procesos pueden ser parte de diferentes aplicaciones • Aplicaciones pueden vivir en diferentes computadores Monday, July 4, 2011
52. Mensajería • Compartir datos entre procesos • Procesos pueden ser parte de diferentes aplicaciones • Aplicaciones pueden vivir en diferentes computadores • La comunicación es asíncrona Monday, July 4, 2011
53. Conceptos Principales Monday, July 4, 2011
54. Conceptos Principales • Mensajes son enviados por Producers Monday, July 4, 2011
55. Conceptos Principales • Mensajes son enviados por Producers • Mensajes se envían a Consumers Monday, July 4, 2011
56. Conceptos Principales • Mensajes son enviados por Producers • Mensajes se envían a Consumers • Mensajes van a través de un Channel Monday, July 4, 2011
57. RabbitMQ y la Mensajería Monday, July 4, 2011
58. ¿Qué es RabbitMQ? Monday, July 4, 2011
59. RabbitMQ • Sistema de Mensajería Empresarial • Código Libre MPL • Escrito en Erlang/OTP • Soporte Comercial • Mensajería via AMQP Monday, July 4, 2011
60. Features • Confiable y Altamente Escalable • Fácil de Instalar • Fácil de Clusterizar • Corre en: Windows, Solaris, Linux, OSX • AMQP 0.8 - 0.9.1 Monday, July 4, 2011
61. Librerías AMQP • Java • .NET/C# • Erlang • Ruby, Python, PHP, Perl, AS3, Lisp, Scala, Clojure, Haskell Monday, July 4, 2011
62. AMQP • Advanced Message Queuing Protocol • Pensado para la Interoperabilidad • Protocolo Completamente Abierto • Protocol Binario Monday, July 4, 2011
63. Flujo de Mensajes http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/chap-Messaging_Tutorial-Initial_Concepts.html Monday, July 4, 2011
64. Modelo AMQP • Exchanges • Message Queues • Bindings • Rules for binding them Monday, July 4, 2011
65. Tipos de Exchange • Fanout • Direct • Topic Monday, July 4, 2011
66. http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Fanout_Exchange.html Monday, July 4, 2011
67. http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Direct_Exchange.html Monday, July 4, 2011
68. http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Topic_Exchange.html Monday, July 4, 2011
69. Patrones de Mensajería Monday, July 4, 2011
70. Les recomiendo un libro al respecto: http://www.eaipatterns.com/ Monday, July 4, 2011
71. Patrones Básicos Monday, July 4, 2011
72. Competing Consumers How can a messaging client process multiple messages concurrently? Monday, July 4, 2011
73. Competing Consumers Create multiple Competing Consumers on a single channel so that the consumers can process multiple messages concurrently. Monday, July 4, 2011
74. Competing Consumers Monday, July 4, 2011
75. Código Publisher init(Exchange, Queue) -> #'exchange.declare'{exchange = Exchange, type = <<"direct">>, durable = true}, #'queue.declare'{queue = Queue, durable = false}, #'queue.bind'{queue = Queue, exchange = Exchange}. publish_msg(Exchange, Payload) -> Props = #'P_basic'{content_type = <<"application/json">>, delivery_mode = 2}, %% persistent publish(Exchange, #amqp_msg{props = Props, payload = Payload}). Monday, July 4, 2011
76. Código Consumer init_consumer(Exchange, Queue) -> init(Exchange, Queue), #'basic.consume'{queue = Queue}. on(#'basic.deliver'{delivery_tag = DeliveryTag}, #amqp_msg{} = Msg) -> do_something_with_msg(Msg), #'basic.ack'{delivery_tag = DeliveryTag}. Monday, July 4, 2011
77. Publish/Subscribe How can the sender broadcast an event to all interested receivers? Monday, July 4, 2011
78. Publish/Subscribe Send the event on a Publish-Subscribe Channel, which delivers a copy of a particular event to each receiver. Monday, July 4, 2011
79. Publish/Subscribe Monday, July 4, 2011
80. Código Pulbisher init(Exchange, Queue) -> #'exchange.declare'{exchange = Exchange, type = <<"fanout">>, %% different type durable = true} %% same as before ... publish_msg(Exchange, Payload) -> Props = #'P_basic'{content_type = <<"application/json">>, delivery_mode = 2}, %% persistent publish(Exchange, #amqp_msg{props = Props, payload = Payload}). Monday, July 4, 2011
81. Código Consumer A init_consumer(Exchange, ResizeImageQueue) -> init(Exchange, ResizeImageQueue), #'basic.consume'{queue = ResizeImageQueue}. on(#'basic.deliver'{delivery_tag = DeliveryTag}, #amqp_msg{} = Msg) -> resize_message(Msg), #'basic.ack'{delivery_tag = DeliveryTag}. Monday, July 4, 2011
82. Código Consumer B init_consumer(Exchange, NotifyFriendsQueue) -> init(Exchange, NotifyFriendsQueue), #'basic.consume'{queue = NotifyFriendsQueue}. on(#'basic.deliver'{delivery_tag = DeliveryTag}, #amqp_msg{} = Msg) -> notify_friends(Msg), #'basic.ack'{delivery_tag = DeliveryTag}. Monday, July 4, 2011
83. Código Consumer C init_consumer(Exchange, LogImageUpload) -> init(Exchange, LogImageUpload), #'basic.consume'{queue = LogImageUpload}. on(#'basic.deliver'{delivery_tag = DeliveryTag}, #amqp_msg{} = Msg) -> log_image_upload(Msg), #'basic.ack'{delivery_tag = DeliveryTag}. Monday, July 4, 2011
84. Request/Reply When an application sends a message, how can it get a response from the receiver? Monday, July 4, 2011
85. Request/Reply Send a pair of Request-Reply messages, each on its own channel. Monday, July 4, 2011
86. Request/Reply Monday, July 4, 2011
87. Return Address How does a replier know where to send the reply? Monday, July 4, 2011
88. Return Address The request message should contain a Return Address that indicates where to send the reply message. Monday, July 4, 2011
89. Return Address Monday, July 4, 2011
90. Correlation Identifier How does a requestor that has received a reply know which request this is the reply for? Monday, July 4, 2011
91. Correlation Identifier Each reply message should contain a Correlation Identifier, a unique identifier that indicates which request message this reply is for. Monday, July 4, 2011
92. Correlation Identifier Monday, July 4, 2011
93. Atando Cabos Monday, July 4, 2011
94. Cliente RPC init() -> #'queue.declare_ok'{queue = SelfQueue} = #'queue.declare'{exclusive = true, auto_delete = true}, #'basic.consume'{queue = SelfQueue, no_ack = true}, SelfQueue. Monday, July 4, 2011
95. Cliente RPC init() -> #'queue.declare_ok'{queue = SelfQueue} = #'queue.declare'{exclusive = true, auto_delete = true}, #'basic.consume'{queue = SelfQueue, no_ack = true}, SelfQueue. request(Payload, RequestId) -> Props = #'P_basic'{correlation_id = RequestId, reply_to = SelfQueue}, publish(ServerExchange, #amqp_msg{props = Props, payload = Payload}). Monday, July 4, 2011
96. Cliente RPC init() -> #'queue.declare_ok'{queue = SelfQueue} = #'queue.declare'{exclusive = true, auto_delete = true}, #'basic.consume'{queue = SelfQueue, no_ack = true}, SelfQueue. request(Payload, RequestId) -> Props = #'P_basic'{correlation_id = RequestId, reply_to = SelfQueue}, publish(ServerExchange, #amqp_msg{props = Props, payload = Payload}). on(#'basic.deliver'{}, #amqp_msg{props = Props, payload = Payload}) -> CorrelationId = Props.correlation_id, do_something_with_reply(Payload). Monday, July 4, 2011
97. Servidor RPC on(#'basic.deliver'{}, #amqp_msg{props = Props, payload = Payload}) -> CorrelationId = Props.correlation_id, ReplyTo = Props.reply_to, Reply = process_request(Payload), NewProps = #'P_basic'{correlation_id = CorrelationId}, publish("", %% anonymous exchange #amqp_msg{props = NewProps, payload = Reply}, ReplyTo). %% routing key Monday, July 4, 2011
98. Patrones Avanzados Monday, July 4, 2011
99. Control Bus How can we effectively administer a messaging system that is distributed across multiple platforms and a wide geographic area? Monday, July 4, 2011
100. Control Bus Use a Control Bus to manage an enterprise integration system. Monday, July 4, 2011
101. Control Bus • Send Configuration Messages • Start/Stop Services • Inject Test Messages • Collect Statistics Monday, July 4, 2011
102. Control Bus Monday, July 4, 2011
103. Control Bus Make Services “Control Bus” Enabled Monday, July 4, 2011
104. Detour How can you route a message through intermediate steps to perform validation, testing or debugging functions? Monday, July 4, 2011
105. Detour Construct a Detour with a context-based router controlled via the Control Bus. In one state the router routes incoming messages through additional steps while in the other it routes messages directly to the destination channel. Monday, July 4, 2011
106. Detour Monday, July 4, 2011
107. Wire Tap How do you inspect messages that travel on a point-to-point channel? Monday, July 4, 2011
108. Wire Tap Insert a simple Recipient List into the channel that publishes each incoming message to the main channel and a secondary channel. Monday, July 4, 2011
109. Wire Tap Monday, July 4, 2011
110. Smart Proxy How can you track messages on a service that publishes reply messages to the Return Address specified by the requestor? Monday, July 4, 2011
111. Smart Proxy Use a Smart Proxy to store the Return Address supplied by the original requestor and replace it with the address of the Smart Proxy. When the service sends the reply message route it to the original Return Address. Monday, July 4, 2011
112. Smart Proxy Monday, July 4, 2011
113. Créditos Imágenes y descripciones de los patrones tomadas de: http://www.eaipatterns.com/ Monday, July 4, 2011
114. ¡Gracias! @old_sound http://vimeo.com/user1169087 http://www.slideshare.net/old_sound Monday, July 4, 2011