Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Reduciendo el acoplamiento entre aplicaciones con RabbitMQ

B3eb24dc767e178a2c7d67f1ee1af11f?s=47 Alvaro Videla
September 26, 2011

Reduciendo el acoplamiento entre aplicaciones con RabbitMQ

Charla que muestra como utilizar RabbitMQ para desacoplar nuestras aplicaciones a la vez que hacer más fácil la escalabilidad de las mismas.

B3eb24dc767e178a2c7d67f1ee1af11f?s=128

Alvaro Videla

September 26, 2011
Tweet

Transcript

  1. Monday, July 4, 2011

  2. Monday, July 4, 2011

  3. About Me • Desarrollador en Liip AG • Blog: http://videlalvaro.github.com/

    • Twitter: @old_sound Monday, July 4, 2011
  4. About Me Escribiendo RabbitMQ in Action http://bit.ly/rabbitmq Monday, July 4,

    2011
  5. ¿Por qué necesito usar Mensajería? Monday, July 4, 2011

  6. Veamos un ejemplo Monday, July 4, 2011

  7. Implementar una Galería de Imágenes Monday, July 4, 2011

  8. Dos Partes: Monday, July 4, 2011

  9. ¿Bastante fácil no? Monday, July 4, 2011

  10. Hasta que nuevos requerimientos comienzan a llegar Monday, July 4,

    2011
  11. El Propietario del Producto Monday, July 4, 2011

  12. ¿Podemos notificar a los amigos del usuario sobre nuevas imágenes?

    Monday, July 4, 2011
  13. ¿Podemos notificar a los amigos del usuario sobre nuevas imágenes?

    Me olvidé de decirles que lo necesito para mañana Monday, July 4, 2011
  14. El “Social Media Guru” Monday, July 4, 2011

  15. Necesitamos premiar a los usuarios por cada foto que suben

    Monday, July 4, 2011
  16. Necesitamos premiar a los usuarios por cada foto que suben

    y enviar notificaciones a Twitter Monday, July 4, 2011
  17. El Administrador de Sistemas Monday, July 4, 2011

  18. ¡Dolobu! Estamos sirviendo imágenes sin achicar. ¡La cuenta de ancho

    de banda a triplicado! Monday, July 4, 2011
  19. ¡Necesitamos arreglar esto para mañana! ¡Dolobu! Estamos sirviendo imágenes sin

    achicar. ¡La cuenta de ancho de banda a triplicado! Monday, July 4, 2011
  20. El Desarrollador en el otro equipo Monday, July 4, 2011

  21. Necesito llamar tus sistemas PHP pero desde Python Monday, July

    4, 2011
  22. Necesito llamar tus sistemas PHP pero desde Python Y la

    semana que viene Java también Monday, July 4, 2011
  23. El Usuario Monday, July 4, 2011

  24. No quiero tener que esperar que tu aplicación procese la

    imagen Monday, July 4, 2011
  25. Tu Monday, July 4, 2011

  26. FML! Monday, July 4, 2011

  27. Veamos la evolución del código Monday, July 4, 2011

  28. %% image_controller handle('PUT', "/user/image", ReqData) -> image_handler:do_upload(ReqData:get_file()), ok. Primera implementación:

    Monday, July 4, 2011
  29. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()),

    resize_image(Image), ok. Segunda implementación: Monday, July 4, 2011
  30. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()),

    resize_image(Image), notify_friends(ReqData:get_user()), ok. Tercera implementación: Monday, July 4, 2011
  31. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()),

    resize_image(Image), notify_friends(ReqData:get_user()), add_points_to_user(ReqData:get_user()), ok. Cuarta implementación: Monday, July 4, 2011
  32. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()),

    resize_image(Image), notify_friends(ReqData:get_user()), add_points_to_user(ReqData:get_user()), tweet_new_image(User, Image), ok. Implementación final: Monday, July 4, 2011
  33. ¿Escala nuestro código a nuevos requerimientos? Monday, July 4, 2011

  34. Qué pasaría si… Monday, July 4, 2011

  35. Qué pasaría si… • Necesitamos acelerar la conversión de imágenes

    Monday, July 4, 2011
  36. Qué pasaría si… • Necesitamos acelerar la conversión de imágenes

    • Las notificaciones a los usuarios tienen que ser enviadas por email Monday, July 4, 2011
  37. Qué pasaría si… • Necesitamos acelerar la conversión de imágenes

    • Las notificaciones a los usuarios tienen que ser enviadas por email • Tenemos que dejar de twittear sobre nuevas imágenes Monday, July 4, 2011
  38. Qué pasaría si… • Necesitamos acelerar la conversión de imágenes

    • Las notificaciones a los usuarios tienen que ser enviadas por email • Tenemos que dejar de twittear sobre nuevas imágenes • Convertir imágenes a diferentes formatos Monday, July 4, 2011
  39. ¿Podemos hacerlo mejor? Monday, July 4, 2011

  40. Por supuesto. Usando Mensajería. Monday, July 4, 2011

  41. Diseño Publish / Subscribe Pattern Monday, July 4, 2011

  42. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()),

    Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message('new_image', Msg). Primera Implementación: Monday, July 4, 2011
  43. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()),

    Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message('new_image', Msg). Primera Implementación: %% friends notifier on('new_image', Msg) -> notify_friends(Msg.user, Msg.image). Monday, July 4, 2011
  44. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()),

    Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message('new_image', Msg). Primera Implementación: %% friends notifier on('new_image', Msg) -> notify_friends(Msg.user, Msg.image). %% points manager on('new_image', Msg) -> add_points(Msg.user, 'new_image'). Monday, July 4, 2011
  45. %% image_controller handle('PUT', "/user/image", ReqData) -> {ok, Image} = image_handler:do_upload(ReqData:get_file()),

    Msg = #msg{user = ReqData:get_user(), image = Image}, publish_message('new_image', Msg). Primera Implementación: %% friends notifier on('new_image', Msg) -> notify_friends(Msg.user, Msg.image). %% points manager on('new_image', Msg) -> add_points(Msg.user, 'new_image'). %% resizer on('new_image', Msg) -> resize_image(Msg.image). Monday, July 4, 2011
  46. Segunda Implementación: Monday, July 4, 2011

  47. Segunda Implementación: THIS PAGE INTENTIONALLY LEFT BLANK Monday, July 4,

    2011
  48. Mensajería Monday, July 4, 2011

  49. Mensajería • Compartir datos entre procesos Monday, July 4, 2011

  50. Mensajería • Compartir datos entre procesos • Procesos pueden ser

    parte de diferentes aplicaciones Monday, July 4, 2011
  51. Mensajería • Compartir datos entre procesos • Procesos pueden ser

    parte de diferentes aplicaciones • Aplicaciones pueden vivir en diferentes computadores Monday, July 4, 2011
  52. Mensajería • Compartir datos entre procesos • Procesos pueden ser

    parte de diferentes aplicaciones • Aplicaciones pueden vivir en diferentes computadores • La comunicación es asíncrona Monday, July 4, 2011
  53. Conceptos Principales Monday, July 4, 2011

  54. Conceptos Principales • Mensajes son enviados por Producers Monday, July

    4, 2011
  55. Conceptos Principales • Mensajes son enviados por Producers • Mensajes

    se envían a Consumers Monday, July 4, 2011
  56. Conceptos Principales • Mensajes son enviados por Producers • Mensajes

    se envían a Consumers • Mensajes van a través de un Channel Monday, July 4, 2011
  57. RabbitMQ y la Mensajería Monday, July 4, 2011

  58. ¿Qué es RabbitMQ? Monday, July 4, 2011

  59. RabbitMQ • Sistema de Mensajería Empresarial • Código Libre MPL

    • Escrito en Erlang/OTP • Soporte Comercial • Mensajería via AMQP Monday, July 4, 2011
  60. Features • Confiable y Altamente Escalable • Fácil de Instalar

    • Fácil de Clusterizar • Corre en: Windows, Solaris, Linux, OSX • AMQP 0.8 - 0.9.1 Monday, July 4, 2011
  61. Librerías AMQP • Java • .NET/C# • Erlang • Ruby,

    Python, PHP, Perl, AS3, Lisp, Scala, Clojure, Haskell Monday, July 4, 2011
  62. AMQP • Advanced Message Queuing Protocol • Pensado para la

    Interoperabilidad • Protocolo Completamente Abierto • Protocol Binario Monday, July 4, 2011
  63. Flujo de Mensajes http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/chap-Messaging_Tutorial-Initial_Concepts.html Monday, July 4, 2011

  64. Modelo AMQP • Exchanges • Message Queues • Bindings •

    Rules for binding them Monday, July 4, 2011
  65. Tipos de Exchange • Fanout • Direct • Topic Monday,

    July 4, 2011
  66. http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Fanout_Exchange.html Monday, July 4, 2011

  67. http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Direct_Exchange.html Monday, July 4, 2011

  68. http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts- Topic_Exchange.html Monday, July 4, 2011

  69. Patrones de Mensajería Monday, July 4, 2011

  70. Les recomiendo un libro al respecto: http://www.eaipatterns.com/ Monday, July 4,

    2011
  71. Patrones Básicos Monday, July 4, 2011

  72. Competing Consumers How can a messaging client process multiple messages

    concurrently? Monday, July 4, 2011
  73. Competing Consumers Create multiple Competing Consumers on a single channel

    so that the consumers can process multiple messages concurrently. Monday, July 4, 2011
  74. Competing Consumers Monday, July 4, 2011

  75. Código Publisher init(Exchange, Queue) -> #'exchange.declare'{exchange = Exchange, type =

    <<"direct">>, durable = true}, #'queue.declare'{queue = Queue, durable = false}, #'queue.bind'{queue = Queue, exchange = Exchange}. publish_msg(Exchange, Payload) -> Props = #'P_basic'{content_type = <<"application/json">>, delivery_mode = 2}, %% persistent publish(Exchange, #amqp_msg{props = Props, payload = Payload}). Monday, July 4, 2011
  76. Código Consumer init_consumer(Exchange, Queue) -> init(Exchange, Queue), #'basic.consume'{queue = Queue}.

    on(#'basic.deliver'{delivery_tag = DeliveryTag}, #amqp_msg{} = Msg) -> do_something_with_msg(Msg), #'basic.ack'{delivery_tag = DeliveryTag}. Monday, July 4, 2011
  77. Publish/Subscribe How can the sender broadcast an event to all

    interested receivers? Monday, July 4, 2011
  78. Publish/Subscribe Send the event on a Publish-Subscribe Channel, which delivers

    a copy of a particular event to each receiver. Monday, July 4, 2011
  79. Publish/Subscribe Monday, July 4, 2011

  80. Código Pulbisher init(Exchange, Queue) -> #'exchange.declare'{exchange = Exchange, type =

    <<"fanout">>, %% different type durable = true} %% same as before ... publish_msg(Exchange, Payload) -> Props = #'P_basic'{content_type = <<"application/json">>, delivery_mode = 2}, %% persistent publish(Exchange, #amqp_msg{props = Props, payload = Payload}). Monday, July 4, 2011
  81. Código Consumer A init_consumer(Exchange, ResizeImageQueue) -> init(Exchange, ResizeImageQueue), #'basic.consume'{queue =

    ResizeImageQueue}. on(#'basic.deliver'{delivery_tag = DeliveryTag}, #amqp_msg{} = Msg) -> resize_message(Msg), #'basic.ack'{delivery_tag = DeliveryTag}. Monday, July 4, 2011
  82. Código Consumer B init_consumer(Exchange, NotifyFriendsQueue) -> init(Exchange, NotifyFriendsQueue), #'basic.consume'{queue =

    NotifyFriendsQueue}. on(#'basic.deliver'{delivery_tag = DeliveryTag}, #amqp_msg{} = Msg) -> notify_friends(Msg), #'basic.ack'{delivery_tag = DeliveryTag}. Monday, July 4, 2011
  83. Código Consumer C init_consumer(Exchange, LogImageUpload) -> init(Exchange, LogImageUpload), #'basic.consume'{queue =

    LogImageUpload}. on(#'basic.deliver'{delivery_tag = DeliveryTag}, #amqp_msg{} = Msg) -> log_image_upload(Msg), #'basic.ack'{delivery_tag = DeliveryTag}. Monday, July 4, 2011
  84. Request/Reply When an application sends a message, how can it

    get a response from the receiver? Monday, July 4, 2011
  85. Request/Reply Send a pair of Request-Reply messages, each on its

    own channel. Monday, July 4, 2011
  86. Request/Reply Monday, July 4, 2011

  87. Return Address How does a replier know where to send

    the reply? Monday, July 4, 2011
  88. Return Address The request message should contain a Return Address

    that indicates where to send the reply message. Monday, July 4, 2011
  89. Return Address Monday, July 4, 2011

  90. Correlation Identifier How does a requestor that has received a

    reply know which request this is the reply for? Monday, July 4, 2011
  91. Correlation Identifier Each reply message should contain a Correlation Identifier,

    a unique identifier that indicates which request message this reply is for. Monday, July 4, 2011
  92. Correlation Identifier Monday, July 4, 2011

  93. Atando Cabos Monday, July 4, 2011

  94. Cliente RPC init() -> #'queue.declare_ok'{queue = SelfQueue} = #'queue.declare'{exclusive =

    true, auto_delete = true}, #'basic.consume'{queue = SelfQueue, no_ack = true}, SelfQueue. Monday, July 4, 2011
  95. Cliente RPC init() -> #'queue.declare_ok'{queue = SelfQueue} = #'queue.declare'{exclusive =

    true, auto_delete = true}, #'basic.consume'{queue = SelfQueue, no_ack = true}, SelfQueue. request(Payload, RequestId) -> Props = #'P_basic'{correlation_id = RequestId, reply_to = SelfQueue}, publish(ServerExchange, #amqp_msg{props = Props, payload = Payload}). Monday, July 4, 2011
  96. Cliente RPC init() -> #'queue.declare_ok'{queue = SelfQueue} = #'queue.declare'{exclusive =

    true, auto_delete = true}, #'basic.consume'{queue = SelfQueue, no_ack = true}, SelfQueue. request(Payload, RequestId) -> Props = #'P_basic'{correlation_id = RequestId, reply_to = SelfQueue}, publish(ServerExchange, #amqp_msg{props = Props, payload = Payload}). on(#'basic.deliver'{}, #amqp_msg{props = Props, payload = Payload}) -> CorrelationId = Props.correlation_id, do_something_with_reply(Payload). Monday, July 4, 2011
  97. Servidor RPC on(#'basic.deliver'{}, #amqp_msg{props = Props, payload = Payload}) ->

    CorrelationId = Props.correlation_id, ReplyTo = Props.reply_to, Reply = process_request(Payload), NewProps = #'P_basic'{correlation_id = CorrelationId}, publish("", %% anonymous exchange #amqp_msg{props = NewProps, payload = Reply}, ReplyTo). %% routing key Monday, July 4, 2011
  98. Patrones Avanzados Monday, July 4, 2011

  99. Control Bus How can we effectively administer a messaging system

    that is distributed across multiple platforms and a wide geographic area? Monday, July 4, 2011
  100. Control Bus Use a Control Bus to manage an enterprise

    integration system. Monday, July 4, 2011
  101. Control Bus • Send Configuration Messages • Start/Stop Services •

    Inject Test Messages • Collect Statistics Monday, July 4, 2011
  102. Control Bus Monday, July 4, 2011

  103. Control Bus Make Services “Control Bus” Enabled Monday, July 4,

    2011
  104. Detour How can you route a message through intermediate steps

    to perform validation, testing or debugging functions? Monday, July 4, 2011
  105. Detour Construct a Detour with a context-based router controlled via

    the Control Bus. In one state the router routes incoming messages through additional steps while in the other it routes messages directly to the destination channel. Monday, July 4, 2011
  106. Detour Monday, July 4, 2011

  107. Wire Tap How do you inspect messages that travel on

    a point-to-point channel? Monday, July 4, 2011
  108. Wire Tap Insert a simple Recipient List into the channel

    that publishes each incoming message to the main channel and a secondary channel. Monday, July 4, 2011
  109. Wire Tap Monday, July 4, 2011

  110. Smart Proxy How can you track messages on a service

    that publishes reply messages to the Return Address specified by the requestor? Monday, July 4, 2011
  111. Smart Proxy Use a Smart Proxy to store the Return

    Address supplied by the original requestor and replace it with the address of the Smart Proxy. When the service sends the reply message route it to the original Return Address. Monday, July 4, 2011
  112. Smart Proxy Monday, July 4, 2011

  113. Créditos Imágenes y descripciones de los patrones tomadas de: http://www.eaipatterns.com/

    Monday, July 4, 2011
  114. ¡Gracias! @old_sound http://vimeo.com/user1169087 http://www.slideshare.net/old_sound Monday, July 4, 2011