Reduciendo el acoplamiento entre aplicaciones con RabbitMQ

Charla que muestra como utilizar RabbitMQ para desacoplar nuestras aplicaciones a la vez que hacer más fácil la escalabilidad de las mismas.

Transcript

1. Monday, July 4, 2011
   

   View Slide

2. Monday, July 4, 2011
   

   View Slide

3. About Me
   • Desarrollador en Liip AG
   • Blog: http://videlalvaro.github.com/
   • Twitter: @old_sound
   Monday, July 4, 2011
   

   View Slide

4. About Me
   Escribiendo
   RabbitMQ in Action
   http://bit.ly/rabbitmq
   Monday, July 4, 2011
   

   View Slide

5. ¿Por qué necesito usar
   Mensajería?
   Monday, July 4, 2011
   

   View Slide

6. Veamos un ejemplo
   Monday, July 4, 2011
   

   View Slide

7. Implementar una
   Galería de Imágenes
   Monday, July 4, 2011
   

   View Slide

8. Dos Partes:
   Monday, July 4, 2011
   

   View Slide

9. ¿Bastante fácil no?
   Monday, July 4, 2011
   

   View Slide

10. Hasta que nuevos
    requerimientos
    comienzan a llegar
    Monday, July 4, 2011
    

    View Slide

11. El Propietario del
    Producto
    Monday, July 4, 2011
    

    View Slide

12. ¿Podemos notificar a
    los amigos del usuario
    sobre nuevas imágenes?
    Monday, July 4, 2011
    

    View Slide

13. ¿Podemos notificar a
    los amigos del usuario
    sobre nuevas imágenes?
    Me olvidé de decirles que lo necesito para mañana
    Monday, July 4, 2011
    

    View Slide

14. El “Social Media Guru”
    Monday, July 4, 2011
    

    View Slide

15. Necesitamos premiar a
    los usuarios por cada
    foto que suben
    Monday, July 4, 2011
    

    View Slide

16. Necesitamos premiar a
    los usuarios por cada
    foto que suben
    y enviar notificaciones a Twitter
    Monday, July 4, 2011
    

    View Slide

17. El Administrador de
    Sistemas
    Monday, July 4, 2011
    

    View Slide

18. ¡Dolobu! Estamos
    sirviendo imágenes sin
    achicar. ¡La cuenta de
    ancho de banda a
    triplicado!
    Monday, July 4, 2011
    

    View Slide

19. ¡Necesitamos arreglar esto para mañana!
    ¡Dolobu! Estamos
    sirviendo imágenes sin
    achicar. ¡La cuenta de
    ancho de banda a
    triplicado!
    Monday, July 4, 2011
    

    View Slide

20. El Desarrollador en el
    otro equipo
    Monday, July 4, 2011
    

    View Slide

21. Necesito llamar tus
    sistemas PHP pero
    desde Python
    Monday, July 4, 2011
    

    View Slide

22. Necesito llamar tus
    sistemas PHP pero
    desde Python
    Y la semana que viene Java también
    Monday, July 4, 2011
    

    View Slide

23. El Usuario
    Monday, July 4, 2011
    

    View Slide

24. No quiero tener que
    esperar que tu
    aplicación procese la
    imagen
    Monday, July 4, 2011
    

    View Slide

25. Tu
    Monday, July 4, 2011
    

    View Slide

26. FML!
    Monday, July 4, 2011
    

    View Slide

27. Veamos la evolución del
    código
    Monday, July 4, 2011
    

    View Slide

28. %% image_controller
    handle('PUT', "/user/image", ReqData) ->
    image_handler:do_upload(ReqData:get_file()),
    ok.
    Primera implementación:
    Monday, July 4, 2011
    

    View Slide

29. %% image_controller
    handle('PUT', "/user/image", ReqData) ->
    {ok, Image} = image_handler:do_upload(ReqData:get_file()),
    resize_image(Image),
    ok.
    Segunda implementación:
    Monday, July 4, 2011
    

    View Slide

30. %% image_controller
    handle('PUT', "/user/image", ReqData) ->
    {ok, Image} = image_handler:do_upload(ReqData:get_file()),
    resize_image(Image),
    notify_friends(ReqData:get_user()),
    ok.
    Tercera implementación:
    Monday, July 4, 2011
    

    View Slide

31. %% image_controller
    handle('PUT', "/user/image", ReqData) ->
    {ok, Image} = image_handler:do_upload(ReqData:get_file()),
    resize_image(Image),
    notify_friends(ReqData:get_user()),
    add_points_to_user(ReqData:get_user()),
    ok.
    Cuarta implementación:
    Monday, July 4, 2011
    

    View Slide

32. %% image_controller
    handle('PUT', "/user/image", ReqData) ->
    {ok, Image} = image_handler:do_upload(ReqData:get_file()),
    resize_image(Image),
    notify_friends(ReqData:get_user()),
    add_points_to_user(ReqData:get_user()),
    tweet_new_image(User, Image),
    ok.
    Implementación final:
    Monday, July 4, 2011
    

    View Slide

33. ¿Escala nuestro código
    a nuevos
    requerimientos?
    Monday, July 4, 2011
    

    View Slide

34. Qué pasaría si…
    Monday, July 4, 2011
    

    View Slide

35. Qué pasaría si…
    • Necesitamos acelerar la conversión de
    imágenes
    Monday, July 4, 2011
    

    View Slide

36. Qué pasaría si…
    • Necesitamos acelerar la conversión de
    imágenes
    • Las notificaciones a los usuarios tienen que
    ser enviadas por email
    Monday, July 4, 2011
    

    View Slide

37. Qué pasaría si…
    • Necesitamos acelerar la conversión de
    imágenes
    • Las notificaciones a los usuarios tienen que
    ser enviadas por email
    • Tenemos que dejar de twittear sobre
    nuevas imágenes
    Monday, July 4, 2011
    

    View Slide

38. Qué pasaría si…
    • Necesitamos acelerar la conversión de
    imágenes
    • Las notificaciones a los usuarios tienen que
    ser enviadas por email
    • Tenemos que dejar de twittear sobre
    nuevas imágenes
    • Convertir imágenes a diferentes formatos
    Monday, July 4, 2011
    

    View Slide

39. ¿Podemos hacerlo
    mejor?
    Monday, July 4, 2011
    

    View Slide

40. Por supuesto.
    Usando Mensajería.
    Monday, July 4, 2011
    

    View Slide

41. Diseño
    Publish / Subscribe Pattern
    Monday, July 4, 2011
    

    View Slide

42. %% image_controller
    handle('PUT', "/user/image", ReqData) ->
    {ok, Image} = image_handler:do_upload(ReqData:get_file()),
    Msg = #msg{user = ReqData:get_user(), image = Image},
    publish_message('new_image', Msg).
    Primera Implementación:
    Monday, July 4, 2011
    

    View Slide

43. %% image_controller
    handle('PUT', "/user/image", ReqData) ->
    {ok, Image} = image_handler:do_upload(ReqData:get_file()),
    Msg = #msg{user = ReqData:get_user(), image = Image},
    publish_message('new_image', Msg).
    Primera Implementación:
    %% friends notifier
    on('new_image', Msg) ->
    notify_friends(Msg.user, Msg.image).
    Monday, July 4, 2011
    

    View Slide

44. %% image_controller
    handle('PUT', "/user/image", ReqData) ->
    {ok, Image} = image_handler:do_upload(ReqData:get_file()),
    Msg = #msg{user = ReqData:get_user(), image = Image},
    publish_message('new_image', Msg).
    Primera Implementación:
    %% friends notifier
    on('new_image', Msg) ->
    notify_friends(Msg.user, Msg.image).
    %% points manager
    on('new_image', Msg) ->
    add_points(Msg.user, 'new_image').
    Monday, July 4, 2011
    

    View Slide

45. %% image_controller
    handle('PUT', "/user/image", ReqData) ->
    {ok, Image} = image_handler:do_upload(ReqData:get_file()),
    Msg = #msg{user = ReqData:get_user(), image = Image},
    publish_message('new_image', Msg).
    Primera Implementación:
    %% friends notifier
    on('new_image', Msg) ->
    notify_friends(Msg.user, Msg.image).
    %% points manager
    on('new_image', Msg) ->
    add_points(Msg.user, 'new_image').
    %% resizer
    on('new_image', Msg) ->
    resize_image(Msg.image).
    Monday, July 4, 2011
    

    View Slide

46. Segunda Implementación:
    Monday, July 4, 2011
    

    View Slide

47. Segunda Implementación:
    THIS PAGE INTENTIONALLY LEFT BLANK
    Monday, July 4, 2011
    

    View Slide

48. Mensajería
    Monday, July 4, 2011
    

    View Slide

49. Mensajería
    • Compartir datos entre procesos
    Monday, July 4, 2011
    

    View Slide

50. Mensajería
    • Compartir datos entre procesos
    • Procesos pueden ser parte de diferentes
    aplicaciones
    Monday, July 4, 2011
    

    View Slide

51. Mensajería
    • Compartir datos entre procesos
    • Procesos pueden ser parte de diferentes
    aplicaciones
    • Aplicaciones pueden vivir en diferentes
    computadores
    Monday, July 4, 2011
    

    View Slide

52. Mensajería
    • Compartir datos entre procesos
    • Procesos pueden ser parte de diferentes
    aplicaciones
    • Aplicaciones pueden vivir en diferentes
    computadores
    • La comunicación es asíncrona
    Monday, July 4, 2011
    

    View Slide

53. Conceptos Principales
    Monday, July 4, 2011
    

    View Slide

54. Conceptos Principales
    • Mensajes son enviados por Producers
    Monday, July 4, 2011
    

    View Slide

55. Conceptos Principales
    • Mensajes son enviados por Producers
    • Mensajes se envían a Consumers
    Monday, July 4, 2011
    

    View Slide

56. Conceptos Principales
    • Mensajes son enviados por Producers
    • Mensajes se envían a Consumers
    • Mensajes van a través de un Channel
    Monday, July 4, 2011
    

    View Slide

57. RabbitMQ
    y la
    Mensajería
    Monday, July 4, 2011
    

    View Slide

58. ¿Qué es RabbitMQ?
    Monday, July 4, 2011
    

    View Slide

59. RabbitMQ
    • Sistema de Mensajería Empresarial
    • Código Libre MPL
    • Escrito en Erlang/OTP
    • Soporte Comercial
    • Mensajería via AMQP
    Monday, July 4, 2011
    

    View Slide

60. Features
    • Confiable y Altamente Escalable
    • Fácil de Instalar
    • Fácil de Clusterizar
    • Corre en: Windows, Solaris, Linux, OSX
    • AMQP 0.8 - 0.9.1
    Monday, July 4, 2011
    

    View Slide

61. Librerías AMQP
    • Java
    • .NET/C#
    • Erlang
    • Ruby, Python, PHP, Perl, AS3, Lisp, Scala,
    Clojure, Haskell
    Monday, July 4, 2011
    

    View Slide

62. AMQP
    • Advanced Message Queuing Protocol
    • Pensado para la Interoperabilidad
    • Protocolo Completamente Abierto
    • Protocol Binario
    Monday, July 4, 2011
    

    View Slide

63. Flujo de Mensajes
    http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/chap-Messaging_Tutorial-Initial_Concepts.html
    Monday, July 4, 2011
    

    View Slide

64. Modelo AMQP
    • Exchanges
    • Message Queues
    • Bindings
    • Rules for binding them
    Monday, July 4, 2011
    

    View Slide

65. Tipos de Exchange
    • Fanout
    • Direct
    • Topic
    Monday, July 4, 2011
    

    View Slide

66. http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts-
    Fanout_Exchange.html
    Monday, July 4, 2011
    

    View Slide

67. http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts-
    Direct_Exchange.html
    Monday, July 4, 2011
    

    View Slide

68. http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_MRG/1.0/html/Messaging_Tutorial/sect-Messaging_Tutorial-Initial_Concepts-
    Topic_Exchange.html
    Monday, July 4, 2011
    

    View Slide

69. Patrones de Mensajería
    Monday, July 4, 2011
    

    View Slide

70. Les recomiendo un
    libro al respecto:
    http://www.eaipatterns.com/
    Monday, July 4, 2011
    

    View Slide

71. Patrones Básicos
    Monday, July 4, 2011
    

    View Slide

72. Competing Consumers
    How can a messaging
    client process multiple
    messages concurrently?
    Monday, July 4, 2011
    

    View Slide

73. Competing Consumers
    Create multiple Competing
    Consumers on a single channel
    so that the consumers can
    process multiple messages
    concurrently.
    Monday, July 4, 2011
    

    View Slide

74. Competing Consumers
    Monday, July 4, 2011
    

    View Slide

75. Código Publisher
    init(Exchange, Queue) ->
    #'exchange.declare'{exchange = Exchange,
    type = <>,
    durable = true},
    #'queue.declare'{queue = Queue, durable = false},
    #'queue.bind'{queue = Queue, exchange = Exchange}.
    publish_msg(Exchange, Payload) ->
    Props = #'P_basic'{content_type = <>,
    delivery_mode = 2}, %% persistent
    publish(Exchange, #amqp_msg{props = Props, payload = Payload}).
    Monday, July 4, 2011
    

    View Slide

76. Código Consumer
    init_consumer(Exchange, Queue) ->
    init(Exchange, Queue),
    #'basic.consume'{queue = Queue}.
    on(#'basic.deliver'{delivery_tag = DeliveryTag},
    #amqp_msg{} = Msg) ->
    do_something_with_msg(Msg),
    #'basic.ack'{delivery_tag = DeliveryTag}.
    Monday, July 4, 2011
    

    View Slide

77. Publish/Subscribe
    How can the sender
    broadcast an event to all
    interested receivers?
    Monday, July 4, 2011
    

    View Slide

78. Publish/Subscribe
    Send the event on a Publish-Subscribe
    Channel, which delivers a copy of a
    particular event to each receiver.
    Monday, July 4, 2011
    

    View Slide

79. Publish/Subscribe
    Monday, July 4, 2011
    

    View Slide

80. Código Pulbisher
    init(Exchange, Queue) ->
    #'exchange.declare'{exchange = Exchange,
    type = <>, %% different type
    durable = true}
    %% same as before ...
    publish_msg(Exchange, Payload) ->
    Props = #'P_basic'{content_type = <>,
    delivery_mode = 2}, %% persistent
    publish(Exchange, #amqp_msg{props = Props, payload = Payload}).
    Monday, July 4, 2011
    

    View Slide

81. Código Consumer A
    init_consumer(Exchange, ResizeImageQueue) ->
    init(Exchange, ResizeImageQueue),
    #'basic.consume'{queue = ResizeImageQueue}.
    on(#'basic.deliver'{delivery_tag = DeliveryTag},
    #amqp_msg{} = Msg) ->
    resize_message(Msg),
    #'basic.ack'{delivery_tag = DeliveryTag}.
    Monday, July 4, 2011
    

    View Slide

82. Código Consumer B
    init_consumer(Exchange, NotifyFriendsQueue) ->
    init(Exchange, NotifyFriendsQueue),
    #'basic.consume'{queue = NotifyFriendsQueue}.
    on(#'basic.deliver'{delivery_tag = DeliveryTag},
    #amqp_msg{} = Msg) ->
    notify_friends(Msg),
    #'basic.ack'{delivery_tag = DeliveryTag}.
    Monday, July 4, 2011
    

    View Slide

83. Código Consumer C
    init_consumer(Exchange, LogImageUpload) ->
    init(Exchange, LogImageUpload),
    #'basic.consume'{queue = LogImageUpload}.
    on(#'basic.deliver'{delivery_tag = DeliveryTag},
    #amqp_msg{} = Msg) ->
    log_image_upload(Msg),
    #'basic.ack'{delivery_tag = DeliveryTag}.
    Monday, July 4, 2011
    

    View Slide

84. Request/Reply
    When an application sends a message,
    how can it get a response from the
    receiver?
    Monday, July 4, 2011
    

    View Slide

85. Request/Reply
    Send a pair of Request-Reply
    messages, each on its own channel.
    Monday, July 4, 2011
    

    View Slide

86. Request/Reply
    Monday, July 4, 2011
    

    View Slide

87. Return Address
    How does a replier know where to
    send the reply?
    Monday, July 4, 2011
    

    View Slide

88. Return Address
    The request message should contain a
    Return Address that indicates where to send
    the reply message.
    Monday, July 4, 2011
    

    View Slide

89. Return Address
    Monday, July 4, 2011
    

    View Slide

90. Correlation Identifier
    How does a requestor that has
    received a reply know which
    request this is the reply for?
    Monday, July 4, 2011
    

    View Slide

91. Correlation Identifier
    Each reply message should contain a Correlation
    Identifier, a unique identifier that indicates which
    request message this reply is for.
    Monday, July 4, 2011
    

    View Slide

92. Correlation Identifier
    Monday, July 4, 2011
    

    View Slide

93. Atando Cabos
    Monday, July 4, 2011
    

    View Slide

94. Cliente RPC
    init() ->
    #'queue.declare_ok'{queue = SelfQueue} =
    #'queue.declare'{exclusive = true, auto_delete = true},
    #'basic.consume'{queue = SelfQueue, no_ack = true},
    SelfQueue.
    Monday, July 4, 2011
    

    View Slide

95. Cliente RPC
    init() ->
    #'queue.declare_ok'{queue = SelfQueue} =
    #'queue.declare'{exclusive = true, auto_delete = true},
    #'basic.consume'{queue = SelfQueue, no_ack = true},
    SelfQueue.
    request(Payload, RequestId) ->
    Props = #'P_basic'{correlation_id = RequestId,
    reply_to = SelfQueue},
    publish(ServerExchange, #amqp_msg{props = Props,
    payload = Payload}).
    Monday, July 4, 2011
    

    View Slide

96. Cliente RPC
    init() ->
    #'queue.declare_ok'{queue = SelfQueue} =
    #'queue.declare'{exclusive = true, auto_delete = true},
    #'basic.consume'{queue = SelfQueue, no_ack = true},
    SelfQueue.
    request(Payload, RequestId) ->
    Props = #'P_basic'{correlation_id = RequestId,
    reply_to = SelfQueue},
    publish(ServerExchange, #amqp_msg{props = Props,
    payload = Payload}).
    on(#'basic.deliver'{},
    #amqp_msg{props = Props, payload = Payload}) ->
    CorrelationId = Props.correlation_id,
    do_something_with_reply(Payload).
    Monday, July 4, 2011
    

    View Slide

97. Servidor RPC
    on(#'basic.deliver'{},
    #amqp_msg{props = Props, payload = Payload}) ->
    CorrelationId = Props.correlation_id,
    ReplyTo = Props.reply_to,
    Reply = process_request(Payload),
    NewProps = #'P_basic'{correlation_id = CorrelationId},
    publish("", %% anonymous exchange
    #amqp_msg{props = NewProps,
    payload = Reply},
    ReplyTo). %% routing key
    Monday, July 4, 2011
    

    View Slide

98. Patrones Avanzados
    Monday, July 4, 2011
    

    View Slide

99. Control Bus
    How can we effectively administer a messaging
    system that is distributed across multiple
    platforms and a wide geographic area?
    Monday, July 4, 2011
    

    View Slide

100. Control Bus
     Use a Control Bus to
     manage an enterprise
     integration system.
     Monday, July 4, 2011
     

     View Slide

101. Control Bus
     • Send Configuration Messages
     • Start/Stop Services
     • Inject Test Messages
     • Collect Statistics
     Monday, July 4, 2011
     

     View Slide

102. Control Bus
     Monday, July 4, 2011
     

     View Slide

103. Control Bus
     Make Services
     “Control Bus” Enabled
     Monday, July 4, 2011
     

     View Slide

104. Detour
     How can you route a message through
     intermediate steps to perform validation,
     testing or debugging functions?
     Monday, July 4, 2011
     

     View Slide

105. Detour
     Construct a Detour with a context-based router
     controlled via the Control Bus.
     In one state the router routes incoming messages
     through additional steps while in the other it routes
     messages directly to the destination channel.
     Monday, July 4, 2011
     

     View Slide

106. Detour
     Monday, July 4, 2011
     

     View Slide

107. Wire Tap
     How do you inspect messages that
     travel on a point-to-point channel?
     Monday, July 4, 2011
     

     View Slide

108. Wire Tap
     Insert a simple Recipient List into the channel that
     publishes each incoming message to the main
     channel and a secondary channel.
     Monday, July 4, 2011
     

     View Slide

109. Wire Tap
     Monday, July 4, 2011
     

     View Slide

110. Smart Proxy
     How can you track messages on a service that
     publishes reply messages to the Return Address
     specified by the requestor?
     Monday, July 4, 2011
     

     View Slide

111. Smart Proxy
     Use a Smart Proxy to store the Return
     Address supplied by the original
     requestor and replace it with the
     address of the Smart Proxy.
     When the service sends the reply
     message route it to the original Return
     Address.
     Monday, July 4, 2011
     

     View Slide

112. Smart Proxy
     Monday, July 4, 2011
     

     View Slide

113. Créditos
     Imágenes y descripciones de los patrones tomadas de:
     http://www.eaipatterns.com/
     Monday, July 4, 2011
     

     View Slide

114. ¡Gracias!
     @old_sound
     http://vimeo.com/user1169087
     http://www.slideshare.net/old_sound
     Monday, July 4, 2011
     

     View Slide