Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Decouple all the things - Asynchronous messaging keeps it simple

Decouple all the things - Asynchronous messaging keeps it simple

Talk at DevconTLV, March 2016

5e8e44a4f6632772c47925006aff31d9?s=128

Kerstin Puschke

March 22, 2016
Tweet

Transcript

  1. Decouple all the things Asynchronous messaging keeps it simple DevconTLV

    2016
  2. Machine-to-Machine Communication

  3. 418 I’m a Teapot

  4. Improve Maintainability and Stability by adding Asynchronous Communication

  5. Outline Disadvantages of Synchronous Communication AMQP Publish-Subscribe Task Queue Summary

    5
  6. Disadvantages of Synchronous Communication

  7. inform about update Updates need to propagate 7 business partners

    business partner out of business orders support contracts
  8. downside of REST: client misses update if temporarily down 8

    business partners orders support contracts
  9. downside of REST: app needs to know its clients 9

    business partners orders support contracts invoices
  10. xing.com – core updates have to propagate 10 user profile

    data jobs events …
  11. Asynchronous Messaging Advanced Message Queuing Protocol

  12. amqp.org/about/what “The capable, commoditized, multi-vendor communications ecosystem which AMQP enables

    “The capable, commoditized, multi-vendor communications ecosystem which AMQP enables creates opportunities for commerce and innovation “The capable, commoditized, multi-vendor communications ecosystem which AMQP enables creates opportunities for commerce and innovation which can transform the way business is done “The capable, commoditized, multi-vendor communications ecosystem which AMQP enables creates opportunities for commerce and innovation which can transform the way business is done on the Internet “The capable, commoditized, multi-vendor communications ecosystem which AMQP enables creates opportunities for commerce and innovation which can transform the way business is done on the Internet, and in the Cloud.”
  13. “open standard application layer protocol for message-oriented middleware” https://en.wikipedia.org/wiki/ Advanced_Message_Queuing_Protocol

    “open standard application layer protocol for message-oriented middleware”
  14. Interoperability 14 message broker message consumer message producer

  15. Messages contain payload and routing key 15 Routing Key Payload

    •  application data – any form, any encoding… …and more •  Structured app-specific data (properties / attributes) •  Headers •  Annotations •  …
  16. Message Flow 16 P Message Broker X Q C Producer

    Exchange Queue Consumer
  17. No information lost if consumer down 17 X Q Producer

    Exchange Queue Consumer P C
  18. Applications are in power 18 X Q Producer Exchange Queue

    Consumer P C
  19. Consumer subscribes, without touching producer 19 P X Q2 Q1

    C1 C2 Producer Exchange Queue Consumer
  20. Publish - Subscribe

  21. Example Message routing key <...>user.deleted payload 21 { “user_id”: 42

    }
  22. Example Message routing key <...>profile.updated payload 22 { “user_id”: 42,

    “fields”: [“last_name”] }
  23. Why not include updated data? routing key <...>profile.updated payload 23

    { “user_id”: 42, “fields”: [“last_name”] }
  24. No guaranteed order – aim for commutativity Order of messages

    not guarantueed Commutativity Complement with synchronous API (e.g. REST) •  most recent data •  leading system •  message consumers follow up with REST if necessary Small payload •  easier on the broker 24
  25. No exactly-once delivery – aim for idem-potent messages Duplicated messages

    •  possible e.g. if producer re-sends message after connection failure •  use idem-potent message handling 25
  26. Add producers without touching the consumer 26 Producer Exchange Queue

    Consumer P1 P2 X Q C
  27. Centralized tracking of decentralized events 27 P1 P2 X Q

    C event to be tracked tracking counter ++
  28. Avoid n-m routing, keep breaking changes manageable 28 P1 P2

    X Q2 Q1 C1 C2 P1 P2 C2 C1
  29. Notifications / Events - producer state changed 29 Message Broker

    X Q P C event / state change
  30. Task / Command – consumer state to change 30 Message

    Broker X Q P C event / state change
  31. Task Queue

  32. Run tasks asynchronously 32 Message Broker X Q P C

    image upload user facing image processing
  33. Improve response time, avoid downtimes 33 Message Broker X Q

    P C image upload user facing image processing
  34. Scaling on the fly 34 X Q C C P

    migration trigger data migration
  35. Migration without downtime 35 Message Broker X Q P C

    old system user facing new system
  36. Test without impacting users 36 Message Broker X Q P

    C old app user facing shadow calls to new app
  37. In a nutshell •  Interoperability •  Improve response times • 

    Avoid downtimes •  Scale on the fly •  Reduce complexity •  Simplify code •  Easy to get started 37
  38. Decouple all the things!

  39. Who am I Contact •  http://www.kpuschke.eu •  twitter: @titanoboa42 • 

    https://www.xing.com/profile/ Kerstin_Puschke Senior Software Engineer at XING Hamburg •  xing.com social network for business professionals •  debian, perl, javascript, ruby on rails, mysql, redis, riak, . . . We’re hiring in Hamburg, Barcelona,… http://corporate.xing.com/ english/company/careers- at-xing/ The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
  40. www.xing.com Thank you for your attention!

  41. Flexible Routing via different Exchange Types Direct Exchange •  routes

    to all queues whose binding_key equals routing_key Fanout Exchange •  broadcasts to every queue it has a binding for •  ignores routing key and queue name Topic Exchange •  dot-separated words as routing keys, plus wildcards •  routing on multiple attributes Header Exchange •  based on headers, not routing keys •  routing on multiple attributes 41