Decouple all the things - Asynchronous messaging keeps it simple

Talk at DevconTLV, March 2016

Transcript

1. Decouple all the things Asynchronous messaging keeps it simple DevconTLV 2016
2. Machine-to-Machine Communication
3. 418 I’m a Teapot
4. Improve Maintainability and Stability by adding Asynchronous Communication
5. Outline Disadvantages of Synchronous Communication AMQP Publish-Subscribe Task Queue Summary 5
6. Disadvantages of Synchronous Communication
7. inform about update Updates need to propagate 7 business partners business partner out of business orders support contracts
8. downside of REST: client misses update if temporarily down 8 business partners orders support contracts
9. downside of REST: app needs to know its clients 9 business partners orders support contracts invoices
10. xing.com – core updates have to propagate 10 user profile data jobs events …
11. Asynchronous Messaging Advanced Message Queuing Protocol
12. amqp.org/about/what “The capable, commoditized, multi-vendor communications ecosystem which AMQP enables “The capable, commoditized, multi-vendor communications ecosystem which AMQP enables creates opportunities for commerce and innovation “The capable, commoditized, multi-vendor communications ecosystem which AMQP enables creates opportunities for commerce and innovation which can transform the way business is done “The capable, commoditized, multi-vendor communications ecosystem which AMQP enables creates opportunities for commerce and innovation which can transform the way business is done on the Internet “The capable, commoditized, multi-vendor communications ecosystem which AMQP enables creates opportunities for commerce and innovation which can transform the way business is done on the Internet, and in the Cloud.”
13. “open standard application layer protocol for message-oriented middleware” https://en.wikipedia.org/wiki/ Advanced_Message_Queuing_Protocol “open standard application layer protocol for message-oriented middleware”
14. Interoperability 14 message broker message consumer message producer
15. Messages contain payload and routing key 15 Routing Key Payload •  application data – any form, any encoding… …and more •  Structured app-specific data (properties / attributes) •  Headers •  Annotations •  …
16. Message Flow 16 P Message Broker X Q C Producer Exchange Queue Consumer
17. No information lost if consumer down 17 X Q Producer Exchange Queue Consumer P C
18. Applications are in power 18 X Q Producer Exchange Queue Consumer P C
19. Consumer subscribes, without touching producer 19 P X Q2 Q1 C1 C2 Producer Exchange Queue Consumer
20. Publish - Subscribe
21. Example Message routing key <...>user.deleted payload 21 { “user_id”: 42 }
22. Example Message routing key <...>profile.updated payload 22 { “user_id”: 42, “fields”: [“last_name”] }
23. Why not include updated data? routing key <...>profile.updated payload 23 { “user_id”: 42, “fields”: [“last_name”] }
24. No guaranteed order – aim for commutativity Order of messages not guarantueed Commutativity Complement with synchronous API (e.g. REST) •  most recent data •  leading system •  message consumers follow up with REST if necessary Small payload •  easier on the broker 24
25. No exactly-once delivery – aim for idem-potent messages Duplicated messages •  possible e.g. if producer re-sends message after connection failure •  use idem-potent message handling 25
26. Add producers without touching the consumer 26 Producer Exchange Queue Consumer P1 P2 X Q C
27. Centralized tracking of decentralized events 27 P1 P2 X Q C event to be tracked tracking counter ++
28. Avoid n-m routing, keep breaking changes manageable 28 P1 P2 X Q2 Q1 C1 C2 P1 P2 C2 C1
29. Notifications / Events - producer state changed 29 Message Broker X Q P C event / state change
30. Task / Command – consumer state to change 30 Message Broker X Q P C event / state change
31. Task Queue
32. Run tasks asynchronously 32 Message Broker X Q P C image upload user facing image processing
33. Improve response time, avoid downtimes 33 Message Broker X Q P C image upload user facing image processing
34. Scaling on the fly 34 X Q C C P migration trigger data migration
35. Migration without downtime 35 Message Broker X Q P C old system user facing new system
36. Test without impacting users 36 Message Broker X Q P C old app user facing shadow calls to new app
37. In a nutshell •  Interoperability •  Improve response times •  Avoid downtimes •  Scale on the fly •  Reduce complexity •  Simplify code •  Easy to get started 37
38. Decouple all the things!
39. Who am I Contact •  http://www.kpuschke.eu •  twitter: @titanoboa42 •  https://www.xing.com/profile/ Kerstin_Puschke Senior Software Engineer at XING Hamburg •  xing.com social network for business professionals •  debian, perl, javascript, ruby on rails, mysql, redis, riak, . . . We’re hiring in Hamburg, Barcelona,… http://corporate.xing.com/ english/company/careers- at-xing/ The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x still appears, you may have to delete the image and then insert it again.
40. www.xing.com Thank you for your attention!
41. Flexible Routing via different Exchange Types Direct Exchange •  routes to all queues whose binding_key equals routing_key Fanout Exchange •  broadcasts to every queue it has a binding for •  ignores routing key and queue name Topic Exchange •  dot-separated words as routing keys, plus wildcards •  routing on multiple attributes Header Exchange •  based on headers, not routing keys •  routing on multiple attributes 41