Amazon SNS/SQS with Go - Paul Borlin - Compassion International

Message brokers are a great way to introduce an asynchronous way of coordinating services that is both reliable and scalable while providing a way to handle backpressure. In this session, we will give an introduction to message brokers and show how Amazon SNS and SQS can be used as a modern cloud-base broker solution.

Transcript

1. Amazon SNS/SQS with Go MAKING HIGH AVAILABILITY, FAILOVER, ASYNCHRONOUS COMMUNICATION, AND RELIABLE DELIVERY COME TO LIFE
2. Volunteer u Mmmmmm, candy….
5. What will you do about it?
6. Paul Borlin
7. Paul Borlin
12. You Can Sponsor a Child! u https://compassion.de (Deutschland) u https://compassionuk.com (United Kingdom) u http://www.compassion.ie (Ireland) u https://compassion.it (Italia, España) u https://compassion.nl (Holland) u https://www.selfrance.org/ (France) u https://compassion.no (Nordic) u https://compassion.se (Sverige) u https://compassion.ch (Schweiz)
13. What are we talking about, today? u What are Message Brokers? u Advantages of using a message broker u What is Amazon SNS u What is Amazon SQS u Learn how we communicate to SNS/SQS via Go
14. What is a Message Broker?
15. Message Broker u A message broker is an intermediary computer program module that translates a message from the formal messaging protocol of the sender to the formal messaging protocol of the receiver. Message brokers are elements in telecommunication or computer networks where software applications communicate by exchanging formally- defined messages. u Analogous to the Postal Service
16. Message Broker
17. Message Broker – Sender
18. Message Broker – Receiver
19. Message Broker
20. Message Broker SNS/SQS
21. Why Use a Message Broker??
22. Asynchronous Communication u The publisher doesn’t need to “block” while waiting u Back-pressure can be regulated
23. Backpressure
24. Backpressure Real world example
25. Backpressure
26. More About Message Brokers, Very Quickly….
27. High Availability u Multiple servers/nodes for one “broker” u Servers can be synchronized u Guaranteed delivery u Load balancing u Alive when your back-end application is not
28. Failover u Other nodes in the HA cluster can take over if one goes down u No data lost
29. Distribution u Multiple applications need the same data u Publisher doesn’t need to know who gets it u No re-deploy or reconfigure u Handles the distribution in various modes u Analogous to passing out candy u Event Driven or Reactive Design Architecture
30. Does no work on data.
31. Applications Contact the Broker, Not the Other Way Around Application Broker
32. How Does a Broker Communicate? u Protocol Standards u STOMP u AMQP u HTTP / REST u MQTT u JMS u RPC u OpenWire u RSS and Atom u More…..
33. Additional Features u Retry logic u Dead Letter Queue u Time-To-Live (TTL) u Auto Delete u Persistence or Transient u Limits u Different Types of Distribution
34. Message Brokers on the Market u Amazon SQS/SNS u ActiveMQ (Apache) u Amazon MQ u Kafka (Apache) u RabbitMQ (Pivitol) u BizTalk Server (Microsoft) u HornetQ (RedHat) u IBM MQ u MSMQ (Microsoft)
35. When Not to Use a Message Broker Big Messages Synchronous Communication Needed or Desired It’s Simply Done By Direct Communication (Don’t over- architect)
36. Great Uses for a Message Broker Offload work to be done later Need a quicker response to user Need to handle backpressure at various times of day Why spin up more nodes if you don’t need to? More than one receiver
37. Amazon SNS (Simple Notification Service)
38. Amazon SQS (Simple Queueing Service)
39. SQS SNS Service Service Service Service Service
40. Advantages of Amazon SNS/SQS Over Other Brokers u HTTP is simple u SDK is well built and ready to use u No need for a “shutdown” u Hosted for you
41. (Potential) Disadvantages of Amazon SNS/SQS Versus Other Brokers u Polling u Cost u May not be as “fast” as other brokers u Messages have a max life of 14 days u Need advanced features in other brokers
42. Now, let’s see something cool… Instead of my PowerPoint.
43. Connect with me Get the code and slides u @PaulBorlin u https://www.linkedin.com/in/paulborlin/ u https://paulborlin.com