Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Amazon SNS/SQS with Go - Paul Borlin - Compassion International

January 23, 2020

Amazon SNS/SQS with Go - Paul Borlin - Compassion International

Message brokers are a great way to introduce an asynchronous way of coordinating services that is both reliable and scalable while providing a way to handle backpressure. In this session, we will give an introduction to message brokers and show how Amazon SNS and SQS can be used as a modern cloud-base broker solution.


January 23, 2020

More Decks by GoDays

Other Decks in Technology



  2. Volunteer u Mmmmmm, candy….

  3. None
  4. None
  5. What will you do about it?

  6. Paul Borlin

  7. Paul Borlin

  8. None
  9. None
  10. None
  11. None
  12. You Can Sponsor a Child! u https://compassion.de (Deutschland) u https://compassionuk.com

    (United Kingdom) u http://www.compassion.ie (Ireland) u https://compassion.it (Italia, España) u https://compassion.nl (Holland) u https://www.selfrance.org/ (France) u https://compassion.no (Nordic) u https://compassion.se (Sverige) u https://compassion.ch (Schweiz)
  13. What are we talking about, today? u What are Message

    Brokers? u Advantages of using a message broker u What is Amazon SNS u What is Amazon SQS u Learn how we communicate to SNS/SQS via Go
  14. What is a Message Broker?

  15. Message Broker u A message broker is an intermediary computer

    program module that translates a message from the formal messaging protocol of the sender to the formal messaging protocol of the receiver. Message brokers are elements in telecommunication or computer networks where software applications communicate by exchanging formally- defined messages. u Analogous to the Postal Service
  16. Message Broker

  17. Message Broker – Sender

  18. Message Broker – Receiver

  19. Message Broker

  20. Message Broker SNS/SQS

  21. Why Use a Message Broker??

  22. Asynchronous Communication u The publisher doesn’t need to “block” while

    waiting u Back-pressure can be regulated
  23. Backpressure

  24. Backpressure Real world example

  25. Backpressure

  26. More About Message Brokers, Very Quickly….

  27. High Availability u Multiple servers/nodes for one “broker” u Servers

    can be synchronized u Guaranteed delivery u Load balancing u Alive when your back-end application is not
  28. Failover u Other nodes in the HA cluster can take

    over if one goes down u No data lost
  29. Distribution u Multiple applications need the same data u Publisher

    doesn’t need to know who gets it u No re-deploy or reconfigure u Handles the distribution in various modes u Analogous to passing out candy u Event Driven or Reactive Design Architecture
  30. Does no work on data.

  31. Applications Contact the Broker, Not the Other Way Around Application

  32. How Does a Broker Communicate? u Protocol Standards u STOMP

    u AMQP u HTTP / REST u MQTT u JMS u RPC u OpenWire u RSS and Atom u More…..
  33. Additional Features u Retry logic u Dead Letter Queue u

    Time-To-Live (TTL) u Auto Delete u Persistence or Transient u Limits u Different Types of Distribution
  34. Message Brokers on the Market u Amazon SQS/SNS u ActiveMQ

    (Apache) u Amazon MQ u Kafka (Apache) u RabbitMQ (Pivitol) u BizTalk Server (Microsoft) u HornetQ (RedHat) u IBM MQ u MSMQ (Microsoft)
  35. When Not to Use a Message Broker Big Messages Synchronous

    Communication Needed or Desired It’s Simply Done By Direct Communication (Don’t over- architect)
  36. Great Uses for a Message Broker Offload work to be

    done later Need a quicker response to user Need to handle backpressure at various times of day Why spin up more nodes if you don’t need to? More than one receiver
  37. Amazon SNS (Simple Notification Service)

  38. Amazon SQS (Simple Queueing Service)

  39. SQS SNS Service Service Service Service Service

  40. Advantages of Amazon SNS/SQS Over Other Brokers u HTTP is

    simple u SDK is well built and ready to use u No need for a “shutdown” u Hosted for you
  41. (Potential) Disadvantages of Amazon SNS/SQS Versus Other Brokers u Polling

    u Cost u May not be as “fast” as other brokers u Messages have a max life of 14 days u Need advanced features in other brokers
  42. Now, let’s see something cool… Instead of my PowerPoint.

  43. Connect with me Get the code and slides u @PaulBorlin

    u https://www.linkedin.com/in/paulborlin/ u https://paulborlin.com