Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Amazon SNS/SQS with Go - Paul Borlin - Compassion International

GoDays
January 23, 2020

Amazon SNS/SQS with Go - Paul Borlin - Compassion International

Message brokers are a great way to introduce an asynchronous way of coordinating services that is both reliable and scalable while providing a way to handle backpressure. In this session, we will give an introduction to message brokers and show how Amazon SNS and SQS can be used as a modern cloud-base broker solution.

GoDays

January 23, 2020
Tweet

More Decks by GoDays

Other Decks in Technology

Transcript

  1. Amazon SNS/SQS
    with Go
    MAKING HIGH AVAILABILITY, FAILOVER, ASYNCHRONOUS
    COMMUNICATION, AND RELIABLE DELIVERY COME TO LIFE

    View full-size slide

  2. Volunteer
    u Mmmmmm, candy….

    View full-size slide

  3. What will you do about it?

    View full-size slide

  4. You Can Sponsor a Child!
    u https://compassion.de (Deutschland)
    u https://compassionuk.com (United Kingdom)
    u http://www.compassion.ie (Ireland)
    u https://compassion.it (Italia, España)
    u https://compassion.nl (Holland)
    u https://www.selfrance.org/ (France)
    u https://compassion.no (Nordic)
    u https://compassion.se (Sverige)
    u https://compassion.ch (Schweiz)

    View full-size slide

  5. What are
    we talking
    about,
    today?
    u What are Message Brokers?
    u Advantages of using a
    message broker
    u What is Amazon SNS
    u What is Amazon SQS
    u Learn how we communicate
    to SNS/SQS via Go

    View full-size slide

  6. What is a Message Broker?

    View full-size slide

  7. Message Broker
    u A message broker is an intermediary computer program module
    that translates a message from the formal messaging protocol of the
    sender to the formal messaging protocol of the receiver. Message
    brokers are elements in telecommunication or computer networks
    where software applications communicate by exchanging formally-
    defined messages.
    u Analogous to the Postal Service

    View full-size slide

  8. Message Broker

    View full-size slide

  9. Message
    Broker –
    Sender

    View full-size slide

  10. Message
    Broker –
    Receiver

    View full-size slide

  11. Message Broker

    View full-size slide

  12. Message Broker
    SNS/SQS

    View full-size slide

  13. Why Use a Message Broker??

    View full-size slide

  14. Asynchronous
    Communication
    u The publisher doesn’t need to
    “block” while waiting
    u Back-pressure can be regulated

    View full-size slide

  15. Backpressure

    View full-size slide

  16. Backpressure
    Real world example

    View full-size slide

  17. Backpressure

    View full-size slide

  18. More About Message Brokers, Very
    Quickly….

    View full-size slide

  19. High Availability
    u Multiple servers/nodes for one
    “broker”
    u Servers can be synchronized
    u Guaranteed delivery
    u Load balancing
    u Alive when your back-end
    application is not

    View full-size slide

  20. Failover
    u Other nodes in the HA cluster
    can take over if one goes
    down
    u No data lost

    View full-size slide

  21. Distribution
    u Multiple applications need
    the same data
    u Publisher doesn’t need to
    know who gets it
    u No re-deploy or reconfigure
    u Handles the distribution in
    various modes
    u Analogous to passing out
    candy
    u Event Driven or Reactive
    Design Architecture

    View full-size slide

  22. Does no work on data.

    View full-size slide

  23. Applications Contact the Broker, Not the Other Way Around
    Application Broker

    View full-size slide

  24. How Does a
    Broker
    Communicate?
    u Protocol Standards
    u STOMP
    u AMQP
    u HTTP / REST
    u MQTT
    u JMS
    u RPC
    u OpenWire
    u RSS and Atom
    u More…..

    View full-size slide

  25. Additional Features
    u Retry logic
    u Dead Letter Queue
    u Time-To-Live (TTL)
    u Auto Delete
    u Persistence or Transient
    u Limits
    u Different Types of Distribution

    View full-size slide

  26. Message Brokers on the Market
    u Amazon SQS/SNS
    u ActiveMQ (Apache)
    u Amazon MQ
    u Kafka (Apache)
    u RabbitMQ (Pivitol)
    u BizTalk Server (Microsoft)
    u HornetQ (RedHat)
    u IBM MQ
    u MSMQ (Microsoft)

    View full-size slide

  27. When Not to
    Use a Message
    Broker
    Big Messages
    Synchronous Communication Needed
    or Desired
    It’s Simply Done By Direct
    Communication (Don’t over-
    architect)

    View full-size slide

  28. Great Uses for
    a Message
    Broker
    Offload work to be done later
    Need a quicker response to user
    Need to handle
    backpressure at various
    times of day
    Why spin up more nodes if you
    don’t need to?
    More than one receiver

    View full-size slide

  29. Amazon SNS
    (Simple Notification Service)

    View full-size slide

  30. Amazon SQS
    (Simple Queueing Service)

    View full-size slide

  31. SQS
    SNS
    Service
    Service
    Service Service Service

    View full-size slide

  32. Advantages of Amazon SNS/SQS
    Over Other Brokers
    u HTTP is simple
    u SDK is well built and ready to use
    u No need for a “shutdown”
    u Hosted for you

    View full-size slide

  33. (Potential) Disadvantages of
    Amazon SNS/SQS Versus Other
    Brokers
    u Polling
    u Cost
    u May not be as “fast” as other brokers
    u Messages have a max life of 14 days
    u Need advanced features in other brokers

    View full-size slide

  34. Now, let’s see
    something
    cool…
    Instead of my
    PowerPoint.

    View full-size slide

  35. Connect with me
    Get the code and slides
    u @PaulBorlin
    u https://www.linkedin.com/in/paulborlin/
    u https://paulborlin.com

    View full-size slide