Introducing SQS - Speaker Deck

Slide 1

Slide 1 text

Introducing SQS

Slide 2

Slide 2 text

Hello! I am Michael Wittig I like to build distributed but simple systems You can find me at @hellomichibye

Slide 3

Slide 3 text

1. WHAT IS SQS? Introducing Amazon Simple Queue Service

Slide 4

Slide 4 text

“ Amazon SQS is a fast, reliable, scalable, fully managed message queuing service. SQS makes it simple and cost- effective to decouple the components of a cloud application. https://aws.amazon.com/sqs

Slide 5

Slide 5 text

What is SQS? Distributed Queue System Scales automatically Fault Tolerant Managed Service

Slide 6

Slide 6 text

Pricing $0.50 per 1 million requests 1 to 10 messages, up to a maximum total payload of 256KB per request payload of 64KB is billed as 1 request.

Slide 7

Slide 7 text

2. WHY USING SQS? Motivation and Use Cases

Slide 8

Slide 8 text

Loose Coupling Decouple services to increase fault tolerance and allow scaling

Slide 9

Slide 9 text

Well Known Approach Load Balancer

Slide 10

Slide 10 text

Synchronous Answer requests immediately Decoupling Asynchronous Process requests sometime

Slide 11

Slide 11 text

Use Cases Scanning Git repositories (AutoScout24) Buffering DynamoDB writes (Shazam) Recording activity within mobile games (Scopely) Invalidating multi-region cache ...

Slide 12

Slide 12 text

Asynchronous Decoupling

Slide 13

Slide 13 text

3. HOW DOES SQS WORK? Looking Under The Hood

Slide 14

Slide 14 text

Parallelism Multiple clients acting as readers or writes are supported.

Slide 15

Slide 15 text

Queue Named bucket containing messages. Terms Message Can be written, read, and deleted from queue.

Slide 16

Slide 16 text

Limitations Message order not guaranteed At least once delivery Eventual consistency

Slide 17

Slide 17 text

Why? Distributed System

Slide 18

Slide 18 text

SQS Architecture Source: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/DistributedQueues.html

Slide 19

Slide 19 text

Solves the problem of “At least once” delivery Idempotence

Slide 20

Slide 20 text

4. MESSAGE LIFECYCLE Deep Dive

Slide 21

Slide 21 text

Simplified Lifecycle Read Delete Create

Slide 22

Slide 22 text

Lifecycle (1) Source: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/MessageLifecycle.html

Slide 23

Slide 23 text

Lifecycle (2) Source: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/MessageLifecycle.html

Slide 24

Slide 24 text

Lifecycle (3) Source: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/MessageLifecycle.html

Slide 25

Slide 25 text

5. COMPARISON Compare with other solutions

Slide 26

Slide 26 text

Kinesis + Kafka Shards for scaling Message order guaranteed within partition key Client manages reading position

Slide 27

Slide 27 text

RabbitMQ Full-featured message broker Flexible routing of messages

Slide 28

Slide 28 text

6. ADVANCED TOPICS Even more insights into SQS

Slide 29

Slide 29 text

Dead Letter Queue Collecting failed messages Message reached Visibility Timeout several times React to messages in DLQ Trigger alerting Forward to human ...

Slide 30

Slide 30 text

Delay Message Message stays invisible after being inserted into queue Configurable based on … Message Queue

Slide 31

Slide 31 text

Long Polling Reduce number of empty read responses Wait between 1 and 20 seconds Configurable based on … Read request Queue

Slide 32

Slide 32 text

Thanks! Any questions?

Slide 33

Slide 33 text

Credits Special thanks to all the people who made and released these awesome resources for free: Presentation template by SlidesCarnival Photographs by Unsplash