Your Lambda function might execute twice. Be prepared! (ServerlessDays Zurich)

Slide 1

Slide 1 text

1 Your Lambda function might execute twice. Be prepared! Michael Wittig cloudonaut.io

Slide 2

Slide 2 text

Michael Wittig 2 ● AWS in Action ● Rapid Docker on AWS ● cloudonaut.io ● Freelancer [email protected] @hellomichibye

Slide 3

Slide 3 text

3 marbot.io - Incident Management via Slack 100% serverless

Slide 4

Slide 4 text

Demo “Something is wrong here” 4 https://github.com/widdix/serverlessdays-zurich-2019

Slide 5

Slide 5 text

5 Cron rate(5 mins)?!

Slide 6

Slide 6 text

6 Cron rate(5 mins)?!

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Retry In case of an (internal) error, Lambda executes your code again. 8

Slide 9

Slide 9 text

Surprised? 9

Slide 10

Slide 10 text

No exactly once Lambda functions / event sources do not guarantee exactly-once delivery! 10

Slide 11

Slide 11 text

Lambda internals on-demand invocation 1

Slide 12

Slide 12 text

12 1 Execution RequestResponse Invoke Success Data Error Error Not 200 Error

Slide 13

Slide 13 text

13 “Event” Retries

Slide 14

Slide 14 text

14 1 Execution 2 1. Retry after random delay Execution 3 2. Retry after random delay Execution Error Error Error Dead Letter Queue Event Invoke Success Success Success Not 202 Error

Slide 15

Slide 15 text

15 Random delays between “Event” retries 1 2 3 1 2 3 Time Invoke A Invoke B

Slide 16

Slide 16 text

“ Invocations occur at least once in response to an event and functions must be idempotent to handle this. - AWS in 2017 16 https://web.archive.org/web/20171208134129/http://docs.aws.amazon.com/de_de/lambda/latest/dg/API_Invoke.html

Slide 17

Slide 17 text

Slide 18

Slide 18 text

“ When an error occurs, your function may be invoked multiple times. Retry behavior varies by error type, client, event source, and invocation type. - AWS in 2019 18 https://docs.aws.amazon.com/de_de/lambda/latest/dg/API_Invoke.html

Slide 19

Slide 19 text

2 Idempotence A powerful concept

Slide 20

Slide 20 text

“ Executing your Lambda function [1...N] times with the same event must eventually lead to the same result (after retries) 20

Slide 21

Slide 21 text

21 Error Success Output = Input Invoke ... { "a": 7 } exports.h = async (e) => { return {b: e.a}; }; { "b": 7 }

Slide 22

Slide 22 text

22 Error Success Output = f(Input) Invoke ... { "a": 7 } exports.h = async (e) => { return {b: e.a*e.a}; }; { "b": 49 }

Slide 23

Slide 23 text

23 Error Success Store data in data store Invoke ... { "mail": "y" } exports.h = async (e) => { u = await db.create(e.mail); return {id: u.id}; }; { "id": ? }

Slide 24

Slide 24 text

24 Error Success Make HTTP POST Invoke ... { "mail": "y" } exports.h = async (e) => { await http.post({ url: 'mailch.imp/add, data: e }); return {}; }; {}

Slide 25

Slide 25 text

25 Error Success Get data from data store Invoke ... { "mail": "x" } exports.h = async (e) => { u = await db.get(e.mail); if (u === null) { return {ok: false}; } else { return {ok: true}; } }; { "ok": ? }

Slide 26

Slide 26 text

“ Executing your Lambda function [1...N] times with the same event must eventually lead to the same result (after retries) when only our functions mutaes the external state 26

Slide 27

Slide 27 text

27 Error Success Get data from data store Invoke ... { "mail": "x" } exports.h = async (e) => { u = await db.get(e.mail); if (u === null) { return {ok: false}; } else { return {ok: true}; } }; { "ok": true } x: u123 y: u456

Slide 28

Slide 28 text

Tricks How to create idempotent functions 3

Slide 29

Slide 29 text

Tricks ￮ Client generated token for uniqueness ￮ Avoid system time ￮ Let the client pass the state into the system 29 https://github.com/widdix/serverlessdays-zurich-2019

Slide 30

Slide 30 text

ConditionExpression DynamoDB supports conditional writes: ￮ AND, OR, NOT ￮ =, <>, <, <=, >, >= ￮ attribute_not_exists(attr) ￮ attribute_exists(attr) ￮ begins_with(attr, str) 30 https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.OperatorsAndFunctions.html

Slide 31

Slide 31 text

31 Serialize work with Kinesis Data Streams A Time Iterator B Iterator C Iterator

Slide 32

Slide 32 text

Credits Special thanks to all the people who made and released these awesome resources for free: ￮ Presentation template by SlidesCarnival ￮ Photographs by Unsplash and Pexels 32

Slide 33

Slide 33 text

33 Michael Wittig ● AWS in Action ● Rapid Docker on AWS ● cloudonaut.io ● Freelancer ● marbot.io [email protected] @hellomichibye