Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Your Lambda function might execute twice. Be pr...

Your Lambda function might execute twice. Be prepared! (ServerlessDays Zurich)

Are you confused when scheduled Lambdas execute twice, SNS messages trigger an invocation three times, your handmade S3 inventory is out of sync because events occurred twice? Bad news: Sooner or later, your Lambda function executes multiple times. You have to prepare yourself! The reasons are system failures, retries on errors, and event sources that guarantee at-least-once delivery (e.g., CloudWatch Events, SNS, …). In this talk, you learn why Lambda functions might execute twice and what you can do about it focusing on Lambda, DynamoDB, and Kinesis Data Streams.

Michael Wittig

April 11, 2019
Tweet

More Decks by Michael Wittig

Other Decks in Technology

Transcript

  1. Michael Wittig 2 • AWS in Action • Rapid Docker

    on AWS • cloudonaut.io • Freelancer [email protected] @hellomichibye
  2. 7

  3. No exactly once Lambda functions / event sources do not

    guarantee exactly-once delivery! 10
  4. 14 1 Execution 2 1. Retry after random delay Execution

    3 2. Retry after random delay Execution Error Error Error Dead Letter Queue Event Invoke Success Success Success Not 202 Error
  5. “ Invocations occur at least once in response to an

    event and functions must be idempotent to handle this. - AWS in 2017 16 https://web.archive.org/web/20171208134129/http://docs.aws.amazon.com/de_de/lambda/latest/dg/API_Invoke.html
  6. 17

  7. “ When an error occurs, your function may be invoked

    multiple times. Retry behavior varies by error type, client, event source, and invocation type. - AWS in 2019 18 https://docs.aws.amazon.com/de_de/lambda/latest/dg/API_Invoke.html
  8. “ Executing your Lambda function [1...N] times with the same

    event must eventually lead to the same result (after retries) 20
  9. 21 Error Success Output = Input Invoke ... { "a":

    7 } exports.h = async (e) => { return {b: e.a}; }; { "b": 7 }
  10. 22 Error Success Output = f(Input) Invoke ... { "a":

    7 } exports.h = async (e) => { return {b: e.a*e.a}; }; { "b": 49 }
  11. 23 Error Success Store data in data store Invoke ...

    { "mail": "y" } exports.h = async (e) => { u = await db.create(e.mail); return {id: u.id}; }; { "id": ? }
  12. 24 Error Success Make HTTP POST Invoke ... { "mail":

    "y" } exports.h = async (e) => { await http.post({ url: 'mailch.imp/add, data: e }); return {}; }; {}
  13. 25 Error Success Get data from data store Invoke ...

    { "mail": "x" } exports.h = async (e) => { u = await db.get(e.mail); if (u === null) { return {ok: false}; } else { return {ok: true}; } }; { "ok": ? }
  14. “ Executing your Lambda function [1...N] times with the same

    event must eventually lead to the same result (after retries) when only our functions mutaes the external state 26
  15. 27 Error Success Get data from data store Invoke ...

    { "mail": "x" } exports.h = async (e) => { u = await db.get(e.mail); if (u === null) { return {ok: false}; } else { return {ok: true}; } }; { "ok": true } x: u123 y: u456
  16. Tricks ○ Client generated token for uniqueness ○ Avoid system

    time ○ Let the client pass the state into the system 29 https://github.com/widdix/serverlessdays-zurich-2019
  17. ConditionExpression DynamoDB supports conditional writes: ○ AND, OR, NOT ○

    =, <>, <, <=, >, >= ○ attribute_not_exists(attr) ○ attribute_exists(attr) ○ begins_with(attr, str) 30 https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.OperatorsAndFunctions.html
  18. Credits Special thanks to all the people who made and

    released these awesome resources for free: ○ Presentation template by SlidesCarnival ○ Photographs by Unsplash and Pexels 32
  19. 33 Michael Wittig • AWS in Action • Rapid Docker

    on AWS • cloudonaut.io • Freelancer • marbot.io [email protected] @hellomichibye