Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Your Lambda function might execute twice. Be pr...

Your Lambda function might execute twice. Be prepared!

Are you confused when scheduled Lambdas execute twice, SNS messages trigger an invocation three times, your handmade S3 inventory is out of sync because events occurred twice? Bad news: Sooner or later, your Lambda function will be invoked multiple times. You have to be prepared! The reasons are retries on errors and event sources that guarantee at-least-once delivery (e.g., CloudWatch Events, SNS, …).In this talk, you will learn why Lambda functions might execute twice and what you can do about it.

Michael Wittig

September 26, 2018
Tweet

More Decks by Michael Wittig

Other Decks in Programming

Transcript

  1. Michael Wittig ○ Author of AWS in Action ○ https://cloudonaut.io

    ○ freelance AWS consultant and programmer ○ AWS Community Hero 3
  2. 7

  3. 10 A On-Demand Invocation By calling the API directly B

    Event Source Invocation 1 Invocation Type RequestResponse 2 Invocation Type Event 1 Poll based Executed by Lambda a Stream based Kinesis, DynamoDB b Others SQS 2 Push based Executed by Service a Synchronous API Gateway, Cognito b Asynchronous S3, SNS, SES, CW Exactly once guaranteed?
  4. No exactly once Lambda functions / event sourced do not

    guarantee exactly-once delivery! 11
  5. On Handled or Unhandled error: Lambda executes your code once.

    But when StatusCode <> 200 there is no invocation at all. https://docs.aws.amazon.com/lambda/lates t/dg/API_Invoke.html 13 A On-Demand Invocation 1 Invocation Type RequestResponse
  6. On Handled or Unhandled error: Lambda retries executing your code

    twice and therefore executes your code at-least-once [1...3]. But when StatusCode <> 202 there is no invocation at all. https://docs.aws.amazon.com/lambda/lates t/dg/retries-on-errors.html 15 A On-Demand Invocation 1 Invocation Type Event
  7. 17 1 Execution 2 1. Retry after random delay Execution

    3 2. Retry after random delay Execution Error Error Error Dead Letter Queue Event Invoke Success Success Success Not 202 Error
  8. “ If you use [...] Event [...], the function will

    be invoked at least once [...] and the function must be idempotent to handle this. 19 https://docs.aws.amazon.com/de_de/lambda/latest/dg/API_Invoke.html
  9. “ With the same event execute your Lambda function code

    [1...N] times the result is eventually the same (after retries) 21
  10. 22 Error Success Output = Input Invoke ... { "a":

    7 } exports.h = async (e) => { return {b: e.a}; }; { "b": 7 }
  11. 23 Error Success Output = f(Input) Invoke ... { "a":

    7 } exports.h = async (e) => { return {b: e.a*e.a}; }; { "b": 49 }
  12. 24 Error Success Store data in data store Invoke ...

    { "mail": "y" } exports.h = async (e) => { u = await db.create(e.mail); return {id: u.id}; }; { "id": ? }
  13. 25 Error Success Make HTTP POST Invoke ... { "mail":

    "y" } exports.h = async (e) => { await http.post({ url: 'mailch.imp/add, data: e }); return {}; }; {}
  14. 26 Error Success Get data from data store Invoke ...

    { "mail": "x" } exports.h = async (e) => { u = await db.get(e.mail); if (u === null) { return {ok: false}; } else { return {ok: true}; } }; { "ok": ? }
  15. “ with the same event execute your Lambda function code

    [1...N] times the result is eventually the same (after retries) when only our code mutaes the external state 27
  16. 28 Error Success Get data from data store Invoke ...

    { "mail": "x" } exports.h = async (e) => { u = await db.get(e.mail); if (u === null) { return {ok: false}; } else { return {ok: true}; } }; { "ok": true } x: u123 y: u456
  17. Primary Key Partition key plus optional sort key identifies item.

    DynamoDB No enforced schema You can add attributes to an item at any time. 30 Read GetItem Query Scan Mutate PutItem UpdateItem DeleteItem
  18. Idea 1 Status update creation time as an input (part

    of the event) Keep order Idea 2 Previous status update as an input (part of the event) 34 Idea 3 Use Kinesis Data Stream in between Idea 4 Use Kinesis Data Stream in between with sequence numbers
  19. “ PutItem If an item that has the same primary

    key as an existing item, the new item completely replaces the existing item. 35 https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html
  20. ConditionExpression DynamoDB supports conditional writes: ○ AND, OR, NOT ○

    =, <>, <, <=, >, >= ○ attribute_not_exists(attr) ○ attribute_exists(attr) ○ begins_with(attr, str) 39 https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.Op eratorsAndFunctions.html
  21. “ UpdateItem If an item that has the same primary

    key as an existing item, only the attributes in UpdateExpression are touched. 45 https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_UpdateItem.ht ml
  22. “ SequenceNumberForOrdering Set the SequenceNumber ForOrdering of record n to

    the sequence number of record n-1 (as returned in the result when putting record n-1). 48 https://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.html
  23. Credits Special thanks to all the people who made and

    released these awesome resources for free: ○ Presentation template by SlidesCarnival ○ Photographs by Unsplash and Pexels 51
  24. Michael Wittig ○ Author of AWS in Action ○ https://cloudonaut.io

    ○ freelance AWS consultant and programmer ○ AWS Community Hero 52
  25. Place your screenshot here 53 AWS in Action 2nd edition

    now available Use code ctwawscd18 on manning.com and save 40%