Slide 1

Slide 1 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Gunnar Grosch @gunnargrosch October 30, 2020 Continuous verification for serverless applications Serverless WEEK

Slide 2

Slide 2 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Testing to ensure that you can meet your availability goals is the only way you can have confidence that you will meet those goals” Reliability Pillar AWS Well-Architected Framework

Slide 3

Slide 3 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Verification of applications

Slide 4

Slide 4 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Verification of applications Objectives Verify Measures Conditions

Slide 5

Slide 5 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Verification of applications Objectives Verify Measures Conditions

Slide 6

Slide 6 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Conditions for verification Errors in your code Security policy errors Service configuration errors Function disk space failure Downstream service issues Concurrency and throttling Latency Client Amazon Simple Storage Service (Amazon S3) Amazon API Gateway AWS Lambda Amazon DynamoDB AWS Lambda Amazon Simple Storage Service (Amazon S3)

Slide 7

Slide 7 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Tools for adding conditions Chaos-lambda Python Failure-lambda NodeJS

Slide 8

Slide 8 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Failure-lambda NodeJS NPM package for NodeJS Lambdas Configuration using Parameter Store or AWS AppConfig Several failure modes • Latency • Status code • Exception • Disk space • Denylist const failureLambda = require('failure-lambda’) exports.handler = failureLambda(async (event, context) => { ... }) { "isEnabled": false, "failureMode": "latency", "rate": 1, "minLatency": 100, "maxLatency": 400, "exceptionMsg": "Exception message!", "statusCode": 404, "diskSpace": 100, “denylist": [ "s3.*.amazonaws.com", "dynamodb.*.amazonaws.com" ] }

Slide 9

Slide 9 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Verification of applications Objectives Verify Measures Conditions

Slide 10

Slide 10 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Verification example Objective: My purchase API should respond in less than 400 ms Measure: 400 ms threshold Condition: 100-200 ms latency injection to function Verify: Pass or fail Client Amazon API Gateway Amazon DynamoDB AWS Lambda

Slide 11

Slide 11 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Verification example Objective: I want 10 clicks per second Measure: 10 clicks threshold Condition: Inject downstream service failure Verify: Pass or fail Client Amazon API Gateway Amazon DynamoDB AWS Lambda

Slide 12

Slide 12 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Verification demo

Slide 13

Slide 13 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Verification demo

Slide 14

Slide 14 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Verification demo Client Amazon S3 Amazon API Gateway AWS Lambda Amazon DynamoDB AWS Lambda AWS Lambda

Slide 15

Slide 15 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Client Amazon S3 Amazon API Gateway AWS Lambda Amazon DynamoDB AWS Lambda AWS Lambda Verification demo • Condition: Add 100-400 ms latency for each invocation • Condition: Return error codes on some invocations • Condition: Intercept and deny connections to DynamoDB

Slide 16

Slide 16 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Demo

Slide 17

Slide 17 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Making verification continuous

Slide 18

Slide 18 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Making verification continuous

Slide 19

Slide 19 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Making verification continuous Objective: My purchase API should respond in less than 400 ms Measure: 400 ms threshold Condition: 100-200 ms latency injection to function Verify: Pass Default deploy

Slide 20

Slide 20 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Making verification continuous Objective: My purchase API should respond in less than 400 ms Measure: 400 ms threshold Condition: 100-200 ms latency injection to function Verify: Fail Canary deploy

Slide 21

Slide 21 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Making verification continuous Objective: My purchase API should respond in less than 400 ms Measure: 400 ms threshold Condition: 100-200 ms latency injection to function Verify: Pass Feature flag

Slide 22

Slide 22 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Continuous verification demo

Slide 23

Slide 23 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Continuous verification demo

Slide 24

Slide 24 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Continuous verification demo

Slide 25

Slide 25 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Continuous verification demo Objective: Users should frequently get new images Measure: 10 images loaded per minute Condition: Return error codes on invocations Verify: Pass or Fail Canary deploy

Slide 26

Slide 26 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Demo

Slide 27

Slide 27 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Key learnings Verification tells you why it’s important to explore a set of conditions It’s not about breaking things, it’s about learning and building confidence It’s easy to get started.

Slide 28

Slide 28 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Testing to ensure that you can meet your availability goals is the only way you can have confidence that you will meet those goals” Reliability Pillar AWS Well-Architected Framework

Slide 29

Slide 29 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Do you want more? Reliability pillar https://docs.aws.amazon.com/wellarchitected/latest/reliability-pillar/welcome.html Serverless Chaos Demo app https://demo.serverlesschaos.com Failure-lambda https://github.com/gunnargrosch/failure-lambda Chaos-lambda https://github.com/adhorn/aws-lambda-chaos-injection/

Slide 30

Slide 30 text

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you! Gunnar Grosch @gunnargrosch