Smart and efficient ways to test serverless architectures

Serverless architectures allow developers to focus on their code, and not worry about infrastructure. But just because it's "serverless" doesn't mean it doesn't need testing! Whether you're working with API Gateway, AppSync, Step Functions or Event-Driven Architectures, I'll show you how to test it. Testing serverless doesn't have to be hard.

Transcript

1. smart and e ffi cient way to test serverless architectures
2. Yan Cui http://theburningmonk.com @theburningmonk AWS user since 2010
3. http://bit.ly/yubl-serverless
4. Developer Advocate @ Yan Cui http://theburningmonk.com @theburningmonk
5. Independent Consultant advise training delivery Yan Cui http://theburningmonk.com @theburningmonk
6. “how do I test serverless?”
7. Serverless > Lambda
8. “Use Lambda to transform data, not transport data”
9. AppSync DynamoDB AppSync DynamoDB Lambda
10. Step Functions
11. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge SNS
12. Serverless > Lambda, Serverless > APIs
13. Di ff erent services present di ff erent challenges to testing
14. Hard Easy
15. Hard Easy Lambdalith (e.g. express.js in Lambda)
16. Hard Easy Lambdalith (e.g. express.js in Lambda) API Gateway with Lambda
17. Hard Easy Lambdalith (e.g. express.js in Lambda) Step Functions with direct integrations Step Functions with timeouts or wait state(s) API Gateway with Lambda
18. Hard Easy Lambdalith (e.g. express.js in Lambda) Step Functions with direct integrations Step Functions with timeouts or wait state(s) Event-Driven Architecture API Gateway with Lambda
19. https://testserverlessapps.com
20. https://testserverlessapps.com Chapter 1: how to test serverless architectures Chapter 2: testing API Gateway APIs Chapter 3: testing AppSync APIs Chapter 4: testing Step Functions Chapter 5: testing Event-Driven Architectures Chapter 6: testing in production
21. the RECIPE to TESTING serverless architectures
22. #1: “remocal” testing for Lambda functions
23. Local testing Runs code locally Can use debugger Change code without deployment FAST FEEDBACK!
24. Test against mocks Test against real thing “does it work?” “does it do what I expect?”
25. Test against mocks Test against real thing “does it work?” “does it do what I expect?” Expectations/Assumptions are sometimes wrong… Reality!
27. Local testing Runs code locally Can use debugger Change code without deployment FAST FEEDBACK! Limited coverage Prone to false positives LOW CONFIDENCE
28. Your application consists of more than just your code
29. Your application consists of more than just your code Your job is to ensure all of it works
30. HIGH CONFIDENCE! Local testing Remote testing Runs code locally Can use debugger Change code without deployment FAST FEEDBACK! Limited coverage Prone to false positives LOW CONFIDENCE Test in the cloud Realistic tests Better coverage
31. HIGH CONFIDENCE! Local testing Remote testing Runs code locally Can use debugger Change code without deployment FAST FEEDBACK! Limited coverage Prone to false positives LOW CONFIDENCE Test in the cloud Realistic tests Better coverage Slow deployments Every change needs deploying… SLOW FEEDBACK…
32. HIGH CONFIDENCE! Local testing Remote testing Runs code locally Can use debugger Change code without deployment FAST FEEDBACK! Limited coverage Prone to false positives LOW CONFIDENCE Test in the cloud Realistic tests Better coverage Slow deployments Every change needs deploying… SLOW FEEDBACK…
33. Local testing Remote testing
34. REMOCAL testing Runs code locally, talk to real AWS services Can use debugger Change code without deployment Realistic tests
36. Cognito API Gateway AppSync EventBridge
37. Cognito API Gateway AppSync EventBridge IAM, event pattern, etc. IAM, URL path, method, etc. IAM, resolver con fi g, resolver template, etc.
38. Your application consists of more than just your code Your job is to ensure all of it works
39. Auth Cognito AWS_IAM API Key Lambda authorizer Request validation Request transform Integration Lambda Direct service integration Domain logic Integrations Response transform IAM
40. Auth Cognito AWS_IAM API Key Lambda authorizer Request validation Request transform Integration Lambda Direct service integration Domain logic Integrations Response transform IAM Unit Test Remocal Test E2E Test
41. Auth Cognito AWS_IAM API Key Lambda authorizer Request validation Request transform Integration Lambda Direct service integration Domain logic Integrations Response transform IAM Unit Test Remocal Test E2E Test E2E Test E2E Test E2E Test E2E Test
42. Auth Cognito AWS_IAM API Key Lambda authorizer Request validation Request transform Integration Lambda Direct service integration Domain logic Integrations Response transform IAM Unit Test Remocal Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test
43. Auth Cognito AWS_IAM API Key Lambda authorizer Request validation Request transform Integration Lambda Direct service integration Domain logic Integrations Response transform IAM Unit Test Remocal Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test Unit Test Remocal Test
44. DEMO TIME
45. REMOCAL testing Runs code locally, talk to real AWS services Can use debugger Change code without deployment Realistic tests AWS resources need to be provisioned
46. #2: Ephemeral environments
47. npx sls deploy -s dev-my-feature Step 1: (creates a new “dev-my-feature” environment)
48. npx sls deploy -s dev-my-feature Step 1: (creates a new “dev-my-feature” environment) Step 2: Make code changes, iterate, run remocal tests against the “dev-my-feature” environment
49. npx sls deploy -s dev-my-feature Step 1: (creates a new “dev-my-feature” environment) Step 2: Make code changes, iterate, run remocal tests against the “dev-my-feature” environment Step 3: Commit code and send PR (CI pipeline runs all tests, etc.)
50. npx sls deploy -s dev-my-feature Step 1: (creates a new “dev-my-feature” environment) Step 2: Make code changes, iterate, run remocal tests against the “dev-my-feature” environment Step 3: Commit code and send PR Step 4: npx sls remove -s dev-my-feature (destroys the ephemeral environment) (CI pipeline runs all tests, etc.)
51. npx sls deploy -s dev-my-feature Step 1: (creates a new “dev-my-feature” environment) Step 2: Make code changes, iterate, run remocal tests against the “dev-my-feature” environment Step 3: Commit code and send PR Step 4: npx sls remove -s dev-my-feature (destroys the ephemeral environment) (CI pipeline runs all tests, etc.)
52. Insulated environment for development and testing
53. Insulated environment for development and testing (avoids polluting shared dev/test/staging environments with test data)
54. No cost overhead with usage-based pricing
55. https://theburningmonk.com/2023/02/how-to-handle-serverful-resources-when-using-ephemeral-environments
56. npx sls deploy -s dev-my-feature Step 1: (creates a new “dev-my-feature” environment) Step 2: Make code changes, iterate, run remocal tests against the “dev-my-feature” environment Step 3: Commit code and send PR Step 4: npx sls remove -s dev-my-feature (destroys the ephemeral environment) (CI pipeline runs all tests, etc.) npx sls deploy -s ci-<SHA> Step 1: Step 2: npm run tests:all Step 3: npx sls remove -s ci-<SHA>
57. environment !== AWS account
58. Accounts dev test staging prod test dev staging prod
59. Accounts dev test staging prod test dev staging prod dev-yan feature-a feature-b
60. Accounts dev test staging prod test dev staging prod dev-yan feature-a feature-b different stacks
61. Accounts dev test staging prod test dev staging prod dev-yan feature-a feature-b different CDK apps
62. Accounts dev test staging prod test dev staging prod dev-yan feature-a feature-b combination of something
63. “How do I make sure resource names don’t clash?”
64. #1: Don’t explicit name resources (unless you have to) “How do I make sure resource names don’t clash?”
65. #1: Don’t explicit name resources (unless you have to) “How do I make sure resource names don’t clash?” #2: Include environment name in resource names
66. remocal testing + ephemeral environments = test Lambda functions with ease BUT…
67. Serverless > Lambda
68. Step Functions
70. { “provider”: “stripe” }
71. { “provider”: “stripe” }
72. { “provider”: “stripe” } Check for execution result Validate side-effects
73. { “provider”: “stripe” }
74. { “provider”: “stripe” } How to make this step fail/timeout?
75. { “provider”: “stripe” }
76. { “provider”: “stripe” } How to make Stripe return a failed response?
77. The further away a state is, the harder it is to test (di ff i cult to direct execution along the desired path)
78. What I want to test
79. Timeout and Wait states are hard to test
80. #3: Step Functions Local
84. state machine name
85. state name
86. state name response name
87. state name
89. yeah, it’s awkward… but it works and the best we’ve got right now…
90. yeah, it’s awkward… but it works and the best we’ve got right now… But mocked responses doesn’t skip time, and can’t throw timeout errors
91. What I want to test rewrite the Wait state in local state machine
92. DEMO TIME
94. Event-Driven Architecture in a nutshell: Something happened Do something
95. Event publisher Event consumer EventBridge
96. Asynchrony means you don’t know when the event has been processed.
97. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge
98. Hard Easy Lambdalith (e.g. express.js in Lambda) Step Functions with direct integrations Step Functions with timeouts or wait state(s) Event-Driven Architecture API Gateway with Lambda
99. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge Component tests
100. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge Component tests
101. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge Component tests Unit Test Remocal Test
102. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge Component tests Mocks Unit Test
103. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge Component tests Mocks Unit Test
104. Test against mocks Test against real thing “does it work?” “does it do what I expect?” Expectations/Assumptions are sometimes wrong… Reality!
105. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge Component tests Remocal Test
106. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge Component tests How to capture what was sent? Remocal Test
107. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge E2E tests
108. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge E2E tests How to capture what was sent? How to capture what was sent? How to capture what was sent?
109. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge E2E tests SNS How to capture what was sent?
110. #4: Capture async events/messages
111. Lambda EventBridge System under test
112. Lambda EventBridge System under test AppSync (subscription)
113. Lambda EventBridge Push System under test AppSync (subscription)
114. Lambda EventBridge Push Shared infra AppSync (subscription)
115. service service service
116. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge AppSync (subscription)
117. Lambda EventBridge Lambda SNS Lambda DynamoDB Lambda DynamoDB SQS DLQ EventBridge Component tests AppSync (subscription) Remocal Test
118. DEMO TIME
119. #4: Capture async events/messages #3: Step Functions Local #2: Ephemeral environments #1: “remocal” testing for Lambda functions
122. productionreadyserverless.com 20% OFF with promo code “serverlessldn”
123. Questions?