Efficient patterns for serverless development (AWS Summit London)

Serverless is a very different paradigm from how we used to build software, and many people are struggling with the shift in mindset and approach.

But serverless doesn't have to be hard. Once you figure out the "how", it'd be like getting a superpower, you will be able to do much more with less time and effort and build better software faster.

Yan has run serverless workloads in production since 2016 and has made all the mistakes so you don't have to. He's going to show us what works and what doesn't.

He will walk us through a day in the life of a serverless developer, showing you how to write and test serverless applications, advanced architectural patterns and best practices for building scalable, secure and cost-efficient applications in the cloud.

Transcript

1. Patterns for Efficient Serverless Development
2. Yan Cui http://theburningmonk.com @theburningmonk AWS user since 2010
3. Developer Advocate @ Yan Cui http://theburningmonk.com @theburningmonk
4. Independent Consultant advise training delivery Yan Cui http://theburningmonk.com @theburningmonk
5. FAST Feedback Loop
6. Myth: there is no local development for serverless
7. Option 1: Hexagonal Architecture
8. An architectural pattern for creating loosely-coupled application components that can easily connect to their environment via ports and adapters.
9. Port Port Port Port Adapter Adapter Client A Adapter Client B Adapter External system Adapter Application Core Domain External system External system
10. Port Port Port Port Adapter Adapter Client A Adapter Client B Adapter External system Adapter Application Core Domain External system External system
11. Port Port Port Port Adapter Adapter Client A Adapter Client B Adapter External system Adapter Application Core Domain External system External system
12. Port Port Port Port Adapter Adapter Client A Adapter Client B Adapter External system Adapter Application Core Domain External system External system
13. Port Port Port Port Adapter Adapter Client A Adapter Client B Adapter External system Adapter Application Core Domain External system External system
14. Port Port Port Port Adapter Adapter Client A Adapter Client B Adapter External system Adapter Application Core Domain External system External system
15. Port Port Port Port Adapter Adapter Lambda Adapter Client B Adapter External system Adapter Application Core Domain External system External system
16. Port Port Port Port Adapter Adapter Lambda Adapter Client B Adapter External system Adapter Application Core Domain External system External system (event, context) CreateUserCommand
17. Port Port Port Port Adapter Adapter Lambda Adapter Client B Adapter External system Adapter Application Core Domain External system External system
18. Port Port Port Port Adapter Adapter Lambda Adapter Fargate Adapter External system Adapter Application Core Domain External system External system
19. Creates portability that is useful when you’re not sure if a workload should run in Lambda or container
20. Port Port Port Port Adapter Adapter Lambda Adapter Fargate Adapter External system Adapter Application Core Domain External system External system
21. Port Port Port Port Adapter Adapter Lambda Adapter Fargate Adapter External system Adapter Application Core Domain External system DynamoDB
22. Port Port Port Port Adapter Adapter Lambda Adapter Fargate Adapter External system Adapter Application Core Domain External system Mock
23. Port Port Port Port Adapter Adapter Lambda Adapter Fargate Adapter External system Adapter Application Core Domain External system RDS
24. Port Port Port Port Adapter Adapter Client A Adapter Client B Adapter External system Adapter Application Core Domain External system External system
25. Option 1: Hexagonal Architecture Universally applicable (language, framework, etc.) ??? Portability
26. Serverless is more than Lambda
27. Serverless means: 1. No need to manage servers. 2. Scale to zero. 3. Usage-based pricing with no minimum.
28. “Use Lambda to transform data, NOT transport data”
29. API Gateway
30. API Gateway Cognito
31. API Gateway DynamoDB Cognito
32. API Gateway DynamoDB Cognito EventBridge Pipes EventBridge Bus
33. API Gateway DynamoDB Cognito EventBridge Pipes EventBridge Bus 3rd party API
34. Option 1: Hexagonal Architecture Universally applicable (language, framework, etc.) Can’t test direct integrations Portability
35. Option 1: Hexagonal Architecture Universally applicable (language, framework, etc.) Can’t test direct integrations Portability Upfront work to create abstraction layers
36. Option 2: Local simulation
38. API Gateway DynamoDB Cognito EventBridge Pipes EventBridge Bus
39. API Gateway DynamoDB Cognito EventBridge Pipes EventBridge Bus
40. API Gateway DynamoDB Cognito EventBridge Pipes EventBridge Bus IAM enforcement
41. API Gateway DynamoDB Cognito EventBridge Pipes EventBridge Bus Lambda
42. Option 2: Local simulation Not only Lambda. ???
43. Option 2: Local simulation Not only Lambda. Never 100% accurate - missing APIs, bugs, behaviour di ff erences, etc.
44. Option 3: Lambdalith
45. Lambdalith: run web app (e.g. express.js) inside Lambda
46. https://github.com/awslabs/aws-lambda-web-adapter
47. https://github.com/CodeGenieApp/serverless-express
48. https://bref.sh
49. https://github.com/zappa/Zappa
50. Option 3: Lambdalith Familiar to developers ??? Portability
51. Option 3: Lambdalith Familiar to developers Can’t test direct integrations Portability
52. Option 3: Lambdalith Familiar to developers Can’t test direct integrations Portability Cold start penalty with large web framework dependency
53. Option 3: Lambdalith Familiar to developers Can’t test direct integrations Portability Cold start penalty with large web framework dependency Might not work as well for non-API workloads
54. Option 3: Lambdalith Familiar to developers Can’t test direct integrations Portability Cold start penalty with large web framework dependency Might not work as well for non-API workloads Less granular alerting and access controls
55. Option 4: Deployment frameworks
56. sls invoke local sam local invoke sst dev
57. Option 4: Deployment frameworks Framework does the heavy lifting for you ???
58. Option 4: Deployment frameworks Framework does the heavy lifting for you Framework (and sometimes language) speci fi c
59. Option 4: Deployment frameworks Framework does the heavy lifting for you Framework (and sometimes language) speci fi c Can’t test direct integrations
60. Option 4: Deployment frameworks Framework does the heavy lifting for you Framework (and sometimes language) speci fi c Can’t test direct integrations Exploratory testing is NOT the same as automated tests
61. Option 5: “remocal” testing
62. Remocal testing: execute code locally against remote resources
63. Local testing Runs code locally Can use debugger Change code without deployment FAST FEEDBACK!
64. Test against mocks Test against real thing “does it work?” “does it do what I expect?”
65. Expectations/ Assumptions are sometimes wrong… Reality! Test against mocks Test against real thing “does it work?” “does it do what I expect?”
66. Local testing Runs code locally Can use debugger Change code without deployment Limited coverage Prone to false positives FAST FEEDBACK! LOW CONFIDENCE
67. Your application consists of more than just your code
68. Your job is to ensure all of it works Your application consists of more than just your code
69. Local testing Runs code locally Can use debugger Change code without deployment Limited coverage Prone to false positives FAST FEEDBACK! LOW CONFIDENCE HIGH CONFIDENCE! Remote testing Test in the cloud Realistic tests Better coverage
70. Local testing Runs code locally Can use debugger Change code without deployment Limited coverage Prone to false positives FAST FEEDBACK! LOW CONFIDENCE Slow deployments Every change needs deploying… SLOW FEEDBACK… HIGH CONFIDENCE! Remote testing Test in the cloud Realistic tests Better coverage
71. Local testing Remote testing
72. Runs code locally, talk to real AWS services REMOCAL testing Can use debugger Change code without deployment Realistic tests
74. Cognito API Gateway AppSync EventBridge
75. Cognito API Gateway AppSync EventBridge IAM, event pattern, etc. IAM, URL path, method, etc. IAM, resolver con fi g, resolver template, etc.
76. Option 5: Remocal testing Universally applicable (language, framework, etc.) ???
77. Option 5: Remocal testing Universally applicable (language, framework, etc.) Can’t test direct integrations
78. Auth Cognito AWS_IAM API Key Lambda authorizer Request validation Request transform Integration Lambda Direct service integration Domain logic Integrations Response transform IAM
79. Auth Cognito AWS_IAM API Key Lambda authorizer Request validation Request transform Integration Lambda Direct service integration Domain logic Integrations Response transform IAM Unit Test Remocal Test E2E Test
80. Auth Cognito AWS_IAM API Key Lambda authorizer Request validation Request transform Integration Lambda Direct service integration Domain logic Integrations Response transform IAM Unit Test Remocal Test E2E Test E2E Test E2E Test E2E Test E2E Test
81. Auth Cognito AWS_IAM API Key Lambda authorizer Request validation Request transform Integration Lambda Direct service integration Domain logic Integrations Response transform IAM Unit Test Remocal Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test
82. Auth Cognito AWS_IAM API Key Lambda authorizer Request validation Request transform Integration Lambda Direct service integration Domain logic Integrations Response transform IAM Unit Test Remocal Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test Unit Test
83. Auth Cognito AWS_IAM API Key Lambda authorizer Request validation Request transform Integration Lambda Direct service integration Domain logic Integrations Response transform IAM Unit Test Remocal Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test E2E Test Unit Test Remocal Test
84. DEMO TIME
85. REMOCAL testing AWS resources need to be provisioned Runs code locally, talk to real AWS services Can use debugger Change code without deployment Realistic tests
86. Ephemeral environments
87. Myth: there is no local development for serverless
88. Option 1: Hexagonal Architecture Option 2: Local simulation Option 3: Lambdalith Option 4: Deployment frameworks Option 5: Remocal testing
89. productionreadyserverless.com Join 20+ AWS Heroes & Community Builders and 1000+ happy students in levelling up your serverless game.
90. “You could spend tens of thousands on costly mistakes, and years of dev time to learn these lessons, or you could spend a few days downloading Yan's extensive experience. The choice is clear!”
92. Questions?