From 1 to 101 Lambda Functions in Production: Evolving a Serverless Architecture - Node Congress 2021

Building a serverless function or an API is easy. However, things get a bit more complicated as your application grows. What works for a few functions often don't work for hundreds of functions. As your application grows, you'll need to evolve your architecture, deployment, monitoring, and many other things. This talk is a case study of an evolution of the serverless startup's architecture. We started with a single Lambda function in early 2018 and evolved our application through multiple stages and architectures. At the moment application uses GraphQL and more than 100 Lambda functions and serves millions of requests. We faced and solved many issues during the last three years, learned many things, and managed to keep our infrastructure costs low.

Transcript

2. A story about Vacation Tracker a 100% serverless startup
3. A story about Vacation Tracker a 100% serverless startup
4. A story about Vacation Tracker a 100% serverless startup
5. A story about Vacation Tracker a 100% serverless startup
6. A story about Vacation Tracker a 100% serverless startup
7. A story about Vacation Tracker a 100% serverless startup
8. In 2016 we decided to build a leave tracking system a leave tracking system
9. In 2016 we decided to build a leave tracking system To solve our own leave tracking problem
10. In 2016 we decided to build a leave tracking system 2017 To solve our own leave tracking problem
11. In 2016 we decided to build a leave tracking system 2017 2018 To solve our own leave tracking problem
12. The idea was simple: - track leave requests and number of remaining days - use some SSO, so we do not n!d to remember more passwords - connect to Slack so that we can have the info in Slack - connect to the calendar so we can subscribe to events
13. We were solving our problem but we were not sure if anyone else will use it
14. - startups - sma" companies - schools - non-profits - teams from enterprises - government organizations - churches
16. The number of unique users by December 1st 263623
17. The dashboard
18. Slack
19. Microsoft Teams
20. The architecture v0.1 A simple serverless bot
22. Why serverless?
23. From 1 to 101 Lambda Functions in Production Evolving a Serverless Architecture
24. @slobodan_ Before we continue…
25. Slobodan Stojanović CTO @ Cloud Horizon & CTO @ Vacation Tracker co-author of Serverless Applications with Node.js book AWS Serverless Hero https://slobodan.me
26. Why serverless?
27. s e r v e r l e s s
28. s e r v e r l e s s low
29. s e r v e r l e s s low xpensive
30. s e r v e r l e s s low xpensive endor lock-in
31. s e r v e r l e s s low xpensive endor lock-in
32. Serverless because: - Sma" team without devops experience - AUTO SCALABLE - Cheap - Fast to build a prototype - Secure
33. But it wasn't 100% "serverless"
36. - Quick and independent deployments - Easy to understand and maintain - Easy to onboard new people - Cheap Benefits:
37. The Cost: $0/month
38. Adding new features?
41. - Independent deployments - Hard to manage - Hard to scale - A bottleneck Downsides:
42. ~ 100 paying teams
43. The architecture v1.0 A complex serverless application
44. Infrastructure-as-a-Code
45. Service Service (micro?) (micro?)
47. ~150 Lambda functions
48. FIRST MIGRATIONS (FROM AN OLD SERVICE TO A NEW ONE)
49. Finding a good architecture
50. - Everything is in IaaC CloudFormation - Replaced Node.js server with serverless services - Added TypeScript - Replaced MongoDB with DynamoDB Things we changed:
51. - independent deployments - Auto-scalable - Almost 100% uptime out-of-the-box - Sti" cheap Benefits:
52. - Storing state not events - Wasting time on less important things - Hard to onboard new developers - Many new services Downsides:
53. - Storing state not events - Wasting time on less important things - Hard to onboard new developers - Many new services - Developers don't like YAML :) Downsides:
54. ~ 600 paying teams
55. The architecture v2.0 An event driven system
56. Finding a good architecture v2.0
57. Command Query Responsibility Segregation (CQRS)
58. Why CQRS?
59. Storing the state vs storing the events
60. - Ana created a location and moved John and Mike - Ana assigned Mike as an approver - Ana assigned a leave policy (20 PTO days per year) - John requested a leave, Ana approved - Ro"over, some unused days were transferred to the next year - Ana changed John's working w!k - Mike added some past leaves for John - Alex transferred John to another location with different policy - …
61. A quiz question: Calculate John's remaining PTO days for 2021
62. CQRS to the rescue
63. Mutation Event storage Subscription Query Read-only table EventBridge Business logic Business logic AWS AppSync
64. Mutation Event storage Subscription Query Read-only table EventBridge Business logic Business logic AWS AppSync
65. Mutation Event storage Subscription Query Read-only table EventBridge Business logic Business logic AWS AppSync
66. 112 Lambda functions
67. - Fu"y Managed GraphQL - Less code - Better control - A" benefits from the previous architecture - Monorepo Benefits:
68. - More new services to learn - Velocity templates Downsides:
69. Challenges
70. Onboarding new developers
71. Current team - 4 developers (a" fu" stack, one new) - 1 Marketing - 1 Customer support - 1 Product manager - Some fr!lance support (marketing + design)
72. One environment per developer (serverless :))
74. Good to start with
75. Testing
76. @slobodan_
77. @slobodan_
78. @slobodan_
79. @slobodan_
80. @slobodan_
81. @slobodan_
82. @slobodan_
83. @slobodan_
84. @slobodan_
85. Migrations
86. @slobodan_
87. @slobodan_
88. @slobodan_
89. @slobodan_ But, how does this look like?
90. @slobodan_ describe('DynamoDB repository', () => { describe('unit', () => { ... }) describe('integration', () => { beforeAll(() => { // Create test DB }) afterAll(() => { // Destroy test DB }) // Tests }) })
91. @slobodan_ beforeAll(async () => { const params = { ... } await dynamoDb.createTable(params).promise() await dynamoDb.waitFor('tableExists', { TableName: tableName }).promise() })
92. @slobodan_ afterAll(async () => { await dynamoDb.deleteTable({ TableName: tableName }).promise() await dynamoDb.waitFor('tableNotExists', { TableName: tableName }).promise() })
93. Debugging and monitoring
96. Cost
97. Total cost from 2018: ~$7000
98. $5000 AWS credits ~$2000 paid
99. The most expensive bug: ~$3200
101. We have many other smaller challenges, but we are happy with serverless!
102. @slobodan_
103. @slobodan_ Summary
104. @slobodan_ • Evolve your architecture with your product • Pick a good architecture, it helps you to keep your migration and onboarding costs low • Think about onboarding new team members • Hexagonal architecture is a nice fit for serverless apps • CQRS is also A nice fit for serverless apps and excellent fit for Vacation Tracker • Test your integrations (and app in general) • Testing is not enough, you'll need monitoring and error tracking
105. @slobodan_
106. @slobodan_ https://slobodan.me/subscribe