Serverless Office Hours | Serverless for Startups

https://youtu.be/HghO9bT5i3E

Transcript

1. A story about Vacation Tracker a 100% serverless and bootstrapped startup
2. A story about Vacation Tracker a 100% serverless and bootstrapped startup
3. A story about Vacation Tracker a 100% serverless and bootstrapped startup
4. A story about Vacation Tracker a 100% serverless and bootstrapped startup
5. A story about Vacation Tracker a 100% serverless and bootstrapped startup
6. A story about Vacation Tracker a 100% serverless and bootstrapped startup
7. In 2016 we decided to build a leave tracking system to build a leave tracking system
8. In 2016 we decided to build a leave tracking system To solve our own leave tracking problem
9. In 2016 we decided to build a leave tracking system 2017 To solve our own leave tracking problem
10. In 2016 we decided to build a leave tracking system 2017 2018 To solve our own leave tracking problem
11. The idea was simple: - track leave requests and number of remaining days - We do not want to remember more passwords - use SSO - connect to Slack to show requests and info in the chat - connect to the calendar so we can subscribe to events
12. How hard can it be?
13. How hard can it be? Famous last words
14. We were solving our problem but we were not sure if anyone else will use it
15. Everything started with a landing page with pricing and a demo video
16. And then, first customers signed up!
17. - startups - sma! and medium companies - schools - non-profits - teams from enterprises - government organizations - churches
19. The dashboard
20. Slack
21. Microsoft Teams
22. But, you are not here for the product demo. Let's see the fun stuff!
23. The architecture v0.1 A simple serverless bot
25. Why serverless?
26. s e r v e r l e s s
27. s e r v e r l e s s low xpensive endor lock-in esponse latency
28. s e r v e r l e s s low xpensive endor lock-in esponse latency
29. Serverless because: - Sma! team without devops experience - AUTO SCALABLE - Cheap - Fast to build a prototype - Secure
30. But it wasn't 100% "serverless"
32. - Quick and independent deployments - Easy to understand and maintain - Easy to onboard new people - Cheap Benefits:
33. The Cost: $0/month
34. Adding new features?
36. - Independent deployments - Hard to manage - Hard to scale - A bottleneck Downsides:
37. ~ 100 paying teams
38. The architecture v1.0 A complex serverless application
39. Infrastructure-as-a-Code
40. Service Service (micro?) (micro?)
42. ~150 Lambda functions
43. FIRST MIGRATIONS (FROM AN OLD SERVICE TO A NEW ONE)
44. Finding a good architecture
45. - Everything is in IaaC CloudFormation - Replaced Node.js server with serverless services - Added TypeScript - Replaced MongoDB with DynamoDB Things we changed:
46. - independent deployments - Auto-scalable - Almost 100% uptime out-of-the-box - Sti! cheap Benefits:
47. - Storing state not events - Wasting time on less important things - Hard to onboard new developers - Many new services - Developers don't like YAML :) Downsides:
48. ~ 600 paying teams
49. The architecture v2.0 An event driven system
50. Finding a good architecture v2.0
51. Command Query Responsibility Segregation (CQRS)
52. Why CQRS?
53. Storing the state vs storing the events
54. - Ana created a location and moved John and Mike - Ana assigned Mike as an approver - Ana assigned a leave policy (20 PTO days per year) - John requested a leave, Ana approved - Ro!over, some unused days were transferred to the next year - Ana changed John's working w"k - Mike added some past leaves for John - Alex transferred John to another location with different policy - …
55. A quiz question: Calculate John's remaining PTO days for 2021
56. CQRS to the rescue
58. 1. The client sends an API POST request or GraphQL mutation
59. 1. The client sends an API POST request or GraphQL mutation 2. The event is stored in the Events table (append-only, no edits)
60. 1. The client sends an API POST request or GraphQL mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus
61. 1. The client sends an API POST request or GraphQL mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function
62. 1. The client sends an API POST request or GraphQL mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function 5. The business logic Lambda stores the "cached" data to one of the read-only DynamoDB tables
63. 1. The client sends an API POST request or GraphQL mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function 5. The business logic Lambda stores the "cached" data to one of the read-only DynamoDB tables 6. And then triggers the mutation that sends a "fake" mutation
64. 1. The client sends an API POST request or GraphQL mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function 5. The business logic Lambda stores the "cached" data to one of the read-only DynamoDB tables 6. And then triggers the mutation that sends a "fake" mutation 7. A "fake" mutation triggers the GraphQl subscription to notify the clients
65. 1. The client sends an API POST request or GraphQL mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function 5. The business logic Lambda stores the "cached" data to one of the read-only DynamoDB tables 6. And then triggers the mutation that sends a "fake" mutation 7. A "fake" mutation triggers the GraphQl subscription to notify the clients
66. 1. The client sends an API POST request or GraphQL mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function 5. The business logic Lambda stores the "cached" data to one of the read-only DynamoDB tables 6. And then triggers the mutation that sends a "fake" mutation 7. A "fake" mutation triggers the GraphQl subscription to notify the clients 8. App uses an event bus to "route" the response to the user's platform
68. Client then query the read-only tables directly using GraphQl
69. Client then query the read-only tables directly using GraphQl Or using the RESTful API in case of bots
70. 193 Lambda functions
71. - Fu!y Managed GraphQL - Less code - Better control - A! benefits from the previous architecture - Monorepo Benefits:
72. - More new services to learn - Velocity templates Downsides:
73. Challenges
74. Onboarding new developers
75. Current team - 3 fu! stack developers + 1 WP +1 QA - 1 Product manager - 2 Marketing - 2 Customer support - 2 Founders - Some fr"lance support (marketing + design)
76. One environment per developer ~$3.75 per environment, because of serverless :)
77. "Use the force, Luke!" a! developers are fu!-stack, but some are better with front end and some are better with back end
79. Good to start with (front end)
80. Good to start with (front end) Good to start with (back end)
81. Testing
82. @slobodan_
83. @slobodan_
84. @slobodan_
85. @slobodan_
86. @slobodan_
87. @slobodan_
88. @slobodan_
89. @slobodan_
90. @slobodan_
91. @slobodan_ export async function getPresignedUrl<T>({ event, bucket, region, userImagesManager, parser, }: IGetPresignedUrlParams<T>): Promise<IGetPresignedUrlresponse> { try { const { userId, mimeType } = parser(event) const extension = getFileExtension(mimeType) const fileName = `${userId}-${Date.now()}.${extension}` const uploadUrl = await userImagesManager.getPresignedUrl(fileName) const imageUrl = `https://${bucket}.s3.${region}.amazonaws.com/${fileName}` return { uploadUrl, imageUrl, } } catch (error) { logger.error(error) throw generateError(PresignedUrlFailed) } } main.ts
92. @slobodan_ export async function getPresignedUrl<T>({ event, bucket, region, userImagesManager, parser, }: IGetPresignedUrlParams<T>): Promise<IGetPresignedUrlresponse> { try { const { userId, mimeType } = parser(event) const extension = getFileExtension(mimeType) const fileName = `${userId}-${Date.now()}.${extension}` const uploadUrl = await userImagesManager.getPresignedUrl(fileName) const imageUrl = `https://${bucket}.s3.${region}.amazonaws.com/${fileName}` return { uploadUrl, imageUrl, } } catch (error) { logger.error(error) throw generateError(PresignedUrlFailed) } } main.ts
93. @slobodan_ export async function handler (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> { try { const { imageUrl, uploadUrl } = await getPresignedUrl<APIGatewayProxyEvent>({ event, userImagesManager, parser: parseApiEvent, bucket: process.env.USER_IMAGES_BUCKET, region: process.env.AWS_REGION, }) return httpResponse({ body: { imageUrl, uploadUrl, }}) } catch (error) { return httpResponse({ statusCode: 400, body: error, }) } } lambda.ts
94. @slobodan_ export async function handler (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> { try { const { imageUrl, uploadUrl } = await getPresignedUrl<APIGatewayProxyEvent>({ event, userImagesManager, parser: parseApiEvent, bucket: process.env.USER_IMAGES_BUCKET, region: process.env.AWS_REGION, }) return httpResponse({ body: { imageUrl, uploadUrl, }}) } catch (error) { return httpResponse({ statusCode: 400, body: error, }) } } lambda.ts
95. Testing VTL templates? Yes, you can and should do that too.
97. Example: serverlesspub/upollo s" appsync/test folder
98. Migrations
99. @slobodan_
100. @slobodan_
101. @slobodan_
102. @slobodan_ But, how does this look like?
103. @slobodan_ describe('DynamoDB repository', () => { describe('unit', () => { ... }) describe('integration', () => { beforeAll(() => { // Create test DB }) afterAll(() => { // Destroy test DB }) // Tests }) })
104. @slobodan_ beforeAll(async () => { const params = { ... } await dynamoDb.createTable(params).promise() await dynamoDb.waitFor('tableExists', { TableName: tableName }).promise() })
105. @slobodan_ afterAll(async () => { await dynamoDb.deleteTable({ TableName: tableName }).promise() await dynamoDb.waitFor('tableNotExists', { TableName: tableName }).promise() })
106. Debugging and monitoring
107. Following best practices can be expensive! You do not n"d to use a! the fancy tools. At least not at the beginning.
111. Cost
112. Monthly AWS Bill: ~$600 Less than 1% of our MRR
113. Cost per non-prod environment $3.75 ~ $15 when we turn on WAF
115. The most expensive bug: ~$3200
117. Security
118. AWS SSO is awesome! Microsoft 365 users -> AWS SSO + Ben Kehoe's awesome OS tool
119. » aws-sso-util login --profile vtslobodan
121. » aws-sso-util login --profile vtslobodan Logging in https://notrealurl.awsapps.com/start AWS SSO login required. Attempting to open the SSO authorization page in your default browser. If the browser does not open or you wish to use a different device to authorize this request, open the following URL: https://device.sso.eu-central-1.amazonaws.com/ Then enter the code: ABCD-EFGH Login succeeded, valid until 2022-06-22 02:26 CEST
122. Use the power of IAM Granular permissions. Hard to set up, huge benefits.
123. We have many other smaller challenges, but we are happy with serverless!
124. @slobodan_
125. @slobodan_ Summary
126. @slobodan_ • Evolve your architecture with your product • Pick a good architecture, it helps you to keep your migration and onboarding costs low • Think about onboarding new team members • Hexagonal architecture is a nice fit for serverless apps • CQRS is also A nice fit for serverless apps and excellent fit for Vacation Tracker • Test your integrations (and app in general) • Testing is not enough, you'll need monitoring and error tracking
127. @slobodan_
128. @slobodan_ Running Serverless Realtime GraphQL Applications with AppSync slobodan.me/books Subscribe at: