Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Office Hours | Serverless for Startups

Serverless Office Hours | Serverless for Startups

2663a9ac90cc6ed420e0e1560db57782?s=128

Slobodan Stojanović

June 21, 2022
Tweet

More Decks by Slobodan Stojanović

Other Decks in Programming

Transcript

  1. A story about Vacation Tracker a 100% serverless and bootstrapped

    startup
  2. A story about Vacation Tracker a 100% serverless and bootstrapped

    startup
  3. A story about Vacation Tracker a 100% serverless and bootstrapped

    startup
  4. A story about Vacation Tracker a 100% serverless and bootstrapped

    startup
  5. A story about Vacation Tracker a 100% serverless and bootstrapped

    startup
  6. A story about Vacation Tracker a 100% serverless and bootstrapped

    startup
  7. In 2016 we decided to build a leave tracking system

    to build a leave tracking system
  8. In 2016 we decided to build a leave tracking system

    To solve our own leave tracking problem
  9. In 2016 we decided to build a leave tracking system

    2017 To solve our own leave tracking problem
  10. In 2016 we decided to build a leave tracking system

    2017 2018 To solve our own leave tracking problem
  11. The idea was simple: - track leave requests and number

    of remaining days - We do not want to remember more passwords - use SSO - connect to Slack to show requests and info in the chat - connect to the calendar so we can subscribe to events
  12. How hard can it be?

  13. How hard can it be? Famous last words

  14. We were solving our problem but we were not sure

    if anyone else will use it
  15. Everything started with a landing page with pricing and a

    demo video
  16. And then, first customers signed up!

  17. - startups - sma! and medium companies - schools -

    non-profits - teams from enterprises - government organizations - churches
  18. None
  19. The dashboard

  20. Slack

  21. Microsoft Teams

  22. But, you are not here for the product demo. Let's

    see the fun stuff!
  23. The architecture v0.1 A simple serverless bot

  24. None
  25. Why serverless?

  26. s e r v e r l e s s

  27. s e r v e r l e s s

    low xpensive endor lock-in esponse latency
  28. s e r v e r l e s s

    low xpensive endor lock-in esponse latency
  29. Serverless because: - Sma! team without devops experience - AUTO

    SCALABLE - Cheap - Fast to build a prototype - Secure
  30. But it wasn't 100% "serverless"

  31. None
  32. - Quick and independent deployments - Easy to understand and

    maintain - Easy to onboard new people - Cheap Benefits:
  33. The Cost: $0/month

  34. Adding new features?

  35. None
  36. - Independent deployments - Hard to manage - Hard to

    scale - A bottleneck Downsides:
  37. ~ 100 paying teams

  38. The architecture v1.0 A complex serverless application

  39. Infrastructure-as-a-Code

  40. Service Service (micro?) (micro?)

  41. None
  42. ~150 Lambda functions

  43. FIRST MIGRATIONS (FROM AN OLD SERVICE TO A NEW ONE)

  44. Finding a good architecture

  45. - Everything is in IaaC CloudFormation - Replaced Node.js server

    with serverless services - Added TypeScript - Replaced MongoDB with DynamoDB Things we changed:
  46. - independent deployments - Auto-scalable - Almost 100% uptime out-of-the-box

    - Sti! cheap Benefits:
  47. - Storing state not events - Wasting time on less

    important things - Hard to onboard new developers - Many new services - Developers don't like YAML :) Downsides:
  48. ~ 600 paying teams

  49. The architecture v2.0 An event driven system

  50. Finding a good architecture v2.0

  51. Command Query Responsibility Segregation (CQRS)

  52. Why CQRS?

  53. Storing the state vs storing the events

  54. - Ana created a location and moved John and Mike

    - Ana assigned Mike as an approver - Ana assigned a leave policy (20 PTO days per year) - John requested a leave, Ana approved - Ro!over, some unused days were transferred to the next year - Ana changed John's working w"k - Mike added some past leaves for John - Alex transferred John to another location with different policy - …
  55. A quiz question: Calculate John's remaining PTO days for 2021

  56. CQRS to the rescue

  57. None
  58. 1. The client sends an API POST request or GraphQL

    mutation
  59. 1. The client sends an API POST request or GraphQL

    mutation 2. The event is stored in the Events table (append-only, no edits)
  60. 1. The client sends an API POST request or GraphQL

    mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus
  61. 1. The client sends an API POST request or GraphQL

    mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function
  62. 1. The client sends an API POST request or GraphQL

    mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function 5. The business logic Lambda stores the "cached" data to one of the read-only DynamoDB tables
  63. 1. The client sends an API POST request or GraphQL

    mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function 5. The business logic Lambda stores the "cached" data to one of the read-only DynamoDB tables 6. And then triggers the mutation that sends a "fake" mutation
  64. 1. The client sends an API POST request or GraphQL

    mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function 5. The business logic Lambda stores the "cached" data to one of the read-only DynamoDB tables 6. And then triggers the mutation that sends a "fake" mutation 7. A "fake" mutation triggers the GraphQl subscription to notify the clients
  65. 1. The client sends an API POST request or GraphQL

    mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function 5. The business logic Lambda stores the "cached" data to one of the read-only DynamoDB tables 6. And then triggers the mutation that sends a "fake" mutation 7. A "fake" mutation triggers the GraphQl subscription to notify the clients
  66. 1. The client sends an API POST request or GraphQL

    mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function 5. The business logic Lambda stores the "cached" data to one of the read-only DynamoDB tables 6. And then triggers the mutation that sends a "fake" mutation 7. A "fake" mutation triggers the GraphQl subscription to notify the clients 8. App uses an event bus to "route" the response to the user's platform
  67. None
  68. Client then query the read-only tables directly using GraphQl

  69. Client then query the read-only tables directly using GraphQl Or

    using the RESTful API in case of bots
  70. 193 Lambda functions

  71. - Fu!y Managed GraphQL - Less code - Better control

    - A! benefits from the previous architecture - Monorepo Benefits:
  72. - More new services to learn - Velocity templates Downsides:

  73. Challenges

  74. Onboarding new developers

  75. Current team - 3 fu! stack developers + 1 WP

    +1 QA - 1 Product manager - 2 Marketing - 2 Customer support - 2 Founders - Some fr"lance support (marketing + design)
  76. One environment per developer ~$3.75 per environment, because of serverless

    :)
  77. "Use the force, Luke!" a! developers are fu!-stack, but some

    are better with front end and some are better with back end
  78. None
  79. Good to start with (front end)

  80. Good to start with (front end) Good to start with

    (back end)
  81. Testing

  82. @slobodan_

  83. @slobodan_

  84. @slobodan_

  85. @slobodan_

  86. @slobodan_

  87. @slobodan_

  88. @slobodan_

  89. @slobodan_

  90. @slobodan_

  91. @slobodan_ export async function getPresignedUrl<T>({ event, bucket, region, userImagesManager, parser,

    }: IGetPresignedUrlParams<T>): Promise<IGetPresignedUrlresponse> { try { const { userId, mimeType } = parser(event) const extension = getFileExtension(mimeType) const fileName = `${userId}-${Date.now()}.${extension}` const uploadUrl = await userImagesManager.getPresignedUrl(fileName) const imageUrl = `https://${bucket}.s3.${region}.amazonaws.com/${fileName}` return { uploadUrl, imageUrl, } } catch (error) { logger.error(error) throw generateError(PresignedUrlFailed) } } main.ts
  92. @slobodan_ export async function getPresignedUrl<T>({ event, bucket, region, userImagesManager, parser,

    }: IGetPresignedUrlParams<T>): Promise<IGetPresignedUrlresponse> { try { const { userId, mimeType } = parser(event) const extension = getFileExtension(mimeType) const fileName = `${userId}-${Date.now()}.${extension}` const uploadUrl = await userImagesManager.getPresignedUrl(fileName) const imageUrl = `https://${bucket}.s3.${region}.amazonaws.com/${fileName}` return { uploadUrl, imageUrl, } } catch (error) { logger.error(error) throw generateError(PresignedUrlFailed) } } main.ts
  93. @slobodan_ export async function handler (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> { try

    { const { imageUrl, uploadUrl } = await getPresignedUrl<APIGatewayProxyEvent>({ event, userImagesManager, parser: parseApiEvent, bucket: process.env.USER_IMAGES_BUCKET, region: process.env.AWS_REGION, }) return httpResponse({ body: { imageUrl, uploadUrl, }}) } catch (error) { return httpResponse({ statusCode: 400, body: error, }) } } lambda.ts
  94. @slobodan_ export async function handler (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> { try

    { const { imageUrl, uploadUrl } = await getPresignedUrl<APIGatewayProxyEvent>({ event, userImagesManager, parser: parseApiEvent, bucket: process.env.USER_IMAGES_BUCKET, region: process.env.AWS_REGION, }) return httpResponse({ body: { imageUrl, uploadUrl, }}) } catch (error) { return httpResponse({ statusCode: 400, body: error, }) } } lambda.ts
  95. Testing VTL templates? Yes, you can and should do that

    too.
  96. None
  97. Example: serverlesspub/upollo s" appsync/test folder

  98. Migrations

  99. @slobodan_

  100. @slobodan_

  101. @slobodan_

  102. @slobodan_ But, how does this look like?

  103. @slobodan_ describe('DynamoDB repository', () => { describe('unit', () => {

    ... }) describe('integration', () => { beforeAll(() => { // Create test DB }) afterAll(() => { // Destroy test DB }) // Tests }) })
  104. @slobodan_ beforeAll(async () => { const params = { ...

    } await dynamoDb.createTable(params).promise() await dynamoDb.waitFor('tableExists', { TableName: tableName }).promise() })
  105. @slobodan_ afterAll(async () => { await dynamoDb.deleteTable({ TableName: tableName }).promise()

    await dynamoDb.waitFor('tableNotExists', { TableName: tableName }).promise() })
  106. Debugging and monitoring

  107. Following best practices can be expensive! You do not n"d

    to use a! the fancy tools. At least not at the beginning.
  108. None
  109. None
  110. None
  111. Cost

  112. Monthly AWS Bill: ~$600 Less than 1% of our MRR

  113. Cost per non-prod environment $3.75 ~ $15 when we turn

    on WAF
  114. None
  115. The most expensive bug: ~$3200

  116. None
  117. Security

  118. AWS SSO is awesome! Microsoft 365 users -> AWS SSO

    + Ben Kehoe's awesome OS tool
  119. » aws-sso-util login --profile vtslobodan

  120. None
  121. » aws-sso-util login --profile vtslobodan Logging in https://notrealurl.awsapps.com/start AWS SSO

    login required. Attempting to open the SSO authorization page in your default browser. If the browser does not open or you wish to use a different device to authorize this request, open the following URL: https://device.sso.eu-central-1.amazonaws.com/ Then enter the code: ABCD-EFGH Login succeeded, valid until 2022-06-22 02:26 CEST
  122. Use the power of IAM Granular permissions. Hard to set

    up, huge benefits.
  123. We have many other smaller challenges, but we are happy

    with serverless!
  124. @slobodan_

  125. @slobodan_ Summary

  126. @slobodan_ • Evolve your architecture with your product • Pick

    a good architecture, it helps you to keep your migration and onboarding costs low • Think about onboarding new team members • Hexagonal architecture is a nice fit for serverless apps • CQRS is also A nice fit for serverless apps and excellent fit for Vacation Tracker • Test your integrations (and app in general) • Testing is not enough, you'll need monitoring and error tracking
  127. @slobodan_

  128. @slobodan_ Running Serverless Realtime GraphQL Applications with AppSync slobodan.me/books Subscribe

    at: