Tweet
Tweet

Slide 1

Slide 1 text

A story about building a profitable bootstrapped startup using serverless.

Slide 2

Slide 2 text

@slobodan_ Our journey starts with a prototype, continues with an MVP, and follows the evolution to a real product.

Slide 3

Slide 3 text

@slobodan_ We faced many issues and learned a lot along the way.

Slide 4

Slide 4 text

Evolving a serverless startup architecture From 1 to 101 Lambda functions in production

Slide 5

Slide 5 text

Evolving a serverless startup architecture From 1 to 101 Lambda functions in production

Slide 6

Slide 6 text

@slobodan_ The problem & the idea

Slide 7

Slide 7 text

@slobodan_ Everything started in 2016 with a problem in our other company.

Slide 8

Slide 8 text

@slobodan_ Everything started in 2017 with an idea, a prototype, and a landing page.

Slide 9

Slide 9 text

@slobodan_ Everything really started in 2018 with an MVP.

Slide 10

Slide 10 text

@slobodan_ A simple idea •Track leave requests and a number of remaining PTO days •Use SSO to avoid another username and password to remember •Integrate with Slack to make and approve PTO requests •Show events in a Google calendar

Slide 11

Slide 11 text

@slobodan_ How hard can it be?

Slide 12

Slide 12 text

@slobodan_ Slobodan Stojanović CTO and co-founder of the product I am talking about co-author of Serverless Apps with Node.js book AWS Serverless Hero JS Belgrade meetup organizer

Slide 13

Slide 13 text

@slobodan_ Architecture

Slide 14

Slide 14 text

@slobodan_ A prototype

Slide 15

Slide 15 text

@slobodan_ An MVP architecture: a simple serverless bot*

Slide 16

Slide 16 text

@slobodan_ Why serverless? •Faster - it was fast to build a prototype and an MVP •Less - we outsourced scaling, maintenance, and security •Focused - we focused on the business logic and minimized time spent on everything else •Cheaper - the cost scales with users, and it starts with $0

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

@slobodan_ Benefits •Quick and independent deployments •Easy to understand and maintain •Easy to onboard new people •Cheap

Slide 19

Slide 19 text

@slobodan_ The cost of the infrastructure: $0,00

Slide 20

Slide 20 text

@slobodan_ Adding new features

Slide 21

Slide 21 text

@slobodan_ Downsides •Independent deployment for each Lambda function •Hard to manage a!er we added 10 Lambda functions •Hard to scale, as a critical part wasn't serverless •An obvious bottleneck

Slide 22

Slide 22 text

@slobodan_ ~100 paying teams

Slide 23

Slide 23 text

@slobodan_ The first architecture iteration: a complex serverless spaghetti

Slide 24

Slide 24 text

@slobodan_ The first architecture iteration: a complex serverless application

Slide 25

Slide 25 text

@slobodan_ Infrastructure as code (IaC) & first microservices

Slide 26

Slide 26 text

@slobodan_ Microservices Service Service (micro?) (micro?)

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

@slobodan_ We had ~150 Lambda functions

Slide 29

Slide 29 text

@slobodan_ First migrations from an old service to a new one

Slide 30

Slide 30 text

@slobodan_ We realized that we'll need to find the right architecture for our problem

Slide 31

Slide 31 text

@slobodan_ Most significant changes •(Almost) everything was in AWS CloudFormation (IaC) •We replaced Node.js server with serverless services •We started using TypeScript instead of JavaScript •We started a migration from MongoDB to DynamoDB

Slide 32

Slide 32 text

@slobodan_ Benefits •Easier deployments •App was auto-scalable •We had almost 100% uptime out-of-the-box •Still very cheap (our infrastructure cost was less than $100/ month)

Slide 33

Slide 33 text

@slobodan_ Downsides •The big flaw in our system design: we were storing a state and not events in our database •We were still wasting a lot of time on less important things •Project complexity and the number of new services increased, and it was harder to onboard new developers •Developers don't like YAML

Slide 34

Slide 34 text

@slobodan_ ~600 paying teams

Slide 35

Slide 35 text

@slobodan_ The second architecture iteration: an event-driven system with microservices

Slide 36

Slide 36 text

@slobodan_ The quest for an architecture that solves our problem

Slide 37

Slide 37 text

@slobodan_ In the end, we decided to use Event Sourcing & Command Query Responsibility Segregation (CQRS)

Slide 38

Slide 38 text

@slobodan_ Why Event Sourcing and CQRS?

Slide 39

Slide 39 text

@slobodan_ Storing state vs storing events

Slide 40

Slide 40 text

@slobodan_ A common flow in our app •Ana created a new location and moved John and Mike to that location •Ana assigned Mike as an approver •Ana made a leave policy (20 PTO days per year) •John requested leave, and Ana approved it •Brought forward event happened and five unused days are transferred to the next year balance •Ana changed John's working week •Mike added some past leaves for John •Alex moved John to another location with a different policy

Slide 41

Slide 41 text

@slobodan_ How do we calculate John's remaining PTO days? Events and CQRS to the rescue. As a bonus, we got everything we need for the audit logs.

Slide 42

Slide 42 text

1. The client sends an API POST request or GraphQL mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function 5. The business logic Lambda stores the "cached" data to one of the read-only DynamoDB tables 6. And then triggers the mutation that sends a "fake" mutation 7. A "fake" mutation triggers the GraphQl subscription to notify the clients

Slide 43

Slide 43 text

1. The client sends an API POST request or GraphQL mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function 5. The business logic Lambda stores the "cached" data to one of the read-only DynamoDB tables 6. And then triggers the mutation that sends a "fake" mutation 7. A "fake" mutation triggers the GraphQl subscription to notify the clients 8. App uses an event bus to "route" the response to the user's platform

Slide 44

Slide 44 text

Client then query the read-only tables directly using GraphQl

Slide 45

Slide 45 text

Client then query the read-only tables directly using GraphQl Or using the RESTful API in case of bots

Slide 46

Slide 46 text

We decreased the number of Lambda functions to 101! But now we have 197 functions in production.

Slide 47

Slide 47 text

@slobodan_ Benefits •Fully Managed GraphQL •Less code •Better control •All benefits from the previous architecture (without the spaghetti part) •Monorepo and shared types

Slide 48

Slide 48 text

@slobodan_ Downsides •YAML is still very important (but we also added CDK for some smaller services) •Many new AWS services to learn •Velocity templates

Slide 49

Slide 49 text

Lambda functions → business logic Other services → transformation and orchestration

Slide 50

Slide 50 text

@slobodan_ ~1600 paying teams

Slide 51

Slide 51 text

@slobodan_ The cost of the infrastructure: <1% of MRR ~$750

Slide 52

Slide 52 text

@slobodan_ Architecture should evolve with your product. Don't waste your time making it perfect for the MVP.

Slide 53

Slide 53 text

@slobodan_ Development & Testing

Slide 54

Slide 54 text

@slobodan_ Common questions during onboarding •How do I run this locally? •How do we debug errors? •What is a DynamoDB single-table design, and why do we store all events in the same table? •What did we just do?

Slide 55

Slide 55 text

@slobodan_ It's impossible to run the whole serverless application locally But if you are learning to be a pilot, you don't start with driving a plane in your backyard !

Slide 56

Slide 56 text

@slobodan_ You can simulate parts of your application locally Think of it as an early version of a "Flight Simulator" It's helpful, but probably far from the complete set of tools to learn how to fly the plane

Slide 57

Slide 57 text

@slobodan_ What can we do? •A Lambda function is just a function - you can run it locally as any other function* •You can use SAM Local or similar tools to run (a simulation of) a Lambda function locally in the Docker container •You can test in the cloud •You can waste your time trying to simulate everything locally

Slide 58

Slide 58 text

@slobodan_ Or you can write tests

Slide 59

Slide 59 text

@slobodan_ How do we test a common client-server app?

Slide 60

Slide 60 text

@slobodan_ How do we test a serverless app?

Slide 61

Slide 61 text

@slobodan_ How do we test a serverless app?

Slide 62

Slide 62 text

@slobodan_ Hexagonal architecture

Slide 63

Slide 63 text

@slobodan_ Anatomy of a Lambda function

Slide 64

Slide 64 text

@slobodan_ Anatomy of a Lambda function

Slide 65

Slide 65 text

@slobodan_ Anatomy of a Lambda function

Slide 66

Slide 66 text

@slobodan_ TypeScript example (business logic) interface IParams { event: T parser: (T) => IParsedData repositories: { users: IUserRepository notifications: INotificationRepository } } export async function businessLogic(params: IParams): Promise { const {event, parser, repositories} = params try { const parsedData = parser(event) await repositories.users.invite(parsedData.email) await repositories.notifications.invitationSent() } catch (err) { await repositories.notifications.failure(err) } }

Slide 67

Slide 67 text

@slobodan_ TypeScript example (business logic) interface IParams { event: T parser: (T) => IParsedData repositories: { users: IUserRepository notifications: INotificationRepository } } export async function businessLogic(params: IParams): Promise { const {event, parser, repositories} = params try { const parsedData = parser(event) await repositories.users.invite(parsedData.email) await repositories.notifications.invitationSent() } catch (err) { await repositories.notifications.failure(err) } }

Slide 68

Slide 68 text

@slobodan_ TypeScript example (business logic) interface IParams { event: T parser: (T) => IParsedData repositories: { users: IUserRepository notifications: INotificationRepository } } export async function businessLogic(params: IParams): Promise { const {event, parser, repositories} = params try { const parsedData = parser(event) await repositories.users.invite(parsedData.email) await repositories.notifications.invitationSent() } catch (err) { await repositories.notifications.failure(err) } }

Slide 69

Slide 69 text

@slobodan_ TypeScript example (business logic) interface IParams { event: T parser: (T) => IParsedData repositories: { users: IUserRepository notifications: INotificationRepository } } export async function businessLogic(params: IParams): Promise { const {event, parser, repositories} = params try { const parsedData = parser(event) await repositories.users.invite(parsedData.email) await repositories.notifications.invitationSent() } catch (err) { await repositories.notifications.failure(err) } }

Slide 70

Slide 70 text

@slobodan_ TypeScript example (business logic) interface IParams { event: T parser: (T) => IParsedData repositories: { users: IUserRepository notifications: INotificationRepository } } export async function businessLogic(params: IParams): Promise { const {event, parser, repositories} = params try { const parsedData = parser(event) await repositories.users.invite(parsedData.email) await repositories.notifications.invitationSent() } catch (err) { await repositories.notifications.failure(err) } }

Slide 71

Slide 71 text

@slobodan_ TypeScript example (business logic) interface IParams { event: T parser: (T) => IParsedData repositories: { users: IUserRepository notifications: INotificationRepository } } export async function businessLogic(params: IParams): Promise { const {event, parser, repositories} = params try { const parsedData = parser(event) await repositories.users.invite(parsedData.email) await repositories.notifications.invitationSent() } catch (err) { await repositories.notifications.failure(err) } }

Slide 72

Slide 72 text

@slobodan_ TypeScript example (business logic) interface IParams { event: T parser: (T) => IParsedData repositories: { users: IUserRepository notifications: INotificationRepository } } export async function businessLogic(params: IParams): Promise { const {event, parser, repositories} = params try { const parsedData = parser(event) await repositories.users.invite(parsedData.email) await repositories.notifications.invitationSent() } catch (err) { await repositories.notifications.failure(err) } }

Slide 73

Slide 73 text

@slobodan_ TypeScript example (business logic) interface IParams { event: T parser: (T) => IParsedData repositories: { users: IUserRepository notifications: INotificationRepository } } export async function businessLogic(params: IParams): Promise { const {event, parser, repositories} = params try { const parsedData = parser(event) await repositories.users.invite(parsedData.email) await repositories.notifications.invitationSent() } catch (err) { await repositories.notifications.failure(err) } }

Slide 74

Slide 74 text

@slobodan_ TypeScript example (handler) export async function handler(event: APIGatewayProxyEvent): Promise { try { const userRepository = new UsersDynamoDBRepository(process.env.USERS_TABLE) const eventBridgeRepository = new EventBridgeRepository(process.env.EVENT_BUS) await businessLogic({ event, parser: parseApiGatewayEvent, repositories: { users: userRepository, notifications: eventBridgeRepository } }) await apiGatewaySuccessResponse() } catch(err) { await apiGatewayErrorResponse() } }

Slide 75

Slide 75 text

@slobodan_ TypeScript example (handler) export async function handler(event: APIGatewayProxyEvent): Promise { try { const userRepository = new UsersDynamoDBRepository(process.env.USERS_TABLE) const eventBridgeRepository = new EventBridgeRepository(process.env.EVENT_BUS) await businessLogic({ event, parser: parseApiGatewayEvent, repositories: { users: userRepository, notifications: eventBridgeRepository } }) await apiGatewaySuccessResponse() } catch(err) { await apiGatewayErrorResponse() } }

Slide 76

Slide 76 text

@slobodan_ TypeScript example (handler) export async function handler(event: APIGatewayProxyEvent): Promise { try { const userRepository = new UsersDynamoDBRepository(process.env.USERS_TABLE) const eventBridgeRepository = new EventBridgeRepository(process.env.EVENT_BUS) await businessLogic({ event, parser: parseApiGatewayEvent, repositories: { users: userRepository, notifications: eventBridgeRepository } }) await apiGatewaySuccessResponse() } catch(err) { await apiGatewayErrorResponse() } }

Slide 77

Slide 77 text

@slobodan_

Slide 78

Slide 78 text

@slobodan_

Slide 79

Slide 79 text

@slobodan_

Slide 80

Slide 80 text

@slobodan_

Slide 81

Slide 81 text

@slobodan_

Slide 82

Slide 82 text

@slobodan_ Also useful for migrations

Slide 83

Slide 83 text

@slobodan_

Slide 84

Slide 84 text

@slobodan_

Slide 85

Slide 85 text

@slobodan_

Slide 86

Slide 86 text

@slobodan_ Integration tests

Slide 87

Slide 87 text

@slobodan_ Testing in the cloud

Slide 88

Slide 88 text

@slobodan_ describe('DynamoDB repository', () => { describe('unit', () => { ... }) describe('integration', () => { beforeAll(() => { // Create test DB }) afterAll(() => { // Destroy test DB }) // Tests }) })

Slide 89

Slide 89 text

@slobodan_ beforeAll(async () => { const params = { ... } await dynamoDb.createTable(params).promise() await dynamoDb.waitFor('tableExists', { TableName: tableName }).promise() })

Slide 90

Slide 90 text

@slobodan_ afterAll(async () => { await dynamoDb.deleteTable({ TableName: tableName }).promise() await dynamoDb.waitFor('tableNotExists', { TableName: tableName }).promise() })

Slide 91

Slide 91 text

@slobodan_ Or we deploy the full app and run integration tests

Slide 92

Slide 92 text

@slobodan_ Deploy where? Serverless environments are often cheap!

Slide 93

Slide 93 text

@slobodan_ Our environments 12 similar environments total cost: $750/month

Slide 94

Slide 94 text

@slobodan_ It's hard to run or simulate a serverless app locally. Make small trade-o"s to make your app testable, and you'll be able to move fast.

Slide 95

Slide 95 text

@slobodan_ Security

Slide 96

Slide 96 text

@slobodan_ Accounts are cheap Create one AWS sub-account per environment Create an AWS sub-account for each developer But remember to set budget notifications!

Slide 97

Slide 97 text

@slobodan_ Tools for management and easy repeatability CloudFormation AWS Organization Formation

Slide 98

Slide 98 text

@slobodan_ Use AWS SSO

Slide 99

Slide 99 text

@slobodan_ Use AWS IAM Identity Center (Successor to AWS SSO)

Slide 100

Slide 100 text

@slobodan_ Learn AWS Identity & Access Management (IAM) Apply the minimal permissions model for all AWS services i.e., a Lambda function can do one specific operation only

Slide 101

Slide 101 text

@slobodan_ Use WAF to protect your application

Slide 102

Slide 102 text

@slobodan_ But what about distributed denial-of-service (DDOS) attacks?

Slide 103

Slide 103 text

@slobodan_ But what about distributed denial-of-wallet (DDOW) attacks? "All AWS customers benefit from the automatic protections of AWS Shield Standard, at no additional charge."

Slide 104

Slide 104 text

@slobodan_ You still need to be sure that your code is secure

Slide 105

Slide 105 text

@slobodan_ Learn about security best practices for your service provider and use them from day one.

Slide 106

Slide 106 text

@slobodan_ Philosophy

Slide 107

Slide 107 text

@slobodan_ Focus on the important things only, and outsource everything else

Slide 108

Slide 108 text

@slobodan_ But how do we know what's important?

Slide 109

Slide 109 text

@slobodan_ See "Why the Fuss about Serverless?" (https://youtu.be/SPsaqiegOP4)

Slide 110

Slide 110 text

@slobodan_ See "Why the Fuss about Serverless?" (https://youtu.be/SPsaqiegOP4)

Slide 111

Slide 111 text

@slobodan_ See "Why the Fuss about Serverless?" (https://youtu.be/SPsaqiegOP4)

Slide 112

Slide 112 text

@slobodan_ See "Why the Fuss about Serverless?" (https://youtu.be/SPsaqiegOP4)

Slide 113

Slide 113 text

@slobodan_ See "Why the Fuss about Serverless?" (https://youtu.be/SPsaqiegOP4)

Slide 114

Slide 114 text

@slobodan_ See "Why the Fuss about Serverless?" (https://youtu.be/SPsaqiegOP4)

Slide 115

Slide 115 text

@slobodan_ See "Why the Fuss about Serverless?" (https://youtu.be/SPsaqiegOP4)

Slide 116

Slide 116 text

@slobodan_ See "Why the Fuss about Serverless?" (https://youtu.be/SPsaqiegOP4)

Slide 117

Slide 117 text

@slobodan_ Does this apply only to the infrastructure?

Slide 118

Slide 118 text

@slobodan_ Do not waste your time or energy building from scratch components in the product or commodity phases

Slide 119

Slide 119 text

@slobodan_ Focus on your business logic and outsource everything else.

Slide 120

Slide 120 text

@slobodan_ Joe Armstrong “Make it work, then make it beautiful, then if you really, really have to, make it fast. 90% of the time, if you make it beautiful, it will already be fast. So really, just make it beautiful!” creator of Erlang programming language

Slide 121

Slide 121 text

No content

Slide 122

Slide 122 text

Thank you! twitter: @slobodan_