From 1 to 201 Lambda functions in production: Evolving a serverless startup architecture

Transcript

A story about building a proﬁtable bootstrapped startup using serverless.

@slobodan_ Our journey starts with a prototype, continues with an

MVP, and follows the evolution to a real product.

@slobodan_ We faced many issues and learned a lot along

the way.

Evolving a serverless startup architecture From 1 to 201 Lambda

functions in production

Evolving a serverless startup architecture From 1 to 201 Lambda

functions in production

@slobodan_ The problem & the idea

@slobodan_ Everything started in 2016 with a problem in our

other company.

@slobodan_ Everything started in 2017 with an idea, a prototype,

and a landing page.

@slobodan_ Everything really started in 2018 with an MVP.

@slobodan_ A simple idea •Track leave requests and a number

of remaining PTO days •Use SSO to avoid another username and password to remember •Integrate with Slack to make and approve PTO requests •Show events in a Google calendar

@slobodan_ How hard can it be?

@slobodan_ Slobodan Stojanović CTO and co-founder of the product I

am talking about co-author of Serverless Apps with Node.js book AWS Serverless Hero JS Belgrade meetup organizer

@slobodan_ An MVP architecture: a simple serverless bot*

@slobodan_ Why serverless? •Faster - it was fast to build

a prototype and an MVP •Less - we outsourced scaling, maintenance, and security •Focused - we focused on the business logic and minimized time spent on everything else •Cheaper - the cost scales with users, and it starts with $0

None

@slobodan_ Benefits •Quick and independent deployments •Easy to understand and

maintain •Easy to onboard new people •Cheap

@slobodan_ The cost of the infrastructure: $0,00

@slobodan_ Adding new features

@slobodan_ Downsides •Independent deployment for each Lambda function •Hard to

manage a!er we added 10 Lambda functions •Hard to scale, as a critical part wasn't serverless •An obvious bottleneck

@slobodan_ ~100 paying teams

@slobodan_ The ﬁrst architecture iteration: a complex serverless spaghetti

@slobodan_ The ﬁrst architecture iteration: a complex serverless application

@slobodan_ Infrastructure as code (IaC) & ﬁrst microservices

@slobodan_ Microservices

None

@slobodan_ We had ~150 Lambda functions

@slobodan_ First migrations from an old service to a new

one

@slobodan_ We realized that we'll need to ﬁnd the right

architecture for our problem

@slobodan_ Most significant changes •(Almost) everything was in AWS CloudFormation

(IaC) •We replaced Node.js server with serverless services •We started using TypeScript instead of JavaScript •We started a migration from MongoDB to DynamoDB

@slobodan_ Benefits •Easier deployments •App was auto-scalable •We had almost

100% uptime out-of-the-box •Still very cheap (our infrastructure cost was less than $100/ month)

@slobodan_ Downsides •The big flaw in our system design: we

were storing a state and not events in our database •We were still wasting a lot of time on less important things •Project complexity and the number of new services increased, and it was harder to onboard new developers •Developers don't like YAML

@slobodan_ ~600 paying teams

@slobodan_ The second architecture iteration: an event-driven system with microservices

@slobodan_ The quest for an architecture that solves our problem

@slobodan_ In the end, we decided to use Event Sourcing

& Command Query Responsibility Segregation (CQRS)

@slobodan_ Why Event Sourcing and CQRS?

@slobodan_ Storing state vs storing events

@slobodan_ A common flow in our app •Ana created a

new location and moved John and Mike to that location •Ana assigned Mike as an approver •Ana made a leave policy (20 PTO days per year) •John requested leave, and Ana approved it •Brought forward event happened and five unused days are transferred to the next year balance •Ana changed John's working week •Mike added some past leaves for John •Alex moved John to another location with a diﬀerent policy

@slobodan_ How do we calculate John's remaining PTO days? Events

and CQRS to the rescue. As a bonus, we got everything we need for the audit logs.

None

1. The client sends an API POST request or GraphQL

mutation 2. The event is stored in the Events table (append-only, no edits) 3. The DynamoDB streams the event to the Lambda function that sends it to the EventBridge event bus 4. EventBridge triggers the specific business logic Lambda function 5. The business logic Lambda stores the "cached" data to one of the read-only DynamoDB tables 6. And then triggers the mutation that sends a "fake" mutation 7. A "fake" mutation triggers the GraphQl subscription to notify the clients 8. App uses an event bus to "route" the response to the user's platform

Client then query the read-only tables directly using GraphQl

Client then query the read-only tables directly using GraphQl Or

using the RESTful API in case of bots

We decreased the number of Lambda functions to 101! But

now we have 230 functions in production !

@slobodan_ Benefits •Fully Managed GraphQL •Less code •Better control •All

benefits from the previous architecture (without the spaghetti part) •Monorepo and shared types

@slobodan_ Downsides •YAML is still very important (but we also

added CDK for some smaller services) •Many new AWS services to learn •Velocity templates

Lambda functions → business logic Other services → transformation and

orchestration

@slobodan_ ~2000 paying teams

@slobodan_ The cost of the infrastructure: ~1% of MRR ~$1250

@slobodan_ Architecture should evolve with your product. Don't waste your

time making it perfect for the MVP.

@slobodan_ Development & Testing

@slobodan_ Common questions during onboarding •How do I run this

locally? •How do we debug errors? •What is a DynamoDB single-table design, and why do we store all events in the same table? •What did we just do?

@slobodan_ It's impossible to run the whole serverless application locally

But if you are learning to be a pilot, you don't start with driving a plane in your backyard !

@slobodan_ You can simulate parts of your application locally Think

of it as an early version of a "Flight Simulator" It's helpful, but probably far from the complete set of tools to learn how to fly the plane

@slobodan_ What can we do? •A Lambda function is just

a function - you can run it locally as any other function* •You can use SAM Local or similar tools to run (a simulation of) a Lambda function locally in the Docker container •You can test in the cloud •You can waste your time trying to simulate everything locally

@slobodan_ Or you can write tests

@slobodan_ How do we test a common client-server app?

@slobodan_ How do we test a serverless app?

@slobodan_ Hexagonal architecture

@slobodan_ Anatomy of a Lambda function

@slobodan_ TypeScript example (business logic) interface IParams<T> { event: T

parser: (T) => IParsedData repositories: { users: IUserRepository notifications: INotificationRepository } } export async function businessLogic<T>(params: IParams<T>): Promise<void> { const {event, parser, repositories} = params try { const parsedData = parser(event) await repositories.users.invite(parsedData.email) await repositories.notifications.invitationSent() } catch (err) { await repositories.notifications.failure(err) } }

@slobodan_ TypeScript example (business logic) interface IParams<T> { event: T

@slobodan_ TypeScript example (handler) export async function handler(event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult>

{ try { const userRepository = new UsersDynamoDBRepository(process.env.USERS_TABLE) const eventBridgeRepository = new EventBridgeRepository(process.env.EVENT_BUS) await businessLogic<APIGatewayProxyEvent>({ event, parser: parseApiGatewayEvent, repositories: { users: userRepository, notifications: eventBridgeRepository } }) await apiGatewaySuccessResponse() } catch(err) { await apiGatewayErrorResponse() } }

From 1 to 201 Lambda functions in production: E...

From 1 to 201 Lambda functions in production: Evolving a serverless startup architecture

More Decks by Slobodan Stojanović

Other Decks in Programming

Featured

Transcript