Slide 1

Slide 1 text

Serverless Serverless The "Middy" way The "Middy" way — Workshop loige.link/middy-way Community Day Dublin October , 1st 2019 Luciano Mammino @loige Wi-Fi: DC_Conference October-2019 1

Slide 2

Slide 2 text

2

Slide 3

Slide 3 text

Hello, I am Luciano! Hello, I am Luciano! 2

Slide 4

Slide 4 text

Hello, I am Luciano! Hello, I am Luciano! 2

Slide 5

Slide 5 text

Hello, I am Luciano! Hello, I am Luciano! 2

Slide 6

Slide 6 text

Hello, I am Luciano! Hello, I am Luciano! 2

Slide 7

Slide 7 text

Hello, I am Luciano! Hello, I am Luciano! Cloud Architect 2

Slide 8

Slide 8 text

Hello, I am Luciano! Hello, I am Luciano! Cloud Architect Blog: Twitter: GitHub: loige.co @loige @lmammino 2

Slide 9

Slide 9 text

Hello, I am Luciano! Hello, I am Luciano! Cloud Architect Blog: Twitter: GitHub: loige.co @loige @lmammino 2

Slide 10

Slide 10 text

Hello, I am Luciano! Hello, I am Luciano! Cloud Architect Blog: Twitter: GitHub: loige.co @loige @lmammino 2

Slide 11

Slide 11 text

loige.link/middy-way loige.link/middy-way @loige 3

Slide 12

Slide 12 text

Let's chat on: Let's chat on: tlk.io/middy-way tlk.io/middy-way @loige 4

Slide 13

Slide 13 text

What is serverless What is serverless Compute as functions (FaaS) Compute as functions (FaaS) Event-based model Event-based model @loige 5

Slide 14

Slide 14 text

Why serverless is good Why serverless is good Focus on business logic Focus on business logic Scalability Scalability Pay per usage model (compute time * memory) Pay per usage model (compute time * memory) Managed infrastructure Managed infrastructure Forces you to think micro-services Forces you to think micro-services @loige 6

Slide 15

Slide 15 text

My Serverless experience My Serverless experience Open source A semi-automated weekly newsletter ( ) A middleware framework for AWS Lambda ( ) Enterprise Trading, Billing engine, Market data aggregator solutions ( ) Big data pipeline to make network metadata searchable ( ) Organised various workshops around Europe ( ) Fullstack bulletin middy.js.org ESB Vectra.ai Serverlesslab.com @loige 7

Slide 16

Slide 16 text

Let's get this party started Let's get this party started @loige 8

Slide 17

Slide 17 text

Requirements! Requirements! ♂ ♂ @loige 9

Slide 18

Slide 18 text

A nice laptop A nice laptop @loige 10

Slide 19

Slide 19 text

An AWS account An AWS account aws.amazon.com/free aws.amazon.com/free @loige 11

Slide 20

Slide 20 text

Have an Have an admin user admin user in your AWS account in your AWS account loige.link/aws-admin-user loige.link/aws-admin-user @loige 12

Slide 21

Slide 21 text

install the AWS CLI install the AWS CLI aws.amazon.com/cli aws.amazon.com/cli @loige 13

Slide 22

Slide 22 text

configure the AWS CLI configure the AWS CLI loige.link/aws-cli-config loige.link/aws-cli-config @loige 14

Slide 23

Slide 23 text

Verify Verify aws sts get-caller-identity { "Account": "123456789012", "UserId": "ABCDEFGHIJKLMNOPQRSTU", "Arn": "arn:aws:iam::123456789012:user/super-mario" } @loige 15

Slide 24

Slide 24 text

Install Git and Node.js Install Git and Node.js git-scm.com git-scm.com nodejs.org nodejs.org @loige 16

Slide 25

Slide 25 text

You are ready! You are ready! @loige 17

Slide 26

Slide 26 text

What are we going to build? What are we going to build? @loige 18

Slide 27

Slide 27 text

@loige A realtime dashboard for public transport! A realtime dashboard for public transport! 19

Slide 28

Slide 28 text

loige.link/middy-way-starter loige.link/middy-way-starter Read... the README! Clone the project locally Deploy it to your AWS account Play with the APIs @loige 20

Slide 29

Slide 29 text

loige.link/middy-way-starter loige.link/middy-way-starter Read... the README! Clone the project locally Deploy it to your AWS account Play with the APIs I need staaarz! @loige 20

Slide 30

Slide 30 text

⏱ 20 mins warm up ♀ ⏱ 20 mins warm up ♀ 1. Use the APIs to create a dashboard that contains: 1. Use the APIs to create a dashboard that contains: An IrishRail station widget A LUAS stop widget A DublinBus stop widget (there are utility scripts to list all the stops and stations, check the README) 2. Have a look at the code 2. Have a look at the code If you find bugs, please send a or report an PR issue @loige 21

Slide 31

Slide 31 text

AWS Lambda & Middy AWS Lambda & Middy ❤ ❤ @loige 22

Slide 32

Slide 32 text

The problem with Lambdas The problem with Lambdas @loige 23

Slide 33

Slide 33 text

exports.myLambda = function ( event, context, callback ) { // get input from event and context // use callback to return output or errors } Anatomy of a Node.js lambda on AWS Anatomy of a Node.js lambda on AWS @loige 24

Slide 34

Slide 34 text

(event, context, callback) => { // decrypt environment variables with KMS // deserialize the content of the event // validate input, authentication, authorization // REAL BUSINESS LOGIC // (process input, generate output) // validate output // serialize response // handle errors } A typical "REAL" Lambda function A typical "REAL" Lambda function @loige 25

Slide 35

Slide 35 text

(event, context, callback) => { // decrypt environment variables with KMS // deserialize the content of the event // validate input, authentication, authorization // REAL BUSINESS LOGIC // (process input, generate output) // validate output // serialize response // handle errors } A typical "REAL" Lambda function A typical "REAL" Lambda function @loige 25

Slide 36

Slide 36 text

(event, context, callback) => { // decrypt environment variables with KMS // deserialize the content of the event // validate input, authentication, authorization // REAL BUSINESS LOGIC // (process input, generate output) // validate output // serialize response // handle errors } A typical "REAL" Lambda function A typical "REAL" Lambda function @loige 25

Slide 37

Slide 37 text

(event, context, callback) => { // decrypt environment variables with KMS // deserialize the content of the event // validate input, authentication, authorization // REAL BUSINESS LOGIC // (process input, generate output) // validate output // serialize response // handle errors } A typical "REAL" Lambda function A typical "REAL" Lambda function LOTS of BOILERPLATE @loige 25

Slide 38

Slide 38 text

The solution The solution @loige 26

Slide 39

Slide 39 text

npm install @middy/core Note: using preview (alpha) version 1.0.0 @loige 27

Slide 40

Slide 40 text

npm install @middy/core Note: using preview (alpha) version 1.0.0 Give it moar love @loige 27

Slide 41

Slide 41 text

Usage Usage const middy = require('@middy/core') const { middleware1, middleware2, middleware3 } = require('some-middlewares') const originalHandler = (event, context, callback) => { /* your business logic */ } const handler = middy(originalHandler) handler .use(middleware1()) .use(middleware2()) .use(middleware3()) module.exports = { handler } @loige 28

Slide 42

Slide 42 text

Usage Usage const middy = require('@middy/core') const { middleware1, middleware2, middleware3 } = require('some-middlewares') const originalHandler = (event, context, callback) => { /* your business logic */ } const handler = middy(originalHandler) handler .use(middleware1()) .use(middleware2()) .use(middleware3()) module.exports = { handler } 1. define handler @loige 28

Slide 43

Slide 43 text

Usage Usage const middy = require('@middy/core') const { middleware1, middleware2, middleware3 } = require('some-middlewares') const originalHandler = (event, context, callback) => { /* your business logic */ } const handler = middy(originalHandler) handler .use(middleware1()) .use(middleware2()) .use(middleware3()) module.exports = { handler } 2. "middify" the handler @loige 28

Slide 44

Slide 44 text

Usage Usage const middy = require('@middy/core') const { middleware1, middleware2, middleware3 } = require('some-middlewares') const originalHandler = (event, context, callback) => { /* your business logic */ } const handler = middy(originalHandler) handler .use(middleware1()) .use(middleware2()) .use(middleware3()) module.exports = { handler } 3. attach middlewares @loige 28

Slide 45

Slide 45 text

Usage Usage const middy = require('@middy/core') const { middleware1, middleware2, middleware3 } = require('some-middlewares') const originalHandler = (event, context, callback) => { /* your business logic */ } const handler = middy(originalHandler) handler .use(middleware1()) .use(middleware2()) .use(middleware3()) module.exports = { handler } 4. export "middyfied" handler @loige 28

Slide 46

Slide 46 text

const middy = require('@middy/core') const urlEncodedBodyParser = require('@middy/http-urlencode-body-parser') const validator = require('@middy/validator') const httpErrorHandler = require('@middy/http-error-handler') const processPaymentHandler = (event, context, callback) => { const { creditCardNumber, expiryMonth, expiryYear, cvc, nameOnCard, amount } = event.body // do stuff with this data ... return callback(null, { result: 'success', message: 'payment processed correctly'} ) } const inputSchema = { // define validation schema here ... } const handler = middy(processPaymentHandler) .use(urlEncodedBodyParser()) .use(validator(inputSchema)) .use(httpErrorHandler()) module.exports = { handler } @loige 29

Slide 47

Slide 47 text

const middy = require('@middy/core') const urlEncodedBodyParser = require('@middy/http-urlencode-body-parser') const validator = require('@middy/validator') const httpErrorHandler = require('@middy/http-error-handler') const processPaymentHandler = (event, context, callback) => { const { creditCardNumber, expiryMonth, expiryYear, cvc, nameOnCard, amount } = event.body // do stuff with this data ... return callback(null, { result: 'success', message: 'payment processed correctly'} ) } const inputSchema = { // define validation schema here ... } const handler = middy(processPaymentHandler) .use(urlEncodedBodyParser()) .use(validator(inputSchema)) .use(httpErrorHandler()) module.exports = { handler } Handler @loige 29

Slide 48

Slide 48 text

const middy = require('@middy/core') const urlEncodedBodyParser = require('@middy/http-urlencode-body-parser') const validator = require('@middy/validator') const httpErrorHandler = require('@middy/http-error-handler') const processPaymentHandler = (event, context, callback) => { const { creditCardNumber, expiryMonth, expiryYear, cvc, nameOnCard, amount } = event.body // do stuff with this data ... return callback(null, { result: 'success', message: 'payment processed correctly'} ) } const inputSchema = { // define validation schema here ... } const handler = middy(processPaymentHandler) .use(urlEncodedBodyParser()) .use(validator(inputSchema)) .use(httpErrorHandler()) module.exports = { handler } Attach middlewares @loige 29

Slide 49

Slide 49 text

const middy = require('@middy/core') const urlEncodedBodyParser = require('@middy/http-urlencode-body-parser') const validator = require('@middy/validator') const httpErrorHandler = require('@middy/http-error-handler') const processPaymentHandler = (event, context, callback) => { const { creditCardNumber, expiryMonth, expiryYear, cvc, nameOnCard, amount } = event.body // do stuff with this data ... return callback(null, { result: 'success', message: 'payment processed correctly'} ) } const inputSchema = { // define validation schema here ... } const handler = middy(processPaymentHandler) .use(urlEncodedBodyParser()) .use(validator(inputSchema)) .use(httpErrorHandler()) module.exports = { handler } Export enhanced handler @loige 29

Slide 50

Slide 50 text

Why? Why? @loige 30

Slide 51

Slide 51 text

Why? Why? Simplify code @loige 30

Slide 52

Slide 52 text

Why? Why? Simplify code Reusability input parsing input & output validation output serialization error handling ... @loige 30

Slide 53

Slide 53 text

Why? Why? Simplify code Reusability input parsing input & output validation output serialization error handling ... Focus (even) MORE on business logic @loige 30

Slide 54

Slide 54 text

How it works How it works @loige 31

Slide 55

Slide 55 text

Execution order Execution order 32 @loige

Slide 56

Slide 56 text

Execution order Execution order 32 @loige

Slide 57

Slide 57 text

Execution order Execution order 1. middleware1 (before) 32 @loige

Slide 58

Slide 58 text

Execution order Execution order 1. middleware1 (before) 2. middleware2 (before) 32 @loige

Slide 59

Slide 59 text

Execution order Execution order 1. middleware1 (before) 2. middleware2 (before) 3. middleware3 (before) 32 @loige

Slide 60

Slide 60 text

Execution order Execution order 1. middleware1 (before) 2. middleware2 (before) 3. middleware3 (before) 4. handler 32 @loige

Slide 61

Slide 61 text

Execution order Execution order 1. middleware1 (before) 2. middleware2 (before) 3. middleware3 (before) 4. handler 5. middleware3 (after) 32 @loige

Slide 62

Slide 62 text

Execution order Execution order 1. middleware1 (before) 2. middleware2 (before) 3. middleware3 (before) 4. handler 5. middleware3 (after) 6. middleware2 (after) 32 @loige

Slide 63

Slide 63 text

Execution order Execution order 1. middleware1 (before) 2. middleware2 (before) 3. middleware3 (before) 4. handler 5. middleware3 (after) 6. middleware2 (after) 7. middleware1 (after) 32 @loige

Slide 64

Slide 64 text

Execution order Execution order 1. middleware1 (before) 2. middleware2 (before) 3. middleware3 (before) 4. handler 5. middleware3 (after) 6. middleware2 (after) 7. middleware1 (after) 32 @loige

Slide 65

Slide 65 text

When an error happens... When an error happens... Flow is stopped First middleware implementing `onError` gets control It can choose to handle the error, or delegate it to the next handler If the error is handler a response is returned If the error is not handled the execution fails reporting the unhandled error @loige 33

Slide 66

Slide 66 text

Writing a middleware Writing a middleware const myMiddleware = (config) => { // might set default options in config return ({ before: (handler, next) => { // might read options from `config` }, after: (handler, next) => { // might read options from `config` }, onError: (handler, next) => { // might read options from `config` } }) } module.exports = myMiddleware @loige 34

Slide 67

Slide 67 text

Inline middlewares Inline middlewares const middy = require('@middy/core') const handler = middy((event, context, callback) => { // do stuff }) handler.before((handler, next) => { // do something in the before phase next() }) handler.after((handler, next) => { // do something in the after phase next() }) handler.onError((handler, next) => { // do something in the on error phase next() }) module.exports = { handler } @loige 35

Slide 68

Slide 68 text

It supports async handlers! It supports async handlers! const middy = require('@middy/core') const handler = middy( async (event, context) => { // do stuff return { some: 'response' } } ) module.exports = { handler } @loige 36

Slide 69

Slide 69 text

Exercise 0 Exercise 0: let's middify! : let's middify! In the next exercises we will need the full power of middy, so to get started let's just "middify" all our handlers — Note: handlers are defined in "src/handlers", and they get used in "src/handler.js". You can either middify every single handler file or middify them when used in src/handler.js. In the first case, make sure you middify the actual handler function and not the "handler factory". ( ) a solution @loige 37

Slide 70

Slide 70 text

Exercise 1 Exercise 1: body parsing : body parsing We are manually deserializing the JSON input in our APIs. We can simplify the code a bit by using the middleware. — Install @middy/http-json-body-parser and remove all the instances of JSON.parse() in our handlers. ( ) http-json-body-parser a solution @loige 38

Slide 71

Slide 71 text

What happens when one of our handlers crashes? We are actually not even dealing with errors right now... by using the middleware we will be able to handle all HTTP errors and return codes consistently. — Install @middy/http-error-handler and apply that to all our handlers. ( ) http-error-handler a solution Exercise 2 Exercise 2: what if we fail? : what if we fail? @loige 39

Slide 72

Slide 72 text

We want to make sure that data is validated before being written to the database. We can use the middleware for this task! — Install @middy/validator and apply that to all our POST API handlers. You can use to define your schemas or, if you are struggling, you can find some schemas ready-made for you :) Note: The validator should be added after the json-body-parser, so you can validate individual fields of the input data. ( ) validator jsonschema.net here a solution Exercise 3 Exercise 3: validate! : validate! @loige 40

Slide 73

Slide 73 text

We have our dashboards fully working, but consuming them in JSON is definitely not the friendliest experience! It would be much better to do in a properly rendered HTML so that we could easily visualize them on our mobile device! — You can implement this in (at least) 2 ways : Using the middleware and building a frontend somewhere like or Using the middleware and implementing a specific rule for accept "text/html" ( - ) http-cors codesandbox codepen http-content-negotiation a CORS solution a CodeSandbox React UI Exercise 4 Exercise 4: mobile rendering : mobile rendering @loige 41

Slide 74

Slide 74 text

Did you notice that our APIs are fully open? Knowing the endpoint and the id, anyone can basically change or delete the dashboards! That's unacceptable! — Find a way to secure the APIs. One easy way could be to create a secret random token when a dashboard is created the first time. Then you can create a custom middleware that checks the secret every time someone is trying to modify or delete that dashboard! Note: if you feel fancy you can use JWT tokens, in such case you could take inspiration from the middleware! jwt-auth Exercise 5 Exercise 5: better safe than sorry : better safe than sorry @loige 42

Slide 75

Slide 75 text

Sometimes our APIs are very slow. They take literally seconds to answer. This is because of a common serverless issue known as " ". If you are running for your train every morning, this is unacceptable! We should fix this... — Use the middleware to mitigate the cold start issue Note: make sure to also setup the proper event for every API. cold start problem warmup schedule 2 @loige Exercise 6 Exercise 6: stop the cold!

Slide 76

Slide 76 text

Some of your Java friends would love to use these APIs to build a shiny SWING UI... The problem is that they don't really like JSON and they would rather have a good old XML-based API. You might think that this is actually their problem, but also, you don't want to lose your friends... — Use the middleware to provide responses in XML for accept "application/xml". Note: You could use a library like to automate the conversion of JavaScript objects to XML. http-content-negotiation json2xml @loige Exercise 7 Exercise 7: legacy friendship : legacy friendship 44

Slide 77

Slide 77 text

In summary In summary Serverless is cool, it helps you to build apps quickly and with a greater focus on business logic, rather than on infrastructure! Middy helps you to keep focusing on your business logic first You can add extra behaviours with very minimal changes to your core logic by introducing dedicated middlewares You can easily share common functionality through middlewares 45

Slide 78

Slide 78 text

If you like Middy If you like Middy Use it Use it Give feedback Give feedback Contribute (I am looking for co-maintainers) Contribute (I am looking for co-maintainers) Version 1.0 (stable) coming soon Version 1.0 (stable) coming soon 46

Slide 79

Slide 79 text

Thank you Thank you @loige 47

Slide 80

Slide 80 text

Credits Credits Cover Image by from A special thank you to all the amazing Middy users and ! Thanks to for reviewing these slides and to for finding lots of bugs! TotumRevolutum Pixabay contributors @StefanoAbalsamo @organicdelight @loige 48