Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless, The Middy Way - Workshop

Serverless, The Middy Way - Workshop

An introduction workshop about Serverless on AWS using Middy, The stylish Node.js middleware engine for AWS Lambda!

Luciano Mammino

October 01, 2019
Tweet

More Decks by Luciano Mammino

Other Decks in Technology

Transcript

  1. Serverless Serverless The "Middy" way The "Middy" way — Workshop

    loige.link/middy-way Community Day Dublin October , 1st 2019 Luciano Mammino @loige Wi-Fi: DC_Conference October-2019 1
  2. 2

  3. Hello, I am Luciano! Hello, I am Luciano! Cloud Architect

    Blog: Twitter: GitHub: loige.co @loige @lmammino 2
  4. Hello, I am Luciano! Hello, I am Luciano! Cloud Architect

    Blog: Twitter: GitHub: loige.co @loige @lmammino 2
  5. Hello, I am Luciano! Hello, I am Luciano! Cloud Architect

    Blog: Twitter: GitHub: loige.co @loige @lmammino 2
  6. What is serverless What is serverless Compute as functions (FaaS)

    Compute as functions (FaaS) Event-based model Event-based model @loige 5
  7. Why serverless is good Why serverless is good Focus on

    business logic Focus on business logic Scalability Scalability Pay per usage model (compute time * memory) Pay per usage model (compute time * memory) Managed infrastructure Managed infrastructure Forces you to think micro-services Forces you to think micro-services @loige 6
  8. My Serverless experience My Serverless experience Open source A semi-automated

    weekly newsletter ( ) A middleware framework for AWS Lambda ( ) Enterprise Trading, Billing engine, Market data aggregator solutions ( ) Big data pipeline to make network metadata searchable ( ) Organised various workshops around Europe ( ) Fullstack bulletin middy.js.org ESB Vectra.ai Serverlesslab.com @loige 7
  9. Have an Have an admin user admin user in your

    AWS account in your AWS account loige.link/aws-admin-user loige.link/aws-admin-user @loige 12
  10. ⏱ 20 mins warm up ♀ ⏱ 20 mins warm

    up ♀ 1. Use the APIs to create a dashboard that contains: 1. Use the APIs to create a dashboard that contains: An IrishRail station widget A LUAS stop widget A DublinBus stop widget (there are utility scripts to list all the stops and stations, check the README) 2. Have a look at the code 2. Have a look at the code If you find bugs, please send a or report an PR issue @loige 21
  11. exports.myLambda = function ( event, context, callback ) { //

    get input from event and context // use callback to return output or errors } Anatomy of a Node.js lambda on AWS Anatomy of a Node.js lambda on AWS @loige 24
  12. (event, context, callback) => { // decrypt environment variables with

    KMS // deserialize the content of the event // validate input, authentication, authorization // REAL BUSINESS LOGIC // (process input, generate output) // validate output // serialize response // handle errors } A typical "REAL" Lambda function A typical "REAL" Lambda function @loige 25
  13. (event, context, callback) => { // decrypt environment variables with

    KMS // deserialize the content of the event // validate input, authentication, authorization // REAL BUSINESS LOGIC // (process input, generate output) // validate output // serialize response // handle errors } A typical "REAL" Lambda function A typical "REAL" Lambda function @loige 25
  14. (event, context, callback) => { // decrypt environment variables with

    KMS // deserialize the content of the event // validate input, authentication, authorization // REAL BUSINESS LOGIC // (process input, generate output) // validate output // serialize response // handle errors } A typical "REAL" Lambda function A typical "REAL" Lambda function @loige 25
  15. (event, context, callback) => { // decrypt environment variables with

    KMS // deserialize the content of the event // validate input, authentication, authorization // REAL BUSINESS LOGIC // (process input, generate output) // validate output // serialize response // handle errors } A typical "REAL" Lambda function A typical "REAL" Lambda function LOTS of BOILERPLATE @loige 25
  16. Usage Usage const middy = require('@middy/core') const { middleware1, middleware2,

    middleware3 } = require('some-middlewares') const originalHandler = (event, context, callback) => { /* your business logic */ } const handler = middy(originalHandler) handler .use(middleware1()) .use(middleware2()) .use(middleware3()) module.exports = { handler } @loige 28
  17. Usage Usage const middy = require('@middy/core') const { middleware1, middleware2,

    middleware3 } = require('some-middlewares') const originalHandler = (event, context, callback) => { /* your business logic */ } const handler = middy(originalHandler) handler .use(middleware1()) .use(middleware2()) .use(middleware3()) module.exports = { handler } 1. define handler @loige 28
  18. Usage Usage const middy = require('@middy/core') const { middleware1, middleware2,

    middleware3 } = require('some-middlewares') const originalHandler = (event, context, callback) => { /* your business logic */ } const handler = middy(originalHandler) handler .use(middleware1()) .use(middleware2()) .use(middleware3()) module.exports = { handler } 2. "middify" the handler @loige 28
  19. Usage Usage const middy = require('@middy/core') const { middleware1, middleware2,

    middleware3 } = require('some-middlewares') const originalHandler = (event, context, callback) => { /* your business logic */ } const handler = middy(originalHandler) handler .use(middleware1()) .use(middleware2()) .use(middleware3()) module.exports = { handler } 3. attach middlewares @loige 28
  20. Usage Usage const middy = require('@middy/core') const { middleware1, middleware2,

    middleware3 } = require('some-middlewares') const originalHandler = (event, context, callback) => { /* your business logic */ } const handler = middy(originalHandler) handler .use(middleware1()) .use(middleware2()) .use(middleware3()) module.exports = { handler } 4. export "middyfied" handler @loige 28
  21. const middy = require('@middy/core') const urlEncodedBodyParser = require('@middy/http-urlencode-body-parser') const validator

    = require('@middy/validator') const httpErrorHandler = require('@middy/http-error-handler') const processPaymentHandler = (event, context, callback) => { const { creditCardNumber, expiryMonth, expiryYear, cvc, nameOnCard, amount } = event.body // do stuff with this data ... return callback(null, { result: 'success', message: 'payment processed correctly'} ) } const inputSchema = { // define validation schema here ... } const handler = middy(processPaymentHandler) .use(urlEncodedBodyParser()) .use(validator(inputSchema)) .use(httpErrorHandler()) module.exports = { handler } @loige 29
  22. const middy = require('@middy/core') const urlEncodedBodyParser = require('@middy/http-urlencode-body-parser') const validator

    = require('@middy/validator') const httpErrorHandler = require('@middy/http-error-handler') const processPaymentHandler = (event, context, callback) => { const { creditCardNumber, expiryMonth, expiryYear, cvc, nameOnCard, amount } = event.body // do stuff with this data ... return callback(null, { result: 'success', message: 'payment processed correctly'} ) } const inputSchema = { // define validation schema here ... } const handler = middy(processPaymentHandler) .use(urlEncodedBodyParser()) .use(validator(inputSchema)) .use(httpErrorHandler()) module.exports = { handler } Handler @loige 29
  23. const middy = require('@middy/core') const urlEncodedBodyParser = require('@middy/http-urlencode-body-parser') const validator

    = require('@middy/validator') const httpErrorHandler = require('@middy/http-error-handler') const processPaymentHandler = (event, context, callback) => { const { creditCardNumber, expiryMonth, expiryYear, cvc, nameOnCard, amount } = event.body // do stuff with this data ... return callback(null, { result: 'success', message: 'payment processed correctly'} ) } const inputSchema = { // define validation schema here ... } const handler = middy(processPaymentHandler) .use(urlEncodedBodyParser()) .use(validator(inputSchema)) .use(httpErrorHandler()) module.exports = { handler } Attach middlewares @loige 29
  24. const middy = require('@middy/core') const urlEncodedBodyParser = require('@middy/http-urlencode-body-parser') const validator

    = require('@middy/validator') const httpErrorHandler = require('@middy/http-error-handler') const processPaymentHandler = (event, context, callback) => { const { creditCardNumber, expiryMonth, expiryYear, cvc, nameOnCard, amount } = event.body // do stuff with this data ... return callback(null, { result: 'success', message: 'payment processed correctly'} ) } const inputSchema = { // define validation schema here ... } const handler = middy(processPaymentHandler) .use(urlEncodedBodyParser()) .use(validator(inputSchema)) .use(httpErrorHandler()) module.exports = { handler } Export enhanced handler @loige 29
  25. Why? Why? Simplify code Reusability input parsing input & output

    validation output serialization error handling ... @loige 30
  26. Why? Why? Simplify code Reusability input parsing input & output

    validation output serialization error handling ... Focus (even) MORE on business logic @loige 30
  27. Execution order Execution order 1. middleware1 (before) 2. middleware2 (before)

    3. middleware3 (before) 4. handler 5. middleware3 (after) 32 @loige
  28. Execution order Execution order 1. middleware1 (before) 2. middleware2 (before)

    3. middleware3 (before) 4. handler 5. middleware3 (after) 6. middleware2 (after) 32 @loige
  29. Execution order Execution order 1. middleware1 (before) 2. middleware2 (before)

    3. middleware3 (before) 4. handler 5. middleware3 (after) 6. middleware2 (after) 7. middleware1 (after) 32 @loige
  30. Execution order Execution order 1. middleware1 (before) 2. middleware2 (before)

    3. middleware3 (before) 4. handler 5. middleware3 (after) 6. middleware2 (after) 7. middleware1 (after) 32 @loige
  31. When an error happens... When an error happens... Flow is

    stopped First middleware implementing `onError` gets control It can choose to handle the error, or delegate it to the next handler If the error is handler a response is returned If the error is not handled the execution fails reporting the unhandled error @loige 33
  32. Writing a middleware Writing a middleware const myMiddleware = (config)

    => { // might set default options in config return ({ before: (handler, next) => { // might read options from `config` }, after: (handler, next) => { // might read options from `config` }, onError: (handler, next) => { // might read options from `config` } }) } module.exports = myMiddleware @loige 34
  33. Inline middlewares Inline middlewares const middy = require('@middy/core') const handler

    = middy((event, context, callback) => { // do stuff }) handler.before((handler, next) => { // do something in the before phase next() }) handler.after((handler, next) => { // do something in the after phase next() }) handler.onError((handler, next) => { // do something in the on error phase next() }) module.exports = { handler } @loige 35
  34. It supports async handlers! It supports async handlers! const middy

    = require('@middy/core') const handler = middy( async (event, context) => { // do stuff return { some: 'response' } } ) module.exports = { handler } @loige 36
  35. Exercise 0 Exercise 0: let's middify! : let's middify! In

    the next exercises we will need the full power of middy, so to get started let's just "middify" all our handlers — Note: handlers are defined in "src/handlers", and they get used in "src/handler.js". You can either middify every single handler file or middify them when used in src/handler.js. In the first case, make sure you middify the actual handler function and not the "handler factory". ( ) a solution @loige 37
  36. Exercise 1 Exercise 1: body parsing : body parsing We

    are manually deserializing the JSON input in our APIs. We can simplify the code a bit by using the middleware. — Install @middy/http-json-body-parser and remove all the instances of JSON.parse() in our handlers. ( ) http-json-body-parser a solution @loige 38
  37. What happens when one of our handlers crashes? We are

    actually not even dealing with errors right now... by using the middleware we will be able to handle all HTTP errors and return codes consistently. — Install @middy/http-error-handler and apply that to all our handlers. ( ) http-error-handler a solution Exercise 2 Exercise 2: what if we fail? : what if we fail? @loige 39
  38. We want to make sure that data is validated before

    being written to the database. We can use the middleware for this task! — Install @middy/validator and apply that to all our POST API handlers. You can use to define your schemas or, if you are struggling, you can find some schemas ready-made for you :) Note: The validator should be added after the json-body-parser, so you can validate individual fields of the input data. ( ) validator jsonschema.net here a solution Exercise 3 Exercise 3: validate! : validate! @loige 40
  39. We have our dashboards fully working, but consuming them in

    JSON is definitely not the friendliest experience! It would be much better to do in a properly rendered HTML so that we could easily visualize them on our mobile device! — You can implement this in (at least) 2 ways : Using the middleware and building a frontend somewhere like or Using the middleware and implementing a specific rule for accept "text/html" ( - ) http-cors codesandbox codepen http-content-negotiation a CORS solution a CodeSandbox React UI Exercise 4 Exercise 4: mobile rendering : mobile rendering @loige 41
  40. Did you notice that our APIs are fully open? Knowing

    the endpoint and the id, anyone can basically change or delete the dashboards! That's unacceptable! — Find a way to secure the APIs. One easy way could be to create a secret random token when a dashboard is created the first time. Then you can create a custom middleware that checks the secret every time someone is trying to modify or delete that dashboard! Note: if you feel fancy you can use JWT tokens, in such case you could take inspiration from the middleware! jwt-auth Exercise 5 Exercise 5: better safe than sorry : better safe than sorry @loige 42
  41. Sometimes our APIs are very slow. They take literally seconds

    to answer. This is because of a common serverless issue known as " ". If you are running for your train every morning, this is unacceptable! We should fix this... — Use the middleware to mitigate the cold start issue Note: make sure to also setup the proper event for every API. cold start problem warmup schedule 2 @loige Exercise 6 Exercise 6: stop the cold!
  42. Some of your Java friends would love to use these

    APIs to build a shiny SWING UI... The problem is that they don't really like JSON and they would rather have a good old XML-based API. You might think that this is actually their problem, but also, you don't want to lose your friends... — Use the middleware to provide responses in XML for accept "application/xml". Note: You could use a library like to automate the conversion of JavaScript objects to XML. http-content-negotiation json2xml @loige Exercise 7 Exercise 7: legacy friendship : legacy friendship 44
  43. In summary In summary Serverless is cool, it helps you

    to build apps quickly and with a greater focus on business logic, rather than on infrastructure! Middy helps you to keep focusing on your business logic first You can add extra behaviours with very minimal changes to your core logic by introducing dedicated middlewares You can easily share common functionality through middlewares 45
  44. If you like Middy If you like Middy Use it

    Use it Give feedback Give feedback Contribute (I am looking for co-maintainers) Contribute (I am looking for co-maintainers) Version 1.0 (stable) coming soon Version 1.0 (stable) coming soon 46
  45. Credits Credits Cover Image by from A special thank you

    to all the amazing Middy users and ! Thanks to for reviewing these slides and to for finding lots of bugs! TotumRevolutum Pixabay contributors @StefanoAbalsamo @organicdelight @loige 48