Share
Embed
The future will be Serverless - JSDay Verona 2018
Software development is on the verge of a new revolution that will change the rules of the Cloud Computing game... again! The new wind of change is called "Serverless" and you should definitely get ready for it! In this talk I will illustrate why I believe Serverless will be a game changer in the industry, how did we get to have Serverless and how to get started with it to build real products.
Luciano Mammino
May 09, 2018
Transcript
66 slides
-
THE FUTURE WILL BE THE FUTURE WILL BE SERVERLESS SERVERLESS
Verona, 9 May 2018 Luciano Mammino ( ) @loige loige.link/serverless-jsday 1 -
LUCIANO... WHO LUCIANO... WHO Find me online: - (@loige) -
(lmammino) - - (loige.co) Twitter GitHub Linkedin Blog Solution Architect at with @mariocasciaro with @andreaman87 with @ Podgeypoos79 3 -
AGENDA AGENDA CHAPTER 1: CHAPTER 1: FROM BARE METAL TO
SERVERLESS FROM BARE METAL TO SERVERLESS CHAPTER 2: CHAPTER 2: SERVERLESS, WTF?! SERVERLESS, WTF?! CHAPTER 3: CHAPTER 3: UNDERSTANDING SERVERLESS UNDERSTANDING SERVERLESS CHAPTER 4: CHAPTER 4: A SERVERLESS USE CASE A SERVERLESS USE CASE CHAPTER 5: CHAPTER 5: PROS N' CONS PROS N' CONS CHAPTER 6: CHAPTER 6: IT'S TIME TO GET STARTED IT'S TIME TO GET STARTED 4 -
CHAPTER 1 CHAPTER 1 FROM FROM BARE METAL BARE METAL
TO TO SERVERLESS SERVERLESS loige.co/from-bare-metal-to-serverless 5 -
“YOU HAVE TO KNOW THE PAST TO UNDERSTAND THE “YOU
HAVE TO KNOW THE PAST TO UNDERSTAND THE PRESENT.” PRESENT.” ― Carl Sagan 6 -
1997 — Grid computing: 2 machines are better than 1!
Ian Foster 10 -
1999 — Salesforce introduces the concept of Software as a
Service (SaaS) Marc Benioff 11 -
2002-2006 — AWS is born (IaaS), people talk about "Cloud
computing" 13 -
2009 — Heroku and the invention of the "Platform as
a Service" (PaaS) James Lindenbaum Adam Wiggins Orion Henry 14 -
2011 — Envolve/Firebase, real time database as a service (RTDaaS???)
James Tamplin and Andrew Lee 15 -
2012 — Parse.com and the first Backend as a Service
(BaaS) Tikhon Bernstam Ilya Sukhar James Yu Kevin Lacker 16 -
2013 — Docker, "linux containers are better than virtual machines"
Solomon Hykes 17 -
"Serverless most often refers to serverless applications. Serverless applications are
ones that don't require you to provision or manage any servers. You can focus on your core product and business logic instead of responsibilities like operating system (OS) access control, OS patching, provisioning, right-sizing, scaling, and availability. By building your application on a serverless platform, the platform manages these responsibilities for you." — Amazon Web Services loige.link/serverless-apps-lambda 21 -
“ The essence of the serverless trend is the absence
of the server concept during software development. — Auth0 loige.link/what-is-serverless 22 -
THE 4 PILLARS OF SERVERLESS THE 4 PILLARS OF SERVERLESS
(TLDR; IT'S NOT ONLY ABOUT SERVERS) (TLDR; IT'S NOT ONLY ABOUT SERVERS) No server management You don't know how many and how they are configured Flexible scaling If you need more resources, they will be allocated for you High availability Redundancy and fault tolerance are built in Never pay for idle Unused resources cost $0 26 -
THE SERVERLESS LAYERS THE SERVERLESS LAYERS (TLDR; IT'S NOT ONLY
"FAAS") (TLDR; IT'S NOT ONLY "FAAS") COMPUTE COMPUTE DATA DATA MESSAGING AND STREAMING MESSAGING AND STREAMING 27 USER MANAGEMENT AND IDENTITY USER MANAGEMENT AND IDENTITY MONITORING AND DEPLOYMENT MONITORING AND DEPLOYMENT EDGE EDGE -
STUFF THAT WE CAN BUILD STUFF THAT WE CAN BUILD
Mobile Backends APIs & Microservices Data Processing pipelines ⚡ Webhooks Bots and integrations ⚙ IoT Backends Single page web applications 28 -
SOME SERVERLESS APPS I HELPED BUILD SOME SERVERLESS APPS I
HELPED BUILD Open source A "semi-automated" weekly newsletter ( ) A middleware framework for AWS Lambda ( ) Enterprise Various solutions for the UK Energy industry: Trading platform, Billing engine, Market data aggregator ( ) Big data pipeline (~1-5TB/day/customer) to make network data searchable ( ) Fullstack bulletin middy.js.org Planet 9 Energy Vectra.ai 29 -
IF ________________________________ THEN ________________________________ "IF "IF THIS THIS THEN THEN
THAT THAT" MODEL " MODEL 31 -
SERVERLESS AND JAVASCRIPT SERVERLESS AND JAVASCRIPT Frontend Serverless Web hosting
is static, but you can build SPAs (React, Angular, Vue, etc.) Backend Node.js is supported by every provider ⚡ Fast startup (as opposed to Java) Use all the modules on NPM Support other languages/dialects (TypeScript, ClojureScript, ESNext...) 32 -
exports.myLambda = function ( event, context, callback ) { //
get input from event and context // use callback to return output or errors } ANATOMY OF A NODE.JS LAMBDA ON AWS ANATOMY OF A NODE.JS LAMBDA ON AWS 33 -
Search for threats using network metadata Security researcher Have an
API to be able to search across network metadata files available on the shared FTP drive I can find and validate potential security threats on the network 35 -
A SERVERLESS IMPLEMENTATION (ON AWS) A SERVERLESS IMPLEMENTATION (ON AWS)
Network Metadata FTP Storage Search API (API Gateway) Network metadata search service User or other services Sync Lambda Parse/Load Lambda API Lambda Network Metadata S3 Bucket /metadata elastic search index 36 scheduled event new object API Request -
SYNC LAMBDA SYNC LAMBDA { "account": "123456789012", "region": "us-east-1", "detail":
{}, "detail-type": "Scheduled Event", "source": "aws.events", "time": "2018-05-09T14:30:21Z", "id": "cdc73f9d-aea9-1234-9d5a-835b769c0d9c", "resources": [ "arn:aws:events:us-east-1:123456789012:rule/my-schedule" ] } Cloudwatch scheduled event 37 -
exports.syncLambda = ( event, context, callback ) => { //
1. connect to the FTP server // 2. get list of files from FTP // 3. get list of files from S3 // 4. make a diff to find new files in FTP // 5. read the new files and save them to S3 // 6. invoke the callback to stop lambda } SYNC LAMBDA SYNC LAMBDA 38 -
LOAD LAMBDA LOAD LAMBDA { "Records": [ { "eventTime": "2018-05-09T14:30:21Z",
"s3": { "object": { "eTag": "0123456789abcdef0123456789abcdef", "sequencer": "0A1B2C3D4E5F678901", "key": "2018-05-09-metadata.csv.gz", "size": 1024 }, "bucket": { "arn": "arn:aws:s3:::metadata-sync-files", "name": "metadata-sync-files", } }, "awsRegion": "us-east-1", "eventName": "ObjectCreated:Put", "eventSource": "aws:s3" } ] } S3 new object event new object 39 -
exports.loadLambda = ( event, context, callback ) => { //
1. get the new file details from the `event` // 2. read the file and deserialize the records // 3. send the records to elastic search using // the bulk API // 4. invoke the callback to stop lambda } LOAD LAMBDA LOAD LAMBDA 40 -
API LAMBDA API LAMBDA { "body": "", "resource": "/metadata/search", "requestContext":
{ "resourceId": "123456", "apiId": "1234567890", "resourcePath": "/metadata/search", "httpMethod": "GET", "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef", "stage": "prod" }, "queryStringParameters": { "q": "srcIp:127.0.0.1 AND host:google.com" }, "headers": { "User-Agent": "Custom User Agent String", }, "httpMethod": "GET", "path": "/metadata/search" } API Gateway proxy event API Request 41 -
exports.apiLambda = ( event, context, callback ) => { //
1. read the HTTP details (query) from // the `event` // 2. issue the query to ElasticSearch to the // given index // 3. reformat the results into an API gateway // response // 4. invoke the callback passing the response } API LAMBDA API LAMBDA 42 -
ORCHESTRATION / CONFIGURATION ORCHESTRATION / CONFIGURATION WHO DEFINES THE EVENTS
AND WHERE DO WE STORE THE CONFIG? WHO DEFINES THE EVENTS AND WHERE DO WE STORE THE CONFIG? 43 -
ORCHESTRATION / CONFIGURATION ORCHESTRATION / CONFIGURATION MANY OPTIONS MANY OPTIONS
Manually a.k.a. "clicky-clicky" from the Web UI Declarative code-based approaches: or AWS Cloudformation SAM Serverless framework Terraform 44 -
service: metadataSearch provider: name: aws runtime: nodejs8.10 functions: sync: handler:
index.syncLambda environment: FTP_HOST: "28.0.1.22:23" FTP_USERNAME: admin FTP_PASSWORD: unicorns S3_BUCKET: metadata-sync-files events: - schedule: rate(2 hours) load: handler: index.loadLambda environment: ES_INDEX: metadata events: - s3: bucket: metadata-sync-files event: "s3:ObjectCreated:*" api: handler: index.apiLambda environment: ES_INDEX: metadata events: - http: GET metadata/search SERVERLESS FRAMEWORK EXAMPLE SERVERLESS FRAMEWORK EXAMPLE serverless.yml sls deploy Reads the serverless.yml and parses the resources Creates a zip file containing the source code Creates a Cloudformation stack with all the resources listed in serverless.yml Deploys the stack (including Lambda code) $ 45 -
(MANY) THINGS I DIDN'T HAVE TO WORRY ABOUT... (MANY) THINGS
I DIDN'T HAVE TO WORRY ABOUT... What type of virtual machine do I need? What operating system? How to keep OS/System updated? How much disk space do I need? How do I handle/retry failures? How do I collect and rotate logs? What about metrics? What machine do I need to run the database? How do I backup the database? How do I scale the database? Which web server should I use and how to configure it? Throttling? Managing API Keys? API caching? 46 -
FOCUS ON DELIVERING BUSINESS VALUE / FAST TIME FOCUS ON
DELIVERING BUSINESS VALUE / FAST TIME TO MARKET TO MARKET 48 -
LESS "TECH-FREEDOM™ " / LESS "TECH-FREEDOM™ " / TIGHT VENDOR
LOCK-IN! TIGHT VENDOR LOCK-IN! 49 -
NOT-MAGIC™! NOT-MAGIC™! YOU STILL HAVE TO WRITE CONFIGURATION YOU STILL
HAVE TO WRITE CONFIGURATION 51 -
OLDER TECHNOLOGIES OLDER TECHNOLOGIES MIGHT NOT INTEGRATE WELL MIGHT NOT
INTEGRATE WELL 59 -
CHAPTER 6 CHAPTER 6 IT'S TIME TO GET STARTED IT'S
TIME TO GET STARTED 60 -
PICK A FAAS AND START TO HAVE FUN! PICK A
FAAS AND START TO HAVE FUN! IBM Cloud Functions AWS Lambda Azure Functions Google Cloud Functions Auth0 Webtask Iron.io FaaS Spotinst Functions Apache OpenWhisk Fission stdlib service Functions 62 Fn Kubeless CLOUD BASED CLOUD BASED SELF-HOSTED / OPEN SOURCE SELF-HOSTED / OPEN SOURCE F effe LeverOS Open FaaS -
WHY IS SERVERLESS THE FUTURE? WHY IS SERVERLESS THE FUTURE?
MY 2 FAVOURITE REASONS: MY 2 FAVOURITE REASONS: Opportunity to deliver value to the customer quickly Pay only for the used resources 63 -
TOMORROW I'LL MIGRATE ALL TOMORROW I'LL MIGRATE ALL MY APPS
TO SERVERLESS MY APPS TO SERVERLESS APPROACH THIS APPROACH THIS WITH CARE... WITH CARE... 64 -
THANKS! THANKS! QUESTIONS? QUESTIONS? Now or later to :) @loige
loige.link/serverless-jsday If your company wants to get started with serverless on AWS, be sure to check out serverlesslab.com 65 -
CREDITS CREDITS High Res Emojis by emojiisland.com World Wide Web
Foundation, history of the web Web hosting on Wikipedia The history of web hosting: how things have changed since Tibus started in 1996 Grid computing on Wikipedia Globus toolkit on Wikipedia Globus Alliance on Wikipedia Globus: A Metacomputing Infrastructure Toolkit (slides) The Anatomy of the Grid: Enabling Scalable Virtual Organizations (paper) Computer Weekly, the history of Cloud Computing Brief history of Salesforce.com With long history of virtualization behind it, IBM looks to the future About Amazon Web Services (official) Fullstack python: What is Serverless Parse (platform) on Wikipedia Firebase on Wikipedia Kubernetes on Wikipedia CoreOS on Wikipedia AWS Lambda on Wikipedia Serverless Computing on Wikipedia "AWS Serverless Applications Lens" white paper (pdf) "Serverless Architectures with AWS Lambda" white paper (pdf) AWS Lambda VS the World Serverless: Looking Back to See Forward Awesome Serverless (GitHub repository) Discussion about the history of Serverless on Lobste.rs a HUGE thanks to: @vladholubiev @mariocasciaro @katavic_d @Podgeypoos79 PLEASE GIVE FEEDBACK loige.link/vote-jsday 66