Serverless applications on AWS, presented by Julien Simon at AWS Sofia 2018

A quick refresh on AWS Lambda, Amazon API Gateway and serverless architectures as well as demos of several development frameworks (AWS SAM, Serverless, or Chalice)

http://aws.techhuddle.com

Transcript

1. ©2018, Amazon Web Services, Inc. or its affiliates. All rights reserved Building serverless applications Julien Simon, Principal Technical Evangelist, AWS @julsimon April 2018
2. Agenda • Serverless? • AWS Lambda • Simplifying development • Simplifying deployment • Serverless architectures
3. Werner Vogels, CTO, Amazon.com AWS re:Invent 2015
4. Serverless architecture = Managed services + AWS Lambda
5. AWS Lambda • Announced at re:Invent 2014 • Deploy functions in Java, Python, Node.js ,C# and Go. • Just code, without the infrastructure drama • Built-in scalability and high availability • Integrated with many AWS services • Pay as you go • Combination of execution time (100ms slots) & memory used. • Starts at $0.20 per million requests. • Free tier available: first 1 million requests per month are free. • Orchestration with AWS Step Functions. http://aws.amazon.com/lambda
6. What can you build with serverless architectures? • Automate your AWS infrastructure • Build event-driven applications • Build APIs together with Amazon API Gateway
7. Building a serverless data pipeline Lambda DynamoDB Kinesis Firehose API Gateway HTTP POST /prod/logger writeTo Kinesis DynamoDB ToFirehose S3 eventTable DynamoDB streams bucket EMR, Redshift, … firehoseToS3 Kinesis Streams Lambda Lambda KinesisTo DynamoDB Web apps http://www.slideshare.net/JulienSIMON5/building-a-serverless-pipeline
8. DEMO
9. Building a serverless data pipeline Lambda DynamoDB Kinesis Firehose API Gateway HTTP POST /prod/logger writeTo Kinesis DynamoDB ToFirehose S3 eventTable DynamoDB streams bucket EMR, Redshift, … firehoseToS3 Kinesis Streams Lambda Lambda KinesisTo DynamoDB Web apps Lines of code: 16 Number of servers: zero Performance & scalability: maximum
10. https://read.acloud.guru/serverless-the-future-of-software-architecture-d4473ffed864 A Cloud Guru: 100% Serverless
11. Simplifying Development Code available at https://github.com/juliensimon/aws/tree/master/lambda_frameworks
12. The Serverless framework formerly known as JAWS: Just AWS Without Servers • Announced at re:Invent 2015 • Auto-deploys and runs Lambda functions, locally or remotely • Auto-deploys your Lambda event sources: API Gateway, S3, DynamoDB, etc. • Creates all required infrastructure with CloudFormation • Simple configuration in YML http://github.com/serverless/serverless https://serverless.com
13. Serverless: “Hello World” API $ serverless create Edit handler.js, serverless.yml and event.json $ serverless deploy [--stage stage_name] $ serverless invoke [local] --function function_name $ serverless info $ http $URL
14. AWS Chalice Think of it as a serverless framework for Flask apps • Released in July 2016 • Just add your Python code – Deploy with a single call and zero config – The API is created automatically, the IAM policy is auto-generated • Run APIs locally on port 8000 (similar to Flask) https://github.com/awslabs/chalice
15. AWS Chalice: PUT/GET in S3 bucket $ chalice new-project s3test Write your function in app.py $ chalice local $ http put http://localhost:8000/objects/doc.json value1=5 value2=8 $ http get http://localhost:8000/objects/doc.json $ chalice deploy [stage_name] $ export URL=`chalice url` $ http put $URL/objects/doc.json value1=5 value2=8 $ http get $URL/objects/doc.json
16. DEMO
17. More development tools https://java.awsblog.com/post/TxWZES6J1RSQ2Z/Testing-Lambda-functions-using-the-AWS-Toolkit-for-Eclipse https://aws.amazon.com/blogs/developer/aws-toolkit-for-eclipse-serverless-application https://github.com/awslabs/aws-serverless-java-container https://github.com/awslabs/aws-serverless-express Eclipse plug-in • Code, test and deploy Lambdas from Eclipse • Run your functions locally and remotely • Test with local events and Junit4 Serverless Java Container • Run Java RESTful APIs as-is • Default implementation of the Java servlet HttpServletRequest HttpServletResponse • Support for Java frameworks such as Spring, Jersey or Spark Serverless Express Use your existing Node.js application framework on top of Lambda and API Gateway.
18. Simplifying Deployment
19. AWS Serverless Application Model (SAM) • CloudFormation extension to bundle Lambda functions, APIs & events • 3 new CloudFormation resource types – AWS::Serverless::Function – AWS::Serverless::Api – AWS::Serverless::SimpleTable • 2 new CloudFormation CLI commands – ‘aws cloudformation package’ – ‘aws cloudformation deploy’ • Integration with CodeBuild and CodePipeline for CI/CD https://aws.amazon.com/fr/blogs/compute/introducing-simplified-serverless-application-deplyoment-and-management https://github.com/awslabs/serverless-application-model/
20. AWSTemplateFormatVersion: '2010-09-09' Transform: AWS::Serverless-2016-10-31 Description: Get items from a DynamoDB table. Resources: GetFunction: Type: AWS::Serverless::Function Properties: Handler: index.get Runtime: nodejs4.3 Policies: AmazonDynamoDBReadOnlyAccess Environment: Variables: TABLE_NAME: !Ref Table Events: GetResource: Type: Api Properties: Path: /resource/{resourceId} Method: get Table: Type: AWS::Serverless::SimpleTable Sample SAM template for: • Lambda function • HTTP GET API • DynamoDB table
21. DEMO
22. AWS SAM Local • Test functions locally. • Start a local API Gateway from a SAM template. • Validate a SAM template. • Generate sample payloads for various event sources. https://github.com/awslabs/aws-sam-local https://aws.amazon.com/blogs/aws/new-aws-sam-local-beta-build-and-test-serverless-applications-locally/
23. AWS Serverless Application Repository https://aws.amazon.com/serverless/serverlessrepo/ https://aws.amazon.com/blogs/aws/now-available-aws-serverless-application-repository/
24. Serverless architectures
25. Web application Data stored in Amazon DynamoDB Dynamic content in AWS Lambda Amazon API Gateway Browser Amazon CloudFront Amazon S3 Amazon Cognito
26. Search and Data Catalog • DynamoDB as Metadata repository • Amazon Elasticsearch AWS Lambda AWS Lambda Metadata Index (DynamoDB) Search Index (Amazon ES) ObjectCreated ObjectDeleted PutItem Update Stream Update Index Extract Search Fields S3 Bucket https://aws.amazon.com/answers/big-data/data-lake-solution/
27. Serverless batch processing AWS Lambda: Splitter Amazon S3 Object Amazon DynamoDB: Mapper Results AWS Lambda: Mappers …. …. AWS Lambda: Reducer Amazon S3 Results
28. Streaming data ingestion Amazon CloudWatch: Delivery metrics Amazon S3: Buffered files Kinesis Agent Record Producers Amazon Redshift: Table loads Amazon Elasticsearch Service: Domain loads Amazon S3: Source record backup AWS Lambda: Transformations & enrichment Amazon DynamoDB: Lookup tables Raw records Lookup Transformed records Transformed records Raw records Amazon Kinesis Firehose: Delivery stream
29. Real-time analytics Amazon Kinesis Streams: Ingest stream Amazon Kinesis Analytics: Time window aggregation Amazon Kinesis Streams: Aggregates stream Amazon Kinesis Firehose: Error stream Amazon S3: Error records Record Producers AWS Lambda: Alert function Amazon DynamoDB: Device thresholds AWS SNS: Notifications
30. Image recognition and processing Web App Amazon DynamoDB: Image meta-data & tags Amazon Cognito: User authentication Amazon S3: Image uploads AWS Step Functions: Workflow orchestration Start state machine execution 1 Extract image meta-data 2 Amazon Rekognition: Object detection Invoke Amazon Rekognition Generate image thumbnail 3 3 Store meta-data and tags 4 https://github.com/awslabs/lambda-refarch-imagerecognition
31. Going further
32. The only Lambda book you need to read Written by AWS Technical Evangelist Danilo Poccia https://www.amazon.com/Aws-Lambda-Action-Event- driven-Applications/dp/1617293717/
33. AWS Whitepapers on serverless architectures • Optimizing Enterprise Economics with Serverless Architectures https://d0.awsstatic.com/whitepapers/optimizing-enterprise-economics-serverless-architectures.pdf • Serverless Architectures with AWS Lambda https://d1.awsstatic.com/whitepapers/serverless-architectures-with-aws-lambda.pdf • Serverless Applications Lens - AWS Well-Architected Framework https://d1.awsstatic.com/whitepapers/architecture/AWS-Serverless-Applications-Lens.pdf • Streaming Data Solutions on AWS with Amazon Kinesis https://d1.awsstatic.com/whitepapers/whitepaper-streaming-data-solutions-on-aws-with-amazon- kinesis.pdf • AWS Serverless Multi-Tier Architectures https://d1.awsstatic.com/whitepapers/AWS_Serverless_Multi-Tier_Archiectures.pdf
34. Thank you! Julien Simon, Principal Technical Evangelist, AWS @julsimon