Как построить приложение, не создав ни одного сервера

DevOpsDaysMoscow, 07-12-2019, Роман Бойко

В своем докладе я расскажу о подходах к построению serverless приложений на AWS. Я расскажу:
– как локально разрабатывать и отлаживать AWS Lambda функции при помощи AWS SAM;
– деплоить их с AWS CDK;
– мониторить на AWS CloudWatch;
– и автоматизировать весь процесс при помощи AWS Code.

Transcript

1. © 2019, Amazon Web Services, Inc. or its Affiliates. Roman Boiko Solutions Architect Amazon Web Services How to build modern applications without creating servers
2. © 2019, Amazon Web Services, Inc. or its Affiliates. Agenda Serverless foundations Web application Stream processing Data lake Machine learning CI/CD for Serverless
3. © 2019, Amazon Web Services, Inc. or its Affiliates. Serverless means … No server or container management Flexible scaling No idle capacity $ High availability
4. © 2019, Amazon Web Services, Inc. or its Affiliates. AWS Lambda AWS Fargate Amazon API Gateway Amazon SNS Amazon SQS AWS Step Functions COMPUTE DATA STORES INTEGRATION Amazon Aurora Serverless Amazon S3 Amazon DynamoDB AWS AppSync
5. © 2019, Amazon Web Services, Inc. or its Affiliates. Common Serverless use cases Web Applications • Static websites • Complex web apps • Packages for Flask and Express Data Processing • Real time • MapReduce • Batch Chatbots • Powering chatbot logic Backends • Apps & services • Mobile • IoT </> </> Amazon Alexa • Powering voice-enabled apps • Alexa Skills Kit IT Automation • Policy engines • Extending AWS services • Infrastructure management
6. © 2019, Amazon Web Services, Inc. or its Affiliates. Web app/microservice/API
7. © 2019, Amazon Web Services, Inc. or its Affiliates. Web application Data stored in Amazon DynamoDB Dynamic content in AWS Lambda Amazon API Gateway Browser Amazon CloudFront Amazon S3 Amazon Cognito
8. © 2019, Amazon Web Services, Inc. or its Affiliates. Web application Data stored in Amazon DynamoDB Dynamic content in AWS Lambda Browser Amazon CloudFront Amazon S3 Amazon Cognito ALB
9. © 2019, Amazon Web Services, Inc. or its Affiliates. Web application Data stored in Amazon DynamoDB Dynamic content in AWS Lambda Amazon API Gateway Browser Amazon CloudFront Amazon S3 Amazon Cognito Lambda@Edge
10. © 2019, Amazon Web Services, Inc. or its Affiliates. Amazon API Gateway AWS Lambda Amazon DynamoDB Amazon S3 Amazon CloudFront • Bucket Policies • ACLs • OAI • Geo-restriction • Signed cookies • Signed URLs • DDoS protection IAM AuthZ Serverless web app security • Cross-account Lambda • Throttling (per method) • Resource policies • Usage plans Static content Browser Amazon Cognito • Encryption at rest • VPC endpoint • Function policies • Env variables
11. © 2019, Amazon Web Services, Inc. or its Affiliates. AWS AppSync DynamoDB table Lambda function Amazon ES GraphQL schema Upload schema GraphQL Query mutation subscription Real-time online/offline AppSync API Amazon Cognito user pool Legacy application Amazon RDS https:// HTTP resolver
12. © 2019, Amazon Web Services, Inc. or its Affiliates. Streaming processing
13. © 2019, Amazon Web Services, Inc. or its Affiliates. Streaming with Amazon Kinesis Easily collect, process, and analyze video and data streams in real time Capture, process, and store video streams Kinesis Video Streams Load data streams into data stores Kinesis Data Firehose SQL Analyze data streams with SQL Kinesis Data Analytics Capture, process, and store data streams Kinesis Data Streams
14. © 2019, Amazon Web Services, Inc. or its Affiliates. Streaming data ingestion Amazon CloudWatch: Delivery metrics Amazon S3: Buffered files Kinesis Agent Record producers Amazon Redshift: Table loads Amazon Elasticsearch Service: Domain loads Amazon S3: Source record backup AWS Lambda: Transformations & enrichment Amazon DynamoDB: Lookup tables Raw records Lookup Transformed records Transformed records Raw records Kinesis Data Firehose: Delivery stream
15. © 2019, Amazon Web Services, Inc. or its Affiliates. Fan-out pattern Fan-out pattern trades strict message ordering versus higher throughput & lower latency Kinesis Data Streams: Stream Lambda: Dispatcher function Lambda: Processor function Increase throughput, reduce processing latency Streaming source
16. © 2019, Amazon Web Services, Inc. or its Affiliates. Real-time analytics Kinesis Data Streams: Ingest stream Kinesis Data Analytics: Time window aggregation Kinesis Data Firehose: Error stream Amazon S3: Error records Record producers AWS Lambda: Alert function Amazon DynamoDB: Device thresholds Amazon SNS: Notifications
17. © 2019, Amazon Web Services, Inc. or its Affiliates. Data lake
18. © 2019, Amazon Web Services, Inc. or its Affiliates. AWS serverless data lake Amazon S3 bucket(s) Amazon ES AWS Glue Amazon DynamoDB Catalog & Search AWS KMS AWS CloudTrail IAM Amazon Macie Security & Auditing Amazon Cognito Amazon API Gateway IAM API/UI Amazon Athena Amazon QuickSight Amazon Redshift Spectrum Analytics & Processing AWS Glue AWS Lambda Kinesis Data Streams Kinesis Data Firehose AWS Direct Connect Ingest AWS IoT
19. © 2019, Amazon Web Services, Inc. or its Affiliates. AWS Glue Crawlers AWS Glue Data Catalog Amazon QuickSight Amazon Athena S3 Bucket(s) Instantly query your data lake on Amazon S3
20. © 2019, Amazon Web Services, Inc. or its Affiliates. Serverless batch processing AWS Lambda: Splitter Amazon S3 Object Amazon DynamoDB: Mapper Results AWS Lambda: Mappers …. …. AWS Lambda: Reducer Amazon S3 Results
21. © 2019, Amazon Web Services, Inc. or its Affiliates. AWS Lake Formation
22. © 2019, Amazon Web Services, Inc. or its Affiliates. Machine learning
23. © 2019, Amazon Web Services, Inc. or its Affiliates. 2. Submit image 4. DetectFaces 7. DetectText 1. Upload 3. Store image Lambda AWS Step Functions 5. DetectLabels 6. DetectModerationLabels 8. Store metadata & analysis DynamoDB Amazon ES Image processing with Amazon Rekognition Image
24. © 2019, Amazon Web Services, Inc. or its Affiliates. Intelligent call center chatbot Amazon Connect Customer Amazon Lex AWS Lambda: Chatbot Processing DynamoDB: Customer Data Amazon SNS: SMS Messaging Customer calls Connect to reschedule an appointment Connect calls Lex chatbot Lex chatbot calls Lambda function to get customer preferences and fulfil Intents Lambda function sends text message confirmation via SNS Customer receives appointment confirmation text message Lambda function writes updates to DynamoDB
25. © 2019, Amazon Web Services, Inc. or its Affiliates. CI/CD for Serverless
26. © 2019, Amazon Web Services, Inc. or its Affiliates. Release process stages Source Build Test Production
27. © 2019, Amazon Web Services, Inc. or its Affiliates. Author and debug Lambda applications on AWS using your favorite IDEs AWS Cloud9 Python, Node AWS Toolkit for PyCharm Python AWS Toolkit for IntelliJ Java, Python AWS Toolkit for Visual Studio Code .NET, Node
28. © 2019, Amazon Web Services, Inc. or its Affiliates. Model function environments with AWS Serverless Application Model (SAM) https://aws.amazon.com/serverless/sam/ • Open source framework for building serverless applications on AWS • Shorthand syntax to express functions, APIs, databases, and event source mappings • Transforms and expands SAM syntax into AWS CloudFormation syntax on deployment • Supports all AWS CloudFormation resource types
29. © 2019, Amazon Web Services, Inc. or its Affiliates. Model container environments with AWS Cloud Development Kit (CDK) • Open source framework to define cloud infrastructure in Typescript • Provides library of higher-level resource types (“construct” classes) that have AWS best practices built in by default, packaged as npm modules • Provisions resources with CloudFormation • Supports all CloudFormation resource types AWS CDK https://awslabs.github.io/aws-cdk
30. © 2019, Amazon Web Services, Inc. or its Affiliates. AWS CodePipeline • Continuous delivery service for fast and reliable application updates • Model and visualize your software release process • Builds, tests, and deploys your code every time there is a code change • Integrates with third-party tools and AWS
31. © 2019, Amazon Web Services, Inc. or its Affiliates. AWS CodeBuild • Fully managed build service that compiles source code, runs tests, and produces software packages • Scales continuously and processes multiple builds concurrently • No build servers to manage • Pay by the minute, only for the compute resources you use • Monitor builds through CloudWatch Events
32. © 2019, Amazon Web Services, Inc. or its Affiliates. AWS CodeBuild: Lambda buildspec version: 0.2 phases: build: commands: - npm ci - npm test - > aws cloudformation package --template-file template.yml --output-template template-output.yml --s3_bucket $BUCKET artifacts: type: zip files: - template-output.yml
33. © 2019, Amazon Web Services, Inc. or its Affiliates. AWS CodeDeploy • Automates code deployments to any instance and Lambda • Handles the complexity of updating your applications • Avoid downtime during application deployment • Roll back automatically if failure detected • Deploy to Amazon EC2, Lambda, or on-premises servers
34. © 2019, Amazon Web Services, Inc. or its Affiliates. CodeDeploy-Lambda deployments Enable in your serverless application template Resources: GetFunction: Type: AWS::Serverless::Function Properties: DeploymentPreference: Type: Canary10Percent10Minutes Alarms: - !Ref ErrorsAlarm Hooks: PreTraffic: !Ref PreTrafficHook
35. © 2019, Amazon Web Services, Inc. or its Affiliates. CodeDeploy-Lambda canary deployment 100%
36. © 2019, Amazon Web Services, Inc. or its Affiliates. CodeDeploy-Lambda canary deployment 100% Run hook against v2 code before it receives traffic 0%
37. © 2019, Amazon Web Services, Inc. or its Affiliates. CodeDeploy-Lambda canary deployment 90% Wait for 10 minutes, roll back in case of alarm 10%
38. © 2019, Amazon Web Services, Inc. or its Affiliates. CodeDeploy-Lambda canary deployment 0% Complete deployment 100%
39. © 2019, Amazon Web Services, Inc. or its Affiliates. AWS X-Ray Integration with Serverless • Lambda instruments incoming requests for all supported languages and can capture calls made in code • API Gateway inserts a tracing header into HTTP calls as well as reports data back to X-Ray itself
40. © 2019, Amazon Web Services, Inc. or its Affiliates. Summary “No server is easier to manage than no server.” - Werner Vogels, Amazon CTO Never pay for idle & scale as you grow Apply serverless patterns for common use-cases: Web applications Stream processing Data lake Machine learning What will you build with serverless?
41. © 2019, Amazon Web Services, Inc. or its Affiliates. Thanks! Q&A romboiko@amazon.de