Advanced Serverless Architectural Patterns on AWS

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda Serverless foundations (quickly, I promise!) Advanced serverless patterns: 1. Web application 2. Stream processing 3. Data lake 4. Machine learning

Slide 4

Slide 4 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Compute Spectrum AWS Lambda Amazon Kinesis Amazon S3 Amazon API Gateway Amazon SQS Amazon DynamoDB AWS IoT Amazon EMR Amazon ElastiCache Amazon RDS Amazon Redshift Amazon Elasticsearch Managed Serverless Amazon EC2 Microsoft SQL Server “On Amazon EC2” Amazon Cognito Amazon CloudWatch Amazon Athena AWS X-Ray AWS Step Functions Amazon MQ Amazon SageMaker Amazon Neptune AWS Fargate Amazon DocumentDB

Slide 5

Slide 5 text

Slide 6

Slide 6 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Bootstrap the runtime Start your code Lambda: The execution lifecycle Cold start Warm start Download your code Start new container Time

Slide 7

Slide 7 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Tune your function’s resources Only a memory control - % of CPU core and network capacity allocated to a function proportionally Is your code CPU, network or memory-bound? If so, it could be cheaper to choose more memory > Memory, > Cores, > Network

Slide 8

Slide 8 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. “AWS Lambda Power Tuning” Data-driven cost & performance optimization for AWS Lambda github.com/alexcasalboni/aws-lambda-power-tuning Lambda Power Tuning

Slide 9

Slide 9 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lambda best practices Minimize your package size & use only needed SDK modules Put your dependency (e.g. .jar files) in a separate directory Improve dependency injection with smaller and simpler IoC frameworks that load quickly on startup, like Dagger2 Leverage smaller and faster frameworks like jackson-jr for Java data binding Use environment variables to modify operational behavior Secure secrets/tokens/passwords with Parameter Store and AWS Secrets Manager

Slide 10

Slide 10 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Serverless Application Model (SAM) AWS CloudFormation extension (Macro) to simplify serverless apps New serverless resource types: functions, APIs, and tables Local testing with SAM CLI github.com/awslabs/serverless-application-model

Slide 11

Slide 11 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Source Build Test Deploy AWS CodeCommit AWS CodeBuild Third Party Tooling AWS CodeDeploy AWS CodePipeline AWS CodeStar AWS code services

Slide 12

Slide 12 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lambda alias traffic shifting & AWS SAM AutoPublishAlias By adding this property AWS SAM will do the following: • Detect when new code is being deployed based on changes to the Amazon S3 URI of the Lambda function • Create and publish an updated version of that function with the latest code • Create an alias with a name you provide and point to the updated version of the Lambda function Deployment preference type Canary10Percent30Minutes Canary10Percent5Minutes Canary10Percent10Minutes Canary10Percent15Minutes Linear10PercentEvery10Minutes Linear10PercentEvery1Minute Linear10PercentEvery2Minutes Linear10PercentEvery3Minutes AllAtOnce

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Choose the right API endpoint type Edge optimized: reduce latency from anywhere on the Internet AWS Region API Gateway Internet edge location edge location edge location CloudFront Distribution API Gateway Managed

Slide 16

Slide 16 text

Slide 17

Slide 17 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lambda@Edge use cases & blueprints Content customization • Based on user attributes, device properties Visitor session validation • User-agent validation—add an Access- Control-Allow-Header • Validate access token to confirm authentication URL customization • Re-write URLs, pretty URLs A/B testing and cookie-based sticky sessions • “Flip a coin” to select a version of content displayed to each user Security • Security header insertions (HSTS, X-Content- Type-Options, and more) • Bot handling

Slide 18

Slide 18 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Choose the right API endpoint type Regional AWS us-east-2 API Gateway Internet AWS us-west-2 API Gateway Route 53 Lambda DynamoDB Lambda DynamoDB Global Tables

Slide 19

Slide 19 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Regional AWS us-east-2 API Gateway Internet AWS us-west-2 API Gateway Amazon CloudFront Amazon CloudFront Lambda DynamoDB Lambda DynamoDB Global Tables Choose the right API endpoint type Route 53

Slide 20

Slide 20 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Regional API Gateway Internet API Gateway Route 53 Lambda DynamoDB Lambda DynamoDB Global Tables Lambda@Edge CloudFront Choose the right API endpoint type AWS us-east-2 AWS us-west-2

Slide 21

Slide 21 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Private: expose APIs only inside your VPC AWS Region API Gateway Your VPC AWS Direct Connect On-premises Choose the right API endpoint type

Slide 22

Slide 22 text

Slide 23

Slide 23 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. DynamoDB Lambda API Gateway Browser CloudFront S3 Cognito Serverless web app security Static Content • Geo-Restrictions • Signed Cookies • Signed URLs • DDOS Protection • Bucket Policies • ACLs AuthZ • Cross Account • Throttling per method • Resource Policies • Usage Plans • Encryption at Rest • VPC Endpoint • Function policies • Env Variables • Parameters/Secrets

Slide 24

Slide 24 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lambda Authorizer Client Lambda API Gateway DynamoDB IAM Lambda authorizers Two types: TOKEN―authorization token passed in a header REQUEST―all headers, query strings, paths, stage variables, or context variables

Slide 25

Slide 25 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. GraphQL: A query language for APIs Resources defined by a GraphQL schema Client sends query, server orchestrates data Multiple transports such as HTTP, MQTT, WebSockets Efficient for network bandwidth & dev time Self-documenting - introspection tooling

Slide 26

Slide 26 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS AppSync DynamoDB Lambda Elasticsearch GraphQL Schema Upload Schema Query Mutation Subscription Real-time Online/Offline AWS AppSync Cognito User Pool Legacy Application Amazon RDS HTTP Resolver

Slide 27

Slide 27 text

Slide 28

Slide 28 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Streaming with Amazon Kinesis Collect, process, and analyze video and data streams in real time Kinesis Data Firehose SQL Kinesis Data Analytics Kinesis Data Streams Kinesis Video Streams

Slide 29

Slide 29 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Streaming data ingestion Amazon S3: Buffered files Kinesis Agent Record producers Amazon Redshift: Table loads Amazon Elasticsearch Service: Domain loads Amazon S3: Source record backup Transformed records Put Records Kinesis Firehose: Delivery stream AWS Lambda: Transformations & enrichment Amazon DynamoDB: Lookup tables Raw Lookup Transformed

Slide 30

Slide 30 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Streaming data ingestion (HTTP) HTTP POST/PUT API Gateway Browser Amazon S3: Buffered files Amazon Redshift: Table loads Amazon Elasticsearch Service: Domain loads Amazon S3: Source record backup AWS Lambda: Transformations & enrichment Amazon DynamoDB: Lookup tables Raw Lookup Transformed Transformed records Kinesis Firehose: Delivery stream

Slide 31

Slide 31 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Streaming data ingestion (at the edge) Amazon S3: Buffered files Amazon Redshift: Table loads Amazon Elasticsearch Service: Domain loads Amazon S3: Source record backup AWS Lambda: Transformations & enrichment Amazon DynamoDB: Lookup tables Raw Lookup Transformed Transformed records Kinesis Firehose: Delivery stream HTTP POST/PUT CloudFront Lambda@Edge Browser

Slide 32

Slide 32 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kinesis Best practices Tune Firehose buffer size and buffer interval • Larger objects = fewer Lambda invocations & Amazon S3 PUTs Enable compression to reduce storage costs Enable Parquet format transformation (columnar) Enable Source Record Backup for transformations • Recover from transformation errors

Slide 33

Slide 33 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Kinesis Data Streams and Lambda # of shards corresponds to concurrent invocations of Lambda function Batch size sets maximum # of records per invocation (min 1, max 10K) Data Stream Processor Function Streaming source Other AWS services

Slide 34

Slide 34 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Fan-out pattern Trade strict message ordering for higher throughput & lower latency Kinesis Data Streams: Stream Lambda: Dispatcher function Lambda: Processor function Increase throughput, reduce processing latency Streaming source github.com/aws-samples/aws-lambda-fanout

Slide 35

Slide 35 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Real-time analytics Data Stream Kinesis Data Analytics: Time window aggregation Kinesis Data Firehose: Error stream S3: Error records Record producers Lambda: Alert function DynamoDB SNS: Notifications

Slide 36

Slide 36 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. CREATE OR REPLACE PUMP "STREAM_PUMP" AS INSERT INTO "DESTINATION_SQL_STREAM" SELECT STREAM "device_id", STEP("SOURCE_SQL_STREAM_001".ROWTIME BY INTERVAL '10' MINUTE) as "window_ts", SUM("measurement") as "sample_sum", COUNT(*) AS "sample_count" FROM "SOURCE_SQL_STREAM_001" GROUP BY "device_id", STEP("SOURCE_SQL_STREAM_001".ROWTIME BY INTERVAL '10' MINUTE); Kinesis Data Analytics Aggregation 10-minute tumbling window Kinesis Data Analytics: Time window aggregation Source stream Destination stream(s)

Slide 37

Slide 37 text

Slide 38

Slide 38 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data lake characteristics Collect, store, process, consume, and analyze organizational data Structured, semi-structured, and unstructured data Decoupled compute and storage Fast automated ingestion Schema on-read Complementary to data warehouses

Slide 39

Slide 39 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Serverless data lake S3 Elasticsearch Glue DynamoDB Catalog & search Cognito API Gateway API/UI Athena QuickSight Redshift Spectrum Analytics & processing Lambda Kinesis Streams Kinesis Firehose Direct Connect Ingest AWS IoT KMS CloudTrail IAM Macie Security & auditing

Slide 40

Slide 40 text

Slide 41

Slide 41 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Analytics options S3 Select on CSV, JSON and Apache Parquet objects Amazon QuickSight (data exploration) Amazon Athena (SQL queries) AWS Lambda (BYOL) Amazon SageMaker (predictions) Amazon EMR (Hadoop) AWS Glue (ETL)

Slide 42

Slide 42 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Athena―Serverless interactive query service Query duration: 44.66 seconds Data scanned: 169.53GB Cost*: $0.85 * $5/TB or $0.005/GB SELECT gram, year, sum(count) FROM ngram WHERE gram = 'just say no' GROUP BY gram, year ORDER BY year ASC;

Slide 43

Slide 43 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Athena best practices Partition data s3://my-bucket/my-data/parquet/year=2018/month=11/day=25/ Use columnar formats – Apache Parquet, AVRO, ORC Compress files with splittable compression (bzip2) Optimize file sizes aws.amazon.com/blogs/big-data/top-10-performance-tuning-tips-for-amazon-athena

Slide 44

Slide 44 text

Slide 45

Slide 45 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Serverless batch processing (Map/Reduce) Lambda: Splitter S3 Object DynamoDB: Mapper Results Lambda: Mappers …. …. Lambda: Reducer S3 Results

Slide 46

Slide 46 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Pywren Python library developed by University of California, Berkeley 10 TFLOPS of peak compute power (default of 1000 concurrent functions) Over 80 GB/sec of read and 60 GB/sec of write performance using S3 http://pywren.io

Slide 47

Slide 47 text

Slide 48

Slide 48 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. M L F R A M E W O R K S & I N F R A S T R U C T U R E The Amazon ML Stack: Broadest & Deepest Set of Capabilities A I S E R V I C E S R E K O G N I T I O N I M A G E P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D C O M P R E H E N D M E D I C A L L E X R E K O G N I T I O N V I D E O Vision Speech Chatbots A M A Z O N S A G E M A K E R B U I L D T R A I N F O R E C A S T T E X T R A C T P E R S O N A L I Z E D E P L O Y Pre-built algorithms & notebooks Data labeling (G R O U N D T R U T H ) One-click model training & tuning Optimization ( N E O ) One-click deployment & hosting M L S E R V I C E S F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e E C 2 P 3 & P 3 d n E C 2 C 5 F P G A s G R E E N G R A S S E L A S T I C I N F E R E N C E Models without training data (REINFORCEMENT LEARNING) Algorithms & models ( A W S M A R K E T P L A C E ) Language Forecasting Recommendations NEW NEW NEW NEW NEW NEW NEW NEW NEW

Slide 49

Slide 49 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 1. Upload 2. Submit image Image processing with Amazon Rekognition Image Step Functions 3. Store image Lambda DynamoDB Elasticsearch 8. Store metadata & analysis 4. DetectFaces 7. DetectText 5. DetectLabels 6. DetectModeration

Slide 50

Slide 50 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Media analysis solution S3: Web interface Cognito Amazon Rekognition Video: Detect objects, scenes, faces, & celebrities Elasticsearch: Search index API Gateway: REST APIs https://aws.amazon.com/answers/media-entertainment/media-analysis-solution/ AWS Elemental MediaConvert: Transcode videos S3: Media storage Step Functions: Orchestrate analysis Transcribe Comprehend

Slide 51

Slide 51 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Connect (Serverless contact center) Real time and historical analytics High-quality voice capability Call recording Skills-based routing [Automatic Call Distribution (ACD)]

Slide 52

Slide 52 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Intelligent call center chatbot Amazon Connect Customer Amazon Lex Lambda: Chatbot Processing DynamoDB: Customer Data SNS: SMS Messaging Customer calls Connect to reschedule an appointment Connect calls Lex chatbot Lex chatbot calls Lambda function to get customer preferences and fulfil Intents Lambda function sends text message confirmation via SNS Customer receives appointment confirmation text message Lambda function writes updates to DynamoDB

Slide 53

Slide 53 text

© 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Call center analytics Amazon Connect Customers Agents Call recordings S3: Call recordings S3: Call transcripts Step Functions Transcribe Lambda S3: Sentiment, key phrases, entities Step Functions S3 Notifications for call transcripts Comprehend Lambda Athena QuickSight Contact trace records (CTRs) Kinesis Data Streams Kinesis Data Firehose S3: CTRs