Serverless in Production with CI/CD and End-to-End Tracing

London JavaScript Community, London, July 11th, 2017

In this session I will share some of the best practices to use serverless applications in production on AWS, using AWS CodeStar to set up your entire continuous integration and delivery toolchain in minutes, allowing you to start releasing code faster and supporting agile development. As your distributed application scales in size and complexity, we’ll see how to use AWS X-Ray, either in development or in production, to introduce end-to-end tracing capabilities and help identify performance bottlenecks, edge case errors, and other hard to detect issues .

Transcript

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
   Danilo Poccia, Technical Evangelist
   @danilop
   Serverless in Production
   With CI/CD and End-to-End Tracing
   

   View Slide

2. What is Serverless?
   

   View Slide

3. Operations and management Scaling
   Provisioning and utilization Availability and fault tolerance
   Owning servers means dealing with ...
   

   View Slide

4. AWS Lambda: Run code in response to events
   FUNCTION SERVICES (ANYTHING)
   Changes in
   data state
   Requests to
   endpoints
   Changes in
   resource state
   Node
   Python
   Java
   C#
   EVENT SOURCE
   

   View Slide

5. No servers to provision
   or manage
   Scales with usage
   Never pay for idle Availability and fault
   tolerance built in
   Benefits of Lambda and serverless compute
   

   View Slide

6. Example Serverless Application Architecture
   

   View Slide

7. Web Applications and Backends
   

   View Slide

8. Data Processing
   

   View Slide

9. Data Processing
   

   View Slide

10. Serverless IoT with AWS Lambda at iRobot
    

    View Slide

11. AWS
    Lambda
    Amazon
    CloudFront
    Amazon
    S3
    Amazon
    DynamoDB
    Amazon
    Redshift
    Amazon
    Route 53 Amazon VPC
    Amazon
    CloudWatch
    AWS
    CloudFormation
    AWS
    CloudTrail
    AWS
    Config AWS IAM
    AWS KMS
    AWS WAF
    Amazon Cognito Amazon
    SNS
    Amazon API
    Gateway
    Amazon
    SQS
    Amazon
    Elasticsearch Service
    Amazon
    Kinesis
    Amazon
    QuickSight
    AWS IoT
    How iRobot leverages AWS
    

    View Slide

12. API Gateway
    App requests
    over HTTPS
    CRUD
    operations
    

    View Slide

13. IoT
    MQTT
    CRUD
    operations
    

    View Slide

14. IoT
    MQTT
    MQTT
    Presigned
    URL
    Staging
    bucket
    

    View Slide

15. Infrastructure as Code
    

    View Slide

16. AWS CloudFormation brings:
    • Infrastructure as code
    • Easy to provision and manage a collection of related AWS resources
    • Input .yaml file and output provisioned AWS resources
    • Optimized for infrastructure
    AWS SAM:
    • CloudFormation extension optimized for serverless
    • New serverless resources: functions, APIs, and tables
    • Supports anything CloudFormation supports
    • Open specification (Apache 2.0)
    AWS Serverless Application Model (AWS SAM)
    

    View Slide

17. AWSTemplateFormatVersion: '2010-09-09'
    Resources:
    GetHtmlFunctionGetHtmlPermissionProd:
    Type: AWS::Lambda::Permission
    Properties:
    Action: lambda:invokeFunction
    Principal: apigateway.amazonaws.com
    FunctionName:
    Ref: GetHtmlFunction
    SourceArn:
    Fn::Sub: arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${ServerlessRestApi}/Prod/ANY/*
    ServerlessRestApiProdStage:
    Type: AWS::ApiGateway::Stage
    Properties:
    DeploymentId:
    Ref: ServerlessRestApiDeployment
    RestApiId:
    Ref: ServerlessRestApi
    StageName: Prod
    ListTable:
    Type: AWS::DynamoDB::Table
    Properties:
    ProvisionedThroughput:
    WriteCapacityUnits: 5
    ReadCapacityUnits: 5
    AttributeDefinitions:
    - AttributeName: id
    AttributeType: S
    KeySchema:
    - KeyType: HASH
    AttributeName: id
    GetHtmlFunction:
    Type: AWS::Lambda::Function
    Properties:
    Handler: index.gethtml
    Code:
    S3Bucket: flourish-demo-bucket
    S3Key: todo_list.zip
    Role:
    Fn::GetAtt:
    - GetHtmlFunctionRole
    - Arn
    Runtime: nodejs4.3
    GetHtmlFunctionRole:
    Type: AWS::IAM::Role
    Properties:
    ManagedPolicyArns:
    - arn:aws:iam::aws:policy/AmazonDynamoDBReadOnlyAccess
    - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
    AssumeRolePolicyDocument:
    Version: '2012-10-17'
    Statement:
    - Action:
    - sts:AssumeRole
    Effect: Allow
    Principal:
    Service:
    - lambda.amazonaws.com
    ServerlessRestApiDeployment:
    Type: AWS::ApiGateway::Deployment
    Properties:
    RestApiId:
    Ref: ServerlessRestApi
    Description: 'RestApi deployment id: 127e3fb91142ab1ddc5f5446adb094442581a90d'
    StageName: Stage
    GetHtmlFunctionGetHtmlPermissionTest:
    Type: AWS::Lambda::Permission
    Properties:
    Action: lambda:invokeFunction
    Principal: apigateway.amazonaws.com
    FunctionName:
    Ref: GetHtmlFunction
    SourceArn:
    Fn::Sub: arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${ServerlessRestApi}/*/ANY/*
    ServerlessRestApi:
    Type: AWS::ApiGateway::RestApi
    Properties:
    Body:
    info:
    version: '1.0'
    title:
    Ref: AWS::StackName
    paths:
    "/{proxy+}":
    x-amazon-apigateway-any-method:
    x-amazon-apigateway-integration:
    httpMethod: ANY
    type: aws_proxy
    uri:
    Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-
    31/functions/${GetHtmlFunction.Arn}/invocations
    responses: {}
    swagger: '2.0'
    CF template example – API triggering Lambda
    AWSTemplateFormatVersion: '2010-09-09’
    Transform: AWS::Serverless-2016-10-31
    Resources:
    GetHtmlFunction:
    Type: AWS::Serverless::Function
    Properties:
    CodeUri: s3://flourish-demo-bucket/todo_list.zip
    Handler: index.gethtml
    Runtime: nodejs4.3
    Policies: AmazonDynamoDBReadOnlyAccess
    Events:
    GetHtml:
    Type: Api
    Properties:
    Path: /{proxy+}
    Method: ANY
    ListTable:
    Type: AWS::Serverless::SimpleTable
    AWS SAM example – API triggering Lambda
    AWS SAM: Less complexity, more power
    

    View Slide

18. What is Continuous Integration?
    

    View Slide

19. What is Continuous Integration?
    Automation Component + Cultural Component
    

    View Slide

20. Continuous Integration Benefits
    Improve Developer
    Productivity
    Find and Address
    Bugs Quicker
    Deliver Updates
    Faster
    

    View Slide

21. Set Up a Serverless CI/CD Pipeline on AWS
    AWS
    CodeCommit
    AWS
    CodeBuild
    AWS
    CodePipeline
    AWS
    CloudFormation
    Amazon
    S3
    GitHub
    AWS
    SAM
    Commit Build Deploy
    + Tests (CodeBuild or other services)
    + Manuals Approval (CodePipeline)
    + Multiple Environments (CodePipeline + CloudFormation)
    

    View Slide

22. How to make it easier?
    

    View Slide

23. AWS CodeStar New!
    

    View Slide

24. New!
    AWS X-Ray
    

    View Slide

25. Time for a demo J
    

    View Slide

26. Build a Serverless Web Application
    https://aws.amazon.com/getting-started/serverless-web-app/
    

    View Slide

27. JavaScript + AWS Mobile Hub
    Ionic + AWS MobileHub Starter Project
    https://github.com/ionic-team/ionic2-starter-aws
    Enhanced JavaScript development with AWS Mobile Hub
    https://aws.amazon.com/blogs/mobile/enhanced-javascript-development-with-aws-mobile-hub/
    Deploy a React App to S3 and CloudFront with AWS Mobile Hub
    https://aws.amazon.com/blogs/mobile/deploy-a-react-app-to-s3-and-cloudfront-with-aws-mobile-hub/
    Integrate the AWS SDK for JavaScript into a React App
    https://aws.amazon.com/blogs/mobile/integrate-the-aws-sdk-for-javascript-into-a-react-app/
    

    View Slide

28. Conclusion
    Lambda is a fundamental
    component of modern
    application architectures
    It has a place in everything
    from data processing to
    simple web apps
    

    View Slide

29. Thank you!
    @danilop
    

    View Slide