Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless Development Deep Dive

Serverless Development Deep Dive

Builders' Day, Edinburgh, February 21st, 2018

Danilo Poccia

February 21, 2018
Tweet

More Decks by Danilo Poccia

Other Decks in Programming

Transcript

  1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Serverless Development
    Deep Dive
    Danilo Poccia
    Technical Evangelist
    [email protected]
    @danilop
    danilop

    View Slide

  2. SERVICES (ANYTHING)
    Changes in
    data state
    Requests to
    endpoints
    Changes in
    resource state
    EVENT SOURCE FUNCTION
    Node.js
    Python
    Java
    C#
    Go
    Serverless applications
    © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    New

    View Slide

  3. Common serverless use cases
    Web
    applications
    • Static
    websites
    • Complex web
    apps
    • Packages for
    Flask and
    Express
    Data
    processing
    • Real-time
    • MapReduce
    • Batch
    Chatbots
    • Powering
    chatbot logic
    Backends
    • Apps and
    services
    • Mobile
    • IoT
    >
    >
    Amazon
    Alexa
    • Powering
    voice-enabled
    apps
    • Alexa Skills
    Kit
    IT
    automation
    • Policy engines
    • Extending
    AWS services
    • Infrastructure
    management
    © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

    View Slide

  4. Fine-grained pricing
    Buy compute time in 100-ms increments
    Low request charge
    No hourly, daily, or monthly minimums
    No per-device fees
    Never pay for idle
    Free Tier
    1 M requests and 400,000 GB-s of compute
    Every month, every customer
    © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

    View Slide

  5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    SMART RESOURCE ALLOCATION
    Match resource allocation (up to 3 GB!) to logic
    Stats for Lambda function that calculates 1000 times all prime
    numbers <= 1000000
    128 MB 11.722965 sec $0.024628
    256 MB 6.678945 sec $0.028035
    512 MB 3.194954 sec $0.026830
    1024 MB 1.465984 sec $0.024638

    View Slide

  6. Amazon S3 Amazon
    DynamoDB
    Amazon
    Kinesis
    AWS
    CloudFormation
    AWS CloudTrail Amazon
    CloudWatch
    Amazon
    Cognito
    Amazon SNS
    Amazon
    SES
    Cron events
    DATA STORES ENDPOINTS
    DEVELOPMENT AND MANAGEMENT TOOLS EVENT/MESSAGE SERVICES
    Event sources that trigger AWS Lambda
    …and more!
    AWS
    CodeCommit
    Amazon
    API Gateway
    Amazon
    Alexa
    AWS IoT AWS Step
    Functions
    © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

    View Slide

  7. Lambda execution model
    Synchronous (push) Asynchronous (event) Stream-based
    Amazon
    API Gateway
    AWS Lambda
    function
    Amazon
    DynamoDB
    Amazon
    SNS
    /order
    AWS Lambda
    function
    Amazon
    S3
    reqs
    Amazon
    Kinesis
    changes
    AWS Lambda
    service
    function
    © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

    View Slide

  8. Lambda permissions model
    Fine-grained security controls for both
    execution and invocation
    Execution policies:
    • Define what AWS resources/API calls this
    function can access via IAM
    • Used in streaming invocations
    • For example, "Lambda function A can read
    from DynamoDB table users"
    Function policies:
    • Used for sync and async invocations
    • For example, "Actions on bucket X can invoke
    Lambda function Z"
    • Resource policies allow for cross-account
    access
    © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

    View Slide

  9. Amazon API Gateway
    Internet
    Mobile Apps
    Websites
    Services
    AWS Lambda
    functions
    AWS
    All private (VPC) or
    publicly accessible
    endpoints
    Amazon
    CloudWatch
    Monitoring
    Amazon
    CloudFront
    Any other
    AWS service
    Endpoints on
    Amazon EC2
    AWS Step
    Functions
    © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

    View Slide

  10. Create a unified
    API front end for
    multiple
    microservices
    Authenticate and
    authorize
    requests to a
    backend
    DDoS protection
    and throttling for
    your backend
    Throttle, meter,
    and monetize API
    usage by third-
    party developers
    Amazon API Gateway
    © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

    View Slide

  11. Amazon API Gateway – Lambda Proxy Integration
    {
    "resource": "Resource path",
    "path": "Path parameter",
    "httpMethod": "Incoming request's method name",
    "headers": {Incoming request headers},
    "queryStringParameters": {Query string parameters},
    "pathParameters":{Path parameters},
    "stageVariables": {Applicable stage variables},
    "requestContext": {Request context, including authorizer-returned key-value pairs},
    "body": "...",
    "isBase64Encoded": true|false
    }
    {
    "statusCode": httpStatusCode,
    "headers": { "headerName": "headerValue", ... },
    "body": "...”,
    "isBase64Encoded": true|false
    }
    Input Format of a Lambda Function for Proxy Integration
    Output Format of a Lambda Function for Proxy Integration

    View Slide

  12. New and improved Lambda console
    Cloud9 editor within the Lambda console
    Function graph
    Persisted test events
    Monitoring view (jump to logs for any timeframe)
    New

    View Slide

  13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Demo #1:
    AWS Lambda Console

    View Slide

  14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Tips - Lambda console editor
    - Useful keyboard shortcuts
    - Full screen Cmd/Ctrl + Shift + F
    - Cache file locally Cmd/Ctrl + S
    - Save (UpdateFunctionCode) Cmd/Ctrl + Shift + U
    - Test Cmd/Ctrl + I
    - Configure test events Cmd/Ctrl + J

    View Slide

  15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    CloudFormation
    Provision and manage a collection of related AWS resources.
    Your application = CloudFormation stack
    Input .yaml file and output provisioned AWS resources

    View Slide

  16. Meet
    SAM!

    View Slide

  17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Serverless Application Model (SAM)
    CloudFormation extension optimized for serverless
    New serverless resource types: functions, APIs, and tables
    Supports anything CloudFormation supports
    Open specification (Apache 2.0)

    View Slide

  18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    AWSTemplateFormatVersion: '2010-09-09’
    Transform: AWS::Serverless-2016-10-31
    Resources:
    GetHtmlFunction:
    Type: AWS::Serverless::Function
    Properties:
    CodeUri: s3://demo-bucket/todo_list.zip
    Handler: index.js
    Runtime: nodejs6.1
    Policies: AmazonDynamoDBReadOnlyAccess
    Events:
    GetHtml:
    Type: Api
    Properties:
    Path: /{proxy+}
    Method: ANY
    SAM template

    View Slide

  19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    AWSTemplateFormatVersion: '2010-09-09’
    Transform: AWS::Serverless-2016-10-31
    Resources:
    GetHtmlFunction:
    Type: AWS::Serverless::Function
    Properties:
    CodeUri: s3://demo-bucket/todo_list.zip
    Handler: index.js
    Runtime: nodejs6.1
    Policies: AmazonDynamoDBReadOnlyAccess
    Events:
    GetHtml:
    Type: Api
    Properties:
    Path: /{proxy+}
    Method: ANY
    SAM template
    AWS::Lambda::Function
    AWS::IAM::Role
    AWS::IAM::Policy
    AWS::ApiGateway::RestApi
    AWS::ApiGateway::Stage
    AWS::ApiGateway::Deployment
    AWS::Lambda::Permission

    View Slide

  20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    CloudFormation template
    AWSTemplateFormatVersion: '2010-09-09'
    Resources:
    GetHtmlFunctionGetHtmlPermissionProd:
    Type: AWS::Lambda::Permission
    Properties:
    Action: lambda:invokeFunction
    Principal: apigateway.amazonaws.com
    FunctionName:
    Ref: GetHtmlFunction
    SourceArn:
    Fn::Sub: arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${ServerlessRestApi}/Prod/ANY/*
    ServerlessRestApiProdStage:
    Type: AWS::ApiGateway::Stage
    Properties:
    DeploymentId:
    Ref: ServerlessRestApiDeployment
    RestApiId:
    Ref: ServerlessRestApi
    StageName: Prod
    ListTable:
    Type: AWS::DynamoDB::Table
    Properties:
    ProvisionedThroughput:
    WriteCapacityUnits: 5
    ReadCapacityUnits: 5
    AttributeDefinitions:
    - AttributeName: id
    AttributeType: S
    KeySchema:
    - KeyType: HASH
    AttributeName: id
    GetHtmlFunction:
    Type: AWS::Lambda::Function
    Properties:
    Handler: index.gethtml
    Code:
    S3Bucket: flourish-demo-bucket
    S3Key: todo_list.zip
    Role:
    Fn::GetAtt:
    - GetHtmlFunctionRole
    - Arn
    Runtime: nodejs4.3
    GetHtmlFunctionRole:
    Type: AWS::IAM::Role
    ManagedPolicyArns:
    - arn:aws:iam::aws:policy/AmazonDynamoDBReadOnlyAccess
    - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
    AssumeRolePolicyDocument:
    Version: '2012-10-17'
    Statement:
    - Action:
    - sts:AssumeRole
    Effect: Allow
    Principal:
    Service:
    - lambda.amazonaws.com
    ServerlessRestApiDeployment:
    Type: AWS::ApiGateway::Deployment
    Properties:
    RestApiId:
    Ref: ServerlessRestApi
    Description: 'RestApi deployment id: 127e3fb91142ab1ddc5f5446adb094442581a90d'
    StageName: Stage
    GetHtmlFunctionGetHtmlPermissionTest:
    Type: AWS::Lambda::Permission
    Properties:
    Action: lambda:invokeFunction
    Principal: apigateway.amazonaws.com
    FunctionName:
    Ref: GetHtmlFunction
    SourceArn:
    Fn::Sub: arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${ServerlessRestApi}/*/ANY/*
    ServerlessRestApi:
    Type: AWS::ApiGateway::RestApi
    Properties:
    Body:
    info:
    version: '1.0'
    title:
    Ref: AWS::StackName
    paths:
    "/{proxy+}":
    x-amazon-apigateway-any-method:
    x-amazon-apigateway-integration:
    httpMethod: ANY
    type: aws_proxy
    uri:
    Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-
    31/functions/${GetHtmlFunction.Arn}/invocations

    View Slide

  21. CloudFormation Package/Deploy
    aws cloudformation package \
    --s3-bucket danilop \
    --s3-prefix packages \
    --template-file template.yaml \
    --output-template-file output-template.json
    aws cloudformation deploy \
    --template-file ./output-template.json \
    --stack-name buildersday \
    --capabilities CAPABILITY_IAM

    View Slide

  22. Serverless by Design

    View Slide

  23. Serverless by Design
    https://sbd.danilop.net/

    View Slide

  24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Demo #2:
    AWS SAM

    View Slide

  25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

    View Slide

  26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    AWS Cloud9
    Cloud-based dev environment
    Write, test and debug with just a browser
    Optimized for serverless
    Used by the Lambda console
    New

    View Slide

  27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Testing serverless apps - challenges
    - Test in an environment that resembles Lambda:
    - OS
    - Libraries
    - Runtime
    - Configured limits (memory, timeout)
    - Mimic response and log outputs

    View Slide

  28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Testing serverless apps - challenges
    - Test events need to be:
    - Syntactically accurate
    - Different for each trigger

    View Slide

  29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Testing serverless apps - challenges
    {
    "Records": [
    {
    "eventVersion": "2.0",
    "eventTime": "1970-01-01T00:00:00.000Z",
    "requestParameters": {
    "sourceIPAddress": "127.0.0.1"
    },
    "s3": {
    "configurationId": "testConfigRule",
    "object": {
    "eTag":
    "0123456789abcdef0123456789abcdef",
    "sequencer": "0A1B2C3D4E5F678901",
    "key": "myKey",
    "size": 1024
    },
    "bucket": {
    "arn": "arn:aws:s3:::myBucket",
    "name": "myBucket",
    "ownerIdentity": {
    "principalId": "EXAMPLE"
    }
    },
    "s3SchemaVersion": "1.0"
    },
    "responseElements": {
    "x-amz-id-2":
    "EXAMPLE123/5678abcdefghijklambdaisawesome/mnop
    qrstuvwxyzABCDEFGH",
    "x-amz-request-id": "EXAMPLE123456789"
    },
    "awsRegion": "us-east-1",
    "eventName": "ObjectCreated:Put",
    "userIdentity": {
    "principalId": "EXAMPLE"
    },
    "eventSource": "aws:s3” } ] }

    View Slide

  30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Introducing SAM Local
    CLI tool for local testing of serverless apps
    Leverages Docker images to mimic Lambda’s
    execution environment
    Emulates Lambda functions and APIs
    Event generator to help you generate event
    payload for common Lambda triggers
    sam local generate-event s3 --bucket --key

    View Slide

  31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Introducing SAM Local
    Response object and function logs available
    on your local machine
    Supports live debugging
    Currently supports Java, Node.js and Python
    SAM Local is open source & accepting pull
    requests!
    https://github.com/awslabs/aws-sam-local
    npm install –g aws-sam-local

    View Slide

  32. $ sam --help
    NAME:
    sam -
    ___ _____ ___ _ __ __
    /_\ \ / / __| / __| /_\ | \/ |
    / _ \ \/\/ /\__ \ \__ \/ _ \| |\/| |
    /_/ \_\_/\_/ |___/ |___/_/ \_\_| |_|
    AWS Serverless Application Model (SAM) CLI
    The AWS Serverless Application Model extends AWS CloudFormation to provide a simplified way of defining the Amazon API Gateway APIs, AWS Lambda functions,
    and Amazon DynamoDB tables needed by your serverless application. You can find more in-depth guide about the SAM specification
    here:\nhttps://github.com/awslabs/serverless-application-model.
    USAGE:
    sam [global options] command [command options] [arguments...]
    VERSION:
    0.2.0
    COMMANDS:
    local Run your Serverless application locally for quick development & testing
    validate Validates an AWS SAM template. If valid, will print a summary of the resources found within the SAM template. If the template is invalid, returns
    a non-zero exit code.
    package Package an AWS SAM application. This is an alias for 'aws cloudformation package'.
    deploy Deploy an AWS SAM application. This is an alias for 'aws cloudformation deploy'.
    help, h Shows a list of commands or help for one command
    GLOBAL OPTIONS:
    --help, -h show help
    --version, -v print the version

    View Slide

  33. $ sam local --help
    ..
    USAGE:
    sam local command [command options] [arguments...]
    COMMANDS:
    start-api Allows you to run your Serverless application locally for quick development & testing. When run in a
    directory that contains your Serverless functions and your AWS SAM template, it will create a local HTTP server
    hosting all of your functions. When accessed (via browser, cli etc), it will launch a Docker container locally to
    invoke the function. It will read the CodeUri property of AWS::Serverless::Function resource to find the path in your
    file system containing the Lambda Function code. This could be the project's root directory for interpreted languages
    like Node & Python, or a build directory that stores your compiled artifacts or a JAR file. If you are using a
    interpreted language, local changes will be available immediately in Docker container on every invoke. For more
    compiled languages or projects requiring complex packing support, we recommended you run your own building solution
    and point SAM to the directory or file containing build artifacts.
    invoke Invokes a local Lambda function once and quits after invocation completes.
    Useful for developing serverless functions that handle asynchronous events (such as S3/Kinesis etc), or if you want
    to compose a script of test cases. Event body can be passed in either by stdin (default), or by using the --event
    parameter. Runtime output (logs etc) will be outputted to stderr, and the Lambda function result will be outputted to
    stdout.
    generate-event Generates Lambda events (e.g. for S3/Kinesis etc) that can be piped to 'sam local invoke'
    Available out of the box
    in AWS Cloud9!

    View Slide

  34. >
    GitHub
    Amazon S3
    AWS CodeCommit
    AWS CodeBuild AWS CodeBuild
    Third-party tools
    AWS CloudFormation
    Source Build Test Deploy
    Deploying serverless applications
    AWS CodePipeline

    View Slide

  35. Use AWS CodeStar to set up a project with CI/CD

    View Slide

  36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Safe deployments baked into SAM!
    Lambda aliases now enable traffic shifting
    CodeDeploy integration for deployment automation
    Deployment automation natively supported in SAM
    New

    View Slide

  37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Safe deployments baked into SAM!
    AWSTemplateFormatVersion: '2010-09-09’
    Transform: AWS::Serverless-2016-10-31
    Resources:
    GetHtmlFunction:
    Type: AWS::Serverless::Function
    Properties:
    CodeUri: s3://demo-bucket/todo_list.zip
    Handler: index.js
    Runtime: nodejs6.1
    New

    View Slide

  38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Safe deployments baked into SAM!
    AWSTemplateFormatVersion: '2010-09-09’
    Transform: AWS::Serverless-2016-10-31
    Globals:
    Function:
    AutoPublishAlias: Live
    DeploymentPreference:
    Type: Canary10Percent10Minutes
    Resources:
    GetHtmlFunction:
    Type: AWS::Serverless::Function
    Properties:
    CodeUri: s3://demo-bucket/todo_list.zip
    Handler: index.js
    Runtime: nodejs6.1
    Policies: AmazonDynamoDBReadOnlyAccess
    New

    View Slide

  39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Safe deployments baked into SAM!
    AWSTemplateFormatVersion: '2010-09-09’
    Transform: AWS::Serverless-2016-10-31
    Globals:
    Function:
    AutoPublishAlias: Live
    DeploymentPreference:
    Type: Canary10Percent10Minutes
    Hooks:
    PreTraffic: !Ref CodeDeployHook_PreTest
    PostTraffic: !Ref CodeDeployHook_PostTest
    Alarms:
    - !Ref DurationAlarm
    - !Ref ErrorAlarm
    Resources:
    GetHtmlFunction:
    Type: AWS::Serverless::Function
    Properties:
    CodeUri: s3://demo-bucket/todo_list.zip
    Handler: index.js
    Runtime: nodejs6.1
    Policies: AmazonDynamoDBReadOnlyAccess
    New

    View Slide

  40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Code Deploy console

    View Slide

  41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Demo #3:
    AWS Cloud9 & SAM Local

    View Slide

  42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Takeaways
    1. Use the Lambda console for quick creation and iteration of simple apps
    2. Use AWS SAM to describe your serverless architecture
    3. Plug SAM Local into the IDE of your choice for testing and debugging
    4. "Develop in the cloud" with AWS Cloud9 – optimized for serverless
    applications
    5. Build on SAM for CI/CD capabilities, including canary deployments

    View Slide

  43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
    Serverless Development
    Deep Dive
    Danilo Poccia
    Technical Evangelist
    [email protected]
    @danilop
    danilop

    View Slide