Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CI/CD on AWS

CI/CD on AWS

AWS Summit Benelux, Amsterdam, April 17th, 2019

Danilo Poccia

April 17, 2019
Tweet

More Decks by Danilo Poccia

Other Decks in Programming

Transcript

  1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CI/CD on AWS
    Danilo Poccia
    Principal Evangelist, Serverless
    AWS
    @danilop
    Flynn Bundy
    Consultant, DevOps
    AWS
    @bundyfx

    View full-size slide

  2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Listen
    Iterate
    Experiment
    Innovation
    Flywheel
    Experiments power the engine of rapid innovation

    View full-size slide

  3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Release process stages
    Source Build Test Production

    View full-size slide

  4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Release process stages
    Source Build Test Production

    View full-size slide

  5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications

    View full-size slide

  6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications
    Infrastructure
    as code

    View full-size slide

  7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Infrastructure as code goals
    1. Make infrastructure changes repeatable and predictable
    2. Release infrastructure changes using the same tools as code changes
    3. Replicate production environment in a staging environment to enable
    continuous testing

    View full-size slide

  8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Release infrastructure-as-code with AWS CloudFormation
    “Master”
    branch
    Prepare
    template
    Create & execute
    change set
    Create & execute
    change set

    View full-size slide

  9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Model function environments with AWS
    Serverless Application Model (SAM)
    • Open source framework for building serverless
    applications on AWS
    • Shorthand syntax to express functions, APIs,
    databases, and event source mappings
    • Transforms and expands SAM syntax into AWS
    CloudFormation syntax on deployment
    • Supports all AWS CloudFormation resource types
    https://aws.amazon.com/serverless/sam/

    View full-size slide

  10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Model container environments with AWS
    Cloud Development Kit (CDK)
    Developer
    Preview
    • Open source framework to define cloud
    infrastructure in TypeScript, Java, C#, …
    • Provides library of higher-level resource types
    (“construct” classes) that have AWS best practices
    built in by default, packaged as npm modules
    • Provisions resources with CloudFormation
    • Supports all CloudFormation resource types
    AWS
    CDK
    https://awslabs.github.io/aws-cdk

    View full-size slide

  11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS Cloud Development Kit (CDK)
    npm install -g aws-cdk
    cdk init app --language typescript
    cdk synth
    cdk deploy
    cdk diff
    cdk destroy
    CodePipeline
    Use CloudFormation
    deployment actions with
    any synthesized CDK
    application
    Jenkins
    Use CDK CLI
    D
    eveloper
    Preview
    TypeScript
    C#
    F#
    Java
    Python

    View full-size slide

  12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CDK template
    import ec2 = require('@aws-cdk/aws-ec2');
    import ecs = require('@aws-cdk/aws-ecs');
    import cdk = require('@aws-cdk/cdk');
    class BonjourFargate extends cdk.Stack {
    constructor(parent: cdk.App, name: string, props?: cdk.StackProps) {
    super(parent, name, props);
    const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 });
    const cluster = new ecs.Cluster(this, 'Cluster', { vpc });
    new ecs.LoadBalancedFargateService(
    this, "FargateService", {
    cluster,
    image: ecs.DockerHub.image("amazon/amazon-ecs-sample"),
    });
    }
    }
    const app = new cdk.App();
    new BonjourFargate(app, 'Bonjour');
    app.run();

    View full-size slide

  13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    import ec2 = require('@aws-cdk/aws-ec2');
    import ecs = require('@aws-cdk/aws-ecs');
    import cdk = require('@aws-cdk/cdk');
    class BonjourFargate extends cdk.Stack {
    constructor(parent: cdk.App, name: string, props?: cdk.StackProps) {
    super(parent, name, props);
    const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 });
    const cluster = new ecs.Cluster(this, 'Cluster', { vpc });
    new ecs.LoadBalancedFargateService(
    this, "FargateService", {
    cluster,
    image: ecs.DockerHub.image("amazon/amazon-ecs-sample"),
    });
    }
    }
    const app = new cdk.App();
    new BonjourFargate(app, 'Bonjour');
    app.run();
    CDK template

    View full-size slide

  14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CDK template
    import ec2 = require('@aws-cdk/aws-ec2');
    import ecs = require('@aws-cdk/aws-ecs');
    import cdk = require('@aws-cdk/cdk');
    class BonjourFargate extends cdk.Stack {
    constructor(parent: cdk.App, name: string, props?: cdk.StackProps) {
    super(parent, name, props);
    const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 });
    const cluster = new ecs.Cluster(this, 'Cluster', { vpc });
    new ecs.LoadBalancedFargateService(
    this, "FargateService", {
    cluster,
    image: ecs.DockerHub.image("amazon/amazon-ecs-sample"),
    });
    }
    }
    const app = new cdk.App();
    new BonjourFargate(app, 'Bonjour');
    app.run();

    View full-size slide

  15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Model pipelines with AWS CDK
    • Minimize copy-and-paste by using object-oriented language
    • Define microservice pipeline “shape” in one class, then re-use it across
    many pipelines
    • CDK includes many high-level constructs for modeling a CodePipeline
    pipeline, including automatically configuring IAM role policies

    View full-size slide

  16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CDK pipelines: Construct
    export class MyMicroservicePipeline extends cdk.Construct {
    constructor(parent: cdk.Construct, name: string, props: MyMicroservicePipelineProps) {
    super(parent, name);
    const pipeline = new codepipeline.Pipeline(this, 'Pipeline', {
    pipelineName: props.serviceName,
    });
    const githubAccessToken = new cdk.SecretParameter(this, 'GitHubToken',
    { ssmParameter: 'GitHubToken' });
    new codepipeline.GitHubSourceAction(this, 'GitHubSource', {
    stage: pipeline.addStage('Source'),
    owner: 'myorg',
    repo: props.serviceName,
    oauthToken: githubAccessToken.value
    });

    View full-size slide

  17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CDK pipelines: Stack
    import cdk = require('@aws-cdk/cdk');
    import { MyMicroservicePipeline } from './pipeline';
    class MyMicroservicePipelinesStack extends cdk.Stack {
    constructor(parent: cdk.App, name: string, props?: cdk.StackProps) {
    super(parent, name, props);
    new MyMicroservicePipeline(this, 'Pipeline1', { 'serviceName': 'Microservice1' });
    new MyMicroservicePipeline(this, 'Pipeline2', { 'serviceName': 'Microservice2' });
    new MyMicroservicePipeline(this, 'Pipeline3', { 'serviceName': 'Microservice3' });
    new MyMicroservicePipeline(this, 'Pipeline4', { 'serviceName': 'Microservice4' });
    }
    }
    const app = new cdk.App();
    new MyMicroservicePipelinesStack(app, 'MyMicroservicePipelines');
    app.run();

    View full-size slide

  18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications
    Infrastructure
    as code

    View full-size slide

  19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications

    View full-size slide

  20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications
    Continuous
    integration

    View full-size slide

  21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Continuous integration goals
    Source Build Test Production

    View full-size slide

  22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Continuous integration goals
    1. Automatically kick off a new release when new code is checked in
    2. Build and test code in a consistent, repeatable environment
    3. Continually have an artifact ready for deployment
    4. Continually close feedback loop when build fails

    View full-size slide

  23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodePipeline
    • Continuous delivery service for fast and reliable
    application updates
    • Model and visualize your software release process
    • Builds, tests, and deploys your code every time
    there is a code change
    • Integrates with third-party tools and AWS

    View full-size slide

  24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodePipeline: Supported sources
    Pick branch
    AWS CodeCommit
    GitHub
    Pick object or folder
    Amazon S3
    Pick Docker tag
    Amazon ECR
    Automatically kick off release and pull latest source code

    View full-size slide

  25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodePipeline: ECR source action
    Source code:
    “master” branch
    ECR repository:
    “release” tag

    View full-size slide

  26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodePipeline: Supported triggers
    Automatically kick off release
    Amazon CloudWatch Events
    • Scheduled (nightly release)
    • AWS Health events (Fargate
    platform retirement)
    Available in CloudWatch Events
    console, API, SDK, CLI, and AWS
    CloudFormation
    Webhooks
    • DockerHub
    • Quay
    • Artifactory
    Available in CodePipeline API,
    SDK, CLI, and CloudFormation

    View full-size slide

  27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodeBuild
    • Fully managed build service that compiles source
    code, runs tests, and produces software packages
    • Scales continuously and processes multiple builds
    concurrently
    • No build servers to manage
    • Pay by the minute, only for the compute
    resources you use
    • Monitor builds through CloudWatch Events

    View full-size slide

  28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodeBuild
    • Each build runs in a new Docker container for a
    consistent, immutable environment
    • Docker and AWS CLI are installed in every official
    CodeBuild image
    • Provide custom build environments suited to
    your needs through the use of Docker images

    View full-size slide

  29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodeBuild: Lambda buildspec
    version: 0.2
    phases:
    build:
    commands:
    - npm ci
    - npm test
    - >
    aws cloudformation package
    --template-file template.yaml
    --output-template packaged.yaml
    --s3-bucket $BUCKET
    artifacts:
    type: zip
    files:
    - packaged.yaml

    View full-size slide

  30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodeBuild: Lambda buildspec using SAM CLI
    version: 0.2
    phases:
    install:
    commands:
    - pip install --upgrade awscli aws-sam-cli
    build:
    commands:
    - sam build
    - sam package --s3-bucket $BUCKET --output-template-file packaged.yaml
    artifacts:
    type: zip
    files:
    - packaged.yaml

    View full-size slide

  31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodeBuild: Docker buildspec
    version: 0.2
    phases:
    build:
    commands:
    - $(aws ecr get-login --no-include-email)
    - docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
    - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $ECR_REPO:$IMAGE_TAG
    - docker push $ECR_REPO:$IMAGE_TAG

    View full-size slide

  32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications
    Continuous
    integration

    View full-size slide

  33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications

    View full-size slide

  34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications
    Continuous
    deployment

    View full-size slide

  35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Continuous deployment goals
    Source Build Test Production

    View full-size slide

  36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Continuous deployment goals
    1. Automatically deploy new changes to staging environments for testing
    2. Deploy to production safely without impacting customers
    3. Deliver to customers faster: Increase deployment frequency,
    and reduce change lead time and change failure rate

    View full-size slide

  37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodeDeploy
    • Automates code deployments to any instance
    and Lambda
    • Handles the complexity of updating your
    applications
    • Avoid downtime during application deployment
    • Roll back automatically if failure detected
    • Deploy to Amazon EC2, Lambda, ECS, or on-
    premises servers

    View full-size slide

  38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy – Lambda deployments in SAM templates
    Resources:
    GetFunction:
    Type: AWS::Serverless::Function
    Properties:
    AutoPublishAlias: live
    DeploymentPreference:
    Type: Canary10Percent10Minutes
    Alarms:
    - !Ref ErrorsAlarm
    - !Ref LatencyAlarm
    Hooks:
    PreTraffic: !Ref PreTrafficHookFunction
    PostTraffic: !Ref PostTrafficHookFunction
    Canary10Percent30Minutes
    Canary10Percent5Minutes
    Canary10Percent10Minutes
    Canary10Percent15Minutes
    Linear10PercentEvery10Minutes
    Linear10PercentEvery1Minute
    Linear10PercentEvery2Minutes
    Linear10PercentEvery3Minutes
    AllAtOnce

    View full-size slide

  39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy – Lambda canary deployment
    API
    Gateway
    Lambda
    function
    weighted
    alias “live”
    v1 Lambda
    function
    code
    100%

    View full-size slide

  40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy – Lambda canary deployment
    API
    Gateway
    Lambda
    function
    weighted
    alias “live”
    v1 code
    100%
    Run PreTraffic hook against v2 code before it receives traffic
    v2 code
    0%

    View full-size slide

  41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy – Lambda canary deployment
    API
    Gateway
    Lambda
    function
    weighted
    alias “live”
    v1 code
    90%
    Wait for 10 minutes, roll back in case of alarm
    v2 code
    10%

    View full-size slide

  42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy – Lambda canary deployment
    API
    Gateway
    Lambda
    function
    weighted
    alias “live”
    v1 code
    0%
    Run PostTraffic hook and complete deployment
    v2 code
    100%

    View full-size slide

  43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    API Gateway canary stage
    API
    Gateway
    Production
    stage
    v1 code
    v2 code
    99.5%
    0.5%
    Canary
    stage

    View full-size slide

  44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployments
    • Provisions “green” tasks, then flips traffic at the
    load balancer
    • Validation “hooks” enable testing at each stage of
    the deployment
    • Fast rollback to “blue” tasks in seconds if case of
    hook failure or CloudWatch alarms
    • Monitor deployment status and history via
    console, API, Amazon SNS notifications, and
    CloudWatch Events
    • Use “CodeDeploy-ECS” deploy action in
    CodePipeline or “aws ecs deploy” command in
    Jenkins

    View full-size slide

  45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS appspec
    version: 1.0
    Resources:
    - TargetService:
    Type: AWS::ECS::Service
    Properties:
    - TaskDefinition: "my_task_definition:8"
    LoadBalancerInfos:
    - ContainerName: "SampleApp"
    ContainerPort: 80
    Hooks:
    - BeforeInstall: "LambdaFunctionToExecuteAnythingBeforeNewRevisionInstalltion"
    - AfterInstall: "LambdaFunctionToExecuteAnythingAfterNewRevisionInstallation"
    - AfterAllowTestTraffic: "LambdaFunctionToValidateAfterTestTrafficShift"
    - BeforeAllowTraffic: "LambdaFunctionToValidateBeforeTrafficShift"
    - AfterAllowTraffic: "LambdaFunctionToValidateAfterTrafficShift"

    View full-size slide

  46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployment
    100%
    Prod
    traffic

    View full-size slide

  47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployment
    Target
    group 2
    100%
    Prod
    traffic
    Test traffic listener
    (port 9000)

    View full-size slide

  48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployment
    Green tasks:
    v2 code
    100%
    Prod
    traffic
    Provision green tasks

    View full-size slide

  49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployment
    100%
    Prod
    traffic
    Run hook against test endpoint before green tasks receive prod traffic
    0%
    Prod
    traffic

    View full-size slide

  50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployment
    Flip traffic to green tasks, rollback in case of alarm
    0%
    Prod
    traffic
    100%
    Prod
    traffic

    View full-size slide

  51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployment
    100%
    Prod
    traffic
    Drain blue tasks

    View full-size slide

  52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications
    Continuous
    deployment

    View full-size slide

  53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications

    View full-size slide

  54. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

    View full-size slide

  55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Capital One – Credit Offers API serverless architecture
    Affiliates
    www.capitalone.com/
    credit-cards/prequalify
    AWS Cloud
    Capital One
    API Gateway
    VPC
    Lambda
    Function
    Traces Logs
    Production Support
    Command Center
    COAT
    Credit Offers API Team
    Lambda
    Function
    S3 Bucket
    TTL
    Third-Party
    API

    View full-size slide

  56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Capital One – Credit Offers API CI/CD pipeline
    Continuous Improvement, Continuous Delivery!
    GitHub LGTM Bot Jenkins AWS SAM
    S3 Bucket
    (Versioning)
    Lambda
    Function
    DeploymentType:
    dev: AllAtOnce
    qa: AllAtOnce
    qaw: AllAtOnce
    prod: Canary10Percent10Minutes
    prodw: Canary10Percent10Minutes
    canary5xxGetProductsAlarm:
    Type: AWS::CloudFormation::Alarm
    Properties:
    AlarmActions:
    - !FindInMap:
    - params
    - AdminSNSTopic
    - !Ref Environment
    AlarmDescription: 500 error from product
    listing Lambda.
    ComparisonOperator:
    GreatherThanOrEqualTothreshold
    Period: 300
    Statistic: Sum
    Threshold: 1
    EvaluationPeriod: 1

    View full-size slide

  57. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Capital One – Benefits from taking the API serverless
    Performance gains
    From the time the request
    is received by lambda to
    the time to send the
    response back
    70%
    Cost savings
    By removing EC2, ELB and
    RDS from our solution
    90%
    Increase in team velocity
    Reduce investment in team’s time
    on DevOps and dedicate back to
    feature development!
    30%

    View full-size slide

  58. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

    View full-size slide

  59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Takeaways
    1. Manage your infrastructure as code
    2. Frequently build and integrate your code to get a first feedback
    3. Continuously release in production using canary releases with
    monitoring and automated rollbacks
    4. Use canary releases to get both technical and business feedback

    View full-size slide

  60. Thank you!
    S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Danilo Poccia
    @danilop
    Flynn Bundy
    @bundyfx

    View full-size slide

  61. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

    View full-size slide