Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CI/CD on AWS

CI/CD on AWS

AWS Summit, London, May 8th, 2019

How can you accelerate the delivery of new, high-quality services? How can you be able to experiment and get feedback quickly from your customers? To get the most out of the agility afforded by serverless and containers, it is essential to build CI/CD pipelines that help teams iterate on code and quickly release features. In this talk, we demonstrate how developers can build effective CI/CD release workflows to manage their serverless or containerized deployments on AWS. We cover infrastructure-as-code (IaC) application models, such as AWS Serverless Application Model (AWS SAM) and new imperative IaC tools. We also demonstrate how to set up CI/CD release pipelines with AWS CodePipeline and AWS CodeBuild, and we show you how to automate safer deployments with AWS CodeDeploy.

Danilo Poccia

May 08, 2019
Tweet

More Decks by Danilo Poccia

Other Decks in Programming

Transcript

  1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CI/CD on AWS
    Danilo Poccia
    Principal Evangelist, Serverless
    AWS
    @danilop
    Tonino Greco
    Head of Infrastructure and DevOps
    Dunelm
    @toninog

    View full-size slide

  2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Listen
    Iterate
    Experiment
    Innovation
    Flywheel
    Experiments power the engine of rapid innovation

    View full-size slide

  3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications

    View full-size slide

  4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications
    Infrastructure
    as code

    View full-size slide

  5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Infrastructure as code goals
    1. Make infrastructure changes repeatable and predictable
    2. Release infrastructure changes using the same tools as code changes
    3. Replicate production environment in a staging environment to enable
    continuous testing

    View full-size slide

  6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Release infrastructure-as-code with AWS CloudFormation
    “Master”
    branch
    Prepare
    template
    Create & execute
    change set
    Create & execute
    change set

    View full-size slide

  7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Model function environments with AWS
    Serverless Application Model (SAM)
    • Open source framework for building serverless
    applications on AWS
    • Shorthand syntax to express functions, APIs,
    databases, and event source mappings
    • Transforms and expands SAM syntax into AWS
    CloudFormation syntax on deployment
    • Supports all AWS CloudFormation resource types
    https://aws.amazon.com/serverless/sam/

    View full-size slide

  8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Model container environments with AWS
    Cloud Development Kit (CDK)
    • Open source framework to define cloud
    infrastructure in TypeScript, Java, C#, …
    • Provides library of higher-level resource types
    (“construct” classes) that have AWS best practices
    built in by default, packaged as npm modules
    • Provisions resources with CloudFormation
    • Supports all CloudFormation resource types
    AWS
    CDK
    https://awslabs.github.io/aws-cdk
    D
    eveloper
    Preview

    View full-size slide

  9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS Cloud Development Kit (CDK)
    npm install -g aws-cdk
    cdk init app --language typescript
    cdk synth
    cdk deploy
    cdk diff
    cdk destroy
    CodePipeline
    Use CloudFormation
    deployment actions with
    any synthesized CDK
    application
    Jenkins
    Use CDK CLI
    D
    eveloper
    Preview
    TypeScript
    C#
    F#
    Java
    Python

    View full-size slide

  10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CDK template
    import ec2 = require('@aws-cdk/aws-ec2');
    import ecs = require('@aws-cdk/aws-ecs');
    import cdk = require('@aws-cdk/cdk');
    class BonjourFargate extends cdk.Stack {
    constructor(parent: cdk.App, name: string, props?: cdk.StackProps) {
    super(parent, name, props);
    const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 });
    const cluster = new ecs.Cluster(this, 'Cluster', { vpc });
    new ecs.LoadBalancedFargateService(
    this, "FargateService", {
    cluster,
    image: ecs.DockerHub.image("amazon/amazon-ecs-sample"),
    });
    }
    }
    const app = new cdk.App();
    new BonjourFargate(app, 'Bonjour');
    app.run();
    TypeScript

    View full-size slide

  11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    import ec2 = require('@aws-cdk/aws-ec2');
    import ecs = require('@aws-cdk/aws-ecs');
    import cdk = require('@aws-cdk/cdk');
    class BonjourFargate extends cdk.Stack {
    constructor(parent: cdk.App, name: string, props?: cdk.StackProps) {
    super(parent, name, props);
    const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 });
    const cluster = new ecs.Cluster(this, 'Cluster', { vpc });
    new ecs.LoadBalancedFargateService(
    this, "FargateService", {
    cluster,
    image: ecs.DockerHub.image("amazon/amazon-ecs-sample"),
    });
    }
    }
    const app = new cdk.App();
    new BonjourFargate(app, 'Bonjour');
    app.run();
    CDK template
    TypeScript

    View full-size slide

  12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CDK template
    import ec2 = require('@aws-cdk/aws-ec2');
    import ecs = require('@aws-cdk/aws-ecs');
    import cdk = require('@aws-cdk/cdk');
    class BonjourFargate extends cdk.Stack {
    constructor(parent: cdk.App, name: string, props?: cdk.StackProps) {
    super(parent, name, props);
    const vpc = new ec2.VpcNetwork(this, 'MyVpc', { maxAZs: 2 });
    const cluster = new ecs.Cluster(this, 'Cluster', { vpc });
    new ecs.LoadBalancedFargateService(
    this, "FargateService", {
    cluster,
    image: ecs.DockerHub.image("amazon/amazon-ecs-sample"),
    });
    }
    }
    const app = new cdk.App();
    new BonjourFargate(app, 'Bonjour');
    app.run();
    TypeScript

    View full-size slide

  13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CDK Lambda cron example
    export class LambdaCronStack extends cdk.Stack {
    constructor(app: cdk.App, id: string) {
    super(app, id);
    const lambdaFn = new lambda.Function(this, 'Singleton', {
    code: new lambda.InlineCode(
    fs.readFileSync('lambda-handler.py', { encoding: 'utf-8' })),
    handler: 'index.main’,
    timeout: 300,
    runtime: lambda.Runtime.Python37,
    });
    const rule = new events.EventRule(this, 'Rule', {
    scheduleExpression: 'cron(0 18 ? * MON-FRI *)’,
    });
    rule.addTarget(lambdaFn);
    }
    }
    Lambda function
    CloudWatch Events rule
    CloudFormation Stack
    Set the target
    TypeScript

    View full-size slide

  14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Model pipelines with AWS CDK
    • Minimize copy-and-paste by using object-oriented language
    • Define microservice pipeline “shape” in one class, then re-use it across
    many pipelines
    • CDK includes many high-level constructs for modeling a CodePipeline
    pipeline, including automatically configuring IAM role policies

    View full-size slide

  15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CDK pipelines: Construct
    export class MyMicroservicePipeline extends cdk.Construct {
    constructor(parent: cdk.Construct, name: string, props: MyMicroservicePipelineProps) {
    super(parent, name);
    const pipeline = new codepipeline.Pipeline(this, 'Pipeline', {
    pipelineName: props.serviceName,
    });
    const githubAccessToken = new cdk.SecretParameter(this, 'GitHubToken',
    { ssmParameter: 'GitHubToken' });
    new codepipeline.GitHubSourceAction(this, 'GitHubSource', {
    stage: pipeline.addStage('Source'),
    owner: 'myorg',
    repo: props.serviceName,
    oauthToken: githubAccessToken.value
    });

    View full-size slide

  16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CDK pipelines: Stack
    import cdk = require('@aws-cdk/cdk');
    import { MyMicroservicePipeline } from './pipeline';
    class MyMicroservicePipelinesStack extends cdk.Stack {
    constructor(parent: cdk.App, name: string, props?: cdk.StackProps) {
    super(parent, name, props);
    new MyMicroservicePipeline(this, 'Pipeline1', { 'serviceName': 'Microservice1' });
    new MyMicroservicePipeline(this, 'Pipeline2', { 'serviceName': 'Microservice2' });
    new MyMicroservicePipeline(this, 'Pipeline3', { 'serviceName': 'Microservice3' });
    new MyMicroservicePipeline(this, 'Pipeline4', { 'serviceName': 'Microservice4' });
    }
    }
    const app = new cdk.App();
    new MyMicroservicePipelinesStack(app, 'MyMicroservicePipelines');
    app.run();

    View full-size slide

  17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications
    Infrastructure
    as code

    View full-size slide

  18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications

    View full-size slide

  19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications
    Continuous
    integration

    View full-size slide

  20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Continuous integration goals
    Source Build Test Production

    View full-size slide

  21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Continuous integration goals
    1. Automatically kick off a new release when new code is checked in
    2. Build and test code in a consistent, repeatable environment
    3. Continually have an artifact ready for deployment
    4. Continually close feedback loop when build fails

    View full-size slide

  22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodePipeline
    • Continuous delivery service for fast and reliable
    application updates
    • Model and visualize your software release process
    • Builds, tests, and deploys your code every time
    there is a code change
    • Integrates with third-party tools and AWS

    View full-size slide

  23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodePipeline: Supported sources
    Pick branch
    AWS CodeCommit
    GitHub
    Pick object or folder
    Amazon S3
    Pick Docker tag
    Amazon ECR
    Automatically kick off release and pull latest source code

    View full-size slide

  24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodePipeline: ECR source action
    Source code:
    “master” branch
    ECR repository:
    “release” tag

    View full-size slide

  25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodePipeline: Supported triggers
    Automatically kick off release
    Amazon CloudWatch Events
    • Scheduled (nightly release)
    • AWS Health events (Fargate
    platform retirement)
    Available in CloudWatch Events
    console, API, SDK, CLI, and AWS
    CloudFormation
    Webhooks
    • DockerHub
    • Quay
    • Artifactory
    Available in CodePipeline API,
    SDK, CLI, and CloudFormation

    View full-size slide

  26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodeBuild
    • Fully managed build service that compiles source
    code, runs tests, and produces software packages
    • Scales continuously and processes multiple builds
    concurrently
    • No build servers to manage
    • Pay by the minute, only for the compute
    resources you use
    • Monitor builds through CloudWatch Events

    View full-size slide

  27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodeBuild
    • Each build runs in a new Docker container for a
    consistent, immutable environment
    • Docker and AWS CLI are installed in every official
    CodeBuild image
    • Provide custom build environments suited to
    your needs through the use of Docker images

    View full-size slide

  28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodeBuild: Lambda buildspec
    version: 0.2
    phases:
    build:
    commands:
    - npm ci
    - npm test
    - >
    aws cloudformation package
    --template-file template.yaml
    --output-template packaged.yaml
    --s3-bucket $BUCKET
    artifacts:
    type: zip
    files:
    - packaged.yaml

    View full-size slide

  29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodeBuild: Lambda buildspec using SAM CLI
    version: 0.2
    phases:
    install:
    commands:
    - pip install --upgrade awscli aws-sam-cli
    build:
    commands:
    - sam build
    - sam package --s3-bucket $BUCKET --output-template-file packaged.yaml
    artifacts:
    type: zip
    files:
    - packaged.yaml

    View full-size slide

  30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodeBuild: Docker buildspec
    version: 0.2
    phases:
    build:
    commands:
    - $(aws ecr get-login --no-include-email)
    - docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
    - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $ECR_REPO:$IMAGE_TAG
    - docker push $ECR_REPO:$IMAGE_TAG

    View full-size slide

  31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications
    Continuous
    integration

    View full-size slide

  32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications

    View full-size slide

  33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications
    Continuous
    deployment

    View full-size slide

  34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Continuous deployment goals
    Source Build Test Production

    View full-size slide

  35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Continuous deployment goals
    1. Automatically deploy new changes to staging environments for testing
    2. Deploy to production safely without impacting customers
    3. Deliver to customers faster: Increase deployment frequency,
    and reduce change lead time and change failure rate

    View full-size slide

  36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS CodeDeploy
    • Automates code deployments to any instance
    and Lambda
    • Handles the complexity of updating your
    applications
    • Avoid downtime during application deployment
    • Roll back automatically if failure detected
    • Deploy to Amazon EC2, Lambda, ECS, or on-
    premises servers

    View full-size slide

  37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy – Lambda canary deployment
    API
    Gateway
    Lambda
    function
    weighted
    alias “live”
    v1 Lambda
    function
    code
    100%

    View full-size slide

  38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy – Lambda canary deployment
    API
    Gateway
    Lambda
    function
    weighted
    alias “live”
    v1 code
    100%
    Run PreTraffic hook against v2 code before it receives traffic
    v2 code
    0%

    View full-size slide

  39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy – Lambda canary deployment
    API
    Gateway
    Lambda
    function
    weighted
    alias “live”
    v1 code
    90%
    Wait for 10 minutes, roll back in case of alarm
    v2 code
    10%

    View full-size slide

  40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy – Lambda canary deployment
    API
    Gateway
    Lambda
    function
    weighted
    alias “live”
    v1 code
    0%
    Run PostTraffic hook and complete deployment
    v2 code
    100%

    View full-size slide

  41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy – Lambda deployments in SAM templates
    Resources:
    GetFunction:
    Type: AWS::Serverless::Function
    Properties:
    AutoPublishAlias: live
    DeploymentPreference:
    Type: Canary10Percent10Minutes
    Alarms:
    - !Ref ErrorsAlarm
    - !Ref LatencyAlarm
    Hooks:
    PreTraffic: !Ref PreTrafficHookFunction
    PostTraffic: !Ref PostTrafficHookFunction
    Canary10Percent30Minutes
    Canary10Percent5Minutes
    Canary10Percent10Minutes
    Canary10Percent15Minutes
    Linear10PercentEvery10Minutes
    Linear10PercentEvery1Minute
    Linear10PercentEvery2Minutes
    Linear10PercentEvery3Minutes
    AllAtOnce

    View full-size slide

  42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    API Gateway canary stage
    API
    Gateway
    Production
    stage
    v1 code
    v2 code
    99.5%
    0.5%
    Canary
    stage

    View full-size slide

  43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployments
    • Provisions “green” tasks, then flips
    traffic at the load balancer
    • Validation “hooks” enable testing at
    each stage of the deployment
    • Fast rollback to “blue” tasks in seconds if
    case of hook failure or CloudWatch
    alarms
    • Monitor deployment status and history
    via console, API, Amazon SNS
    notifications, and CloudWatch Events
    • Use “CodeDeploy-ECS” deploy action in
    CodePipeline or “aws ecs deploy”
    command in Jenkins

    View full-size slide

  44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS appspec
    version: 1.0
    Resources:
    - TargetService:
    Type: AWS::ECS::Service
    Properties:
    - TaskDefinition: "my_task_definition:8"
    LoadBalancerInfos:
    - ContainerName: "SampleApp"
    ContainerPort: 80
    Hooks:
    - BeforeInstall: "LambdaFunctionToExecuteAnythingBeforeNewRevisionInstalltion"
    - AfterInstall: "LambdaFunctionToExecuteAnythingAfterNewRevisionInstallation"
    - AfterAllowTestTraffic: "LambdaFunctionToValidateAfterTestTrafficShift"
    - BeforeAllowTraffic: "LambdaFunctionToValidateBeforeTrafficShift"
    - AfterAllowTraffic: "LambdaFunctionToValidateAfterTrafficShift"

    View full-size slide

  45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployment
    100%
    Prod
    traffic

    View full-size slide

  46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployment
    Target
    group 2
    100%
    Prod
    traffic
    Test traffic listener
    (port 9000)

    View full-size slide

  47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployment
    Green tasks:
    v2 code
    100%
    Prod
    traffic
    Provision green tasks

    View full-size slide

  48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployment
    100%
    Prod
    traffic
    Run hook against test endpoint before green tasks receive prod traffic
    0%
    Prod
    traffic

    View full-size slide

  49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployment
    Flip traffic to green tasks, rollback in case of alarm
    0%
    Prod
    traffic
    100%
    Prod
    traffic

    View full-size slide

  50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    CodeDeploy-ECS blue-green deployment
    100%
    Prod
    traffic
    Drain blue tasks

    View full-size slide

  51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    AWS Deployment Framework (ADF)
    https://github.com/awslabs/aws-deployment-framework

    View full-size slide

  52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications
    Continuous
    deployment

    View full-size slide

  53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    S U M M I T
    Pillars of releasing modern applications

    View full-size slide

  54. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

    View full-size slide

  55. CHALLENGES
    • Multiple independent tribes deploying to AWS
    • Complex Serverless deployments
    • Deploy rapid and often
    Our Answer
    • SAM CLI - Wrapped in Ansible
    • Part of a Pipeline library – functions for the developer
    • Jenkins Pipeline as code
    • Extensible, reproducible and simple
    • Slack integrations
    Dunelm Ltd

    View full-size slide

  56. Git Commit
    Jenkins
    Webhook
    Automation Code Product Code
    Code
    Linting
    Prepare
    Worker
    Node
    Install Software
    Security
    Validation
    Build / compile
    Run
    Tests
    Deploy
    Install Software for building
    Vulnerability and Library scan
    Compile the software
    Run Unit and functional tests
    Deploy to AWS

    View full-size slide

  57. THE BENEFITS
    ü Simple, reproducible and security conscious
    ü Scalable for performance – each pipeline has it’s
    own worker node (on Spot instances)
    ü Highly customisable for each tribe – self serve
    ü Centrally managed library of functions – easy to
    adapt to change
    ü Rapid onboarding of new developers – easy to use
    ü Rapid development of CI/CD pipelines
    ü Over 100 developers using the pipelines (across all
    Dunelm technology tribes)
    Dunelm Ltd

    View full-size slide

  58. THE BENEFITS – THE STATS
    • Daily there are
    • >200 pipelines active
    • each deploying > 15 times / day
    • Having > 10 stages per pipeline
    • Productivity gains
    • > 95% of pipelines are successful
    • Allow tribes to be > 30% more effective (based
    on burnup charts and sprint reviews)
    • Allows us to test and security audit automatically
    Dunelm Ltd

    View full-size slide

  59. ADDITIONAL BENEFITS
    • Slack integration with Jenkins
    • Status updates in Tribe channels
    • Kick off adhoc builds from Slack
    • Slack integration with AWS
    • Able to delete stacks (AWS CloudFormation)
    • Add parameters to AWS Systems Manager
    • Create Amazon Route53 DNS entries
    • AWS Lambda function to delete AWS CloudFormation
    stacks once they are removed from BitBucket
    Dunelm Ltd

    View full-size slide

  60. Thank you!
    S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Danilo Poccia
    @danilop
    Tonino Greco
    @toninog

    View full-size slide