Slide 1

Slide 1 text

BUILDING A CI/CD PIPELINE ON AWS https://milancermak.com @milancermak Photo by Robert Bock on Unsplash

Slide 2

Slide 2 text

WHAT TO BUILD Holy grail A git-push-to-production pipeline, with automated rollbacks, with separated environments, with separated accounts, distributed in multiple regions Start simple Automate deployments for EVERYTHING Photo by Kobby Mendez on Unsplash

Slide 3

Slide 3 text

WHY BOTHER? Speed Focus Peace of mind GOOD FOR BUSINESS Photo by Taha Mazandarani on Unsplash

Slide 4

Slide 4 text

TOOLS OF THE TRADE CloudFormation CodeCommit & GitHub CodeBuild CodeDeploy CodePipeline Photo by Rick Mason on Unsplash

Slide 5

Slide 5 text

CODEPIPELINE

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

ACTION CONFIGURATION Action configuration depends on the action category: source, build, test, deploy, approval, invoke

Slide 8

Slide 8 text

CODEPIPELINE ARTIFACTS Artifact is a fancy word for a zip file Code from the repository Output of CodeBuild … Accessible to actions Stored in an S3 bucket

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

AN EXAMPLE Photo by Victor Garcia on Unsplash

Slide 11

Slide 11 text

PULL CODE FROM REPO

Slide 12

Slide 12 text

Name: FetchSource Actions: - Name: CodeCommit ActionTypeId: Category: Source Owner: AWS Provider: CodeCommit Version: '1' Configuration: RepositoryName: !GetAtt CodeRepository.Name BranchName: master OutputArtifacts: - Name: SourceOutput

Slide 13

Slide 13 text

MAKE THE PIPELINE UPDATE ITSELF

Slide 14

Slide 14 text

Name: UpdatePipeline Actions: - Name: PipelineStack ActionTypeId: Category: Deploy Owner: AWS Provider: CloudFormation Version: '1' Configuration: ActionMode: CREATE_UPDATE Capabilities: CAPABILITY_IAM RoleArn: !GetAtt CloudformationRole.Arn StackName: !Ref AWS::StackName TemplatePath: SourceOutput::infrastructure/ pipeline.yml ParameterOverrides: !Sub '{"Service": "$ {Service}"}' InputArtifacts: - Name: SourceOutput

Slide 15

Slide 15 text

PREPARE THE DEPLOYMENT PACKAGE

Slide 16

Slide 16 text

Name: Build Actions: - Name: Functions ActionTypeId: Category: Build Owner: AWS Provider: CodeBuild Version: '1' Configuration: ProjectName: !Ref CodeBuildProject InputArtifacts: - Name: SourceOutput OutputArtifacts: - Name: BuildOutput

Slide 17

Slide 17 text

CODEBUILD PROJECT DECLARATION

Slide 18

Slide 18 text

CodeBuildProject: Type: AWS::CodeBuild::Project Properties: Artifacts: Type: CODEPIPELINE Cache: Location: !Sub '${ArtifactsBucket}/buildcache/ functions' Type: S3 Environment: ComputeType: BUILD_GENERAL1_SMALL EnvironmentVariables: - Name: ARTIFACTS_BUCKET Value: !Ref ArtifactsBucket Type: PLAINTEXT Image: aws/codebuild/python:3.7.1 Type: LINUX_CONTAINER ServiceRole: !GetAtt CodeBuildRole.Arn Source: BuildSpec: infrastructure/buildspec.yml Type: CODEPIPELINE TimeoutInMinutes: 15

Slide 19

Slide 19 text

CODEBUILD TASK DECLARATION

Slide 20

Slide 20 text

BUILDSPEC.YML Collection of build commands in YAML Creates artifacts that are passed further along the pipeline Tips: Print useful progress messages, run verbose programs in quiet mode Use a cache Put “complex” multiline commands in a .sh file Update aws-cli if you need newish services support Debug your build process locally with Docker

Slide 21

Slide 21 text

version: 0.2 env: variables: # ARTIFACTS_BUCKET is available from pipeline.yml PIP_CACHE: pip_cache phases: install: commands: - printenv - pip install -r requirements-dev.txt —cache-dir $PIP_CACHE pre_build: commands: - pytest -vrf tests build: commands: - aws cloudformation package --template-file infrastructure/functions.yml --output-template-file packaged_functions.yml --s3-bucket "${ARTIFACTS_BUCKET}" artifacts: files: - packaged_functions.yml cache: paths: - 'pip_cache/**/*'

Slide 22

Slide 22 text

DEPLOY BETA STACK

Slide 23

Slide 23 text

Name: Deploy Actions: - Name: Beta ActionTypeId: Category: Deploy Owner: AWS Provider: CloudFormation Version: '1' Configuration: ActionMode: CREATE_UPDATE Capabilities: CAPABILITY_IAM,CAPABILITY_AUTO_EXPAND RoleArn: !GetAtt CloudformationRole.Arn StackName: !Sub '${Service}-beta-functions' TemplatePath: BuildOutput::packaged_functions.yml ParameterOverrides: !Sub '{"Service": "${Service}", "Stage": "beta"}' InputArtifacts: - Name: BuildOutput

Slide 24

Slide 24 text

DEPLOY PRODUCTION STACK

Slide 25

Slide 25 text

Actions: - Name: Prod ActionTypeId: Category: Deploy Owner: AWS Provider: CloudFormation Version: '1' Configuration: ActionMode: CREATE_UPDATE Capabilities: CAPABILITY_IAM,CAPABILITY_AUTO_EXPAND RoleArn: !GetAtt CloudformationRole.Arn StackName: !Sub '${Service}-prod-functions' TemplatePath: BuildOutput::packaged_functions.yml ParameterOverrides: !Sub '{"Service": "${Service}", "Stage": “prod"}' InputArtifacts: - Name: BuildOutput

Slide 26

Slide 26 text

LAMBDA CANARY DEPLOYS

Slide 27

Slide 27 text

USING CODEDEPLOY Connect Lambda with CodeDeploy via a CloudWatch Alarm Typically monitor the error rate Easy with AWS SAM, but there’s too much magic Tips: Make sure the alarms works as expected! Skip gradual rollout for non-production stacks Deploy “smaller regions” (i.e. not us-east-1) first

Slide 28

Slide 28 text

Parameters: Stage: Description: Environment stage (deployment phase) Type: String AllowedValues: - beta - prod Conditions: IsBeta: !Equals [!Ref Stage, beta] Resources: HelloWorld: Type: AWS::Serverless::Function Properties: # ... AutoPublishAlias: !Ref Stage DeploymentPreference: Type: !If [IsBeta, AllAtOnce, Canary10Percent5Minutes] Alarms: - !Ref HelloWorldErrorsAlarm

Slide 29

Slide 29 text

HelloWorldErrorsAlarm: Type: AWS::CloudWatch::Alarm Properties: ComparisonOperator: GreaterThanThreshold Dimensions: - Name: Resource Value: !Sub '${HelloWorld}:${Stage}' - Name: FunctionName Value: !Ref HelloWorld - Name: ExecutedVersion Value: !GetAtt HelloWorld.Version.Version EvaluationPeriods: 2 MetricName: Errors Namespace: AWS/Lambda Period: 60 Statistic: Sum Threshold: 0

Slide 30

Slide 30 text

DEMO

Slide 31

Slide 31 text

WHAT’S NEXT? Dynamic feature pipelines Separate accounts per stage Multi-region deployments Pipeline performance metrics Deploying dependent services Photo by Samuel Zeller on Unsplash

Slide 32

Slide 32 text

THANK YOU Photo by Joel & Jasmin Førestbird on Unsplash