BUILDING A CI/CD PIPELINE ON AWS https://milancermak.com @milancermak Photo by Robert Bock on Unsplash 

WHAT TO BUILD Holy grail A git-push-to-production pipeline, with automated rollbacks, with separated environments, with separated accounts, distributed in multiple regions Start simple Automate deployments for EVERYTHING Photo by Kobby Mendez on Unsplash 

WHY BOTHER? Speed Focus Peace of mind GOOD FOR BUSINESS Photo by Taha Mazandarani on Unsplash 

TOOLS OF THE TRADE CloudFormation CodeCommit & GitHub CodeBuild CodeDeploy CodePipeline Photo by Rick Mason on Unsplash 

CODEPIPELINE 

No content

ACTION CONFIGURATION Action configuration depends on the action category: source, build, test, deploy, approval, invoke 

CODEPIPELINE ARTIFACTS Artifact is a fancy word for a zip file Code from the repository Output of CodeBuild … Accessible to actions Stored in an S3 bucket 

No content

AN EXAMPLE Photo by Victor Garcia on Unsplash 

PULL CODE FROM REPO 

Name: FetchSource Actions: - Name: CodeCommit ActionTypeId: Category: Source Owner: AWS Provider: CodeCommit Version: '1' Configuration: RepositoryName: !GetAtt CodeRepository.Name BranchName: master OutputArtifacts: - Name: SourceOutput 

MAKE THE PIPELINE UPDATE ITSELF 

Name: UpdatePipeline Actions: - Name: PipelineStack ActionTypeId: Category: Deploy Owner: AWS Provider: CloudFormation Version: '1' Configuration: ActionMode: CREATE_UPDATE Capabilities: CAPABILITY_IAM RoleArn: !GetAtt CloudformationRole.Arn StackName: !Ref AWS::StackName TemplatePath: SourceOutput::infrastructure/ pipeline.yml ParameterOverrides: !Sub '{"Service": "$ {Service}"}' InputArtifacts: - Name: SourceOutput 

PREPARE THE DEPLOYMENT PACKAGE 

Name: Build Actions: - Name: Functions ActionTypeId: Category: Build Owner: AWS Provider: CodeBuild Version: '1' Configuration: ProjectName: !Ref CodeBuildProject InputArtifacts: - Name: SourceOutput OutputArtifacts: - Name: BuildOutput 

CODEBUILD PROJECT DECLARATION 

CodeBuildProject: Type: AWS::CodeBuild::Project Properties: Artifacts: Type: CODEPIPELINE Cache: Location: !Sub '${ArtifactsBucket}/buildcache/ functions' Type: S3 Environment: ComputeType: BUILD_GENERAL1_SMALL EnvironmentVariables: - Name: ARTIFACTS_BUCKET Value: !Ref ArtifactsBucket Type: PLAINTEXT Image: aws/codebuild/python:3.7.1 Type: LINUX_CONTAINER ServiceRole: !GetAtt CodeBuildRole.Arn Source: BuildSpec: infrastructure/buildspec.yml Type: CODEPIPELINE TimeoutInMinutes: 15 

CODEBUILD TASK DECLARATION 

BUILDSPEC.YML Collection of build commands in YAML Creates artifacts that are passed further along the pipeline Tips: Print useful progress messages, run verbose programs in quiet mode Use a cache Put “complex” multiline commands in a .sh file Update aws-cli if you need newish services support Debug your build process locally with Docker 

version: 0.2 env: variables: # ARTIFACTS_BUCKET is available from pipeline.yml PIP_CACHE: pip_cache phases: install: commands: - printenv - pip install -r requirements-dev.txt —cache-dir $PIP_CACHE pre_build: commands: - pytest -vrf tests build: commands: - aws cloudformation package --template-file infrastructure/functions.yml --output-template-file packaged_functions.yml --s3-bucket "${ARTIFACTS_BUCKET}" artifacts: files: - packaged_functions.yml cache: paths: - 'pip_cache/**/*' 

DEPLOY BETA STACK 

Name: Deploy Actions: - Name: Beta ActionTypeId: Category: Deploy Owner: AWS Provider: CloudFormation Version: '1' Configuration: ActionMode: CREATE_UPDATE Capabilities: CAPABILITY_IAM,CAPABILITY_AUTO_EXPAND RoleArn: !GetAtt CloudformationRole.Arn StackName: !Sub '${Service}-beta-functions' TemplatePath: BuildOutput::packaged_functions.yml ParameterOverrides: !Sub '{"Service": "${Service}", "Stage": "beta"}' InputArtifacts: - Name: BuildOutput 

DEPLOY PRODUCTION STACK 

Actions: - Name: Prod ActionTypeId: Category: Deploy Owner: AWS Provider: CloudFormation Version: '1' Configuration: ActionMode: CREATE_UPDATE Capabilities: CAPABILITY_IAM,CAPABILITY_AUTO_EXPAND RoleArn: !GetAtt CloudformationRole.Arn StackName: !Sub '${Service}-prod-functions' TemplatePath: BuildOutput::packaged_functions.yml ParameterOverrides: !Sub '{"Service": "${Service}", "Stage": “prod"}' InputArtifacts: - Name: BuildOutput 

LAMBDA CANARY DEPLOYS 

USING CODEDEPLOY Connect Lambda with CodeDeploy via a CloudWatch Alarm Typically monitor the error rate Easy with AWS SAM, but there’s too much magic Tips: Make sure the alarms works as expected! Skip gradual rollout for non-production stacks Deploy “smaller regions” (i.e. not us-east-1) first 

Parameters: Stage: Description: Environment stage (deployment phase) Type: String AllowedValues: - beta - prod Conditions: IsBeta: !Equals [!Ref Stage, beta] Resources: HelloWorld: Type: AWS::Serverless::Function Properties: # ... AutoPublishAlias: !Ref Stage DeploymentPreference: Type: !If [IsBeta, AllAtOnce, Canary10Percent5Minutes] Alarms: - !Ref HelloWorldErrorsAlarm 

HelloWorldErrorsAlarm: Type: AWS::CloudWatch::Alarm Properties: ComparisonOperator: GreaterThanThreshold Dimensions: - Name: Resource Value: !Sub '${HelloWorld}:${Stage}' - Name: FunctionName Value: !Ref HelloWorld - Name: ExecutedVersion Value: !GetAtt HelloWorld.Version.Version EvaluationPeriods: 2 MetricName: Errors Namespace: AWS/Lambda Period: 60 Statistic: Sum Threshold: 0 

DEMO 

WHAT’S NEXT? Dynamic feature pipelines Separate accounts per stage Multi-region deployments Pipeline performance metrics Deploying dependent services Photo by Samuel Zeller on Unsplash 

THANK YOU Photo by Joel & Jasmin Førestbird on Unsplash