Introduction to DevOps on AWS

This is my re:Invent DOp209 session
https://www.youtube.com/watch?v=wugkTArXBYo&list=PLrZoMXSbKmNg766qbitDzvGo1N_4D2iCt&index=7&t=0s

Transcript

1. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved. Introduction to DevOps on AWS Sébastien Stormacq Senior Developer Advocate Amazon Web Services, EMEA
2. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.
4. How it all started?
5. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.
6. Cloud benefits
7. References architectures VPC AWS Cloud Availability Zone 1 Auto Scaling group Availability Zone 2 NAT Gateway NAT Gateway Amazon EC2 instance Master database Replica database Application Load Balancer Amazon EC2 instance
8. Infrastructure as click
9. You choose your IDE Python Java, Python .NET, Node .NET
10. You choose your IDE Python Java, Python .NET, Node .NET Node.js .NET New
12. Your IDE is in the cloud A cloud IDE for writing, running, and debugging code
13. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.
14. You move your code to a code repository EBS volumes SSH or HTTPS Secure, scalable, and managed Git source control git push Amazon EC2
15. You move your code to a code repository Git objects in Amazon S3 Git index in Amazon DynamoDB Encryption keys in AWS KMS SSH or HTTPS Secure, scalable, and managed Git source control git push AWS CodeCommit
16. Getting started with CodeCommit & ssh $ ssh-keygen $ vi ~/.ssh Host git-codecommit.*.amazonaws.com User APKAEiBAERJR2EXAMPLE identityFile ~/.ssh/codecommit_rsa $ git clone \ ssh://git-codecommit.<region>.amazonaws.com/v1/repos/<repo> \ <dir>
17. Branching strategy Whole dev team share a branch called Trunk (or Master)
18. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.
19. Continuous integration workflow Version control Continuous integration server Commit to dev branch Pull code Send build report to development team; stop everything if build fails Distributed builds; run tests in parallel Hook Developer Test types Integration Unit Code coverage
20. Continuous integration workflow Commit to dev branch Pull code Send build report to development team; stop everything if build fails Distributed builds; run tests in parallel Hook Test types Integration Unit Code coverage Developer AWS CodeCommit AWS CodeBuild
21. Anatomy of a buildspec file version: 0.2 phases: pre_build: commands: - echo Logging in to Amazon ECR... - aws --version - $(aws ecr get-login --region eu-west-1 --no-include-email) - REPOSiTORY_URi=486652066693.dkr.ecr.eu-west-1.amazonaws.com/nginx - iMAGE_TAG=$(echo $CODEBUiLD_RESOLVED_SOURCE_VERSiON | cut -c 1-7) build: commands: - echo Build started on `date` - echo Building the Docker image... - docker build -t $REPOSiTORY_URi:latest nginx/. - docker tag $REPOSiTORY_URi:latest $REPOSiTORY_URi:$iMAGE_TAG post_build: commands: - echo Build completed on `date` - echo Pushing the Docker images... - docker push $REPOSiTORY_URi:latest - docker push $REPOSiTORY_URi:$iMAGE_TAG - echo Writing image definitions file... - printf '[{"name":"nginx","imageUri":"%s"}]’ $REPOSiTORY_URi:$iMAGE_TAG > imagedefinitions.json artifacts: files: imagedefinitions.json
22. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.
23. One dev environment does not scale
24. One dev/integration environment per developer
25. Infrastructure as code—avoid snowflakes
26. Cloud Development Kit (CDK) AWS CloudFormation template Resources AWS CDK application Stack(s) Construct Construct
27. CDK: Package your application CDK: Create a VPC // // create VPC w/ public and private subnets in 2 AZ // this also creates a NAT Gateway // const vpc = new ec2.Vpc(this, 'NewsBlogVPC', { maxAzs : 2 }); // // create static web site as S3 assets // var path = require('path'); const asset = new assets.Asset(this, ’YourSampleApp', { path: path.join(__dirname, '../html') }); // define a user data script to install & launch our app const userData = UserData.forLinux(); userData.addCommands('yum install -y nginx’, 'chkconfig nginx on', 'service nginx start’); userData.addCommands(`aws s3 cp s3://${asset.s3BucketName}/${asset.s3ObjectKey} .`, `unzip *.zip`, `/bin/cp -r -n ${env}/* /usr/share/nginx/html/`); CDK: Bootstrap your servers // create an auto scaling group for each environment const asg = new autoscaling.AutoScalingGroup(this, 'YourAppgAutoScalingGroup ' , { vpc, instanceType: ec2.instanceType.of(ec2.instanceClass.BURSTABLE3, ec2.instanceSize.MiCRO), machineimage: new ec2.AmazonLinuximage(), desiredCapacity: 2, role: role, userData: userData }); CDK: Create an Auto Scaling group
28. CDK: Deploy your own dev environment CloudFormation Template “compiler” CDK CLI “processor” “assembly language” “source” synthesize deploy executes
29. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.
30. Continuous deployment Merge PR into trunk Hook pull code Developer
31. Complex pipeline example—Trek10
32. IaC also for dev infrastructure // create the source action (github) const sourceOutput = new pipeline.Artifact(); const sourceAction = new pipeline_actions.GitHubSourceAction({ actionName: "GitHubTrigger", owner: github.owner, repo: github.repo, oauthToken: cdk.SecretValue.secretsManager(github.secret_manager_secret_name), output: sourceOutput, branch: 'master' }); // create the build action const buildProject = new codebuild.PipelineProject(pipelineStack, 'CodeBuildProje ct', { projectName: 'DockerBuild', buildSpec: BuildSpec.fromSourceFilename('nginx/buildspec.yml'), environment: { buildimage: codebuild.LinuxBuildimage.STANDARD_2_0, privileged: true } }); // add codebuild permissions to access ECR (to push the image to the repo) const role = <Role>buildProject.role; role.addManagedPolicy(ManagedPolicy.fromAwsManagedPolicyName('AmazonEC2ContainerR egistryPowerUser')); const buildOutput = new pipeline.Artifact(); const buildAction = new pipeline_actions.CodeBuildAction({ actionName: 'CodeBuildDockerimage', project: buildProject, input: sourceOutput, outputs: [buildOutput] }); const deployAction = new irEcsDeployAction({ actionName: 'Deploy', serviceName: ecs.serviceName, clusterName: ecs.clusterName, input: buildOutput, }); // finally, create the pipeline const codePipeline = new pipeline.Pipeline(pipelineStack, 'Pipeline', { pipelineName: 'ECSDeploy', stages: [ { stageName: 'GetSource', actions: [sourceAction], }, { stageName: 'BuildDockerimage', actions: [buildAction] }, { stageName: 'DeployToEcs', actions: [deployAction] } ], });
33. Blue-green deployment 100% Prod traffic
34. Blue-green deployment Target group 2 100% Prod traffic
35. Blue-green deployment Green tasks: v2 code Provision green tasks 100% Prod traffic
36. Blue-green deployment Run hook against test endpoint before green tasks receive prod traffic 0% Prod traffic 100% Prod traffic
37. Blue-green deployment Flip traffic to green tasks, rollback in case of alarm 80% Prod traffic 20% Prod traffic
38. Blue-green deployment Drain blue tasks 0% Prod traffic 100% Prod traffic
39. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.
41. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.
42. Debugging modern apps locally is hard New developer patterns Mockups are not perfect Applications are large
43. You are debugging in the cloud AWS Cloud
44. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.
46. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.
47. ChatOps
48. AWS Chatbot Receive notifications from AWS services about infrastructure events, billing, security, and more Easily integrate with Slack Built-in security templates for common use cases simplify configuration and enable best practices AWS Chatbot Events
49. AWS Chatbot can now run commands Interactive agent for ChatOps
50. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.
52. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.
53. How Amazon does DevOps? (microservices, 2 pizza teams) (governance, templates)
54. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.
55. What we built Trunk-based source code control AWS CDK Developers Services Delivery pipelines Monitor Build Test Release Monitor Build Test Release Monitor Build Test Release Monitor Build Test Release Monitor Build Test Release
56. Think big—impossibly big Start small Iterate
57. impact on dev hiring and retention
58. Start anywhere but start somewhere
59. Thank you! © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved. Sébastien Stormacq @sebsto
60. © 2019, Amazon Web Services, inc. or its affiliates. All rights reserved.