Infrastructure IS Code on AWS

In the age of The Cloud, Serverless, Containers and Microservices, our infrastructure is part of our application more than ever. So why do we still treat it as something else? Our Application is Code, our Business Logic is code, well Infrastructure also IS code. One of the parts of doing things properly at scale is being able to describe your infrastructure as code and deploy it as such. If we already treat our infrastructure as code, why not apply all the best practices of software delivery to infrastructure delivery. In this session we look into Infrastructure as Code solutions, best practices and patterns on AWS.

Transcript

1. © 2020, Amazon Web Services, Inc. or its Affiliates. Infrastructure IS Code on AWS Darko Meszaros Developer Advocate – Amazon Web Services @darkosubotica ln/darko-mesaros twitch.tv/ruptwelve youtube.com/ruptwelve
2. © 2020, Amazon Web Services, Inc. or its Affiliates. Agenda for today • What is Infrastructure as Code • AWS CloudFormation • AWS Serverless Application Model (SAM) • AWS Cloud Development Kit (CDK) • Other Tools out there • Some Best Practices • Wrap up
3. © 2020, Amazon Web Services, Inc. or its Affiliates. What is Infrastructure as Code?
4. © 2020, Amazon Web Services, Inc. or its Affiliates.
5. © 2020, Amazon Web Services, Inc. or its Affiliates.
6. © 2020, Amazon Web Services, Inc. or its Affiliates. Infrastructure as code ✓ Make infrastructure changes repeatable and predictable ✓ Release infrastructure changes using the same tools as code changes ✓ Replicate production in a staging environment to enable continuous testing
7. © 2020, Amazon Web Services, Inc. or its Affiliates. Infrastructure as code Declarative I tell you what I need I tell you what to do Imperative
8. © 2020, Amazon Web Services, Inc. or its Affiliates. $(whoami) Darko Mesaroš / Darko Meszaros / Дарко Месарош ! → " → # → $ → % Berlin ! @darkosubotica ln/darko-mesaros twitch.tv/ruptwelve youtube.com/ruptwelve
9. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS CloudFormation
10. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS CloudFormation • Infrastructure as code (IaC) • Provides a common language for you to describe and provision all the infrastructure resources in your cloud environment • Build and rebuild your infrastructure and applications, without having to perform manual actions or write custom scripts https://aws.amazon.com/cloudformation/
11. © 2020, Amazon Web Services, Inc. or its Affiliates. Sample AWS CloudFormation code • Code is written in files called templates • A stack is generated from a template • Templates primarily define resources for an application • AWS CloudFormation can create over 490 types of resources • Each resource is configured based on its available properties • Dependencies can be explicitly declared or implicitly discovered AWSTemplateFormatVersion: "2010-09-09" Description: A CodeCommit Repo and Cloud9 Environment Resources: MyRepo: Type: "AWS::CodeCommit::Repository" Properties: RepositoryName: MyRepo RepositoryDescription: Sample Repository for Demo MyC9Environment: Type: "AWS::Cloud9::EnvironmentEC2" Properties: Repositories: - PathComponent: /cfn RepositoryUrl: !GetAtt MyRepo.CloneUrlHttp InstanceType: t2.micro
12. © 2020, Amazon Web Services, Inc. or its Affiliates. Anatomy of an AWS CloudFormation template • Resources • Parameters and Mappings • Conditions • Outputs
13. © 2020, Amazon Web Services, Inc. or its Affiliates. Testing via pipelines • As you would with other application code, templates should be version controlled and tested via CI/CD pipelines • The linter can be run in an AWS CodeBuild step to ensure that teams comply with rules and standards • Additional tools, like taskcat (available on GitHub), allow tests across regions Git push Templates AWS CodeCommit AWS CodePipeline AWS CodeBuild AWS CloudFormation Region AWS CodeDeploy
14. © 2020, Amazon Web Services, Inc. or its Affiliates. But I use resources outside of AWS!
15. © 2020, Amazon Web Services, Inc. or its Affiliates. CloudFormation registry Open CLI Open providers Introducing the AWS CloudFormation registry An open approach to managing external resources
16. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS CloudFormation registry and CLI • Allows AWS CloudFormation to support native and non-AWS resources while inheriting many core benefits like rollbacks • Use the AWS CloudFormation CLI tool to create resource providers using JSON schema-driven development, generating many of the code assets for you • Use third-party resource providers as you would use native AWS resource types
17. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS Serverless Application Model (SAM)
18. © 2020, Amazon Web Services, Inc. or its Affiliates. Model function environments with AWS Serverless Application Model (SAM) • Open source framework for building serverless applications on AWS • Shorthand syntax to express functions, APIs, databases, and event source mappings • Transforms and expands SAM syntax into AWS CloudFormation syntax on deployment • Supports all AWS CloudFormation resource types https://aws.amazon.com/serverless/sam/
19. © 2020, Amazon Web Services, Inc. or its Affiliates. SAM template AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: MySimpleTableFunction: Type: AWS::Serverless::Function Properties: Handler: mySimpleTableFunction.handler Runtime: nodejs12.x CodeUri: ./functions Policies: - DynamoDBReadPolicy: TableName: !Ref MySimpleTable Events: MySimpleFunctionApi: Type: Api Properties: Path: /simpleTable Method: GET MySimpleTable: Type: AWS::Serverless::SimpleTable Just 20 lines to create: • Lambda function • IAM role • API Gateway
20. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS SAM CLI • Create, build, test, and deploy AWS SAM applications • Step-through debugging and IDE support • Open source! • https://github.com/awslabs/aw s-sam-cli
21. © 2020, Amazon Web Services, Inc. or its Affiliates. Use SAM CLI to package and deploy SAM templates pip install --user aws-sam-cli # Or even better use native installers sam init --name my-app --runtime python cd my-app/ sam local ... # generate-event/invoke/start-api/start-lambda sam validate # The SAM template sam build # Depending on the runtime sam package --s3-bucket my-packages-bucket \ --output-template-file packaged.yaml sam deploy --template-file packaged.yaml \ --stack-name my-stack-prod sam logs -n MyFunction --stack-name my-stack-prod -t # Tail sam publish # To the Serverless Application Repository CodePipeline Use CloudFormation deployment actions with any SAM application Jenkins Use SAM CLI plugin O pen Source
22. © 2020, Amazon Web Services, Inc. or its Affiliates. Did you just say local tests?
23. © 2020, Amazon Web Services, Inc. or its Affiliates.
24. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS Cloud Development Kit (CDK)
26. © 2020, Amazon Web Services, Inc. or its Affiliates.
27. © 2020, Amazon Web Services, Inc. or its Affiliates. Shorten the learning curve with AWS Cloud Development Kit (CDK) Late 2020 Brings cloud infrastructure to developers in ways the can understand • Build cloud infrastructure with the languages they already know • Use their existing tools and workflows • Helpful abstractions that remove the need to learn the details • Vibrant and fast-growing community of developers
28. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS Cloud Development Kit (AWS CDK) A multi-language development framework for modeling infrastructure as reusable components
29. © 2020, Amazon Web Services, Inc. or its Affiliates. From constructs to the cloud
30. © 2020, Amazon Web Services, Inc. or its Affiliates. AWS CDK Constructs
31. © 2020, Amazon Web Services, Inc. or its Affiliates. All the tests!
32. © 2020, Amazon Web Services, Inc. or its Affiliates. How do we do testing with CDK? • Snapshot tests • Fine-grained assertions • Validation tests npm install --save-dev jest @types/jest @aws-cdk/assert
33. © 2020, Amazon Web Services, Inc. or its Affiliates. Additional Tools and Frameworks
34. © 2020, Amazon Web Services, Inc. or its Affiliates.
35. © 2020, Amazon Web Services, Inc. or its Affiliates.
36. © 2020, Amazon Web Services, Inc. or its Affiliates. npm install -g cdktf-cli
37. © 2020, Amazon Web Services, Inc. or its Affiliates. Some Terraform to my CDK
38. © 2020, Amazon Web Services, Inc. or its Affiliates. Best Practices
39. © 2020, Amazon Web Services, Inc. or its Affiliates. Best practices (1/5) • Layer your application to reduce blast radius when updating resources • Use multiple, isolated environments for testing, production, development, staging, etc. • Smaller files are easier to write, test, and troubleshoot Instances, Auto Scaling groups API endpoints, functions Alarms, dashboards VPCs, NAT gateways, VPNs, subnets IAM users, groups, roles, policies Front-end resources Backend services Stateful resources Base network Identity & security Monitoring resources Databases and clusters, queues
40. © 2020, Amazon Web Services, Inc. or its Affiliates. Quotes “Please for the love of everything you hold dear, separate critical persistent storage from the rest of your IAC so you don’t accidentally remove it. Please.” - An Infrastructure as Code Developer with scars to prove it
41. © 2020, Amazon Web Services, Inc. or its Affiliates. Best practices (2/5) • Start small and don’t try to boil the ocean • Work out simple resources first to get the hang of it. • Do not specify every little detail right from the start.
42. © 2020, Amazon Web Services, Inc. or its Affiliates. Best practices (3/5) • It’s okay to repeat yourself • Do not engineer a whole new construct/library just so you prevent yourself from typing twice. • Do not overengineer things – this will help you out in the long run
43. © 2020, Amazon Web Services, Inc. or its Affiliates. Quotes “It’s okay to repeat yourself in CDK. It’s not normal code, Don’t engineer a whole new construct just to prevent yourself from typing something twice” - An experience CDK developer “Keep in mind the operational aspects coming after you build the infra and make it as simple as possible to support. And more importantly, straightforward to troubleshoot. Operations will pay dearly for crazy abstractions and dependencies in your code” - Someone who had a run in with Operations
44. © 2020, Amazon Web Services, Inc. or its Affiliates. Best practices (4/5) • Parameters and Mappings • Secrets Manager and SSM Parameter store • Do not hardcode sensitive information Resources: MyRDSDB: Type: "AWS::RDS::DBInstance" Properties: DBInstanceClass: db.t2.medium AllocatedStorage: ’20’ Engine: mariadb EngineVersion: ’10.2’ MasterUsername: appadmin MasterUserPassword: ‘{{resolve:ssm-secure:ssbRDSmEcntl:1}}'
45. © 2020, Amazon Web Services, Inc. or its Affiliates. Best practices (5/5) • Test, Test, Test • Put proper guidelines in place before. • Introduce peer reviews of your infrastructure code.
46. © 2020, Amazon Web Services, Inc. or its Affiliates. Quotes “Automation makes it easy to destroy your entire org with a few lines of yaml. Be paranoid about peer review, promoting changes through test environments, and privilege segregation" - An IaC Developer who, apparently, destroyed an entire org. “Automation let’s you mess up, at scale!” - Developer whor an the ‘* destroy’ command
47. © 2020, Amazon Web Services, Inc. or its Affiliates. Takeaways! • Get into Infrastructure as Code Early, it will help you manage scale in thelong run. ☁ • Treat infrastructure as code as any other code! • Use the tools that best fit your needs! ⚒
48. © 2020, Amazon Web Services, Inc. or its Affiliates.
49. © 2020, Amazon Web Services, Inc. or its Affiliates. Yes
50. © 2020, Amazon Web Services, Inc. or its Affiliates. @darkosubotica ln/darko-mesaros twitch.tv/ruptwelve youtube.com/ruptwelve