AWS CloudFormation Level-Up

Slide 1

Slide 1 text

AWS CloudFormation Level-Up Chuck Meyer, Sr. Dev Advocate AWS CloudFormation AWS CloudFormation Level-Up 1 / 42

Slide 2

Slide 2 text

Agenda Intro Authoring Deploying Testing Demos along the way AWS CloudFormation Level-Up 2 / 42

Slide 3

Slide 3 text

Intro AWS CloudFormation Level-Up 3 / 42

Slide 4

Slide 4 text

Who I am Chuck Meyer cmmeyer@amazon.com Sr Developer Advocate, AWS CloudFormation 5+ years at AWS Major: Infrastructure as code and DevOps Minor: Security automation / DevSecOps 20+ Years in Technology Bass player @chuckm AWS CloudFormation Level-Up 4 / 42

Slide 5

Slide 5 text

Infrastructure as code Declarative or imperative statements describing hardware, software and services and their relationships. AWS CloudFormation Level-Up 5 / 42

Slide 6

Slide 6 text

Infrastructure as code Declarative or imperative statements describing hardware, software and services and their relationships. Resource: MyWebServer Class: Server Type: ExtraBig Ports: - 443 AWS CloudFormation Level-Up 5 / 42

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Infrastructure as code Single source of truth for provisioning and conﬁguration Infrastructure that you can replicate, re-deploy, and re-purpose Control versioning on your infrastructure and your application together Roll back to the last good state on failures Build and deploy your infrastructure through your CI/CD pipeline AWS CloudFormation Level-Up 6 / 42

Slide 9

Slide 9 text

A simpliﬁed way to create and manage a collection of AWS resources Enables orderly and predictable provisioning and updating of resources Fully managed service Integrates with the AWS Management Console, the AWS Command Line Interface (CLI), or AWS APIs Only pay for the resources you create AWS CloudFormation AWS CloudFormation Level-Up 7 / 42

Slide 10

Slide 10 text

CloudFormation at a glance Enables provisioning and management of your infrastructure as code AWS CloudFormation Level-Up 8 / 42

Slide 11

Slide 11 text

Authoring AWS CloudFormation Level-Up 9 / 42

Slide 12

Slide 12 text

JSON JavaScript Object Notation Attribute/Value pairs Similar to XML Designed to be machine readable CloudFormation syntax AWS CloudFormation Level-Up 10 / 42

Slide 13

Slide 13 text

YAML YAML ain't a markup language Human readable data serialization standard Comments (use #) No } or ; CloudFormation syntax AWS CloudFormation Level-Up 11 / 42

Slide 14

Slide 14 text

Template Anatomy 1. Format version 2. Transforms 3. Description 4. Metadata 5. Parameters 6. Mappings 7. Conditions 8. Resources* (required) 9. Outputs CloudFormation syntax https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html AWS CloudFormation Level-Up 12 / 42

Slide 15

Slide 15 text

--- AWSTemplateFormatVersion: '2010-09-09' Description: Create an EC2 instance running the latest Amazon Linux AMI. Parameters: KeyPair: Description: The EC2 Key Pair to allow SSH access to the instance Type: String Resources: Ec2Instance: Properties: ImageId: ami-9d23aeea InstanceType: m3.medium KeyName: !Ref 'KeyPair' Type: AWS::EC2::Instance Outputs: InstanceId: Description: The InstanceId of the newly created EC2 instance Value: !Ref 'Ec2Instance' AWS CloudFormation Level-Up 13 / 42

Slide 16

Slide 16 text

Simple template – create EC2 instance Parameters: KeyPair: Description: 'The EC2 Key Pair to allow SSH access to the instance' Type: 'AWS::EC2::KeyPair::KeyName' You enter a value for the KeyPair parameter when you create your stack. AWS CloudFormation Level-Up 14 / 42

Slide 17

Slide 17 text

Simple template – create EC2 instance Resources: Ec2Instance: Type: 'AWS::EC2::Instance' Properties: ImageId: 'ami-9d23aeea' InstanceType: 'm3.medium' KeyName: !Ref 'KeyPair' Includes statically deﬁned properties (ImageId and InstanceType) and a reference to the KeyPair parameter. ImageId is the AMI speciﬁc to the region that you want to launch this stack in (eu- west-1 region in this example). AWS CloudFormation Level-Up 15 / 42

Slide 18

Slide 18 text

Simple template – create EC2 instance Outputs: InstanceId: Description: 'The InstanceId of the newly created EC2 instance' Value: !Ref 'Ec2Instance' These outputs are returned after the template has completed execution. AWS CloudFormation Level-Up 16 / 42

Slide 19

Slide 19 text

CloudFormation syntax – Resources The only section of the template that is required AWS services that will be created, updated, or deleted from your account Supports 342 resource types (and growing) Resources: Ec2Instance: Type: 'AWS::EC2::Instance' Properties: ImageId: 'ami-9d23aeea' InstanceType: 'm3.medium' KeyName: !Ref 'KeyPair' AWS CloudFormation Level-Up 17 / 42

Slide 20

Slide 20 text

CloudFormation syntax – Parameters Enable you to input custom values to your template each time you create or update a stack with input validation and restrictions Parameter types: String, Number, List, CommaDelimitedList, Parameter Store values, and AWS-speciﬁc types (AWS::EC2::Image::Id, AWS::Route53::HostedZone::Id) Use the Ref and Fn::Sub intrinsic functions to reference parameters Pseudo-Parameters are predeﬁned by AWS CloudFormation and used just like normal parameters (AWS::Region) AWS CloudFormation Level-Up 18 / 42

Slide 21

Slide 21 text

Intrinsic functions Basic programmatic functions available in-line for your declarative templates. Retrieve external values (Ref, Fn::Sub, Fn::FindInMap, Fn::GetAtt, Fn::GetAZs) Manipulate strings (Fn::Sub, Fn::Split, Fn::Join, Fn::Base64, Fn::Transform) Conditional logic (Fn::If, Fn::Equals, Fn::Not) AWS CloudFormation Level-Up 19 / 42

Slide 22

Slide 22 text

Dynamic references Inject values from SSM Parameter Store and Secrets Manager KMS encrypted strings Versioned and secured by IAM MyIAMUser: Type: AWS::IAM::User Properties: UserName: 'MyUserName’ LoginProfile: Password: '{{resolve:ssm-secure:IAMUserPassword:10}}' AWS CloudFormation Level-Up 20 / 42

Slide 23

Slide 23 text

CloudFormation syntax - Conditions Resource creation can depend on logical conditions: Conditions: isProd: Fn::Equals [ !Ref EnvType, prod ] Resources: EC2Instance: Type: "AWS::EC2::Instance" Condition: isProd Properties: ImageId: Fn::FindInMap [RegionMap, !Ref "AWS::Region", AMI] You can use conditions with intrinsic functions (Fn::If, Fn::Equals, Fn::Not) to create complex logic for property values. AWS CloudFormation Level-Up 21 / 42

Slide 24

Slide 24 text

CloudFormation syntax - Outputs Outputs from successful operations View them in the console or pass them along as inputs to other stacks Used with nested stacks and cross stack references Outputs: Environment: Description: 'Environment type' Value: Fn:If: [ isProd, 'Production', 'Development' ] AWS CloudFormation Level-Up 22 / 42

Slide 25

Slide 25 text

Authoring Level-up Use a linter (cfn-python-lint) AWS CloudFormation Level-Up 23 / 42

Slide 26

Slide 26 text

Authoring Level-up Use a linter (cfn-python-lint) Decompose architecture by lifecycle (short vs. long lived) AWS CloudFormation Level-Up 23 / 42

Slide 27

Slide 27 text

Authoring Level-up Use a linter (cfn-python-lint) Decompose architecture by lifecycle (short vs. long lived) Isolate stateful resources (databases, caches) AWS CloudFormation Level-Up 23 / 42

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Authoring Level-up Use a linter (cfn-python-lint) Decompose architecture by lifecycle (short vs. long lived) Isolate stateful resources (databases, caches) Don't write, Recycle! AWS Quick Starts Documentation examples AWS CloudFormation Level-Up 23 / 42

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Deploying AWS CloudFormation Level-Up 24 / 42

Slide 32

Slide 32 text

CloudFormation uses your template as a blueprint to provision resources into a construct called a stack. On create, the CloudFormation service: 1. Retrieves template from S3 (or API) 2. Parses template and validates parameters 3. Provisions resources in parallel or serial based on dependencies 4. Waits for resources to stabilize 5. Populates outputs and signals completion -or- Rolls back and signals failure Creating a stack AWS CloudFormation Level-Up 25 / 42

Slide 33

Slide 33 text

CloudFormation evaluates your changes to the template or parameters against the last known state of the provisioned resources in the stack. At then modiﬁes the resources as needed. On update, the CloudFormation service: 1. Retrieves template from S3 (or API) 2. Compares the template ands parameters to the last known state 3. Changes resources in place or creates new immutable resources 4. Waits for resources to stabilize 5. Updates outputs and signals completion -or- Rolls back and signals failure Updating a stack Deleting a stack AWS CloudFormation Level-Up 26 / 42

Slide 34

Slide 34 text

CloudFormation change sets Preview the impact to your stack of changes by comparing the new template and parameters to the last known state of the stack. CloudFormation makes the changes to your stack only when you decide to execute the change set. AWS CloudFormation Level-Up 27 / 42

Slide 35

Slide 35 text

Network Stack Outputs: VPC Description: 'VPC ID' Value: !Ref 'VPC' Export: Name: 'ProdVPC' =======> App Stack Resources: myTargetGroup: Type: 'AWS::ELBV2::TargetGroup' Properties: VpcId: Fn::ImportValue: 'ProdVPC' Cross-stack references (Exports) Allows you to share information between independent stacks. Export a stack’s output values. Other stacks in the same account and region can import the exported values. AWS CloudFormation Level-Up 28 / 42

Slide 36

Slide 36 text

Application Resources: NetworkResources: Type: 'AWS::CloudFormation::Stack' ContainerResources: Type: 'AWS::CloudFormation::Stack' Network Resources Resources: MyVPC Type: 'AWS::EC2::VPC' Create a hieararchy of stacks composed of multiple templates. Re-use templates with frequently used resources. Reference resources across stacks. Nested stacks AWS CloudFormation Level-Up 29 / 42

Slide 37

Slide 37 text

Drift detection Compares the last known state of the stack to current resource conﬁgurations. Shows if conﬁguration changes were made to your stack resources outside of CloudFormation. AWS CloudFormation Level-Up 30 / 42

Slide 38

Slide 38 text

CloudFormation StackSets Create, update, and delete stacks in multiple accounts and regions using a single operation AWS CloudFormation Level-Up 31 / 42

Slide 39

Slide 39 text

Deployment Level-up Build in guard rails Termination protection Stack policies UpdateReplace and Deletion policies IAM AWS CloudFormation Level-Up 32 / 42

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Deployment Level-up Build in guard rails Termination protection Stack policies UpdateReplace and Deletion policies IAM Use an orchestration tool to promote environments Jenkins, CodePipeline Use other services to manage conﬁguration SSM Parameter Store or Secrets Manager AWS CloudFormation Level-Up 32 / 42

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Testing AWS CloudFormation Level-Up 33 / 42

Slide 44

Slide 44 text

Infrastructure as is code! Template code should be in a repo Track issues and history Commits can trigger test suites and builds Use tools and utilities for validation Hook into Jenkins, Bamboo, Ansible, Chef, Puppet... AWS CloudFormation Level-Up 34 / 42

Slide 45

Slide 45 text

The challenge If this is ”infrastructure as code” why are we still testing by continually deploying and ﬁxing failures? Our goal: Catch errors early to reduce authoring time. AWS CloudFormation Level-Up 35 / 42

Slide 46

Slide 46 text

We are living in a golden age of tools cfn-lint Validate AWS CloudFormation yaml/json templates against the AWS CloudFormation spec and additional checks cfn-nag Look for patterns in templates that may indicate insecure infrastructure. Taskcat Catch problems that aren’t obvious in a single template/stack AWS CloudFormation Level-Up 36 / 42

Slide 47

Slide 47 text

cfn-lint "Can I deploy this template?" Community-driven open source tool to validate CloudFormation YAML/JSON templates against the CloudFormation resource speciﬁcation + additional checks. IDE plugins (VS Code, Atom, Sublime, IntelliJ, vim) https://github.com/awslabs/cfn-python-lint pip install cfn-lint AWS CloudFormation Level-Up 37 / 42

Slide 48

Slide 48 text

cfn-nag "Should I deploy this template?" Looks for patterns in CloudFormation templates that may indicate insecure infrastructure. IAM rule wildcards Security group wildcards Access logs that aren't enabled Encryption that isn't enabled https://github.com/stelligent/cfn_nag gem install cfn-nag AWS CloudFormation Level-Up 38 / 42

Slide 49

Slide 49 text

taskcat "Will this template deploy everywhere?" Catches problems that aren’t obvious in a single template/stack. Tests your templates by creating stacks in multiple AWS regions simultaneously. Generates a report with a pass/fail grade for each region AWS CloudFormation Level-Up 39 / 42

Slide 50

Slide 50 text

Validation pipeline AWS CloudFormation Level-Up 40 / 42

Slide 51

Slide 51 text

Testing Level-up Static analysis saves time (cfn-lint, cfn_nag)... AWS CloudFormation Level-Up 41 / 42

Slide 52

Slide 52 text

Testing Level-up Static analysis saves time (cfn-lint, cfn_nag)... ...but can't catch everything (taskcat) AWS CloudFormation Level-Up 41 / 42

Slide 53

Slide 53 text

Testing Level-up Static analysis saves time (cfn-lint, cfn_nag)... ...but can't catch everything (taskcat) Shorter templates are easier to test and maintain AWS CloudFormation Level-Up 41 / 42

Slide 54

Slide 54 text

Testing Level-up Static analysis saves time (cfn-lint, cfn_nag)... ...but can't catch everything (taskcat) Shorter templates are easier to test and maintain More reuse == less testing AWS CloudFormation Level-Up 41 / 42

Slide 55

Slide 55 text

AWS CloudFormation Level-Up @chuckm | cmmeyer@amazon.com 42 / 42