AWS CloudFormation

Evgeny Zislis

{ "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "EC2 Instance and a Security Group", "Parameters" : { "KeyName": { "Description" : "An existing EC2 KeyPair to enable SSH access", "Type": "AWS::EC2::KeyPair::KeyName" }, "InstanceType" : { "Description" : "EC2 instance type", "Type" : "String", "Default" : "t2.small", "AllowedValues" : [ "t2.small", "t2.medium", "t2.large"] }, "SSHLocation" : { "Description" : "CIDR from where SSH access is allowed", "Type": "String", "MinLength": "9", "MaxLength": "18", "Default": "0.0.0.0/0", "AllowedPattern": "(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})/(\\d{1,2})" } ...

"Resources" : { "EC2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "InstanceType" : { "Ref" : "InstanceType" }, "SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ], "KeyName" : { "Ref" : "KeyName" }, "ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" }, { "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" }, "Arch" ] } ] } } }, "InstanceSecurityGroup" : { "Type" : "AWS::EC2::SecurityGroup", "Properties" : { "GroupDescription" : "Enable SSH access via port 22", "SecurityGroupIngress" : [ { "IpProtocol" : "tcp", "FromPort" : "22", "ToPort" : "22", "CidrIp" : { "Ref" : "SSHLocation"} } ] } }

/prəˈviZHən/ supply with equipment the IT manager was responsible to provision servers for the production release pro·vi·sion

/ˈôdəˌmāt/ convert a process to reduce human intervention to a minimum jobs that used to be done by people are now able to be done through automation au·to·mate

/rəˈpēdəb(ə)l/ able to be done again repeatable experiments under laboratory conditions re·peat·a·ble

/prəˈdiktəb(ə)l/ behaving or occurring in a way that is expected. unsurprising. Murphy's unpredictable rule predictably hits you pre·dict·a·ble

CloudFormation is a tool for repeatable and predictable automation of cloud infrastructure provisioning

No content

Create a VPC aws --region us-east-1 \ cloudformation create-stack \ --stack-name VPC \ --template-url https://aws-quickstart.s3.amazonaws.com/quickstart-aws-vpc/templates/aws-vpc.template \ --parameters \ ParameterKey=KeyPairName,ParameterValue=evgeny-ssh \ ParameterKey=AvailabilityZones,ParameterValue="'us-east-1a,us-east-1b'" { "StackId": "arn:aws:cloudformation:us-east-1:675783095585:stack/VPC/76271620-d0f8-11e8-b3a0-500c28b04cd1" } aws.amazon.com/quickstart/architecture/vpc

No content

Nested Stacks CloudFormation

Using Nested Stacks Resources: NestedStack: Type: AWS::CloudFormation::Stack Properties: TemplateURL: !Ref NestedStackS3File InstanceType: !Ref InstanceType Outputs: NestedStackBucketName: Value: !GetAtt NestedStack.InstanceName

docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/troubleshooting.html

Never, ever, should you actually use Nested Stacks because Nested Stacks are Stuck in UPDATE_COMPLETE_CLEANUP_IN_PROGRESS, UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS, or UPDATE_ROLLBACK_IN_PROGRESS A nested stack failed to roll back. Because of potential resource dependencies between nested stacks, AWS CloudFormation doesn't start cleaning up nested stack resources until all nested stacks have been updated or have rolled back. When a nested stack fails to roll back, AWS CloudFormation cancels all operations, regardless of the state that the other nested stacks are in. A nested stack that completed updating or rolling back but did not receive a signal from AWS CloudFormation to start cleaning up because another nested failed to roll back is in an UPDATE_COMPLETE_CLEANUP_IN_PROGRESS or UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS state. A nested stack that failed to update but did not receive a signal to start rolling back is in an UPDATE_ROLLBACK_IN_PROGRESS state. A nested stack might fail to roll back because of changes that were made outside of AWS CloudFormation, when the stack template doesn't accurately reflect the state of the stack. A nested stack might also fail if an Auto Scaling group in a nested stack had an insufficient resource signal timeout period when the group was created or updated. To fix the stack, contact AWS customer support. WTF? docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/troubleshooting.html

No content

“no one knows for sure who erected these monoliths and why”

Exporting & Importing CloudFormation

Stack B: Resources: EC2Instance: Type: AWS::EC2::Instance Properties: NetworkInterfaces: - AssociatePublicIpAddress: true DeviceIndex: 0 GroupSet: - Fn::ImportValue: Global-SecurityGroup Stack A: Outputs: SecurityGroup: Value: !Ref SecurityGroup Description: ID of security group Export: Name: Global-SecurityGroup Exported Output and Imported Value

Exported Value Dependencies

Custom Resources CloudFormation

Custom Resources - Definition and usage Resources: CustomCWEventsLambda: Type: AWS::Lambda::Function Properties: Runtime: nodejs8.10 Handler: index.handler Role: !GetAtt LambdaExecutionRole.Arn Code: ZipFile: > const Response = require('cfn-response'); exports.handler = function handler(event, context) { Response.send(event, context, Response.FAILED, {}); throw new Error('Not Implemented'); } CloudWatchEventsRule: Type: Custom::Events::Rule Parameters: ServiceToken: !GetAtt CustomCWEventsLambda.Arn Name: 'STRING_VALUE' - A name for this rule. *optional* Description: 'STRING_VALUE' - A description of the rule. State: 'ENABLED' - Indicates whether the rule is enabled or disabled. from github.com/devops-israel/aws-cf-custom-events

Custom Resources - Implementation const Response = require('cfn-response'); const log = { info(...args) { console.log(...args) }, error(...args) { console.error(...args) }, } exports.handler = function handler(event, context) { log.info(event, context); switch (event.RequestType) { case 'Create': return createResource(event, context); case 'Delete': return deleteResource(event, context); case 'Update': return updateResource(event, context); default: log.error(new Error(`ERROR: Unknown event.RequestType provided: ${event.RequestType}`)); Response.send(event, context, Response.FAILED, {}); } } from github.com/devops-israel/aws-cf-custom-events

cfn-init CloudFormation

Resources: AutoScalingGroup: Type: AWS::AutoScaling::AutoScalingGroup Properties: LaunchConfigurationName: !Ref LaunchConfiguration CreationPolicy: ResourceSignal: Timeout: PT15M LaunchConfiguration: Type: AWS::AutoScaling::LaunchConfiguration Properties: UserData: Fn::Base64: !Sub | #cloud-config packages: - aws-cfn-bootstrap bootcmd: - "while ! ping -i 10 -I eth0 -n -w 60 -W 2 -c 1 -q s3.amazonaws.com; do sleep 1; done" runcmd: - /opt/aws/bin/cfn-init -v --region ${AWS::Region} --stack ${AWS::StackName} --resource LaunchConfiguration - /opt/aws/bin/cfn-signal -e $? --region ${AWS::Region} --stack ${AWS::StackName} --resource AutoScalingGrou Updating EC2 Instances in an Auto Scaling Group

LaunchConfiguration: Type: AWS::AutoScaling::LaunchConfiguration Properties: ... Metadata: AWS::CloudFormation::Init: config: commands: ecs_task_sysctl1: command: "sysctl -w net.ipv4.conf.all.route_localnet=1" ecs_task_sysctl2: command: "sysctl -w net.ipv4.ip_local_port_range='1024 65000'" ecs_task_roles1: command: "iptables -t nat -A PREROUTING -p tcp -d 169.254.170.2 --dport 80 -j DNAT --to-destination 127.0.0.1:51679 ecs_task_roles2: command: "iptables -t nat -A OUTPUT -d 169.254.170.2 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 51679" save_iptables: command: "service iptables save" yum: ecs-banner: [] ecs-init: [] Updating EC2 Instances in an Auto Scaling Group

LaunchConfiguration: Type: AWS::AutoScaling::LaunchConfiguration Properties: ... Metadata: AWS::CloudFormation::Init: config: ... files: "/etc/sysctl.d/10-route_localnet.conf": mode: "0444" owner: root group: root content: "net.ipv4.conf.all.route_localnet = 1" "/etc/sysctl.d/10-more_ports.conf": mode: "0444" owner: root group: root content: "net.ipv4.ip_local_port_range = 1024 65000" Updating EC2 Instances in an Auto Scaling Group

"/etc/ecs/ecs.config": mode: 000644 owner: root group: root # http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html # and https://github.com/aws/amazon-ecs-agent#environment-variables content: !Sub | # http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html ECS_CLUSTER=${Cluster} ECS_ENGINE_AUTH_TYPE=${ECSEngineAuthType} ECS_AVAILABLE_LOGGING_DRIVERS=["json-file","awslogs","syslog","journald","gelf","fluentd","splunk"] ECS_ENABLE_TASK_IAM_ROLE=true ECS_DISABLE_PRIVILEGED=true ECS_INSTANCE_ATTRIBUTES={"env":"${EnvironmentName}"} ECS_ENGINE_TASK_CLEANUP_WAIT_DURATION=10m # http://docs.aws.amazon.com/AmazonECS/latest/developerguide/automated_image_cleanup.html ECS_IMAGE_CLEANUP_INTERVAL=10m ECS_IMAGE_MINIMUM_CLEANUP_AGE=15m ECS_NUM_IMAGES_DELETE_PER_CYCLE=10 AWS_DEFAULT_REGION=${AWS::Region} Updating EC2 Instances in an Auto Scaling Group

"/etc/cfn/cfn-hup.conf": mode: 000400 owner: root group: root content: !Sub | [main] stack=${AWS::StackId} region=${AWS::Region} "/etc/cfn/hooks.d/cfn-auto-reloader.conf": content: !Sub | [cfn-auto-reloader-hook] triggers=post.update path=Resources.LaunchConfiguration.Metadata.AWS::CloudFormation::Init action=/opt/aws/bin/cfn-init -v --region ${AWS::Region} --stack ${AWS::StackName} --resource LaunchCon Updating EC2 Instances in an Auto Scaling Group The cfn-hup helper is a daemon that detects changes in resource metadata and runs user-specified actions when a change is detected. This allows you to make configuration updates on your running Amazon EC2 instances through the UpdateStack API action.

Updating EC2 Instances in an Auto Scaling Group services: upstart: ecs: enabled: true ensureRunning: true files: - /etc/init/ecs.conf - /etc/ecs/ecs.config sysvinit: cfn-hup: enabled: true ensureRunning: true files: - /etc/cfn/cfn-hup.conf - /etc/cfn/hooks.d/cfn-auto-reloader.conf

No content

CloudFormation is a tool for repeatable and predictable automation of cloud infrastructure provisioning

We invite you to join Operations Israel Facebook group on on.fb.me/Ops-IL Thank you! www.prodops.io