Slide 1

Slide 1 text

AWS CloudFormation

Slide 2

Slide 2 text

Evgeny Zislis

Slide 3

Slide 3 text

{ "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "EC2 Instance and a Security Group", "Parameters" : { "KeyName": { "Description" : "An existing EC2 KeyPair to enable SSH access", "Type": "AWS::EC2::KeyPair::KeyName" }, "InstanceType" : { "Description" : "EC2 instance type", "Type" : "String", "Default" : "t2.small", "AllowedValues" : [ "t2.small", "t2.medium", "t2.large"] }, "SSHLocation" : { "Description" : "CIDR from where SSH access is allowed", "Type": "String", "MinLength": "9", "MaxLength": "18", "Default": "0.0.0.0/0", "AllowedPattern": "(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})/(\\d{1,2})" } ...

Slide 4

Slide 4 text

"Resources" : { "EC2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "InstanceType" : { "Ref" : "InstanceType" }, "SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ], "KeyName" : { "Ref" : "KeyName" }, "ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" }, { "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" }, "Arch" ] } ] } } }, "InstanceSecurityGroup" : { "Type" : "AWS::EC2::SecurityGroup", "Properties" : { "GroupDescription" : "Enable SSH access via port 22", "SecurityGroupIngress" : [ { "IpProtocol" : "tcp", "FromPort" : "22", "ToPort" : "22", "CidrIp" : { "Ref" : "SSHLocation"} } ] } }

Slide 5

Slide 5 text

/prəˈviZHən/ supply with equipment the IT manager was responsible to provision servers for the production release pro·vi·sion

Slide 6

Slide 6 text

/ˈôdəˌmāt/ convert a process to reduce human intervention to a minimum jobs that used to be done by people are now able to be done through automation au·to·mate

Slide 7

Slide 7 text

/rəˈpēdəb(ə)l/ able to be done again repeatable experiments under laboratory conditions re·peat·a·ble

Slide 8

Slide 8 text

/prəˈdiktəb(ə)l/ behaving or occurring in a way that is expected. unsurprising. Murphy's unpredictable rule predictably hits you pre·dict·a·ble

Slide 9

Slide 9 text

CloudFormation is a tool for repeatable and predictable automation of cloud infrastructure provisioning

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

Create a VPC aws --region us-east-1 \ cloudformation create-stack \ --stack-name VPC \ --template-url https://aws-quickstart.s3.amazonaws.com/quickstart-aws-vpc/templates/aws-vpc.template \ --parameters \ ParameterKey=KeyPairName,ParameterValue=evgeny-ssh \ ParameterKey=AvailabilityZones,ParameterValue="'us-east-1a,us-east-1b'" { "StackId": "arn:aws:cloudformation:us-east-1:675783095585:stack/VPC/76271620-d0f8-11e8-b3a0-500c28b04cd1" } aws.amazon.com/quickstart/architecture/vpc

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

Nested Stacks CloudFormation

Slide 14

Slide 14 text

Using Nested Stacks Resources: NestedStack: Type: AWS::CloudFormation::Stack Properties: TemplateURL: !Ref NestedStackS3File InstanceType: !Ref InstanceType Outputs: NestedStackBucketName: Value: !GetAtt NestedStack.InstanceName

Slide 15

Slide 15 text

docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/troubleshooting.html

Slide 16

Slide 16 text

Never, ever, should you actually use Nested Stacks because Nested Stacks are Stuck in UPDATE_COMPLETE_CLEANUP_IN_PROGRESS, UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS, or UPDATE_ROLLBACK_IN_PROGRESS A nested stack failed to roll back. Because of potential resource dependencies between nested stacks, AWS CloudFormation doesn't start cleaning up nested stack resources until all nested stacks have been updated or have rolled back. When a nested stack fails to roll back, AWS CloudFormation cancels all operations, regardless of the state that the other nested stacks are in. A nested stack that completed updating or rolling back but did not receive a signal from AWS CloudFormation to start cleaning up because another nested failed to roll back is in an UPDATE_COMPLETE_CLEANUP_IN_PROGRESS or UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS state. A nested stack that failed to update but did not receive a signal to start rolling back is in an UPDATE_ROLLBACK_IN_PROGRESS state. A nested stack might fail to roll back because of changes that were made outside of AWS CloudFormation, when the stack template doesn't accurately reflect the state of the stack. A nested stack might also fail if an Auto Scaling group in a nested stack had an insufficient resource signal timeout period when the group was created or updated. To fix the stack, contact AWS customer support. WTF? docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/troubleshooting.html

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

“no one knows for sure who erected these monoliths and why”

Slide 19

Slide 19 text

Exporting & Importing CloudFormation

Slide 20

Slide 20 text

Stack B: Resources: EC2Instance: Type: AWS::EC2::Instance Properties: NetworkInterfaces: - AssociatePublicIpAddress: true DeviceIndex: 0 GroupSet: - Fn::ImportValue: Global-SecurityGroup Stack A: Outputs: SecurityGroup: Value: !Ref SecurityGroup Description: ID of security group Export: Name: Global-SecurityGroup Exported Output and Imported Value

Slide 21

Slide 21 text

Exported Value Dependencies

Slide 22

Slide 22 text

Custom Resources CloudFormation

Slide 23

Slide 23 text

Custom Resources - Definition and usage Resources: CustomCWEventsLambda: Type: AWS::Lambda::Function Properties: Runtime: nodejs8.10 Handler: index.handler Role: !GetAtt LambdaExecutionRole.Arn Code: ZipFile: > const Response = require('cfn-response'); exports.handler = function handler(event, context) { Response.send(event, context, Response.FAILED, {}); throw new Error('Not Implemented'); } CloudWatchEventsRule: Type: Custom::Events::Rule Parameters: ServiceToken: !GetAtt CustomCWEventsLambda.Arn Name: 'STRING_VALUE' - A name for this rule. *optional* Description: 'STRING_VALUE' - A description of the rule. State: 'ENABLED' - Indicates whether the rule is enabled or disabled. from github.com/devops-israel/aws-cf-custom-events

Slide 24

Slide 24 text

Custom Resources - Implementation const Response = require('cfn-response'); const log = { info(...args) { console.log(...args) }, error(...args) { console.error(...args) }, } exports.handler = function handler(event, context) { log.info(event, context); switch (event.RequestType) { case 'Create': return createResource(event, context); case 'Delete': return deleteResource(event, context); case 'Update': return updateResource(event, context); default: log.error(new Error(`ERROR: Unknown event.RequestType provided: ${event.RequestType}`)); Response.send(event, context, Response.FAILED, {}); } } from github.com/devops-israel/aws-cf-custom-events

Slide 25

Slide 25 text

cfn-init CloudFormation

Slide 26

Slide 26 text

Resources: AutoScalingGroup: Type: AWS::AutoScaling::AutoScalingGroup Properties: LaunchConfigurationName: !Ref LaunchConfiguration CreationPolicy: ResourceSignal: Timeout: PT15M LaunchConfiguration: Type: AWS::AutoScaling::LaunchConfiguration Properties: UserData: Fn::Base64: !Sub | #cloud-config packages: - aws-cfn-bootstrap bootcmd: - "while ! ping -i 10 -I eth0 -n -w 60 -W 2 -c 1 -q s3.amazonaws.com; do sleep 1; done" runcmd: - /opt/aws/bin/cfn-init -v --region ${AWS::Region} --stack ${AWS::StackName} --resource LaunchConfiguration - /opt/aws/bin/cfn-signal -e $? --region ${AWS::Region} --stack ${AWS::StackName} --resource AutoScalingGrou Updating EC2 Instances in an Auto Scaling Group

Slide 27

Slide 27 text

LaunchConfiguration: Type: AWS::AutoScaling::LaunchConfiguration Properties: ... Metadata: AWS::CloudFormation::Init: config: commands: ecs_task_sysctl1: command: "sysctl -w net.ipv4.conf.all.route_localnet=1" ecs_task_sysctl2: command: "sysctl -w net.ipv4.ip_local_port_range='1024 65000'" ecs_task_roles1: command: "iptables -t nat -A PREROUTING -p tcp -d 169.254.170.2 --dport 80 -j DNAT --to-destination 127.0.0.1:51679 ecs_task_roles2: command: "iptables -t nat -A OUTPUT -d 169.254.170.2 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 51679" save_iptables: command: "service iptables save" yum: ecs-banner: [] ecs-init: [] Updating EC2 Instances in an Auto Scaling Group

Slide 28

Slide 28 text

LaunchConfiguration: Type: AWS::AutoScaling::LaunchConfiguration Properties: ... Metadata: AWS::CloudFormation::Init: config: ... files: "/etc/sysctl.d/10-route_localnet.conf": mode: "0444" owner: root group: root content: "net.ipv4.conf.all.route_localnet = 1" "/etc/sysctl.d/10-more_ports.conf": mode: "0444" owner: root group: root content: "net.ipv4.ip_local_port_range = 1024 65000" Updating EC2 Instances in an Auto Scaling Group

Slide 29

Slide 29 text

"/etc/ecs/ecs.config": mode: 000644 owner: root group: root # http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html # and https://github.com/aws/amazon-ecs-agent#environment-variables content: !Sub | # http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html ECS_CLUSTER=${Cluster} ECS_ENGINE_AUTH_TYPE=${ECSEngineAuthType} ECS_AVAILABLE_LOGGING_DRIVERS=["json-file","awslogs","syslog","journald","gelf","fluentd","splunk"] ECS_ENABLE_TASK_IAM_ROLE=true ECS_DISABLE_PRIVILEGED=true ECS_INSTANCE_ATTRIBUTES={"env":"${EnvironmentName}"} ECS_ENGINE_TASK_CLEANUP_WAIT_DURATION=10m # http://docs.aws.amazon.com/AmazonECS/latest/developerguide/automated_image_cleanup.html ECS_IMAGE_CLEANUP_INTERVAL=10m ECS_IMAGE_MINIMUM_CLEANUP_AGE=15m ECS_NUM_IMAGES_DELETE_PER_CYCLE=10 AWS_DEFAULT_REGION=${AWS::Region} Updating EC2 Instances in an Auto Scaling Group

Slide 30

Slide 30 text

"/etc/cfn/cfn-hup.conf": mode: 000400 owner: root group: root content: !Sub | [main] stack=${AWS::StackId} region=${AWS::Region} "/etc/cfn/hooks.d/cfn-auto-reloader.conf": content: !Sub | [cfn-auto-reloader-hook] triggers=post.update path=Resources.LaunchConfiguration.Metadata.AWS::CloudFormation::Init action=/opt/aws/bin/cfn-init -v --region ${AWS::Region} --stack ${AWS::StackName} --resource LaunchCon Updating EC2 Instances in an Auto Scaling Group The cfn-hup helper is a daemon that detects changes in resource metadata and runs user-specified actions when a change is detected. This allows you to make configuration updates on your running Amazon EC2 instances through the UpdateStack API action.

Slide 31

Slide 31 text

Updating EC2 Instances in an Auto Scaling Group services: upstart: ecs: enabled: true ensureRunning: true files: - /etc/init/ecs.conf - /etc/ecs/ecs.config sysvinit: cfn-hup: enabled: true ensureRunning: true files: - /etc/cfn/cfn-hup.conf - /etc/cfn/hooks.d/cfn-auto-reloader.conf

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

CloudFormation is a tool for repeatable and predictable automation of cloud infrastructure provisioning

Slide 34

Slide 34 text

We invite you to join Operations Israel Facebook group on on.fb.me/Ops-IL Thank you! www.prodops.io