CloudFormation tips, tricks & best practices

D02b5e0b9a348fa8d9bd2cac025eead3?s=47 Steven Ringo
September 02, 2015

CloudFormation tips, tricks & best practices

Tips and tricks on using complementary tools to help you create and manage CloudFormation templates.

Includes discussion on Ansible and Ruby and using them with CloudFormation.

Organising and partitioning stacks

Getting out of trouble

Migrations

D02b5e0b9a348fa8d9bd2cac025eead3?s=128

Steven Ringo

September 02, 2015
Tweet

Transcript

  1. 2.

    “...an easy way to create and manage a collection of

    related AWS resources, provisioning and updating them in an orderly and predictable fashion.”
  2. 8.

    God stack • Single stack for all resources • Good

    for small deployments • Size limits • Reuse only by using a generator tool
  3. 9.

    Nested stacks ┌─────────────────┐ │ │ │ │ │ Solution │

    │ │ │ │ └─────────────────┘ │ ┌───────────────────┬─────────┴─────────┬───────────────────┐ ▼ ▼ ▼ ▼ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ VPC │ │ Database │ │ Servers │ │ Security Groups │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘
  4. 10.

    Nested stacks • CloudFormation invokes child templates • Atomic execution

    • One fails, all fail • Pass parameters through "references" • Need to pass parameters for child template "through" the parent
  5. 11.

    Pipelined stacks Executor ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ 1 │

    │ 2 │ │ 3 │ │ 4 │ │ │ │ │ │ │ │ │ │ VPC │ │ Database │ │ Servers │ │ Security Groups │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘
  6. 12.

    Pipelined stacks • Execute each template manually, requires orchestration •

    Non-atomic execution • One fails, only that template fails • Pass parameters directly • Chain outputs to inputs
  7. 14.

    Horizontal ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │ │ Database │

    │ Servers │ │ Security Groups │ │ │ ├─────────────────┤ ├─────────────────┤ ├─────────────────┤ │ │ │ Application 1 │ │ Application 1 │ │ Application 1 │ │ VPC │ │ │ ├─────────────────┤ ├─────────────────┤ │ │ ├─────────────────┤ │ Application 2 │ │ Application 2 │ │ │ │ Application 2 │ ├─────────────────┤ ├─────────────────┤ │ │ │ │ │ Application 3 │ │ Application 3 │ └─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘
  8. 15.

    Vertical ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │ │ Application 1

    │ │ Application 2 │ │ Application 3 │ │ │ ├─────────────────┤ ├─────────────────┤ ├─────────────────┤ │ │ │ Servers │ │ Servers │ │ │ │ VPC │ ├─────────────────┤ ├─────────────────┤ │ Servers │ │ │ │ Database │ │ Database │ │ │ │ │ ├─────────────────┤ ├─────────────────┤ ├─────────────────┤ │ │ │ Security Groups │ │ Security Groups │ │ Security Groups │ └─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘
  9. 16.

    Partitioning • Keep templates simple • Reuse parameters and mappings

    via a DSL • Choose vertical or horizontal scaling • Decide if you want to rollback the stack or not
  10. 18.

    Shell • Simple batch script • Use aws-cli to invoke

    CloudFormation templates: aws cloudformation create-stack... • Capture output variables for input variables for next stack • Returns immediately, therefore must check status: aws cloudformation create-describe-stacks...
  11. 19.

    Ansible - name: launch ansible cloudformation example cloudformation: stack_name: ansible-cloudformation

    state: present region: us-east-1 disable_rollback: true template: files/cloudformation-example.json template_parameters: KeyName: jmartin DiskType: ephemeral InstanceType: m1.small ClusterSize: 3 tags: Stack: ansible-cloudformation
  12. 20.

    Ansible - include: key_pairs.yml tags: always - include: compile.yml tags:

    always - include: vpc.yml tags: always - include: nat.yml tags: nat - include: titan.yml tags: titan
  13. 21.

    Ansible - name: delete old cloudformation templates file: path: "{{

    item }}" state: absent with_fileglob: ../cloudformation/compiled/*.json when: clean is defined and clean|bool - name: compile the cloudformation templates shell: "{{ item }} expand --nopretty" with_fileglob: ../cloudformation/dsl/*.rb register: compiled - name: write and validate templates template: src: templates/cloudformation.json dest: cloudformation/compiled/{{ item.item | basename | regex_replace('\.rb$', '.json') }} validate: aws cloudformation validate-template --region {{ region }} --template-body file://%s environment: AWS_ACCESS_KEY_ID: "{{ aws_credentials[env].access_key }}" AWS_SECRET_ACCESS_KEY: "{{ aws_credentials[env].secret_key }}" with_items: compiled.results
  14. 22.

    Ansible - name: execute vpc cloudformation template cloudformation: aws_access_key: "{{

    aws_credentials[env].access_key }}" aws_secret_key: "{{ aws_credentials[env].secret_key }}" stack_name: vpc-{{ env }} region: "{{ region }}" template: ./cloudformation/compiled/vpc.json disable_rollback: true template_parameters: Environment: "{{ env }}" tags: Environment: "{{ env }}" Region: "{{ region }}" register: vpc
  15. 23.

    Ansible - name: execute titan cloudformation template cloudformation: aws_access_key: "{{

    aws_credentials[env].access_key }}" aws_secret_key: "{{ aws_credentials[env].secret_key }}" stack_name: titan-{{ env }} region: "{{ region }}" template: ./cloudformation/compiled/titan.json disable_rollback: true template_parameters: Environment: "{{ env }}" CochlearLinkHostedZone: "{{ vpc.stack_outputs.HostedZone }}" CochlearLinkVpc: "{{ vpc.stack_outputs.Vpc }}" TitanSubnets: "{{ vpc.stack_outputs.TitanSubnets }}" tags: Environment: "{{ env }}" Region: "{{ region }}" register: titan
  16. 26.

    { "ap-southeast-2": { "dev": { "productVersion": "3.0.0", "installationId": "eeeeeeee-f057-40c7-b365-792cc7faeccb" },

    "sit": { "productVersion": "3.1.8", "installationId": "17171717-f057-40c7-b365-792cc7faeccb" } }, "us-east-1": { "prd": { "productVersion": "2.1.0", "installationId": "e047ac6f-f057-40c7-b365-792cc7faeccb" } } }
  17. 28.

    { "versions": { "3.1.8": { "ap-southeast-2": { "image": "ami-9f266aa5" },

    "eu-west-1": { "image": "ami-0914307e" } }, "3.1.2": { "ap-southeast-2": { "image": "ami-170e422d" }, "eu-west-1": { "image": "ami-5dbf9a2a" } } } }
  18. 32.

    The dreaded _FAILED states • Resolve manually • Console will

    give you indication of which resource failed to update • Delete offending resource or its dependencies manually • Run delete stack again • When in real shit, call support
  19. 35.
  20. 37.

    Creating the templates • Use CloudFormer to instrospect your VPC.

    • http://docs.aws.amazon.com/AWSCloudFormation/latest/ UserGuide/cfn-using-cloudformer.html • http://aws.amazon.com/developertools/6460180344805680 • Uses a Micro instance inside your VPC to interrogate and dumps results to S3 • Parse with the ruby cloudformation-ruby-dsl :-)
  21. 38.

    Incorporating templates • Resources must be provisioned by CloudFormation •

    Use a phased approach. Use Ref to refer to existing resources, passed as parameters. • Usually in reverse of your pipeline — machines first, VPCs last • e.g. "VpcId" : { "Ref" : "VpcId" }
  22. 39.

    Policies • Use DependsOn and WaitConditions to control dependency creation

    • Use UpdatePolicy for ASG changes. • Use DeletionPolicy to ensure are not deleted by CloudFormation (instance termination protection on steroids) • Note the Update requires property in the documentation
  23. 41.
  24. 42.

    Other • Test deploys in a sandbox environment that mirrors

    production. Smaller resource types can help reduce costs. • Parameterise everything. • Use the http://docs.aws.amazon.com/AWSCloudFormation/ latest/UserGuide/template-reference.html
  25. 43.