Ansible vs CloudFormation Smackdown!

D02b5e0b9a348fa8d9bd2cac025eead3?s=47 Steven Ringo
October 21, 2015

Ansible vs CloudFormation Smackdown!

A quick comparison of Ansible and CloudFormation followed by some best practices for CloudFormation.

Accompanying code at https://github.com/stevenringo/cloudformation-ruby-ansible-examples

D02b5e0b9a348fa8d9bd2cac025eead3?s=128

Steven Ringo

October 21, 2015
Tweet

Transcript

  1. 3.

    What is CloudFormation? “...an easy way to create and manage

    a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion.” https://aws.amazon.com/cloudformation/
  2. 7.

    |--------------|---------------|----------------| | Feature | Ansible | CloudFormation | |==============|===============|================| |

    Written in | YAML | JSON | |--------------|---------------|----------------| | Grouped as | Playbook | Stack | |--------------|---------------|----------------| | State | Stateless | Keeps state | |--------------|---------------|----------------| | Deletion | Manual | Automatic | |--------------|---------------|----------------| | Dependencies | Manual | Automatic | |--------------|---------------|----------------| | Ordering | Top to bottom | Simultaneous | |--------------|---------------|----------------|
  3. 8.

    |--------------------|------------------|-----------------| | Feature | Ansible | CloudFormation | |====================|==================|=================| |

    UI | CLI/Tower | CLI/AWS Console | |--------------------|------------------|-----------------| | Reusability | Roles / includes | Nested Stacks | |--------------------|------------------|-----------------| | Rollbacks | No | Yes | |--------------------|------------------|-----------------| | Existing resources | Can incorporate | No | |--------------------|------------------|-----------------| | AWS Services | Limited | Most | |--------------------|------------------|-----------------| | Ease | Easier | Harder | |--------------------|------------------|-----------------| | Triggers/Policies | No | Yes | |--------------------|------------------|-----------------|
  4. 10.
  5. 15.

    God stack — Single stack for all resources — Good

    for small deployments — Size limits — Reuse only by using a generator tool
  6. 16.

    Nested stacks ┌─────────────────┐ │ │ │ │ │ Solution │

    │ │ │ │ └─────────────────┘ │ ┌───────────────────┬─────────┴─────────┬───────────────────┐ ▼ ▼ ▼ ▼ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ VPC │ │ Database │ │ Servers │ │ Security Groups │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘
  7. 17.

    Nested stacks — CloudFormation invokes child templates — Atomic execution

    — One fails, all fail — Pass parameters through "references" — Need to pass parameters for child template "through" the parent
  8. 18.

    Pipelined stacks Executor ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ 1 │

    │ 2 │ │ 3 │ │ 4 │ │ │ │ │ │ │ │ │ │ VPC │ │ Database │ │ Servers │ │ Security Groups │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘
  9. 19.

    Pipelined stacks — Execute each template manually, requires orchestration —

    Non-atomic execution — One fails, only that template fails — Pass parameters directly — Chain outputs to inputs
  10. 21.

    Horizontal ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │ │ Database │

    │ Servers │ │ Security Groups │ │ │ ├─────────────────┤ ├─────────────────┤ ├─────────────────┤ │ │ │ Application 1 │ │ Application 1 │ │ Application 1 │ │ VPC │ │ │ ├─────────────────┤ ├─────────────────┤ │ │ ├─────────────────┤ │ Application 2 │ │ Application 2 │ │ │ │ Application 2 │ ├─────────────────┤ ├─────────────────┤ │ │ │ │ │ Application 3 │ │ Application 3 │ └─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘
  11. 22.

    Vertical ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │ │ Application 1

    │ │ Application 2 │ │ Application 3 │ │ │ ├─────────────────┤ ├─────────────────┤ ├─────────────────┤ │ │ │ Servers │ │ Servers │ │ │ │ VPC │ ├─────────────────┤ ├─────────────────┤ │ Servers │ │ │ │ Database │ │ Database │ │ │ │ │ ├─────────────────┤ ├─────────────────┤ ├─────────────────┤ │ │ │ Security Groups │ │ Security Groups │ │ Security Groups │ └─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘
  12. 23.

    Partitioning — Keep templates simple — Reuse parameters and mappings

    via a DSL — Choose vertical or horizontal scaling — Decide if you want to rollback the stack or not
  13. 25.

    Shell — Simple batch script — Use aws-cli to invoke

    CloudFormation templates: aws cloudformation create-stack... — Capture output variables for input variables for next stack — Returns immediately, therefore must check status: aws cloudformation create-describe- stacks...
  14. 26.

    Ansible - name: launch ansible cloudformation example cloudformation: stack_name: ansible-cloudformation

    state: present region: us-east-1 disable_rollback: true template: files/cloudformation-example.json template_parameters: KeyName: jmartin DiskType: ephemeral InstanceType: m1.small ClusterSize: 3 tags: Stack: ansible-cloudformation
  15. 27.

    Ansible - include: key_pairs.yml tags: always - include: compile.yml tags:

    always - include: vpc.yml tags: always - include: nat.yml tags: nat - include: titan.yml tags: titan
  16. 28.

    Ansible - name: delete old cloudformation templates file: path: "{{

    item }}" state: absent with_fileglob: ../cloudformation/compiled/*.json when: clean is defined and clean|bool - name: compile the cloudformation templates shell: "{{ item }} expand --nopretty" with_fileglob: ../cloudformation/dsl/*.rb register: compiled - name: write and validate templates template: src: templates/cloudformation.json dest: cloudformation/compiled/{{ item.item | basename | regex_replace('\.rb$', '.json') }} validate: aws cloudformation validate-template --region {{ region }} --template-body file://%s environment: AWS_ACCESS_KEY_ID: "{{ aws_credentials[env].access_key }}" AWS_SECRET_ACCESS_KEY: "{{ aws_credentials[env].secret_key }}" with_items: compiled.results
  17. 29.

    Ansible - name: execute vpc cloudformation template cloudformation: aws_access_key: "{{

    aws_credentials[env].access_key }}" aws_secret_key: "{{ aws_credentials[env].secret_key }}" stack_name: vpc-{{ env }} region: "{{ region }}" template: ./cloudformation/compiled/vpc.json disable_rollback: true template_parameters: Environment: "{{ env }}" tags: Environment: "{{ env }}" Region: "{{ region }}" register: vpc
  18. 30.

    Ansible - name: execute titan cloudformation template cloudformation: aws_access_key: "{{

    aws_credentials[env].access_key }}" aws_secret_key: "{{ aws_credentials[env].secret_key }}" stack_name: titan-{{ env }} region: "{{ region }}" template: ./cloudformation/compiled/titan.json disable_rollback: true template_parameters: Environment: "{{ env }}" CochlearLinkHostedZone: "{{ vpc.stack_outputs.HostedZone }}" CochlearLinkVpc: "{{ vpc.stack_outputs.Vpc }}" TitanSubnets: "{{ vpc.stack_outputs.TitanSubnets }}" tags: Environment: "{{ env }}" Region: "{{ region }}" register: titan
  19. 33.

    { "ap-southeast-2": { "dev": { "productVersion": "3.0.0", "installationId": "eeeeeeee-f057-40c7-b365-792cc7faeccb" },

    "sit": { "productVersion": "3.1.8", "installationId": "17171717-f057-40c7-b365-792cc7faeccb" } }, "us-east-1": { "prd": { "productVersion": "2.1.0", "installationId": "e047ac6f-f057-40c7-b365-792cc7faeccb" } } }
  20. 35.

    { "versions": { "3.1.8": { "ap-southeast-2": { "image": "ami-9f266aa5" },

    "eu-west-1": { "image": "ami-0914307e" } }, "3.1.2": { "ap-southeast-2": { "image": "ami-170e422d" }, "eu-west-1": { "image": "ami-5dbf9a2a" } } } }
  21. 39.

    The dreaded _FAILED states — Resolve manually — Console will

    give you indication of which resource failed to update — Delete offending resource or its dependencies manually — Run delete stack again — When in real shit, call support
  22. 42.
  23. 44.

    Creating the templates — Use CloudFormer to instrospect your VPC.

    — http://docs.aws.amazon.com/AWSCloudFormation/ latest/UserGuide/cfn-using-cloudformer.html — http://aws.amazon.com/developertools/ 6460180344805680 — Uses a Micro instance inside your VPC to interrogate and dumps results to S3 — Parse with the ruby cloudformation-ruby-
  24. 45.

    Incorporating templates — Resources must be provisioned by CloudFormation —

    Use a phased approach. Use Ref to refer to existing resources, passed as parameters. — Usually in reverse of your pipeline — machines first, VPCs last — e.g. "VpcId" : { "Ref" : "VpcId" }
  25. 46.

    Policies — Use DependsOn and WaitConditions to control dependency creation

    — Use UpdatePolicy for ASG changes. — Use DeletionPolicy to ensure are not deleted by CloudFormation (instance termination protection on steroids) — Note the Update requires property in the documentation
  26. 48.
  27. 49.

    Other — Test deploys in a sandbox environment that mirrors

    production. Smaller resource types can help reduce costs. — Parameterise everything. — Use the http://docs.aws.amazon.com/ AWSCloudFormation/latest/UserGuide/template- reference.html
  28. 50.