Engineering for Failure

In this talk, Chris Grigor and I look at how we improved our way of working with Ansible at IceMobile.

We look at our principles of "Engineering for Failure" and share our insights with the community.

This version of the talk was given at CfgMgmtCamp in February 2016.

Transcript

1. Engineering for Failure Building for the unhappy path
2. Kerim Satirli Senior Engineer, IceMobile
3. Chris Grigor Ops Manager, IceMobile
8. The Cloud (someone else’s computer somewhere, with an API)
9. Engineering for Speed “I know what you did last summer”
10. - name: Check to see if pip is already installed command: "pip --version" ignore_errors: true changed_when: false register: pip_is_installed - name: Get pip get_url: url=https://bootstrap.pypa.io/get-pip.py dest=/home/ec2-user when: pip_is_installed.rc != 0 - name: install pip command: "python /home/ec2-user/get-pip.py" sudo: yes when: pip_is_installed.rc != 0 - name: delete get-pip.py file: state=absent path=~/get-pip.py when: pip_is_installed.rc != 0 - name: Installing docker-py for ansible pip: name=docker-py version=1.1.0 #executable=/usr/local/bin/pip sudo: yes - name: Adding Docker Hub robot login copy: src=../resources/dockercfg dest=/home/ec2-user/.dockercfg owner=ec2-user
11. - name: Check to see if pip is already installed command: "pip --version" ignore_errors: true changed_when: false register: pip_is_installed - name: Get pip get_url: url=https://bootstrap.pypa.io/get-pip.py dest=/home/ec2-user when: pip_is_installed.rc != 0 - name: install pip command: "python /home/ec2-user/get-pip.py" sudo: yes when: pip_is_installed.rc != 0 - name: delete get-pip.py file: state=absent path=~/get-pip.py when: pip_is_installed.rc != 0 - name: Installing docker-py for ansible pip: name=docker-py version=1.1.0 #executable=/usr/local/bin/pip sudo: yes - name: Adding Docker Hub robot login copy: src=../resources/dockercfg dest=/home/ec2-user/.dockercfg owner=ec2-user IT WORKS!
12. Ansible 2.0 early adopter
13. Ansible 2.0 it’s called bleeding edge for a reason
14. Engineering for Failure a beginner’s guide
15. Fail early and don’t waste my time!
16. Fail clearly and tell me where it hurts!
17. Engineering for Failure live-action test
18. A tale of Vaults passwords in the cloud!
25. - include: ./notify-datadog.yml vars: event_type: error api_key: "{{ datadog.api_key }}" event_title: "Checksum mismatch for {{ name }} v{{ version }}" event_text: "{{ fetch_application_package.msg }}" event_priority: "{{ datadog.priority }}" event_tags: "app:{{ name | lower }}, \ version:{{ version }}, \ type:checksum-mismatch" when: fetch_application_package.failed is defined tags: - fhp_datadog_checksum
26. A tale of Vaults and a key-value service
27. - # validate config file include: ../../roles/validate-file/tasks/main.yml vars: validation_file: "{{ config.dirs.etc }}/{{ config.file }}" validation_type: json validation_result_var: consulconfig_validation tags: - pc_validate_configfile
28. --- - shell: "cat {{ validation_file }}" register: res - set_fact: "{{ vldtn_res_var | default(vldtn_res)}}": \ "{{ res.stdout | from_json }}" when: validation_type == "json" - set_fact: "{{ vldtn_res_var | default(vldtn_res)}}": \ "{{ res.stdout | from_yaml }}" when: validation_type == "yaml"
30. Engineering for Failure Building for the unhappy path
31. @grigorc @ksatirli
32. The mobile agency for food retail