Ansible & Docker - The Path to Continuous Delivery - Part 1

Ansible & Docker - The Path to Continuous Delivery - Part 1

The aim of this series is to have an end-to-end process for continuously delivering a Rails application with MySQL and Redis dependencies in production, as a series of Docker containers, across multiple physical hosts. I will be able to preview features from git branches, in production, without affecting the active production instances. It goes without saying, all GitHub commits will trigger builds, as one would expect from a real Continuous Delivery setup.

Presented at London Docker Meetup 29th January 2014

http://gerhard.lazu.co.uk/ansible-docker-the-path-to-continuous-delivery-1

6e1b896f6c4807aa4223acc428c57760?s=128

Gerhard Lazu

January 29, 2014
Tweet

Transcript

  1. ANSIBLE & DOCKER The Path to Continuous Delivery Part 1

  2. Demi ➜ infrastructure git:(master) bin/ap create_droplet.yml hosts -c local PLAY

    [Digital Ocean] ********************************************************** GATHERING FACTS *************************************************************** TASK: [Create DigitalOcean droplet] ******************************************* PLAY RECAP ******************************************************************** localhost : ok=1 changed=1 unreachable=0 failed=0
  3. 55” 1’ 25” Create DO 2GB AMS2 droplet

  4. Don’t write cookbooks, just play

  5. Changed. Skipped. OK.

  6. Demi ➜ infrastructure git:(master) bin/ap dod.yml dohosts -u root PLAY

    [Digital Ocean Docker] *************************************************** GATHERING FACTS *************************************************************** TASK: [base | Install common system packages via apt] ************************* TASK: [base | NTP client] ***************************************************** TASK: [base | Sudo group] ***************************************************** TASK: [base | Docker group] *************************************************** TASK: [base | Create gerhard user] ******************************************** TASK: [base | gerhard SSH key] ************************************************ TASK: [base | root SSH key] *************************************************** TASK: [base | Sudoers don't need a password] ********************************** TASK: [base | Sudoers have their ssh agent forwarded] ************************* TASK: [base | Add github.com to known hosts] ********************************** TASK: [base | Make vim the default editor] ************************************ TASK: [base | Common bash aliases] ******************************************** TASK: [docker | Add docker apt repository key] ******************************** TASK: [docker | Add docker apt repository] ************************************ TASK: [docker | Install latest docker] **************************************** TASK: [docker | Start docker on boot] ***************************************** TASK: [docker | Docker aliases] *********************************************** TASK: [nginx | Add nginx PPA repository] ************************************** TASK: [nginx | Install nginx] ************************************************* TASK: [nginx | Customize nginx default configuration] ************************* TASK: [nginx | Start nginx on boot] ******************************************* PLAY RECAP ******************************************************************** 95.85.53.185 : ok=2 changed=23 unreachable=0 failed=0
  7. 1’ 50” Base system setup

  8. nginx As special as dockerd

  9. Demi ➜ infrastructure git:(master) bin/ap terrabox.yml dohosts -l docker-ams2 PLAY

    [terrabox app - Rails] *************************************************** GATHERING FACTS *************************************************************** TASK: [terrabox | Update app repository] ************************************** TASK: [terrabox | Dependent containers] *************************************** TASK: [terrabox | Check MySQL data container] ********************************* TASK: [terrabox | Build MySQL data container] ********************************* TASK: [terrabox | Check MySQL container status] ******************************* TASK: [terrabox | Remove MySQL container with same name] ********************** TASK: [terrabox | Start new MySQL container] ********************************** TASK: [terrabox | Check Redis container status] ******************************* TASK: [terrabox | Remove Redis container with same name] ********************** TASK: [terrabox | Start new Redis container] ********************************** TASK: [terrabox | Ensure we have the latest app master image] ***************** TASK: [terrabox | Ensure app branch image builds from master image] *********** TASK: [terrabox | Ensure app branch image exists] ***************************** TASK: [terrabox | Build app branch image] ************************************* TASK: [terrabox | Ensure all tests pass] ************************************** TASK: [terrabox | Check app container status] ********************************* TASK: [terrabox | Remove app container if stopped (name collision)] *********** TASK: [terrabox | Start new app container] ************************************ TASK: [terrabox | Find app container public port] ***************************** TASK: [terrabox | Update nginx production vhost ] ***************************** TASK: [terrabox | Update DNS production record ] ****************************** TASK: [terrabox | Notify HipChat of new production container ] **************** TASK: [terrabox | Reload nginx ] ********************************************** PLAY RECAP ******************************************************************** 95.85.53.185 : ok=5 changed=18 unreachable=0 failed=0
  10. 8’ 16” Rails, MySQL & Redis containers (no local Docker

    images)
  11. Dependent containers Branch-specific

  12. Test-only containers Hello CI

  13. Demi ➜ infrastructure git:(master) bin/ap terrabox.yml dohosts -l docker-ams2 -e

    ‘app_branch=gem_updates’ PLAY [terrabox app - Rails] *************************************************** GATHERING FACTS *************************************************************** TASK: [terrabox | Update app repository] ************************************** TASK: [terrabox | Dependent containers] *************************************** TASK: [terrabox | Check MySQL data container] ********************************* TASK: [terrabox | Build MySQL data container] ********************************* TASK: [terrabox | Check MySQL container status] ******************************* TASK: [terrabox | Remove MySQL container with same name] ********************** TASK: [terrabox | Start new MySQL container] ********************************** TASK: [terrabox | Check Redis container status] ******************************* TASK: [terrabox | Remove Redis container with same name] ********************** TASK: [terrabox | Start new Redis container] ********************************** TASK: [terrabox | Ensure we have the latest app master image] ***************** TASK: [terrabox | Ensure app branch image builds from master image] *********** TASK: [terrabox | Ensure app branch image exists] ***************************** TASK: [terrabox | Build app branch image] ************************************* TASK: [terrabox | Ensure all tests pass] ************************************** TASK: [terrabox | Check app container status] ********************************* TASK: [terrabox | Remove app container if stopped (name collision)] *********** TASK: [terrabox | Start new app container] ************************************ TASK: [terrabox | Find app container public port] ***************************** TASK: [terrabox | Update nginx branch vhost ] ********************************* TASK: [terrabox | Update DNS branch record ] ********************************** TASK: [terrabox | Notify HipChat of new production container ] **************** TASK: [terrabox | Reload nginx ] ********************************************** PLAY RECAP ******************************************************************** 95.85.53.185 : ok=4 changed=19 unreachable=0 failed=0
  14. 2’ 23” Rails, MySQL & Redis containers (with local Docker

    images)
  15. 3.46x Rails, MySQL & Redis containers (with local Docker images)

  16. ENVIRONMENTS How many do I really need?

  17. PRODUCTION

  18. PRODUCTION Development

  19. PRODUCTION Development Integration

  20. PRODUCTION Development Staging Integration

  21. PRODUCTION Development Staging Integration QA

  22. PRODUCTION Development Staging Integration Beta QA

  23. PRODUCTION Development Staging Integration Beta QA Mooo

  24. [branch].[fqdn] NOT Production

  25. [fqdn] Production

  26. git branch mentality K.I.S.S.

  27. master == production

  28. branch != production

  29. THINGS I LIKE

  30. container == process Single Responsibility Principle

  31. container != VM Single Responsibility Principle

  32. Data-only containers Single Responsibility Principle

  33. Dependent containers .docker.[service]

  34. Test-only containers Docker, the simple CI

  35. DNS service discovery Instant DNS updates are real

  36. Chat notifications Tell me when it’s done

  37. WHAT’S NEXT?

  38. Clean-up Old containers & images

  39. Private registry Build images once, re-use

  40. Data-only containers Back-up & Restore

  41. Previous instances Backup upstream entries

  42. GitHub hooks Real Continuous Delivery

  43. Container supervision exec it

  44. AND AFTER THAT?

  45. Multi-host Redundancy & Scalability

  46. Chat ops Everyone on the team can do it

  47. gerhardlazu Thank you