Ansible & Docker - The Path to Continuous Delivery - Part 1

The aim of this series is to have an end-to-end process for continuously delivering a Rails application with MySQL and Redis dependencies in production, as a series of Docker containers, across multiple physical hosts. I will be able to preview features from git branches, in production, without affecting the active production instances. It goes without saying, all GitHub commits will trigger builds, as one would expect from a real Continuous Delivery setup.

Presented at London Docker Meetup 29th January 2014

http://gerhard.lazu.co.uk/ansible-docker-the-path-to-continuous-delivery-1

Transcript

1. ANSIBLE & DOCKER The Path to Continuous Delivery Part 1
2. Demi ➜ infrastructure git:(master) bin/ap create_droplet.yml hosts -c local PLAY [Digital Ocean] ********************************************************** GATHERING FACTS *************************************************************** TASK: [Create DigitalOcean droplet] ******************************************* PLAY RECAP ******************************************************************** localhost : ok=1 changed=1 unreachable=0 failed=0
3. 55” 1’ 25” Create DO 2GB AMS2 droplet
4. Don’t write cookbooks, just play
5. Changed. Skipped. OK.
6. Demi ➜ infrastructure git:(master) bin/ap dod.yml dohosts -u root PLAY [Digital Ocean Docker] *************************************************** GATHERING FACTS *************************************************************** TASK: [base | Install common system packages via apt] ************************* TASK: [base | NTP client] ***************************************************** TASK: [base | Sudo group] ***************************************************** TASK: [base | Docker group] *************************************************** TASK: [base | Create gerhard user] ******************************************** TASK: [base | gerhard SSH key] ************************************************ TASK: [base | root SSH key] *************************************************** TASK: [base | Sudoers don't need a password] ********************************** TASK: [base | Sudoers have their ssh agent forwarded] ************************* TASK: [base | Add github.com to known hosts] ********************************** TASK: [base | Make vim the default editor] ************************************ TASK: [base | Common bash aliases] ******************************************** TASK: [docker | Add docker apt repository key] ******************************** TASK: [docker | Add docker apt repository] ************************************ TASK: [docker | Install latest docker] **************************************** TASK: [docker | Start docker on boot] ***************************************** TASK: [docker | Docker aliases] *********************************************** TASK: [nginx | Add nginx PPA repository] ************************************** TASK: [nginx | Install nginx] ************************************************* TASK: [nginx | Customize nginx default configuration] ************************* TASK: [nginx | Start nginx on boot] ******************************************* PLAY RECAP ******************************************************************** 95.85.53.185 : ok=2 changed=23 unreachable=0 failed=0
7. 1’ 50” Base system setup
8. nginx As special as dockerd
9. Demi ➜ infrastructure git:(master) bin/ap terrabox.yml dohosts -l docker-ams2 PLAY [terrabox app - Rails] *************************************************** GATHERING FACTS *************************************************************** TASK: [terrabox | Update app repository] ************************************** TASK: [terrabox | Dependent containers] *************************************** TASK: [terrabox | Check MySQL data container] ********************************* TASK: [terrabox | Build MySQL data container] ********************************* TASK: [terrabox | Check MySQL container status] ******************************* TASK: [terrabox | Remove MySQL container with same name] ********************** TASK: [terrabox | Start new MySQL container] ********************************** TASK: [terrabox | Check Redis container status] ******************************* TASK: [terrabox | Remove Redis container with same name] ********************** TASK: [terrabox | Start new Redis container] ********************************** TASK: [terrabox | Ensure we have the latest app master image] ***************** TASK: [terrabox | Ensure app branch image builds from master image] *********** TASK: [terrabox | Ensure app branch image exists] ***************************** TASK: [terrabox | Build app branch image] ************************************* TASK: [terrabox | Ensure all tests pass] ************************************** TASK: [terrabox | Check app container status] ********************************* TASK: [terrabox | Remove app container if stopped (name collision)] *********** TASK: [terrabox | Start new app container] ************************************ TASK: [terrabox | Find app container public port] ***************************** TASK: [terrabox | Update nginx production vhost ] ***************************** TASK: [terrabox | Update DNS production record ] ****************************** TASK: [terrabox | Notify HipChat of new production container ] **************** TASK: [terrabox | Reload nginx ] ********************************************** PLAY RECAP ******************************************************************** 95.85.53.185 : ok=5 changed=18 unreachable=0 failed=0
10. 8’ 16” Rails, MySQL & Redis containers (no local Docker images)
11. Dependent containers Branch-specific
12. Test-only containers Hello CI
13. Demi ➜ infrastructure git:(master) bin/ap terrabox.yml dohosts -l docker-ams2 -e ‘app_branch=gem_updates’ PLAY [terrabox app - Rails] *************************************************** GATHERING FACTS *************************************************************** TASK: [terrabox | Update app repository] ************************************** TASK: [terrabox | Dependent containers] *************************************** TASK: [terrabox | Check MySQL data container] ********************************* TASK: [terrabox | Build MySQL data container] ********************************* TASK: [terrabox | Check MySQL container status] ******************************* TASK: [terrabox | Remove MySQL container with same name] ********************** TASK: [terrabox | Start new MySQL container] ********************************** TASK: [terrabox | Check Redis container status] ******************************* TASK: [terrabox | Remove Redis container with same name] ********************** TASK: [terrabox | Start new Redis container] ********************************** TASK: [terrabox | Ensure we have the latest app master image] ***************** TASK: [terrabox | Ensure app branch image builds from master image] *********** TASK: [terrabox | Ensure app branch image exists] ***************************** TASK: [terrabox | Build app branch image] ************************************* TASK: [terrabox | Ensure all tests pass] ************************************** TASK: [terrabox | Check app container status] ********************************* TASK: [terrabox | Remove app container if stopped (name collision)] *********** TASK: [terrabox | Start new app container] ************************************ TASK: [terrabox | Find app container public port] ***************************** TASK: [terrabox | Update nginx branch vhost ] ********************************* TASK: [terrabox | Update DNS branch record ] ********************************** TASK: [terrabox | Notify HipChat of new production container ] **************** TASK: [terrabox | Reload nginx ] ********************************************** PLAY RECAP ******************************************************************** 95.85.53.185 : ok=4 changed=19 unreachable=0 failed=0
14. 2’ 23” Rails, MySQL & Redis containers (with local Docker images)
15. 3.46x Rails, MySQL & Redis containers (with local Docker images)
16. ENVIRONMENTS How many do I really need?
17. PRODUCTION
18. PRODUCTION Development
19. PRODUCTION Development Integration
20. PRODUCTION Development Staging Integration
21. PRODUCTION Development Staging Integration QA
22. PRODUCTION Development Staging Integration Beta QA
23. PRODUCTION Development Staging Integration Beta QA Mooo
24. [branch].[fqdn] NOT Production
25. [fqdn] Production
26. git branch mentality K.I.S.S.
27. master == production
28. branch != production
29. THINGS I LIKE
30. container == process Single Responsibility Principle
31. container != VM Single Responsibility Principle
32. Data-only containers Single Responsibility Principle
33. Dependent containers .docker.[service]
34. Test-only containers Docker, the simple CI
35. DNS service discovery Instant DNS updates are real
36. Chat notifications Tell me when it’s done
37. WHAT’S NEXT?
38. Clean-up Old containers & images
39. Private registry Build images once, re-use
40. Data-only containers Back-up & Restore
41. Previous instances Backup upstream entries
42. GitHub hooks Real Continuous Delivery
43. Container supervision exec it
44. AND AFTER THAT?
45. Multi-host Redundancy & Scalability
46. Chat ops Everyone on the team can do it
47. gerhardlazu Thank you