Ansible & Docker - The Path to Continuous Delivery - Part 1

The aim of this series is to have an end-to-end process for continuously delivering a Rails application with MySQL and Redis dependencies in production, as a series of Docker containers, across multiple physical hosts. I will be able to preview features from git branches, in production, without affecting the active production instances. It goes without saying, all GitHub commits will trigger builds, as one would expect from a real Continuous Delivery setup.

Presented at London Docker Meetup 29th January 2014

http://gerhard.lazu.co.uk/ansible-docker-the-path-to-continuous-delivery-1

Transcript

1. ANSIBLE & DOCKER
   The Path to Continuous Delivery
   Part 1
   

   View Slide

2. Demi ➜ infrastructure git:(master) bin/ap create_droplet.yml hosts -c local
   PLAY [Digital Ocean] **********************************************************
   GATHERING FACTS ***************************************************************
   TASK: [Create DigitalOcean droplet] *******************************************
   PLAY RECAP ********************************************************************
   localhost : ok=1 changed=1 unreachable=0 failed=0
   

   View Slide

3. 55” 1’ 25”
   Create DO 2GB AMS2 droplet
   

   View Slide

4. Don’t write cookbooks, just play
   

   View Slide

5. Changed. Skipped. OK.
   

   View Slide

6. Demi ➜ infrastructure git:(master) bin/ap dod.yml dohosts -u root
   PLAY [Digital Ocean Docker] ***************************************************
   GATHERING FACTS ***************************************************************
   TASK: [base | Install common system packages via apt] *************************
   TASK: [base | NTP client] *****************************************************
   TASK: [base | Sudo group] *****************************************************
   TASK: [base | Docker group] ***************************************************
   TASK: [base | Create gerhard user] ********************************************
   TASK: [base | gerhard SSH key] ************************************************
   TASK: [base | root SSH key] ***************************************************
   TASK: [base | Sudoers don't need a password] **********************************
   TASK: [base | Sudoers have their ssh agent forwarded] *************************
   TASK: [base | Add github.com to known hosts] **********************************
   TASK: [base | Make vim the default editor] ************************************
   TASK: [base | Common bash aliases] ********************************************
   TASK: [docker | Add docker apt repository key] ********************************
   TASK: [docker | Add docker apt repository] ************************************
   TASK: [docker | Install latest docker] ****************************************
   TASK: [docker | Start docker on boot] *****************************************
   TASK: [docker | Docker aliases] ***********************************************
   TASK: [nginx | Add nginx PPA repository] **************************************
   TASK: [nginx | Install nginx] *************************************************
   TASK: [nginx | Customize nginx default configuration] *************************
   TASK: [nginx | Start nginx on boot] *******************************************
   PLAY RECAP ********************************************************************
   95.85.53.185 : ok=2 changed=23 unreachable=0 failed=0
   

   View Slide

7. 1’ 50”
   Base system setup
   

   View Slide

8. nginx
   As special as dockerd
   

   View Slide

9. Demi ➜ infrastructure git:(master) bin/ap terrabox.yml dohosts -l docker-ams2
   PLAY [terrabox app - Rails] ***************************************************
   GATHERING FACTS ***************************************************************
   TASK: [terrabox | Update app repository] **************************************
   TASK: [terrabox | Dependent containers] ***************************************
   TASK: [terrabox | Check MySQL data container] *********************************
   TASK: [terrabox | Build MySQL data container] *********************************
   TASK: [terrabox | Check MySQL container status] *******************************
   TASK: [terrabox | Remove MySQL container with same name] **********************
   TASK: [terrabox | Start new MySQL container] **********************************
   TASK: [terrabox | Check Redis container status] *******************************
   TASK: [terrabox | Remove Redis container with same name] **********************
   TASK: [terrabox | Start new Redis container] **********************************
   TASK: [terrabox | Ensure we have the latest app master image] *****************
   TASK: [terrabox | Ensure app branch image builds from master image] ***********
   TASK: [terrabox | Ensure app branch image exists] *****************************
   TASK: [terrabox | Build app branch image] *************************************
   TASK: [terrabox | Ensure all tests pass] **************************************
   TASK: [terrabox | Check app container status] *********************************
   TASK: [terrabox | Remove app container if stopped (name collision)] ***********
   TASK: [terrabox | Start new app container] ************************************
   TASK: [terrabox | Find app container public port] *****************************
   TASK: [terrabox | Update nginx production vhost ] *****************************
   TASK: [terrabox | Update DNS production record ] ******************************
   TASK: [terrabox | Notify HipChat of new production container ] ****************
   TASK: [terrabox | Reload nginx ] **********************************************
   PLAY RECAP ********************************************************************
   95.85.53.185 : ok=5 changed=18 unreachable=0 failed=0
   

   View Slide

10. 8’ 16”
    Rails, MySQL & Redis containers
    (no local Docker images)
    

    View Slide

11. Dependent containers
    Branch-specific
    

    View Slide

12. Test-only containers
    Hello CI
    

    View Slide

13. Demi ➜ infrastructure git:(master) bin/ap terrabox.yml dohosts -l docker-ams2
    -e ‘app_branch=gem_updates’
    PLAY [terrabox app - Rails] ***************************************************
    GATHERING FACTS ***************************************************************
    TASK: [terrabox | Update app repository] **************************************
    TASK: [terrabox | Dependent containers] ***************************************
    TASK: [terrabox | Check MySQL data container] *********************************
    TASK: [terrabox | Build MySQL data container] *********************************
    TASK: [terrabox | Check MySQL container status] *******************************
    TASK: [terrabox | Remove MySQL container with same name] **********************
    TASK: [terrabox | Start new MySQL container] **********************************
    TASK: [terrabox | Check Redis container status] *******************************
    TASK: [terrabox | Remove Redis container with same name] **********************
    TASK: [terrabox | Start new Redis container] **********************************
    TASK: [terrabox | Ensure we have the latest app master image] *****************
    TASK: [terrabox | Ensure app branch image builds from master image] ***********
    TASK: [terrabox | Ensure app branch image exists] *****************************
    TASK: [terrabox | Build app branch image] *************************************
    TASK: [terrabox | Ensure all tests pass] **************************************
    TASK: [terrabox | Check app container status] *********************************
    TASK: [terrabox | Remove app container if stopped (name collision)] ***********
    TASK: [terrabox | Start new app container] ************************************
    TASK: [terrabox | Find app container public port] *****************************
    TASK: [terrabox | Update nginx branch vhost ] *********************************
    TASK: [terrabox | Update DNS branch record ] **********************************
    TASK: [terrabox | Notify HipChat of new production container ] ****************
    TASK: [terrabox | Reload nginx ] **********************************************
    PLAY RECAP ********************************************************************
    95.85.53.185 : ok=4 changed=19 unreachable=0 failed=0
    

    View Slide

14. 2’ 23”
    Rails, MySQL & Redis containers
    (with local Docker images)
    

    View Slide

15. 3.46x
    Rails, MySQL & Redis containers
    (with local Docker images)
    

    View Slide

16. ENVIRONMENTS
    How many do I really need?
    

    View Slide

17. PRODUCTION
    

    View Slide

18. PRODUCTION
    Development
    

    View Slide

19. PRODUCTION
    Development
    Integration
    

    View Slide

20. PRODUCTION
    Development
    Staging
    Integration
    

    View Slide

21. PRODUCTION
    Development
    Staging
    Integration
    QA
    

    View Slide

22. PRODUCTION
    Development
    Staging
    Integration Beta
    QA
    

    View Slide

23. PRODUCTION
    Development
    Staging
    Integration Beta
    QA
    Mooo
    

    View Slide

24. [branch].[fqdn]
    NOT Production
    

    View Slide

25. [fqdn]
    Production
    

    View Slide

26. git branch mentality
    K.I.S.S.
    

    View Slide

27. master == production
    

    View Slide

28. branch != production
    

    View Slide

29. THINGS I LIKE
    

    View Slide

30. container == process
    Single Responsibility Principle
    

    View Slide

31. container != VM
    Single Responsibility Principle
    

    View Slide

32. Data-only containers
    Single Responsibility Principle
    

    View Slide

33. Dependent containers
    .docker.[service]
    

    View Slide

34. Test-only containers
    Docker, the simple CI
    

    View Slide

35. DNS service discovery
    Instant DNS updates are real
    

    View Slide

36. Chat notifications
    Tell me when it’s done
    

    View Slide

37. WHAT’S NEXT?
    

    View Slide

38. Clean-up
    Old containers & images
    

    View Slide

39. Private registry
    Build images once, re-use
    

    View Slide

40. Data-only containers
    Back-up & Restore
    

    View Slide

41. Previous instances
    Backup upstream entries
    

    View Slide

42. GitHub hooks
    Real Continuous Delivery
    

    View Slide

43. Container supervision
    exec it
    

    View Slide

44. AND AFTER THAT?
    

    View Slide

45. Multi-host
    Redundancy & Scalability
    

    View Slide

46. Chat ops
    Everyone on the team can do it
    

    View Slide

47. gerhardlazu
    Thank you
    

    View Slide