Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Build Immutable Servers with Packer & Ansible @ Microsoft OS Camp PT

Build Immutable Servers with Packer & Ansible @ Microsoft OS Camp PT

Slides of my talk about "Build Immutable Servers with Packer & Ansible" at Microsoft Open-Source Camp in Lisbon, Portugal.

The code samples can be found here: https://github.com/dcsg/build-immutable-servers-packer-ansible


Daniel Gomes

April 05, 2017

More Decks by Daniel Gomes

Other Decks in Programming


  1. Build Immutable Servers with Packer & Ansible Daniel Gomes #MSOSCAMP

  2. About me • Senior Software Engineer at Talkdesk • Co-founder

    of phplx • DevOps Enthusiast • @_dcsg • dcsg.me
  3. Once upon a time… @_dcsg #MSOSCAMP

  4. A happy team @_dcsg #MSOSCAMP

  5. Delivers new features in a high pace @_dcsg #MSOSCAMP 0

    35 70 105 140 April May June July
  6. Build/Deploy Processes are manual and time expensive @_dcsg #MSOSCAMP

  7. Deploy 1 per week @_dcsg #MSOSCAMP

  8. The cluster 4 instances @_dcsg #MSOSCAMP

  9. But… problems happen! @_dcsg #MSOSCAMP https://http.cat/500

  10. Instance degraded Out of memory issues @_dcsg #MSOSCAMP

  11. SSH’s in production machine and live fixes are done @_dcsg

  12. Problem Solved @_dcsg #MSOSCAMP

  13. The new cluster state Fixed Instance Instances
 not in same

    state @_dcsg #MSOSCAMP
  14. Configuration Drift • Manual ad-hoc changes and updates to servers

    that are not recorded • Servers in your infrastructure became more and more different from each others @_dcsg #MSOSCAMP
  15. Snowflake Server • Long running servers • Difficult to reproduce

    • No consistency between servers • Lack of confidence in your systems • Hard to spin up another instance in the same state @_dcsg https://martinfowler.com/bliki/SnowflakeServer.html
  16. The cluster state after a few weeks State A @_dcsg

    #MSOSCAMP State C State B State B
  17. How can we solve this? @_dcsg #MSOSCAMP

  18. Configuration Management with Automation Tools @_dcsg #MSOSCAMP

  19. Configuration Management The process of systematically handling changes to a

    system in a way that it maintains integrity over time. @_dcsg #MSOSCAMP
  20. Configuration Management Spin up Base Image Run Config Management Server

    in desire state Changes Edit config file Upgrade/install package … Running Server
  21. Applying to the cluster state State A @_dcsg #MSOSCAMP State

    C State B State B Config Management Apply State D
  22. The cluster state after CM ran State D @_dcsg #MSOSCAMP

    State D State D State D
  23. Automation Tools for CM

  24. Ansible @_dcsg https://www.ansible.com/

  25. How Ansible works? • Agent-less by operating over SSH •

    Ad-hoc commands execution • Playbooks @_dcsg #MSOSCAMP
  26. Ansible “components” • Inventory of hosts - your raw materials

    (servers) • Modules - your tools (apt, yum, etc) • Playbooks - your instructions manual (executes tasks) @_dcsg #MSOSCAMP
  27. Ansible Inventory - Hosts & Groups # /my_project/hosts mail.example.com [webservers]

    foo.example.com bar.example.com [dbservers] one.example.com two.example.com
  28. Ansible Modules - — name: Install vim
 apt: pkg=vim state=latest

    - — name: reboot the server
 command: /sbin/reboot -t now
  29. Ansible Playbook “components” • Hosts • Variables, Conditions, Loops •

    Templating (Jinja2) • Tasks • Handlers • Roles @_dcsg http://docs.ansible.com/ansible/playbooks.html
  30. --- - hosts: webservers vars: http_port: 80 remote_user: root tasks:

    - name: write the apache config file template: src=/srv/httpd.j2 dest=/etc/httpd.conf notify: - restart apache - name: ensure apache is running (and enable it at boot) service: name=httpd state=started enabled=yes handlers: - name: restart apache service: name=httpd state=restarted Ansible Variables, Tasks, Handlers
  31. Ansible Conditions & Loops - name: add several users user:

    name: "{{ item.name }}" state: present groups: "{{ item.groups }}" with_items: - { name: 'testuser1', groups: 'wheel' } - { name: 'testuser2', groups: 'root' } when: - ansible_distribution == "CentOS"
  32. Ansible Roles Example # Folder structure roles/ common/ files/ templates/

    tasks/ main.yml handlers/ vars/ defaults/ meta/ # roles/common/tasks/main.yml - name: install common packages yum: name: "{{ item }}" state: present with_items: - htop - vim
  33. Recap • Configuration Drift • Problems of the Snowflake Servers

    • Configuration Management • Ansible @_dcsg #MSOSCAMP
  34. Phoenix Servers @_dcsg https://martinfowler.com/bliki/PhoenixServer.html

  35. “A server should be like a phoenix, regularly rising from

    the ashes.” - Martin Fowler in PhoenixServer @_dcsg #MSOSCAMP
  36. Phoenix Servers • Avoid configuration drifts • Disposable servers •

    Servers can be built from scratch @_dcsg #MSOSCAMP
  37. The cluster Re-launch an instance State D @_dcsg #MSOSCAMP State

    D State D State D Terminate 
 Instance Config Management Apply State D State D
  38. Spinning up new servers are not a problem anymore! @_dcsg

  39. But… @_dcsg #MSOSCAMP

  40. Is Idempotence guaranteed? @_dcsg #MSOSCAMP

  41. What if the packages repositories are down? @_dcsg #MSOSCAMP

  42. Built Process Spin up Base Image Run Config Management Server

    in desire state Install packages Create folders Create user Upload app etc Run Config Management Repositories unavailable Package not locked to specific version
  43. How can we fix this? @_dcsg #MSOSCAMP

  44. Immutable Servers @_dcsg https://martinfowler.com/bliki/ImmutableServer.html

  45. “An Immutable Server is a server, that once deployed, is

    never modified, merely replaced with a new updated instance.” - Kief Morris in ImmutableServer @_dcsg #MSOSCAMP
  46. Immutable Servers • Final state image with everything baked in.

    • No changes after it’s built. • Include scripts to start the application at boot. • Easy to scale out, deploy and rollback • Trustable and testable • Easy to adopt A/B testing, Canary releases or Blue/Green deployments @_dcsg #MSOSCAMP
  47. Immutable Server Build Process Run Config Management (puppet, chef, ansible)

    Bake In the App App Final Image Server in desire state configure application
 environment Spin up Base Image
  48. Build Image Stages Flow Example Base Image OS Hardening Common

 (vim, htop, etc) etc Application Base Image Install necessary software to run the App Create user/folders Application Final Image Upload App Script to run App at boot System upgrades & Security updates Application security, package, configuration updates
  49. • SnowFlake Servers • Inconsistent states between machine (Config Drifts)

    • Phoenix Serves • Avoids Config Drifts using CM Automation Tools • Can be built from scratch • Immutable Serves • Final image with everything baked in • After built cannot be modified • Can only be replaced with an updated instance Recap
  50. Hello Packer @_dcsg https://www.packer.io/

  51. “Packer is an open source tool for creating identical machine

    images for multiple platforms from a single source configuration.” What’s Packer? @_dcsg #MSOSCAMP
  52. Packer components Builders Provisioners Post-Processors Build machine images from configuration

    management compress, upload, etc
  53. Packer Template // base-image.json { "description": "optional", "min_packer_version": "optional", "variables":

    {}, "builders": [], "provisioners": [], "post-processors": [] }
  54. Validating the template syntax > packer validate -syntax-only base-image.json Syntax-only

    check passed. Everything looks okay.
  55. User Variables Can be defined from: • The command line

    • Environment Variables • From a file @_dcsg https://www.packer.io/docs/templates/user-variables.html
  56. User Variables { "variables": { "client_id": "{{env `CLIENT_ID`}}", "my_var": ""

    }, "builders": [ { "type": "azure-arm", "client_id": "{{ user `client_id` }}", "…": "…" } ] }
  57. Building using variables and Env vars > CLIENT_ID=ID packer build

    -var ‘my_var=foo' \
  58. Building using variables and Env vars > packer build -var-file=vars.json

    base-image.json # vars.json { “client_id": "foo", "my_var": "bar" }
  59. Builders @_dcsg https://www.packer.io/docs/builders/index.html

  60. "builders": [ { "type": "docker", "image": "ansible/ubuntu14.04-ansible", "export_path": "out/base-image.tar" },

    { "type": "virtualbox-ovf", "source_path": "vagrant-base-box/box.ovf", "ssh_username": "vagrant", "ssh_password": "vagrant" } ] Builders
  61. Run packer using only docker Builder > packer build -only=docker

    base-image.json > packer build -parallel=true base-image.json Run packer using both Builders in Parallel
  62. Provisioners @_dcsg https://www.packer.io/docs/provisioners/index.html

  63. Provisioners "provisioners": [ { "type": “shell-local“, "command": “tar -zcvf ./toupload/app.tar.gz

    ../app” }, { "type": “file", "source": “./toupload", "destination": "/tmp/" }, { "type": "ansible", "playbook_file": "ansible/site.yml" } ]
  64. Post-Processors @_dcsg https://www.packer.io/docs/post-processors/index.html

  65. "post-processors": [ { "type": "vagrant", "output": "out/my-app.box", "only": ["virtualbox-ovf"] },

    [ { "type": "docker-import", "repository": "dcsg/my-app", "tag": "0.1", "only": ["docker"] }, { "type": "docker-push", "login": true, "login_username": "{{user `docker_hub_user`}}", "login_password": "{{user `docker_hub_password`}}", "only": ["docker"] } ] ]
  66. Demo @_dcsg https://github.com/dcsg/build-immutable-servers-packer-ansible

  67. Questions 
 Twitter: @_dcsg Blog: dcsg.me github.com/dcsg/build-immutable-servers-packer-ansible 
 ? @_dcsg

    #MSOSCAMP we are hiring!
  68. Thanks! @_dcsg #MSOSCAMP