Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Build Immutable Servers with Packer & Ansible @ Microsoft OS Camp PT

Build Immutable Servers with Packer & Ansible @ Microsoft OS Camp PT

Slides of my talk about "Build Immutable Servers with Packer & Ansible" at Microsoft Open-Source Camp in Lisbon, Portugal.

The code samples can be found here: https://github.com/dcsg/build-immutable-servers-packer-ansible

Daniel Gomes

April 05, 2017
Tweet

More Decks by Daniel Gomes

Other Decks in Programming

Transcript

  1. Build
    Immutable Servers
    with Packer & Ansible
    Daniel Gomes
    #MSOSCAMP @_dcsg

    View Slide

  2. About me
    • Senior Software Engineer at Talkdesk
    • Co-founder of phplx
    • DevOps Enthusiast
    • @_dcsg
    • dcsg.me

    View Slide

  3. Once upon a time…
    @_dcsg
    #MSOSCAMP

    View Slide

  4. A happy team
    @_dcsg
    #MSOSCAMP

    View Slide

  5. Delivers new features in a high pace
    @_dcsg
    #MSOSCAMP
    0
    35
    70
    105
    140
    April May June July

    View Slide

  6. Build/Deploy Processes
    are manual and time expensive
    @_dcsg
    #MSOSCAMP

    View Slide

  7. Deploy
    1 per week
    @_dcsg
    #MSOSCAMP

    View Slide

  8. The cluster
    4 instances
    @_dcsg
    #MSOSCAMP

    View Slide

  9. But… problems happen!
    @_dcsg
    #MSOSCAMP
    https://http.cat/500

    View Slide

  10. Instance degraded
    Out of memory issues
    @_dcsg
    #MSOSCAMP

    View Slide

  11. SSH’s in production machine and live fixes are done
    @_dcsg
    #MSOSCAMP

    View Slide

  12. Problem Solved
    @_dcsg
    #MSOSCAMP

    View Slide

  13. The new cluster state
    Fixed Instance
    Instances

    not in same state
    @_dcsg
    #MSOSCAMP

    View Slide

  14. Configuration Drift
    • Manual ad-hoc changes and updates to servers that
    are not recorded
    • Servers in your infrastructure became more and more
    different from each others
    @_dcsg
    #MSOSCAMP

    View Slide

  15. Snowflake Server
    • Long running servers
    • Difficult to reproduce
    • No consistency between servers
    • Lack of confidence in your systems
    • Hard to spin up another instance in the same state
    @_dcsg
    https://martinfowler.com/bliki/SnowflakeServer.html

    View Slide

  16. The cluster state
    after a few weeks
    State A
    @_dcsg
    #MSOSCAMP
    State C
    State B
    State B

    View Slide

  17. How can we solve this?
    @_dcsg
    #MSOSCAMP

    View Slide

  18. Configuration Management
    with Automation Tools
    @_dcsg
    #MSOSCAMP

    View Slide

  19. Configuration Management
    The process of systematically
    handling changes to a system in a
    way that it maintains integrity over
    time.
    @_dcsg
    #MSOSCAMP

    View Slide

  20. Configuration Management
    Spin up
    Base Image
    Run
    Config Management
    Server
    in desire state
    Changes
    Edit config file
    Upgrade/install package

    Running Server

    View Slide

  21. Applying
    to the cluster state
    State A
    @_dcsg
    #MSOSCAMP
    State C
    State B
    State B
    Config
    Management
    Apply
    State D

    View Slide

  22. The cluster state
    after CM ran
    State D
    @_dcsg
    #MSOSCAMP
    State D
    State D
    State D

    View Slide

  23. Automation Tools for CM

    View Slide

  24. Ansible
    @_dcsg
    https://www.ansible.com/

    View Slide

  25. How Ansible works?
    • Agent-less by operating over SSH
    • Ad-hoc commands execution
    • Playbooks
    @_dcsg
    #MSOSCAMP

    View Slide

  26. Ansible “components”
    • Inventory of hosts - your raw materials (servers)
    • Modules - your tools (apt, yum, etc)
    • Playbooks - your instructions manual (executes tasks)
    @_dcsg
    #MSOSCAMP

    View Slide

  27. Ansible Inventory - Hosts & Groups
    # /my_project/hosts
    mail.example.com
    [webservers]
    foo.example.com
    bar.example.com
    [dbservers]
    one.example.com
    two.example.com

    View Slide

  28. Ansible Modules
    - — name: Install vim

    apt: pkg=vim state=latest
    - — name: reboot the server

    command: /sbin/reboot -t now

    View Slide

  29. Ansible Playbook “components”
    • Hosts
    • Variables, Conditions, Loops
    • Templating (Jinja2)
    • Tasks
    • Handlers
    • Roles
    @_dcsg
    http://docs.ansible.com/ansible/playbooks.html

    View Slide

  30. ---
    - hosts: webservers
    vars:
    http_port: 80
    remote_user: root
    tasks:
    - name: write the apache config file
    template: src=/srv/httpd.j2 dest=/etc/httpd.conf
    notify:
    - restart apache
    - name: ensure apache is running (and enable it at boot)
    service: name=httpd state=started enabled=yes
    handlers:
    - name: restart apache
    service: name=httpd state=restarted
    Ansible Variables, Tasks, Handlers

    View Slide

  31. Ansible Conditions & Loops
    - name: add several users
    user:
    name: "{{ item.name }}"
    state: present
    groups: "{{ item.groups }}"
    with_items:
    - { name: 'testuser1', groups: 'wheel' }
    - { name: 'testuser2', groups: 'root' }
    when:
    - ansible_distribution == "CentOS"

    View Slide

  32. Ansible Roles Example
    # Folder structure
    roles/
    common/
    files/
    templates/
    tasks/
    main.yml
    handlers/
    vars/
    defaults/
    meta/
    # roles/common/tasks/main.yml
    - name: install common packages
    yum:
    name: "{{ item }}"
    state: present
    with_items:
    - htop
    - vim

    View Slide

  33. Recap
    • Configuration Drift
    • Problems of the Snowflake Servers
    • Configuration Management
    • Ansible
    @_dcsg
    #MSOSCAMP

    View Slide

  34. Phoenix Servers
    @_dcsg
    https://martinfowler.com/bliki/PhoenixServer.html

    View Slide

  35. “A server should be like a
    phoenix, regularly rising
    from the ashes.”
    - Martin Fowler in PhoenixServer
    @_dcsg
    #MSOSCAMP

    View Slide

  36. Phoenix Servers
    • Avoid configuration drifts
    • Disposable servers
    • Servers can be built from scratch
    @_dcsg
    #MSOSCAMP

    View Slide

  37. The cluster
    Re-launch an instance
    State D
    @_dcsg
    #MSOSCAMP
    State D State D
    State D
    Terminate 

    Instance
    Config
    Management
    Apply
    State D
    State D

    View Slide

  38. Spinning up new servers
    are not a problem anymore!
    @_dcsg
    #MSOSCAMP

    View Slide

  39. But…
    @_dcsg
    #MSOSCAMP

    View Slide

  40. Is Idempotence guaranteed?
    @_dcsg
    #MSOSCAMP

    View Slide

  41. What if the packages
    repositories are down?
    @_dcsg
    #MSOSCAMP

    View Slide

  42. Built Process
    Spin up
    Base Image Run
    Config Management
    Server
    in desire state
    Install packages
    Create folders
    Create user
    Upload app
    etc
    Run
    Config Management
    Repositories
    unavailable
    Package not locked
    to specific version

    View Slide

  43. How can we fix this?
    @_dcsg
    #MSOSCAMP

    View Slide

  44. Immutable Servers
    @_dcsg
    https://martinfowler.com/bliki/ImmutableServer.html

    View Slide

  45. “An Immutable Server is a
    server, that once deployed, is
    never modified, merely
    replaced with a new updated
    instance.”
    - Kief Morris in ImmutableServer
    @_dcsg
    #MSOSCAMP

    View Slide

  46. Immutable Servers
    • Final state image with everything baked in.
    • No changes after it’s built.
    • Include scripts to start the application at boot.
    • Easy to scale out, deploy and rollback
    • Trustable and testable
    • Easy to adopt A/B testing, Canary releases or Blue/Green
    deployments
    @_dcsg
    #MSOSCAMP

    View Slide

  47. Immutable Server Build Process
    Run
    Config Management
    (puppet, chef, ansible)
    Bake In
    the App
    App
    Final Image
    Server
    in desire state
    configure application

    environment
    Spin up
    Base Image

    View Slide

  48. Build Image Stages Flow Example
    Base Image
    OS Hardening
    Common tools

    (vim, htop, etc)
    etc
    Application
    Base Image
    Install necessary
    software to run the App
    Create user/folders
    Application
    Final Image
    Upload App
    Script to run App at boot
    System upgrades &
    Security updates
    Application security,
    package, configuration
    updates

    View Slide

  49. • SnowFlake Servers
    • Inconsistent states between machine (Config Drifts)
    • Phoenix Serves
    • Avoids Config Drifts using CM Automation Tools
    • Can be built from scratch
    • Immutable Serves
    • Final image with everything baked in
    • After built cannot be modified
    • Can only be replaced with an updated instance
    Recap

    View Slide

  50. Hello Packer
    @_dcsg
    https://www.packer.io/

    View Slide

  51. “Packer is an open source tool for creating
    identical machine images for multiple
    platforms from a single source configuration.”
    What’s Packer?
    @_dcsg
    #MSOSCAMP

    View Slide

  52. Packer components
    Builders Provisioners Post-Processors
    Build machine images from configuration management compress, upload, etc

    View Slide

  53. Packer Template
    // base-image.json
    {
    "description": "optional",
    "min_packer_version": "optional",
    "variables": {},
    "builders": [],
    "provisioners": [],
    "post-processors": []
    }

    View Slide

  54. Validating the template syntax
    > packer validate -syntax-only base-image.json
    Syntax-only check passed. Everything looks okay.

    View Slide

  55. User Variables
    Can be defined from:
    • The command line
    • Environment Variables
    • From a file
    @_dcsg
    https://www.packer.io/docs/templates/user-variables.html

    View Slide

  56. User Variables
    {
    "variables": {
    "client_id": "{{env `CLIENT_ID`}}",
    "my_var": ""
    },
    "builders": [
    {
    "type": "azure-arm",
    "client_id": "{{ user `client_id` }}",
    "…": "…"
    }
    ]
    }

    View Slide

  57. Building using variables and Env vars
    > CLIENT_ID=ID packer build -var ‘my_var=foo' \

    base-image.json

    View Slide

  58. Building using variables and Env vars
    > packer build -var-file=vars.json base-image.json
    # vars.json
    {
    “client_id": "foo",
    "my_var": "bar"
    }

    View Slide

  59. Builders
    @_dcsg
    https://www.packer.io/docs/builders/index.html

    View Slide

  60. "builders": [
    {
    "type": "docker",
    "image": "ansible/ubuntu14.04-ansible",
    "export_path": "out/base-image.tar"
    },
    {
    "type": "virtualbox-ovf",
    "source_path": "vagrant-base-box/box.ovf",
    "ssh_username": "vagrant",
    "ssh_password": "vagrant"
    }
    ]
    Builders

    View Slide

  61. Run packer using only docker Builder
    > packer build -only=docker base-image.json
    > packer build -parallel=true base-image.json
    Run packer using both Builders in Parallel

    View Slide

  62. Provisioners
    @_dcsg
    https://www.packer.io/docs/provisioners/index.html

    View Slide

  63. Provisioners
    "provisioners": [
    {
    "type": “shell-local“,
    "command": “tar -zcvf ./toupload/app.tar.gz ../app”
    },
    {
    "type": “file",
    "source": “./toupload",
    "destination": "/tmp/"
    },
    {
    "type": "ansible",
    "playbook_file": "ansible/site.yml"
    }
    ]

    View Slide

  64. Post-Processors
    @_dcsg
    https://www.packer.io/docs/post-processors/index.html

    View Slide

  65. "post-processors": [
    {
    "type": "vagrant",
    "output": "out/my-app.box",
    "only": ["virtualbox-ovf"]
    },
    [
    {
    "type": "docker-import",
    "repository": "dcsg/my-app",
    "tag": "0.1",
    "only": ["docker"]
    },
    {
    "type": "docker-push",
    "login": true,
    "login_username": "{{user `docker_hub_user`}}",
    "login_password": "{{user `docker_hub_password`}}",
    "only": ["docker"]
    }
    ]
    ]

    View Slide

  66. Demo
    @_dcsg
    https://github.com/dcsg/build-immutable-servers-packer-ansible

    View Slide

  67. Questions

    Twitter: @_dcsg
    Blog: dcsg.me
    github.com/dcsg/build-immutable-servers-packer-ansible

    ?
    @_dcsg
    #MSOSCAMP
    we are hiring!

    View Slide

  68. Thanks!
    @_dcsg
    #MSOSCAMP

    View Slide