Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Build Immutable Servers with Packer & Ansible @ PixelsCamp 2017

0fe2d959c89cf2d9de497a237c4ea99d?s=47 Daniel Gomes
September 29, 2017

Build Immutable Servers with Packer & Ansible @ PixelsCamp 2017

Slides of my talk about "Build Immutable Servers with Packer & Ansible" at Pixels Camp in Lisbon, Portugal.

The code samples can be found here: https://github.com/dcsg/build-immutable-servers-packer-ansible


Daniel Gomes

September 29, 2017

More Decks by Daniel Gomes

Other Decks in Technology


  1. Build Immutable Servers with Packer & Ansible Daniel Gomes #PixelsCamp

  2. About me • Team Lead at Talkdesk • @phplx co-founder

    • @_dcsg • dcsg.me Daniel Gomes
  3. Once upon a time… @_dcsg #PixelsCamp

  4. A happy team @_dcsg #PixelsCamp

  5. Delivers new features in a high pace @_dcsg #PixelsCamp 0

    35 70 105 140 April May June July
  6. Build/Deploy Processes are manual and time expensive @_dcsg #PixelsCamp

  7. Deploy 1 per week @_dcsg #PixelsCamp

  8. The cluster 4 instances @_dcsg #PixelsCamp

  9. But… problems happen! @_dcsg #PixelsCamp https://http.cat/500

  10. Instance degraded Out of memory issues @_dcsg #PixelsCamp

  11. SSH’s in production machine and live fixes are done @_dcsg

  12. Problem Solved @_dcsg #PixelsCamp

  13. The new cluster state Fixed Instance Instances
 not in same

    state @_dcsg #PixelsCamp
  14. Configuration Drift @_dcsg #PixelsCamp

  15. Configuration Drift • Manual ad-hoc changes and updates to servers

    that are not recorded • Servers in your infrastructure became more and more different from each others @_dcsg #PixelsCamp
  16. The cluster state after a few weeks State A @_dcsg

    #PixelsCamp State C State B State B
  17. Snowflake Server @_dcsg #PixelsCamp

  18. Snowflake Server • Long running servers • Difficult to reproduce

    • No consistency between servers • Lack of confidence in your systems • Hard to spin up another instance in the same state @_dcsg https://martinfowler.com/bliki/SnowflakeServer.html
  19. How can we solve this? @_dcsg #PixelsCamp

  20. Configuration Management with Automation Tools @_dcsg #PixelsCamp

  21. Configuration Management The process of systematically handling changes to a

    system in a way that it maintains integrity over time. @_dcsg #PixelsCamp
  22. Configuration Management Spin up Base Image Run Config Management Server

    in desired state Changes Edit config file Upgrade/install package … Running Server
  23. Applying to the cluster state State A @_dcsg #PixelsCamp State

    C State B State B Config Management Apply State D
  24. The cluster state after CM ran State D @_dcsg #PixelsCamp

    State D State D State D
  25. Automation Tools for CM

  26. Ansible @_dcsg https://www.ansible.com/

  27. How does Ansible work? • Agent-less by operating over SSH

    • Ad-hoc commands execution • Playbooks @_dcsg #PixelsCamp
  28. Ansible terminology • Inventory of hosts - your raw materials

    (servers) • Modules - your tools (apt, yum, etc) • Playbooks - your instructions manual (executes tasks) @_dcsg #PixelsCamp
  29. Let’s see the code @_dcsg https://github.com/dcsg/build-immutable-servers-packer-ansible

  30. Quick recap • Configuration Drift • Problems of the Snowflake

    Servers • Configuration Management • Ansible @_dcsg #PixelsCamp
  31. Phoenix Servers @_dcsg https://martinfowler.com/bliki/PhoenixServer.html

  32. “A server should be like a phoenix, regularly rising from

    the ashes.” - Martin Fowler in PhoenixServer @_dcsg #PixelsCamp
  33. Phoenix Servers • Avoid configuration drifts • Disposable servers •

    Servers can be built from scratch @_dcsg #PixelsCamp
  34. The cluster Re-launch an instance State D @_dcsg #PixelsCamp State

    D State D State D Terminate 
 Instance Config Management Apply State D State D
  35. Spinning up new servers is not a problem anymore! @_dcsg

  36. But… @_dcsg #PixelsCamp

  37. Is Idempotence guaranteed? @_dcsg #PixelsCamp

  38. What if the packages repositories are down? @_dcsg #PixelsCamp

  39. Built Process Spin up Base Image Run Config Management Server

    in desire state Install packages Create folders Create user Upload app etc Run Config Management Repositories unavailable
  40. How can we fix this? @_dcsg #PixelsCamp

  41. Immutable Servers @_dcsg https://martinfowler.com/bliki/ImmutableServer.html

  42. "An Immutable Server is a server, that once deployed, is

    never modified, merely replaced with a new updated instance." - Kief Morris in ImmutableServer @_dcsg #PixelsCamp
  43. Immutable Servers • Final state image with everything baked in.

    • No changes after it’s built. • Include scripts to start the application at boot. • Easy to scale out, deploy and rollback. • Trustable and testable. • Easy to adopt A/B testing, Canary releases or Blue/Green deployments. @_dcsg #PixelsCamp
  44. Immutable Server Build Process Bake In the App App Final

    Image Server in desire state configure application
 environment Spin up Base Image Run Config Management (puppet, chef, ansible)
  45. Build Image Stages Flow Example Base Image OS Hardening Common

 (vim, htop, etc) etc Application Base Image Install necessary software to run the App Create user/folders Application Final Image Upload App Script to run App at boot System upgrades & Security updates Application security, package, configuration updates
  46. • SnowFlake Servers • Inconsistent states between machine (Config Drifts)

    • Phoenix Servers • Avoids Config Drifts using CM Automation Tools • Can be built from scratch • Immutable Servers • Final image with everything baked in • After built cannot be modified • Can only be replaced with an updated instance Recap
  47. How can we build those machines? @_dcsg #PixelsCamp

  48. Hello Packer @_dcsg https://www.packer.io/

  49. What’s Packer? • Tool for creating identical machine images •

    For multiple platforms and in Parallel @_dcsg https://www.packer.io/
  50. Packer terminology • Artifact • Builders • Builds • Commands

    • Post-processors • Provisioners • Templates @_dcsg https://www.packer.io/
  51. Packer flow Builders Provisioners Post-Processors Build machine images from configuration

    management compress, upload, etc
  52. Demo @_dcsg https://github.com/dcsg/build-immutable-servers-packer-ansible

  53. Questions 
 Twitter: @_dcsg Blog: dcsg.me github.com/dcsg/build-immutable-servers-packer-ansible 
 ? @_dcsg

    #PixelsCamp we are hiring!
  54. Thanks! @_dcsg #PixelsCamp