Build Immutable Servers with Packer & Ansible @ PixelsCamp 2017

Slides of my talk about "Build Immutable Servers with Packer & Ansible" at Pixels Camp in Lisbon, Portugal.

The code samples can be found here: https://github.com/dcsg/build-immutable-servers-packer-ansible

Transcript

1. Build Immutable Servers with Packer & Ansible Daniel Gomes #PixelsCamp @_dcsg
2. About me • Team Lead at Talkdesk • @phplx co-founder • @_dcsg • dcsg.me Daniel Gomes
3. Once upon a time… @_dcsg #PixelsCamp
4. A happy team @_dcsg #PixelsCamp
5. Delivers new features in a high pace @_dcsg #PixelsCamp 0 35 70 105 140 April May June July
6. Build/Deploy Processes are manual and time expensive @_dcsg #PixelsCamp
7. Deploy 1 per week @_dcsg #PixelsCamp
8. The cluster 4 instances @_dcsg #PixelsCamp
9. But… problems happen! @_dcsg #PixelsCamp https://http.cat/500
10. Instance degraded Out of memory issues @_dcsg #PixelsCamp
11. SSH’s in production machine and live fixes are done @_dcsg #PixelsCamp
12. Problem Solved @_dcsg #PixelsCamp
13. The new cluster state Fixed Instance Instances
 not in same state @_dcsg #PixelsCamp
14. Configuration Drift @_dcsg #PixelsCamp
15. Configuration Drift • Manual ad-hoc changes and updates to servers that are not recorded • Servers in your infrastructure became more and more different from each others @_dcsg #PixelsCamp
16. The cluster state after a few weeks State A @_dcsg #PixelsCamp State C State B State B
17. Snowflake Server @_dcsg #PixelsCamp
18. Snowflake Server • Long running servers • Difficult to reproduce • No consistency between servers • Lack of confidence in your systems • Hard to spin up another instance in the same state @_dcsg https://martinfowler.com/bliki/SnowflakeServer.html
19. How can we solve this? @_dcsg #PixelsCamp
20. Configuration Management with Automation Tools @_dcsg #PixelsCamp
21. Configuration Management The process of systematically handling changes to a system in a way that it maintains integrity over time. @_dcsg #PixelsCamp
22. Configuration Management Spin up Base Image Run Config Management Server in desired state Changes Edit config file Upgrade/install package … Running Server
23. Applying to the cluster state State A @_dcsg #PixelsCamp State C State B State B Config Management Apply State D
24. The cluster state after CM ran State D @_dcsg #PixelsCamp State D State D State D
25. Automation Tools for CM
26. Ansible @_dcsg https://www.ansible.com/
27. How does Ansible work? • Agent-less by operating over SSH • Ad-hoc commands execution • Playbooks @_dcsg #PixelsCamp
28. Ansible terminology • Inventory of hosts - your raw materials (servers) • Modules - your tools (apt, yum, etc) • Playbooks - your instructions manual (executes tasks) @_dcsg #PixelsCamp
29. Let’s see the code @_dcsg https://github.com/dcsg/build-immutable-servers-packer-ansible
30. Quick recap • Configuration Drift • Problems of the Snowflake Servers • Configuration Management • Ansible @_dcsg #PixelsCamp
31. Phoenix Servers @_dcsg https://martinfowler.com/bliki/PhoenixServer.html
32. “A server should be like a phoenix, regularly rising from the ashes.” - Martin Fowler in PhoenixServer @_dcsg #PixelsCamp
33. Phoenix Servers • Avoid configuration drifts • Disposable servers • Servers can be built from scratch @_dcsg #PixelsCamp
34. The cluster Re-launch an instance State D @_dcsg #PixelsCamp State D State D State D Terminate 
 Instance Config Management Apply State D State D
35. Spinning up new servers is not a problem anymore! @_dcsg #PixelsCamp
36. But… @_dcsg #PixelsCamp
37. Is Idempotence guaranteed? @_dcsg #PixelsCamp
38. What if the packages repositories are down? @_dcsg #PixelsCamp
39. Built Process Spin up Base Image Run Config Management Server in desire state Install packages Create folders Create user Upload app etc Run Config Management Repositories unavailable
40. How can we fix this? @_dcsg #PixelsCamp
41. Immutable Servers @_dcsg https://martinfowler.com/bliki/ImmutableServer.html
42. "An Immutable Server is a server, that once deployed, is never modified, merely replaced with a new updated instance." - Kief Morris in ImmutableServer @_dcsg #PixelsCamp
43. Immutable Servers • Final state image with everything baked in. • No changes after it’s built. • Include scripts to start the application at boot. • Easy to scale out, deploy and rollback. • Trustable and testable. • Easy to adopt A/B testing, Canary releases or Blue/Green deployments. @_dcsg #PixelsCamp
44. Immutable Server Build Process Bake In the App App Final Image Server in desire state configure application
 environment Spin up Base Image Run Config Management (puppet, chef, ansible)
45. Build Image Stages Flow Example Base Image OS Hardening Common tools
 (vim, htop, etc) etc Application Base Image Install necessary software to run the App Create user/folders Application Final Image Upload App Script to run App at boot System upgrades & Security updates Application security, package, configuration updates
46. • SnowFlake Servers • Inconsistent states between machine (Config Drifts) • Phoenix Servers • Avoids Config Drifts using CM Automation Tools • Can be built from scratch • Immutable Servers • Final image with everything baked in • After built cannot be modified • Can only be replaced with an updated instance Recap
47. How can we build those machines? @_dcsg #PixelsCamp
48. Hello Packer @_dcsg https://www.packer.io/
49. What’s Packer? • Tool for creating identical machine images • For multiple platforms and in Parallel @_dcsg https://www.packer.io/
50. Packer terminology • Artifact • Builders • Builds • Commands • Post-processors • Provisioners • Templates @_dcsg https://www.packer.io/
51. Packer flow Builders Provisioners Post-Processors Build machine images from configuration management compress, upload, etc
52. Demo @_dcsg https://github.com/dcsg/build-immutable-servers-packer-ansible
53. Questions 
 Twitter: @_dcsg Blog: dcsg.me github.com/dcsg/build-immutable-servers-packer-ansible 
 ? @_dcsg #PixelsCamp we are hiring!
54. Thanks! @_dcsg #PixelsCamp