Taming Terraform Workflow Using Terraform Modules and Github

Abstract

Today’s cloud infrastructure is really complex. What if you could truly make this infrastructure a black box?
What if you could mutate this infrastructure safely and easily? See how to use Terraform efficiently across
hundreds of developers while leveraging a simple Github workflow and Terraform modules.

Additional info

Hootsuite was one of the early adopters of Terraform and faced some challenges early on with Terraform workflow.
* This session will cover how Hootsuite uses Terraform modules and leverages them to bridge the gap between devs and
operations.
* We will discuss how we came up with our own state management system that helps us safely mutate the state of our
infrastructure from one state to the other. How you can organize your Terraform states across multiple tiers of your
infrastructure. We will also talk about our own HTTP remote state store on top of S3 that enables us to edit states
and rollback.
* Most importantly, we came up with our own Terraform workflow that uses github pull requests to do approvals and
state locking so we don’t corrupt Terraform states for our infrastructure.

Proposal: https://github.com/cfgmgmtcamp/2017-pdx-cfp/blob/master/presentations/taming-terraform-workflow.md

Transcript

1. Senior Developer - Production Operations and Delivery @anubhavm Taming Terraform Workflow Anubhav Mishra Using Terraform Modules and Github
2. Anubhav Mishra @anubhavm
5. Talk • Terraform at Hootsuite • Use of Terraform Modules (Dev <--> Ops) • Atlantis - Hootsuite’s Terraform Workflow • Live Demo
6. Terraform at Hootsuite
7. Numbers • 100+ developers • 20+ microservices • 2 cluster schedulers • 1000+ servers • 1000s of AWS resources
8. Timeline • Mid 2015 - First Terraform Repo (v1) • Starting 2016 - Terraform (v2) • End of 2016 - Terraform “Advanced” Workflow
9. Before Terraform
13. What is Terraform? • Terraform allows you to manage, build and visualize your infrastructure as code
17. First Attempt
18. First Attempt
19. First Attempt
20. First Attempt staging != production
21. Terraform State
22. First Attempt
24. Problems with our first attempt • Environment drift • One massive state for every environment • Hard to collaborate on infrastructure
25. Problems with our first attempt • Environment drift • One massive state for every environment • Hard to collaborate on infrastructure
26. Terraform Version 2 (Second Attempt)
27. Terraform Version 2 (Second Attempt)
28. Terraform Version 2 (Second Attempt)
29. Problems with our first attempt • Environment drift • One massive state for every environment • Hard to collaborate on infrastructure
30. Problems with our first attempt • Environment drift • One massive state for every environment • Hard to collaborate on infrastructure
31. Terraform Version 2 (Second Attempt) terraform.tfstate { "version": 3, "terraform_version": "0.8.8", "serial": 175, "lineage": "2cb11085-2e4e-40ff-bf81-e4ddf8bc8d5d", "modules": [ { "path": [ "root" ], "outputs": {}, "resources": { ....
32. Terraform Version 2 (Second Attempt) terraform.tfstate { "version": 3, "terraform_version": "0.8.8", "serial": 175, "lineage": "2cb11085-2e4e-40ff-bf81-e4ddf8bc8d5d", "modules": [ { "path": [ "root" ], "outputs": {}, "resources": { ....
33. Terraform Version 2 (Second Attempt) terraform.tfstate
34. Terraform Version 2 (Second Attempt)
35. Terraform Version 2 (Second Attempt)
36. Terraform Version 2 (Second Attempt)
38. terraform.tfstate { "version": 3, "terraform_version": "0.8.8", "serial": 1, "lineage": "9e42c0ac-9e73-4910-8859-504934f2b499", "backend": { "type": "s3", "config": { "bucket": "terraform", "key": "foo-bar", "region": "us-east-1" }, .... "outputs": {}, "resources": { "null_resource.hello": { "type": "null_resource", "depends_on": [], "primary": { "id": "792399369485920658", "attributes": { "id": "792399369485920658" }, "meta": {}, ....
39. stash
43. Problems with our first attempt • Environment drift • One massive state for every environment • Hard to collaborate on infrastructure
44. Problems with our first attempt • Environment drift • One massive state for every environment • Hard to collaborate on infrastructure
48. module.server Let me make a generic server module I need 2 servers. Now! Terraform Version 2 (Second Attempt)
50. Terraform Version 2 (Second Attempt)
51. Terraform Version 2 (Second Attempt)
52. Inputs Output
53. Terraform Version 2 (Second Attempt)
54. Terraform Version 2 (Second Attempt)
55. Terraform Version 2 (Second Attempt)
56. Terraform Version 2 (Second Attempt)
57. Terraform Version 2 (Second Attempt) 139 Repositories
58. • Multi Environment Terraform Projects • Isolated Terraform States for Services ◦ Smaller Failure Domain ◦ Rapid Iteration on Infrastructure • Use of Terraform Modules ◦ Environment Parity ◦ Increased Collaboration Between Ops and Dev
59. Are we all done?
63. How do we continue to move fast while we add people?
65. atlantis A unified workflow for collaborating on Terraform through GitHub. https://github.com/hootsuite/atlantis
66. Demo
67. • atlantis - https://github.com/hootsuite/atlantis • cfgmgmtcamp-atlantis-demo - https://github.com/anubhavmishra/cfgmgmtcamp-atlantis-demo Links
68. Thank You! Senior Developer - Production Operations and Delivery @anubhavm Anubhav Mishra