Taming Terraform Workflow Using Terraform Modules and Github

Talk for HashiCorp User Group in Vancouver at Hootsuite.
https://www.meetup.com/Vancouver-HashiCorp-User-Group/events/241966250/

Transcript

2. Senior Developer - Production Operations and Delivery @anubhavm Taming Terraform Workflow Anubhav Mishra Using Terraform Modules and Github Lead Developer - Production Operations and Delivery @lkysow Luke Kysow
4. Talk • Terraform at Hootsuite • Use of Terraform Modules (Dev <--> Ops) • Atlantis - Hootsuite’s Terraform Workflow • Live Demo
5. Terraform at Hootsuite
6. Numbers • 100+ developers • 20+ microservices • 2 cluster schedulers • 1000+ servers • 1000s of AWS resources
7. Timeline • Mid 2015 - First Terraform Repo (v1) • Starting 2016 - Terraform (v2) • End of 2016 - Terraform “Advanced” Workflow
8. Before Terraform
12. What is Terraform? • Terraform allows you to manage, build and visualize your infrastructure as code
16. First Attempt
17. First Attempt
18. First Attempt
19. First Attempt staging != production
20. Terraform State
21. First Attempt
23. Problems with our first attempt • Environment drift • One massive state for every environment • Hard to collaborate on infrastructure
24. Problems with our first attempt • Environment drift • One massive state for every environment • Hard to collaborate on infrastructure
25. Terraform Version 2 (Second Attempt)
26. Terraform Version 2 (Second Attempt)
27. Terraform Version 2 (Second Attempt)
28. Problems with our first attempt • Environment drift • One massive state for every environment • Hard to collaborate on infrastructure
29. Problems with our first attempt • Environment drift • One massive state for every environment • Hard to collaborate on infrastructure
30. Terraform Version 2 (Second Attempt) terraform.tfstate { "version": 3, "terraform_version": "0.8.8", "serial": 175, "lineage": "2cb11085-2e4e-40ff-bf81-e4ddf8bc8d5d", "modules": [ { "path": [ "root" ], "outputs": {}, "resources": { ....
31. Terraform Version 2 (Second Attempt) terraform.tfstate { "version": 3, "terraform_version": "0.8.8", "serial": 175, "lineage": "2cb11085-2e4e-40ff-bf81-e4ddf8bc8d5d", "modules": [ { "path": [ "root" ], "outputs": {}, "resources": { ....
32. Terraform Version 2 (Second Attempt) terraform.tfstate
33. Terraform Version 2 (Second Attempt)
34. Terraform Version 2 (Second Attempt)
35. Terraform Version 2 (Second Attempt)
37. terraform.tfstate { "version": 3, "terraform_version": "0.8.8", "serial": 1, "lineage": "9e42c0ac-9e73-4910-8859-504934f2b499", "backend": { "type": "s3", "config": { "bucket": "terraform", "key": "foo-bar", "region": "us-east-1" }, .... "outputs": {}, "resources": { "null_resource.hello": { "type": "null_resource", "depends_on": [], "primary": { "id": "792399369485920658", "attributes": { "id": "792399369485920658" }, "meta": {}, ....
38. stash
42. Problems with our first attempt • Environment drift • One massive state for every environment • Hard to collaborate on infrastructure
43. Problems with our first attempt • Environment drift • One massive state for every environment • Hard to collaborate on infrastructure
47. module.server Let me make a generic server module I need 2 servers. Now! Terraform Version 2 (Second Attempt)
49. Terraform Version 2 (Second Attempt)
50. Terraform Version 2 (Second Attempt)
51. Inputs Output
52. Terraform Version 2 (Second Attempt)
53. Terraform Version 2 (Second Attempt)
54. Terraform Version 2 (Second Attempt)
55. Terraform Version 2 (Second Attempt)
56. Terraform Version 2 (Second Attempt) 139 Repositories
57. • Multi Environment Terraform Projects • Isolated Terraform States for Services ◦ Smaller Failure Domain ◦ Rapid Iteration on Infrastructure • Use of Terraform Modules ◦ Environment Parity ◦ Increased Collaboration Between Ops and Dev
58. Are we all done?
62. How do we continue to move fast while we add people?
64. atlantis A unified workflow for collaborating on Terraform through GitHub. https://github.com/hootsuite/atlantis
65. Demo
66. • atlantis - https://github.com/hootsuite/atlantis • slide deck - https://speakerdeck.com/anubhavmishra/taming-terraform-work flow-using-terraform-modules-and-github • hashicorp-atlantis-demo - https://github.com/lkysow/hashicorp-meetup-atlantis-demo Links
67. Thank You! $ terraform plan -var-file=we-are-hiring.tfvars $ terraform apply hootsuite.com/careers • Atlantis Github: https://github.com/hootsuite/atlantis