Taming Terraform Workflow Using Terraform Modules and Github

Taming Terraform Workflow Using Terraform Modules and Github

Abstract

Today’s cloud infrastructure is really complex. What if you could truly make this infrastructure a black box?
What if you could mutate this infrastructure safely and easily? See how to use Terraform efficiently across
hundreds of developers while leveraging a simple Github workflow and Terraform modules.

Additional info

Hootsuite was one of the early adopters of Terraform and faced some challenges early on with Terraform workflow.
* This session will cover how Hootsuite uses Terraform modules and leverages them to bridge the gap between devs and
operations.
* We will discuss how we came up with our own state management system that helps us safely mutate the state of our
infrastructure from one state to the other. How you can organize your Terraform states across multiple tiers of your
infrastructure. We will also talk about our own HTTP remote state store on top of S3 that enables us to edit states
and rollback.
* Most importantly, we came up with our own Terraform workflow that uses github pull requests to do approvals and
state locking so we don’t corrupt Terraform states for our infrastructure.

Proposal: https://github.com/cfgmgmtcamp/2017-pdx-cfp/blob/master/presentations/taming-terraform-workflow.md

26896287bc831a13e768cea7efe29632?s=128

Anubhav Mishra

August 03, 2017
Tweet

Transcript

  1. Senior Developer - Production Operations and Delivery @anubhavm Taming Terraform

    Workflow Anubhav Mishra Using Terraform Modules and Github
  2. Anubhav Mishra @anubhavm

  3. None
  4. None
  5. Talk • Terraform at Hootsuite • Use of Terraform Modules

    (Dev <--> Ops) • Atlantis - Hootsuite’s Terraform Workflow • Live Demo
  6. Terraform at Hootsuite

  7. Numbers • 100+ developers • 20+ microservices • 2 cluster

    schedulers • 1000+ servers • 1000s of AWS resources
  8. Timeline • Mid 2015 - First Terraform Repo (v1) •

    Starting 2016 - Terraform (v2) • End of 2016 - Terraform “Advanced” Workflow
  9. Before Terraform

  10. None
  11. None
  12. None
  13. What is Terraform? • Terraform allows you to manage, build

    and visualize your infrastructure as code
  14. None
  15. None
  16. None
  17. First Attempt

  18. First Attempt

  19. First Attempt

  20. First Attempt staging != production

  21. Terraform State

  22. First Attempt

  23. None
  24. Problems with our first attempt • Environment drift • One

    massive state for every environment • Hard to collaborate on infrastructure
  25. Problems with our first attempt • Environment drift • One

    massive state for every environment • Hard to collaborate on infrastructure
  26. Terraform Version 2 (Second Attempt)

  27. Terraform Version 2 (Second Attempt)

  28. Terraform Version 2 (Second Attempt)

  29. Problems with our first attempt • Environment drift • One

    massive state for every environment • Hard to collaborate on infrastructure
  30. Problems with our first attempt • Environment drift • One

    massive state for every environment • Hard to collaborate on infrastructure
  31. Terraform Version 2 (Second Attempt) terraform.tfstate { "version": 3, "terraform_version":

    "0.8.8", "serial": 175, "lineage": "2cb11085-2e4e-40ff-bf81-e4ddf8bc8d5d", "modules": [ { "path": [ "root" ], "outputs": {}, "resources": { ....
  32. Terraform Version 2 (Second Attempt) terraform.tfstate { "version": 3, "terraform_version":

    "0.8.8", "serial": 175, "lineage": "2cb11085-2e4e-40ff-bf81-e4ddf8bc8d5d", "modules": [ { "path": [ "root" ], "outputs": {}, "resources": { ....
  33. Terraform Version 2 (Second Attempt) terraform.tfstate

  34. Terraform Version 2 (Second Attempt)

  35. Terraform Version 2 (Second Attempt)

  36. Terraform Version 2 (Second Attempt)

  37. None
  38. terraform.tfstate { "version": 3, "terraform_version": "0.8.8", "serial": 1, "lineage": "9e42c0ac-9e73-4910-8859-504934f2b499",

    "backend": { "type": "s3", "config": { "bucket": "terraform", "key": "foo-bar", "region": "us-east-1" }, .... "outputs": {}, "resources": { "null_resource.hello": { "type": "null_resource", "depends_on": [], "primary": { "id": "792399369485920658", "attributes": { "id": "792399369485920658" }, "meta": {}, ....
  39. stash

  40. None
  41. None
  42. None
  43. Problems with our first attempt • Environment drift • One

    massive state for every environment • Hard to collaborate on infrastructure
  44. Problems with our first attempt • Environment drift • One

    massive state for every environment • Hard to collaborate on infrastructure
  45. None
  46. None
  47. None
  48. module.server Let me make a generic server module I need

    2 servers. Now! Terraform Version 2 (Second Attempt)
  49. None
  50. Terraform Version 2 (Second Attempt)

  51. Terraform Version 2 (Second Attempt)

  52. Inputs Output

  53. Terraform Version 2 (Second Attempt)

  54. Terraform Version 2 (Second Attempt)

  55. Terraform Version 2 (Second Attempt)

  56. Terraform Version 2 (Second Attempt)

  57. Terraform Version 2 (Second Attempt) 139 Repositories

  58. • Multi Environment Terraform Projects • Isolated Terraform States for

    Services ◦ Smaller Failure Domain ◦ Rapid Iteration on Infrastructure • Use of Terraform Modules ◦ Environment Parity ◦ Increased Collaboration Between Ops and Dev
  59. Are we all done?

  60. None
  61. None
  62. None
  63. How do we continue to move fast while we add

    people?
  64. None
  65. atlantis A unified workflow for collaborating on Terraform through GitHub.

    https://github.com/hootsuite/atlantis
  66. Demo

  67. • atlantis - https://github.com/hootsuite/atlantis • cfgmgmtcamp-atlantis-demo - https://github.com/anubhavmishra/cfgmgmtcamp-atlantis-demo Links

  68. Thank You! Senior Developer - Production Operations and Delivery @anubhavm

    Anubhav Mishra