Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Taming Terraform Workflow Using Terraform Modul...

Taming Terraform Workflow Using Terraform Modules and Github

Abstract

Today’s cloud infrastructure is really complex. What if you could truly make this infrastructure a black box?
What if you could mutate this infrastructure safely and easily? See how to use Terraform efficiently across
hundreds of developers while leveraging a simple Github workflow and Terraform modules.

Additional info

Hootsuite was one of the early adopters of Terraform and faced some challenges early on with Terraform workflow.
* This session will cover how Hootsuite uses Terraform modules and leverages them to bridge the gap between devs and
operations.
* We will discuss how we came up with our own state management system that helps us safely mutate the state of our
infrastructure from one state to the other. How you can organize your Terraform states across multiple tiers of your
infrastructure. We will also talk about our own HTTP remote state store on top of S3 that enables us to edit states
and rollback.
* Most importantly, we came up with our own Terraform workflow that uses github pull requests to do approvals and
state locking so we don’t corrupt Terraform states for our infrastructure.

Proposal: https://github.com/cfgmgmtcamp/2017-pdx-cfp/blob/master/presentations/taming-terraform-workflow.md

Anubhav Mishra

August 03, 2017
Tweet

More Decks by Anubhav Mishra

Other Decks in Technology

Transcript

  1. Senior Developer - Production Operations and Delivery @anubhavm Taming Terraform

    Workflow Anubhav Mishra Using Terraform Modules and Github
  2. Talk • Terraform at Hootsuite • Use of Terraform Modules

    (Dev <--> Ops) • Atlantis - Hootsuite’s Terraform Workflow • Live Demo
  3. Numbers • 100+ developers • 20+ microservices • 2 cluster

    schedulers • 1000+ servers • 1000s of AWS resources
  4. Timeline • Mid 2015 - First Terraform Repo (v1) •

    Starting 2016 - Terraform (v2) • End of 2016 - Terraform “Advanced” Workflow
  5. What is Terraform? • Terraform allows you to manage, build

    and visualize your infrastructure as code
  6. Problems with our first attempt • Environment drift • One

    massive state for every environment • Hard to collaborate on infrastructure
  7. Problems with our first attempt • Environment drift • One

    massive state for every environment • Hard to collaborate on infrastructure
  8. Problems with our first attempt • Environment drift • One

    massive state for every environment • Hard to collaborate on infrastructure
  9. Problems with our first attempt • Environment drift • One

    massive state for every environment • Hard to collaborate on infrastructure
  10. Terraform Version 2 (Second Attempt) terraform.tfstate { "version": 3, "terraform_version":

    "0.8.8", "serial": 175, "lineage": "2cb11085-2e4e-40ff-bf81-e4ddf8bc8d5d", "modules": [ { "path": [ "root" ], "outputs": {}, "resources": { ....
  11. Terraform Version 2 (Second Attempt) terraform.tfstate { "version": 3, "terraform_version":

    "0.8.8", "serial": 175, "lineage": "2cb11085-2e4e-40ff-bf81-e4ddf8bc8d5d", "modules": [ { "path": [ "root" ], "outputs": {}, "resources": { ....
  12. terraform.tfstate { "version": 3, "terraform_version": "0.8.8", "serial": 1, "lineage": "9e42c0ac-9e73-4910-8859-504934f2b499",

    "backend": { "type": "s3", "config": { "bucket": "terraform", "key": "foo-bar", "region": "us-east-1" }, .... "outputs": {}, "resources": { "null_resource.hello": { "type": "null_resource", "depends_on": [], "primary": { "id": "792399369485920658", "attributes": { "id": "792399369485920658" }, "meta": {}, ....
  13. Problems with our first attempt • Environment drift • One

    massive state for every environment • Hard to collaborate on infrastructure
  14. Problems with our first attempt • Environment drift • One

    massive state for every environment • Hard to collaborate on infrastructure
  15. module.server Let me make a generic server module I need

    2 servers. Now! Terraform Version 2 (Second Attempt)
  16. • Multi Environment Terraform Projects • Isolated Terraform States for

    Services ◦ Smaller Failure Domain ◦ Rapid Iteration on Infrastructure • Use of Terraform Modules ◦ Environment Parity ◦ Increased Collaboration Between Ops and Dev