Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Taming Terraform Workflow Using Terraform Modules and Github

Taming Terraform Workflow Using Terraform Modules and Github

Abstract

Today’s cloud infrastructure is really complex. What if you could truly make this infrastructure a black box?
What if you could mutate this infrastructure safely and easily? See how to use Terraform efficiently across
hundreds of developers while leveraging a simple Github workflow and Terraform modules.

Additional info

Hootsuite was one of the early adopters of Terraform and faced some challenges early on with Terraform workflow.
* This session will cover how Hootsuite uses Terraform modules and leverages them to bridge the gap between devs and
operations.
* We will discuss how we came up with our own state management system that helps us safely mutate the state of our
infrastructure from one state to the other. How you can organize your Terraform states across multiple tiers of your
infrastructure. We will also talk about our own HTTP remote state store on top of S3 that enables us to edit states
and rollback.
* Most importantly, we came up with our own Terraform workflow that uses github pull requests to do approvals and
state locking so we don’t corrupt Terraform states for our infrastructure.

Proposal: https://github.com/cfgmgmtcamp/2017-pdx-cfp/blob/master/presentations/taming-terraform-workflow.md

Anubhav Mishra

August 03, 2017
Tweet

More Decks by Anubhav Mishra

Other Decks in Technology

Transcript

  1. Senior Developer - Production Operations and Delivery
    @anubhavm
    Taming Terraform
    Workflow
    Anubhav Mishra
    Using Terraform Modules and Github

    View full-size slide

  2. Anubhav Mishra
    @anubhavm

    View full-size slide

  3. Talk
    ● Terraform at Hootsuite
    ● Use of Terraform Modules (Dev <--> Ops)
    ● Atlantis - Hootsuite’s Terraform Workflow
    ● Live Demo

    View full-size slide

  4. Terraform at Hootsuite

    View full-size slide

  5. Numbers
    ● 100+ developers
    ● 20+ microservices
    ● 2 cluster schedulers
    ● 1000+ servers
    ● 1000s of AWS resources

    View full-size slide

  6. Timeline
    ● Mid 2015 - First Terraform Repo (v1)
    ● Starting 2016 - Terraform (v2)
    ● End of 2016 - Terraform “Advanced” Workflow

    View full-size slide

  7. Before Terraform

    View full-size slide

  8. What is Terraform?
    ● Terraform allows you to manage, build and visualize
    your infrastructure as code

    View full-size slide

  9. First Attempt

    View full-size slide

  10. First Attempt

    View full-size slide

  11. First Attempt

    View full-size slide

  12. First Attempt
    staging != production

    View full-size slide

  13. Terraform State

    View full-size slide

  14. First Attempt

    View full-size slide

  15. Problems with our first attempt
    ● Environment drift
    ● One massive state for every environment
    ● Hard to collaborate on infrastructure

    View full-size slide

  16. Problems with our first attempt
    ● Environment drift
    ● One massive state for every environment
    ● Hard to collaborate on infrastructure

    View full-size slide

  17. Terraform Version 2 (Second Attempt)

    View full-size slide

  18. Terraform Version 2 (Second Attempt)

    View full-size slide

  19. Terraform Version 2 (Second Attempt)

    View full-size slide

  20. Problems with our first attempt
    ● Environment drift
    ● One massive state for every environment
    ● Hard to collaborate on infrastructure

    View full-size slide

  21. Problems with our first attempt
    ● Environment drift
    ● One massive state for every environment
    ● Hard to collaborate on infrastructure

    View full-size slide

  22. Terraform Version 2 (Second Attempt)
    terraform.tfstate
    {
    "version": 3,
    "terraform_version": "0.8.8",
    "serial": 175,
    "lineage":
    "2cb11085-2e4e-40ff-bf81-e4ddf8bc8d5d",
    "modules": [
    {
    "path": [
    "root"
    ],
    "outputs": {},
    "resources": {
    ....

    View full-size slide

  23. Terraform Version 2 (Second Attempt)
    terraform.tfstate
    {
    "version": 3,
    "terraform_version": "0.8.8",
    "serial": 175,
    "lineage":
    "2cb11085-2e4e-40ff-bf81-e4ddf8bc8d5d",
    "modules": [
    {
    "path": [
    "root"
    ],
    "outputs": {},
    "resources": {
    ....

    View full-size slide

  24. Terraform Version 2 (Second Attempt)
    terraform.tfstate

    View full-size slide

  25. Terraform Version 2 (Second Attempt)

    View full-size slide

  26. Terraform Version 2 (Second Attempt)

    View full-size slide

  27. Terraform Version 2 (Second Attempt)

    View full-size slide

  28. terraform.tfstate
    {
    "version": 3,
    "terraform_version": "0.8.8",
    "serial": 1,
    "lineage": "9e42c0ac-9e73-4910-8859-504934f2b499",
    "backend": {
    "type": "s3",
    "config": {
    "bucket": "terraform",
    "key": "foo-bar",
    "region": "us-east-1"
    },
    ....
    "outputs": {},
    "resources": {
    "null_resource.hello": {
    "type": "null_resource",
    "depends_on": [],
    "primary": {
    "id": "792399369485920658",
    "attributes": {
    "id": "792399369485920658"
    },
    "meta": {},
    ....

    View full-size slide

  29. Problems with our first attempt
    ● Environment drift
    ● One massive state for every environment
    ● Hard to collaborate on infrastructure

    View full-size slide

  30. Problems with our first attempt
    ● Environment drift
    ● One massive state for every environment
    ● Hard to collaborate on infrastructure

    View full-size slide

  31. module.server
    Let me make a
    generic server
    module
    I need 2 servers.
    Now!
    Terraform Version 2 (Second Attempt)

    View full-size slide

  32. Terraform Version 2 (Second Attempt)

    View full-size slide

  33. Terraform Version 2 (Second Attempt)

    View full-size slide

  34. Inputs Output

    View full-size slide

  35. Terraform Version 2 (Second Attempt)

    View full-size slide

  36. Terraform Version 2 (Second Attempt)

    View full-size slide

  37. Terraform Version 2 (Second Attempt)

    View full-size slide

  38. Terraform Version 2 (Second Attempt)

    View full-size slide

  39. Terraform Version 2 (Second Attempt)
    139 Repositories

    View full-size slide

  40. ● Multi Environment Terraform Projects
    ● Isolated Terraform States for Services
    ○ Smaller Failure Domain
    ○ Rapid Iteration on Infrastructure
    ● Use of Terraform Modules
    ○ Environment Parity
    ○ Increased Collaboration Between Ops and Dev

    View full-size slide

  41. Are we all done?

    View full-size slide

  42. How do we continue to move
    fast while we add people?

    View full-size slide

  43. atlantis
    A unified workflow for collaborating on Terraform through GitHub.
    https://github.com/hootsuite/atlantis

    View full-size slide

  44. ● atlantis - https://github.com/hootsuite/atlantis
    ● cfgmgmtcamp-atlantis-demo -
    https://github.com/anubhavmishra/cfgmgmtcamp-atlantis-demo
    Links

    View full-size slide

  45. Thank You!
    Senior Developer - Production Operations and Delivery
    @anubhavm
    Anubhav Mishra

    View full-size slide