Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Taming Terraform Workflow Using Terraform Modules and Github

Taming Terraform Workflow Using Terraform Modules and Github

Abstract

Today’s cloud infrastructure is really complex. What if you could truly make this infrastructure a black box?
What if you could mutate this infrastructure safely and easily? See how to use Terraform efficiently across
hundreds of developers while leveraging a simple Github workflow and Terraform modules.

Additional info

Hootsuite was one of the early adopters of Terraform and faced some challenges early on with Terraform workflow.
* This session will cover how Hootsuite uses Terraform modules and leverages them to bridge the gap between devs and
operations.
* We will discuss how we came up with our own state management system that helps us safely mutate the state of our
infrastructure from one state to the other. How you can organize your Terraform states across multiple tiers of your
infrastructure. We will also talk about our own HTTP remote state store on top of S3 that enables us to edit states
and rollback.
* Most importantly, we came up with our own Terraform workflow that uses github pull requests to do approvals and
state locking so we don’t corrupt Terraform states for our infrastructure.

Proposal: https://github.com/cfgmgmtcamp/2017-pdx-cfp/blob/master/presentations/taming-terraform-workflow.md

Anubhav Mishra

August 03, 2017
Tweet

More Decks by Anubhav Mishra

Other Decks in Technology

Transcript

  1. Senior Developer - Production Operations and Delivery
    @anubhavm
    Taming Terraform
    Workflow
    Anubhav Mishra
    Using Terraform Modules and Github

    View Slide

  2. Anubhav Mishra
    @anubhavm

    View Slide

  3. View Slide

  4. View Slide

  5. Talk
    ● Terraform at Hootsuite
    ● Use of Terraform Modules (Dev <--> Ops)
    ● Atlantis - Hootsuite’s Terraform Workflow
    ● Live Demo

    View Slide

  6. Terraform at Hootsuite

    View Slide

  7. Numbers
    ● 100+ developers
    ● 20+ microservices
    ● 2 cluster schedulers
    ● 1000+ servers
    ● 1000s of AWS resources

    View Slide

  8. Timeline
    ● Mid 2015 - First Terraform Repo (v1)
    ● Starting 2016 - Terraform (v2)
    ● End of 2016 - Terraform “Advanced” Workflow

    View Slide

  9. Before Terraform

    View Slide

  10. View Slide

  11. View Slide

  12. View Slide

  13. What is Terraform?
    ● Terraform allows you to manage, build and visualize
    your infrastructure as code

    View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. First Attempt

    View Slide

  18. First Attempt

    View Slide

  19. First Attempt

    View Slide

  20. First Attempt
    staging != production

    View Slide

  21. Terraform State

    View Slide

  22. First Attempt

    View Slide

  23. View Slide

  24. Problems with our first attempt
    ● Environment drift
    ● One massive state for every environment
    ● Hard to collaborate on infrastructure

    View Slide

  25. Problems with our first attempt
    ● Environment drift
    ● One massive state for every environment
    ● Hard to collaborate on infrastructure

    View Slide

  26. Terraform Version 2 (Second Attempt)

    View Slide

  27. Terraform Version 2 (Second Attempt)

    View Slide

  28. Terraform Version 2 (Second Attempt)

    View Slide

  29. Problems with our first attempt
    ● Environment drift
    ● One massive state for every environment
    ● Hard to collaborate on infrastructure

    View Slide

  30. Problems with our first attempt
    ● Environment drift
    ● One massive state for every environment
    ● Hard to collaborate on infrastructure

    View Slide

  31. Terraform Version 2 (Second Attempt)
    terraform.tfstate
    {
    "version": 3,
    "terraform_version": "0.8.8",
    "serial": 175,
    "lineage":
    "2cb11085-2e4e-40ff-bf81-e4ddf8bc8d5d",
    "modules": [
    {
    "path": [
    "root"
    ],
    "outputs": {},
    "resources": {
    ....

    View Slide

  32. Terraform Version 2 (Second Attempt)
    terraform.tfstate
    {
    "version": 3,
    "terraform_version": "0.8.8",
    "serial": 175,
    "lineage":
    "2cb11085-2e4e-40ff-bf81-e4ddf8bc8d5d",
    "modules": [
    {
    "path": [
    "root"
    ],
    "outputs": {},
    "resources": {
    ....

    View Slide

  33. Terraform Version 2 (Second Attempt)
    terraform.tfstate

    View Slide

  34. Terraform Version 2 (Second Attempt)

    View Slide

  35. Terraform Version 2 (Second Attempt)

    View Slide

  36. Terraform Version 2 (Second Attempt)

    View Slide

  37. View Slide

  38. terraform.tfstate
    {
    "version": 3,
    "terraform_version": "0.8.8",
    "serial": 1,
    "lineage": "9e42c0ac-9e73-4910-8859-504934f2b499",
    "backend": {
    "type": "s3",
    "config": {
    "bucket": "terraform",
    "key": "foo-bar",
    "region": "us-east-1"
    },
    ....
    "outputs": {},
    "resources": {
    "null_resource.hello": {
    "type": "null_resource",
    "depends_on": [],
    "primary": {
    "id": "792399369485920658",
    "attributes": {
    "id": "792399369485920658"
    },
    "meta": {},
    ....

    View Slide

  39. stash

    View Slide

  40. View Slide

  41. View Slide

  42. View Slide

  43. Problems with our first attempt
    ● Environment drift
    ● One massive state for every environment
    ● Hard to collaborate on infrastructure

    View Slide

  44. Problems with our first attempt
    ● Environment drift
    ● One massive state for every environment
    ● Hard to collaborate on infrastructure

    View Slide

  45. View Slide

  46. View Slide

  47. View Slide

  48. module.server
    Let me make a
    generic server
    module
    I need 2 servers.
    Now!
    Terraform Version 2 (Second Attempt)

    View Slide

  49. View Slide

  50. Terraform Version 2 (Second Attempt)

    View Slide

  51. Terraform Version 2 (Second Attempt)

    View Slide

  52. Inputs Output

    View Slide

  53. Terraform Version 2 (Second Attempt)

    View Slide

  54. Terraform Version 2 (Second Attempt)

    View Slide

  55. Terraform Version 2 (Second Attempt)

    View Slide

  56. Terraform Version 2 (Second Attempt)

    View Slide

  57. Terraform Version 2 (Second Attempt)
    139 Repositories

    View Slide

  58. ● Multi Environment Terraform Projects
    ● Isolated Terraform States for Services
    ○ Smaller Failure Domain
    ○ Rapid Iteration on Infrastructure
    ● Use of Terraform Modules
    ○ Environment Parity
    ○ Increased Collaboration Between Ops and Dev

    View Slide

  59. Are we all done?

    View Slide

  60. View Slide

  61. View Slide

  62. View Slide

  63. How do we continue to move
    fast while we add people?

    View Slide

  64. View Slide

  65. atlantis
    A unified workflow for collaborating on Terraform through GitHub.
    https://github.com/hootsuite/atlantis

    View Slide

  66. Demo

    View Slide

  67. ● atlantis - https://github.com/hootsuite/atlantis
    ● cfgmgmtcamp-atlantis-demo -
    https://github.com/anubhavmishra/cfgmgmtcamp-atlantis-demo
    Links

    View Slide

  68. Thank You!
    Senior Developer - Production Operations and Delivery
    @anubhavm
    Anubhav Mishra

    View Slide