Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Terraform Without The Mess

James Nugent
February 03, 2020

Terraform Without The Mess

The way Terraform configuration is written today is impacted hugely by the history of the project and when features essential to minimising complexity became available

In this talk we'll look at some of the history of the project and make specific recommendations about how to structure Terraform configuration in order to make changes simple, predictable and safe.

James Nugent

February 03, 2020

More Decks by James Nugent

Other Decks in Technology


  1. Terraform Without The Mess James Nugent @jen20

  2. The fewer facilities a programming language has, the more important

    rigour is in everyday use to prevent emergence of catastrophic complexity
  3. Bio • CTO at Event Store, a company which builds

    an open source stream database • Terraform Core maintainer at HashiCorp from vEarly to ~v0.8 • Member of the first SRE team at HashiCorp, developing patterns for scalable Terraform and ran for advanced Terraform training classes • Terraform AWS provider maintainer until ~September 2019 • Pulumi contributor since June 2018 • Tweet @jen20 primarily about Rust and Brexit (ffs)
  4. None
  5. Survey Questions…

  6. None
  7. Infrastructure as Code

  8. Infrastructure as Code?

  9. Infrastructure as Code Infrastructure as Software

  10. Ancient History…

  11. Terraform 0.1

  12. Terraform 0.1 • No modules - everything lived in one

    namespace • No terraform_remote_state resource • No remote state at all! • Very “forgiving” HCL dialect (pre-HCL 1.0)
  13. Terraform 0.3

  14. Terraform 0.3

  15. Terraform 0.3.5

  16. Terraform 0.4.0

  17. Terraform 0.6.0

  18. Terraform 0.6.0

  19. output “private_subnets” { value = “${join(aws_subnet.private.*.id, “,”)}” } resource “aws_instance”

    “servers” { // other fields subnet_id = “${element(split(var.private_subnets, “,”, count.index)}” }
  20. By Terraform 0.6, we had the tools to avoid creating

    a mess…
  21. … but people already had large amounts of Terraform code,

    and the ‘best practices’ were already in the wild.
  22. Scalable Infrastructure Patterns

  23. Microservices

  24. Microservices • This is a tortuous analogy because I don’t

    actually like the name Microservices, or many of the concepts that it has come to embody. • Perhaps Service Oriented Architecture (SOA) is a better name • The gestalt is that we prefer to keep services (“applications”): • Small, so they can easily be understood (“micro”) • Independent, so that issues in one service are less likely to affect another • Similar to the UNIX philosophy of applications doing one thing (and doing it well).
  25. Terraform • We should also prefer to keep Terraform “applications”

    as small as possible: • An “application” is the module where we type terraform {plan,apply} • One “application” represents a single state file • A single state file is: • A Security boundary (RBAC in TFE, ) • A Blast Radius boundary in terms of accidental destruction
  26. Terraform • One state file for an entire infrastructure is

    A Bad Time waiting to happen • Since Terraform 0.4, we don’t have to do this! • Prefer to keep configuration for each aspect of a system in a separate “application” • Aspects of the system live in a hierarchy • Refer to outputs of “applications” lower in the hierarchy by using the terraform_remote_state data source
  27. Bastion Host “Application” Web Site “Application” Network “Application” terraform_remote_state Private

    subnet IDs Public subnet IDs terraform_remote_state Private subnet IDs
  28. Prefer to have more state files which are individually smaller

  29. “organizations which design systems... are constrained to produce designs which

    are copies of the communication structures of these organizations.” — Melvin Conway, 1967
  30. The “Composition Root” Pattern

  31. None
  32. The “Composition Root” Pattern • “Where should we compose object

    module graphs?” • “As close as possible to the application’s entry point” • “A Composition Root is a (preferably) unique location in an application where modules are composed together.” • “Only at the entry point of the application is the entire module graph finally composed” • “Only applications should have composition roots. Libraries and frameworks shouldn’t have them”
  33. Applying this to Terraform…

  34. Modules should either manage resources or compose other modules

  35. There should be one composition root per application, at the

    root, parameterised for environments
  36. Composition Root Configuration • Composition Roots should consist of very

    little code and minimal logic: • Variables - only things that need to change per deployment or workspace at the current time! • State Storage Configuration - to confi gure the backend. • Data Sources - to query the state of the cloud based on variables passed in, and to link to other “applications” in the same system. • Module Declarations - to instantiate the modules which create resources. • Outputs - only the things that need to be provided to other stacks at the current time!
  37. None
  38. Resource Modules

  39. “SOLID” Principles

  40. Resource Modules • The modules instantiated by composition roots should

    be: • Simple - no “clever” hacks unless they absolutely cannot be avoided. • Flat - no nested module dependencies. • Pure - treat them like functions. • More importantly they should be: • Cohesive - resources related to the same aspect of a system should live in the same module. • Decoupled - no dependencies on other modules, only on the variables they are passed (and potentially limited data sources).
  41. Resource Module Structure

  42. Module Design - An Interface

  43. Module Design - Resources repo.tf dns.tf stage.tf

  44. Resource Module Anti-Patterns • Modules to create an individual resource

    or small group of resources which expose every option as a configuration point • Modules are an abstraction - treat them that way • Modules which instantiate other modules • State file gets complex, as do resource paths (especially pre-Terraform 0.12) • Refactoring is hard • Complexity spirals out of control • Golden rule: Only composition roots instantiate modules
  45. This approach scales well both with infrastructure size and number

    of infrastructure developers
  46. Enforcing a level of rigour regarding this in your organisation

    helps given the fact that the language does not provide much.
  47. Testing • Testing has been a thing in application development

    for some time now. Why don’t we apply it to infrastructure? • The “testing pyramid” defines various stages of testing for software: • Unit Tests - Fast, do not communicate without external resources. Verify results “in the small”. Lots of them. • The unit is the test NOT the test subject. • Acceptance Tests - Slow(er), communicate with external resources and verify results throughout a system. Relatively fewer of them.
  48. Applying this to Terraform • HashiCorp Sentinal • GruntWork Terratest

    • ServerSpec • Pulumi Testing Framework and runner • Detailed analysis out of scope for this talk, there are other talks at this conference about this with more information (similarly at FOSDEM with videos)
  49. For a step-change in the usability of Infrastructure of Code,

    we need to instead think of Infrastructure as Software
  50. Resources • Any software engineering book covering functional or object

    oriented design. • Terraform Design Guide on the HashiCorp website • HashiDays NYC 2017 Talk - Open source code illustrating this pattern in AWS for a wide variety of infrastructure pieces: • Code: https://github.com/jen20/hashidays-nyc • For Terraform 0.11 • Pull Requests Accepted™