Patterns to Refactor Terraform

Originally presented at NYC HashiCorp User Group

Transcript

1. Copyright © 2020 HashiCorp
   Patterns to Refactor
   Terraform
   
   
   NYC HashiCorp User Group
   November 2022
   

   View Slide

2. View Slide

3. Developer Advocate at HashiCorp
   

   she/her
   

   

   @joatmon08
   
   
   joatmon08.github.io
   Rosemary Wang
   

   View Slide

4. Write “Cleaner”
   Terraform
   
   
   For happier future refactoring
   

   View Slide

5. Pattern:
   Find and
   Replace
   
   
   😫😫😫
   resource "aws_lb_target_group" "app" {
   port = 3000
   }
   resource "aws_security_group" “app” {
   ingress {
   description = "Access to app."
   from_port = 3000
   to_port = 3000
   protocol = "tcp"
   security_groups =
   aws_security_group.ingress_alb.id
   }
   }
   CODE EDITOR
   

   View Slide

6. Pattern:
   Variables
   with
   Defaults
   
   
   😀😀😀
   
   
   Easier to move into
   modules later
   variable "application_port" {
   default = 3000
   }
   resource "aws_lb_target_group" "app" {
   port = var.application_port
   }
   resource "aws_security_group" “app” {
   ingress {
   description = "Access to app."
   from_port = var.application_port
   to_port = var.application_port
   protocol = "tcp"
   security_groups =
   aws_security_group.ingress_alb.id
   }
   }
   CODE EDITOR
   

   View Slide

7. Pattern:
   Secrets as
   Variables
   
   
   🧐 🧐 🧐
   variable "password" {
   type = string
   sensitive = true
   }
   resource "aws_db_instance" "db" {
   password = var.password
   }
   CODE EDITOR
   

   View Slide

8. Pattern:
   Secrets as
   Dynamic
   Values
   
   
   😀😀😀
   resource "random_password" "db" {
   length = 16
   min_upper = 2
   min_lower = 2
   min_numeric = 2
   min_special = 2
   special = true
   override_special = "`~!#$%^&*?"
   }
   resource "aws_db_instance" "db" {
   password = random_password.db.result
   }
   CODE EDITOR
   

   View Slide

9. Pattern:
   Secrets as
   Data
   Sources
   
   
   🤩🤩🤩
   data "vault_generic_secret" "db" {
   path = "app/database/admin"
   }
   resource "aws_db_instance" "db" {
   password =
   data.vault_generic_secret.db.data["password"]
   }
   CODE EDITOR
   

   View Slide

10. Pattern:
    Prototype
    Module
    
    
    😀😀😀
    module "tags" {
    source = "example/tags/aws"
    organization = "HashiCorp"
    unit = "HUG"
    }
    ## Tags Module with Prototype Pattern
    variable "organization" {}
    variable "unit" {}
    output "tags" {
    value = {
    organization = var.organization
    unit = var.unit
    expiration = timestamp()
    }
    }
    CODE EDITOR
    

    View Slide

11. 1. Variables - names, tags, environments
    
    
    2. Constants - set defaults if overrides, otherwise locals
    
    
    3. Dynamic Values - use interpolation / data sources
    
    
    4. Secrets - generate dynamically / data sources
    

    View Slide

12. Break Down Monoliths
    
    
    There’s no such thing as micro-Terraform
    

    View Slide

13. Monolithic Singleton
    
    
    put everything in one configuration and one state to rule it all.
    

    View Slide

14. View Slide

15. Do I have a monolithic singleton?
    
    
    1. Find and replace
    
    
    2. Long apply time
    

    View Slide

16. Refactor
    
    
    1. Find and replace → modules
    
    
    2. Long apply time → state
    

    View Slide

17. Module Refactor
    

    View Slide

18. Pattern: Use
    moved
    Module Refactor
    Only
    resource "aws_instance" "bastion" {}
    module "bastion" {}
    moved {
    from = aws_instance.bastion
    to = 

    module.bastion.aws_instance.bastion
    }
    CODE EDITOR
    developer.hashicorp.com/terraform/language/modules/develop/refactoring
    

    View Slide

19. Pattern: Use
    import
    Module Refactor
    TERMINAL
    # Build module
    > terraform import \

    module...
    > terraform state rm .
    # Comment out old resources in Terraform
    > terraform plan # check no drift
    # Delete old resources in Terraform
    

    View Slide

20. State Refactor
    

    View Slide

21. Dependency Injection
    
    
    decouple dependencies to handle cross-state interpolation
    

    View Slide

22. Pattern:
    Dependency
    Injection
    with
    Terraform
    Outputs
    # create network in TFC workspace
    data "tfe_outputs" "network" {
    organization = "hashicorp"
    workspace = "network"
    }
    resource "aws_instance" "server" {
    subnet_id =
    data.tfe_outputs.network.values.private_subnets.0
    }
    CODE EDITOR
    

    View Slide

23. Pattern:
    Dependency
    Injection
    with Data
    Sources
    data "aws_vpcs" "network" {
    tags = {
    unit = "hug"
    }
    }
    data "aws_subnet_ids" "private" {
    vpc_id = data.aws_vpcs.network.ids.0
    tags = {
    Type = "private"
    }
    }
    resource "aws_instance" "server" {
    subnet_id = data.aws_subnet_ids.private.ids.0
    }
    CODE EDITOR
    

    View Slide

24. State Refactor
    

    View Slide

25. Pattern: Use
    import
    State Refactor
    TERMINAL
    # Copy resource configurations to new
    working directory
    # Implement dependency injection
    > cd
    > terraform init
    > terraform import .
    > terraform plan # check no drift
    

    View Slide

26. Pattern: Use
    import
    State Refactor
    TERMINAL
    > cd
    > terraform state rm .
    # Comment out old resources in Terraform
    > terraform plan # check no drift
    # Delete old resources in Terraform
    

    View Slide

27. Refactor Rules
    
    
    1. Go from high-level to low-level resources
    
    
    2. Choose resources with fewer dependencies
    
    
    3. terraform plan = test
    
    
    4. terraform state = source of truth
    

    View Slide

28. Use Immutability
    
    
    When all else fails, create everything new.
    

    View Slide

29. Many scenarios…
    
    
    1. High-risk refactors
    
    
    2. Failures
    
    
    3. Terraform upgrades (syntax)
    

    View Slide

30. Pattern:
    Blue-green
    deployment
    Old Network New Network
    Servers Servers
    Modules in
    
    
    same state
    Module in
    
    
    separate state
    Module in
    
    
    separate state
    

    View Slide

31. Pattern:
    Blue-green
    deployment
    
    
    Blue resources share the
    same state
    terraform {
    cloud {
    organization = "hashicorp"
    workspaces {
    name = “hug"
    }
    }
    }
    module "network" {}
    module "servers" {
    count = 3
    subnet = module.network.subnet.0
    }
    CODE EDITOR
    

    View Slide

32. Pattern:
    Blue-green
    deployment
    
    
    Green resources in
    separate states
    
    
    terraform {
    cloud {
    organization = "hashicorp"
    workspaces {
    name = "hug-network"
    }
    }
    }
    module "network" {}
    output “subnets" {
    value = module.network.subnets
    }
    CODE EDITOR
    

    View Slide

33. Pattern:
    Blue-green
    deployment
    
    
    Green resources in
    separate states
    
    
    terraform {
    cloud {
    organization = "hashicorp"
    workspaces {
    name = "hug-servers"
    }
    }
    }
    data "tfe_outputs" "network" {
    organization = "hashicorp"
    workspace = "hug-network"
    }
    module "servers" {
    count = 3
    subnet = data.tfe_outputs.network.values.subnets.0
    }
    CODE EDITOR
    

    View Slide

34. Pattern:
    Blue-green
    deployment
    Old Network New Network
    Servers
    Servers Servers
    Modules in
    
    
    same state
    Module in
    
    
    separate state
    Module in
    
    
    separate state
    Load Balancer
    50%
    50%
    

    View Slide

35. Refactoring is inevitable.
    
    
    1. Takes time and effort.
    
    
    2. Go high-level to low-level.
    
    
    3. Roll forward.
    

    View Slide

36. Rosemary Wang
    

    @joatmon08
    
    
    joatmon08.github.io
    Thanks!
    

    View Slide