Pro Yearly is on sale from $80 to $50! »

What's new in Terraform 0.10 - HUG

What's new in Terraform 0.10 - HUG

In this talk we cover an introduction to Terraform, including it's primary use cases of Infrastructure as Code, composing multiple providers (such as IaaS, SaaS, and PaaS), and enabling self service of infrastructure. We show how Terraform works and the basics of the workflow. Then we talk about new features of Terraform 0.10, with a focus on the split between core and providers, the Kubernetes provider, and workspaces. Terraform 0.9 is reviewed with a focus on destroy provisioners, remote backends, state locking.

11ba9630c9136eef9a70d26473d355d5?s=128

Armon Dadgar

August 16, 2017
Tweet

Transcript

  1. None
  2. @armon ARMON DADGAR

  3. AGENDA HASHICORP USE CASES INTRO TO TERRAFORM NEW FEATURES Terraform

    0.10 Terraform 0.9
  4. PROVISION, SECURE AND RUN ANY INFRASTRUCTURE Nomad Consul Vault Vagrant

    Packer Terraform Consul Enterprise Terraform Enterprise Vault Enterprise PRODUCT SUITE OSS TOOL SUITE RUN Applications SECURE Application Infrastructure PROVISION Infrastructure FOR INDIVIDUALS FOR TEAMS Nomad Enterprise
  5. SEP 18-20, 2017 | AUSTIN, TEXAS One day of training

    followed by two days of talks on the future of infrastructure. Discount Code: HUG20
  6. 6 HashiCorp confidential do not distribute USE CASES Easily combine

    and build complex infrastructure Self-Serve Infrastructure Common Workflow for Any Infrastructure Hybrid Cloud Management Automate Infrastructure Provisioning Infrastructure as Code
  7. 7 HashiCorp confidential do not distribute COLLABORATE on infrastructure as

    code using a version control system (VCS) VALIDATE with a common integrated workflow AUTOMATE public cloud, private cloud, and external services THE IDEAL STATE
  8. Terraform

  9. Terraform Goals • Unified view of infrastructure • Infrastructure as

    code • Compose multiple tiers (IaaS to PaaS to SaaS) • Safely change/iterate infrastructure over time • One workflow
  10. Terraform Features • Open Source • Infrastructure as Code •

    Resource Providers • Plan and Apply • Collaboration, History [Enterprise]
  11. Infrastructure as Code

  12. File resource "google_compute_instance" "server" { name = "server" machine_type =

    "g1-small" zone = "us-central1-a" disk { image = "ubuntu-1404-trusty-v20160114e" } } resource "dnsimple_record" "hello" { domain = "example.com" name = "server" value = "${google_compute_instance.server.network_interface.0.address}" type = "A" }
  13. File resource "google_compute_instance" "server" { name = "server" machine_type =

    "g1-small" zone = "us-central1-a" disk { image = "ubuntu-1404-trusty-v20160114e" } } resource "dnsimple_record" "hello" { domain = "example.com" name = "server" value = "${google_compute_instance.server.network_interface.0.address}" type = "A" }
  14. File resource "google_compute_instance" "server" { name = "server" machine_type =

    "g1-small" zone = "us-central1-a" disk { image = "ubuntu-1404-trusty-v20160114e" } } resource "dnsimple_record" "hello" { domain = "example.com" name = "server" value = "${google_compute_instance.server.network_interface.0.address}" type = "A" }
  15. Infrastructure as Code • Human-readable (HCL), Machine-editable (JSON) • Declarative

    • Text makes it versionable and VCS-friendly • Can be split into multiple files and modules
  16. Resource Providers

  17. Resource Providers • Integration point • Expose resources, such as

    "aws_instance", etc. • CRUD API to implement • Core vs. Providers simplifies logic for external contributors
  18. Resource Providers Amazon BitBucket CenturyLink Cloud CloudFlare CloudStack Cobbler Consul

    Datadog DigitalOcean DNSMadeEasy DNSimple Docker Dyn GitHub Fastly Google Heroku Librato MicrosoB Azure MySQL OpenStack Packet PostgreSQL SoBLayer UltraDNS VMware Sphere and more...
  19. Resource Providers Amazon BitBucket CenturyLink Cloud CloudFlare CloudStack Cobbler Consul

    Datadog DigitalOcean DNSMadeEasy DNSimple Docker Dyn GitHub Fastly Google Heroku Librato MicrosoB Azure MySQL OpenStack Packet PostgreSQL SoBLayer UltraDNS VMware Sphere and more...
  20. Resource Providers Amazon BitBucket CenturyLink Cloud CloudFlare CloudStack Cobbler Consul

    Datadog DigitalOcean DNSMadeEasy DNSimple Docker Dyn GitHub Fastly Google Heroku Librato MicrosoB Azure MySQL OpenStack Packet PostgreSQL SoBLayer UltraDNS VMware Sphere and more...
  21. Resource Providers Amazon BitBucket CenturyLink Cloud CloudFlare CloudStack Cobbler Consul

    Datadog DigitalOcean DNSMadeEasy DNSimple Docker Dyn GitHub Fastly Google Heroku Librato MicrosoB Azure MySQL OpenStack Packet PostgreSQL SoBLayer UltraDNS VMware Sphere and more...
  22. Resource Providers Amazon BitBucket CenturyLink Cloud CloudFlare CloudStack Cobbler Consul

    Datadog DigitalOcean DNSMadeEasy DNSimple Docker Dyn GitHub Fastly Google Heroku Librato MicrosoB Azure MySQL OpenStack Packet PostgreSQL SoBLayer UltraDNS VMware Sphere and more...
  23. Plan and Apply

  24. File resource "google_compute_instance" "server" { name = "server" machine_type =

    "g1-small" zone = "us-central1-a" disk { image = "ubuntu-1404-trusty-v20160114e" } } resource "dnsimple_record" "hello" { domain = "example.com" name = "server" value = "${google_compute_instance.server.network_interface.0.address}" type = "A" }
  25. Plan • Plan shows you what will happen • Plans

    can be saved to guarantee what will happen • Plans show reasons for certain actions (such as re-create) • Not equivalent to "noop" due to the ability to save a plan
  26. Terminal $ terraform plan + google_compute_instance.server can_ip_forward: "false" disk.#: "1"

    disk.0.auto_delete: "true" disk.0.image: "ubuntu-1404-trusty-..." machine_type: "g1-small" metadata_fingerprint: "<computed>" name: "server" network_interface.#: "1" network_interface.0.address: "<computed>" network_interface.0.name: "<computed>" network_interface.0.network: "default" self_link: "<computed>" tags_fingerprint: "<computed>" zone: "us-central1-a" ... Plan: 3 to add, 0 to change, 0 to destroy.
  27. Apply • Executes changes in order based on dependencies •

    Parallelizes changes when possible • Handles and recovers transient errors
  28. Terminal $ terraform apply google_compute_instance.server: Creating... can_ip_forward: "" => "false"

    disk.#: "" => "1" disk.0.auto_delete: "" => "true" disk.0.image: "" => "ubuntu-1404-trusty..." machine_type: "" => "g1-small" metadata_fingerprint: "" => "<computed>" name: "" => "server" network_interface.#: "" => "1" network_interface.0.address: "" => "<computed>" network_interface.0.name: "" => "<computed>" network_interface.0.network: "" => "default" self_link: "" => "<computed>" tags_fingerprint: "" => "<computed>" zone: "" => "us-central1-a" ... Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
  29. Apply for Changes • Not only creation, but changes over

    time • Plan will show you what will happen • The `-target` flag can be used for fine-grained change
  30. NEW FEATURES IN TERRAFORM 0.10 AND 0.9

  31. Terraform 0.10 (August 2, 2017) • Split of Terraform Core

    / Providers • Kubernetes Provider • Workspaces renamed from State Environments
  32. Split of Core / Providers

  33. Terraform Core vs Provider Core AWS Provider Fastly Provider HCL


    Files API Integration Terraform
  34. Terraform Parts • Core Responsible for: • Parsing Configuration •

    Resource Graph • Interpolation • State Management • Execution • Providers responsible for: • Resource Definition & Validation • API Integration
  35. Terraform Repository • Everything lived in a single repository •

    Released ~2 weeks • Version Sprawl (10+ releases per major version)
  36. Mono Repo Issues • Terraform Core moves slowly • Weeks

    to implement new features • Terraform providers move fast • 70+ Providers today • 1000+ Resources • Hours to implement new resources
  37. Mono Repo Issues • Hard to triage issues • 2000+

    issues • Core Bugs to Provider Feature Requests • Hard to implement Core features • Hard to scale providers • Impossible to update Provider without updating Core
  38. Divide and Conquer • Grand Split proposed months ago •

    Split Providers into individual repositories • Release Providers on independent cadence • Download Providers as needed • Version constrain Providers • "terraform init”
  39. None
  40. File provider aws { version = "~> v0.1.3" region =

    "us-west-2" } provider fastly { # Implicit: version = “latest” api_key = “exampleapikey" }
  41. Terminal $ terraform init Initializing provider plugins... - Checking for

    available provider plugins on https:// releases.hashicorp.com... - Downloading plugin for provider "fastly" (0.1.2)... - Downloading plugin for provider "aws" (0.1.4)... The following providers do not have any version constraints in configuration, so the latest version was installed. To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "..." constraints to the corresponding provider blocks in configuration, with the constraint strings suggested below. * provider.fastly: version = "~> 0.1" Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands
  42. Terraform Core vs Provider Core 0.10 AWS Provider 0.1.4 Fastly

    Provider 0.1.2 Plugin Interface
  43. End Goal • Independent Versioning and Release for Providers /

    Core • Live documentation for multiple versions • Make it easier to add and support Providers • Infrastructure as Code for all the things!
  44. Kubernetes Provider

  45. Kubernetes Provider • Terraform manages IaaS, SaaS, and PaaS •

    Kubernetes has sub-resources to manage (Pods, Controllers) • Compose other resources (load balancers, DNS, CDN) with K8S • Common workflow for everything
  46. Kubernetes Provider • Data Source: kubernetes_service • Data Source: kubernetes_storage_class

    • Resource: kubernetes_config_map • Resource: kubernetes_horizontal_pod_autoscalar • Resource: kubernetes_limit_range • Resource: kubernetes_namespace • Resource: kubernetes_persistent_volume • Resource: kubernetes_persistent_volume_claim • Resource: kubernetes_pod • Resource: kubernetes_replication_controller • Resource: kubernetes_resource_quota • Resource: kubernetes_secret • Resource: kubernetes_service • Resource: kubernetes_service_account • Resource: kubernetes_storage_class • …
  47. Composing Resources Provider Google google_container_cluster Provider Kubernetes kubernetes_namespace kubernetes_pod kubernetes_replication_controller

    consul_keys Provider Consul
  48. Using Schedulers with Terraform • Kubernetes • Nomad • Docker

    Swarm • AWS ECS • Manage Application Lifecycle • Compose with other resources
  49. Workspaces Formerly: State Environments

  50. Workspaces • A state namespace • Allows single folder of

    TF config to manage multiple distinct sets of infrastructure resources
  51. Terminal $ terraform workspace list * default $ terraform workspace

    new mitchellh-test Created and switched to workspace "mitchellh-test"! $ terraform workspace list default * mitchellh-test
  52. File resource "aws_instance" "example" { count = “${terraform.workspace == "default"

    ? 5 : 1}" tags { Name = "web - ${terraform.workspace}" } # ... other fields }
  53. Terraform 0.9 (March 15, 2017) • Destroy provisioners • Remote

    backends • State locking • Workspaces (formerly State Environments)
  54. Destroy Provisioners

  55. Provisioners (Terraform <= 0.8) • Run arbitrary code locally or

    remotely on resource creation • If provisioner fails, resource is tainted and scheduled for recreation on the next apply
  56. File resource "null_resource" "example" { provisioner "local-exec" { command =

    "echo foo" } }
  57. File resource "null_resource" "example" { provisioner "local-exec" { command =

    "echo foo" } provisioner "local-exec" { command = "echo destroying" when = "destroy" } }
  58. Terminal $ terraform apply null_resource.example: Creating... null_resource.example: Provisioning with 'local-exec'...

    null_resource.example (local-exec): Executing: /bin/sh -c "echo foo" null_resource.example (local-exec): foo null_resource.example: Creation complete (ID: 1965091882910923448) Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
  59. Terminal $ terraform destroy -force null_resource.example: Refreshing state... (ID: 1965091882910923448)

    null_resource.example: Destroying... (ID: 1965091882910923448) null_resource.example: Provisioning with 'local-exec'... null_resource.example (local-exec): Executing: /bin/sh -c "echo destroying" null_resource.example (local-exec): destroying null_resource.example: Destruction complete
  60. Destroy Provisioners • Configured with when = "destroy" • Run

    on resource destroy (not just "terraform destroy") • Failure cancels physical resource destruction by default • Can allow failure with on_failure = "continue"
  61. File resource "null_resource" "example" { provisioner "local-exec" { command =

    "exit 1" when = "destroy" } }
  62. Terminal $ terraform destroy -force null_resource.example: Refreshing state... (ID: 8665586891184105369)

    null_resource.example: Destroying... (ID: 8665586891184105369) null_resource.example: Provisioning with 'local-exec'... null_resource.example (local-exec): Executing: /bin/sh -c "exit 1" Error applying plan: 1 error(s) occurred: * null_resource.example (destroy): 1 error(s) occurred: * Error running command 'exit 1': exit status 1.
  63. File resource "null_resource" "example" { provisioner "local-exec" { command =

    "exit 1" when = "destroy" on_failure = "continue" } }
  64. Terminal $ terraform destroy -force null_resource.example: Refreshing state... (ID: 8665586891184105369)

    null_resource.example: Destroying... (ID: 8665586891184105369) null_resource.example: Provisioning with 'local-exec'... null_resource.example (local-exec): Executing: /bin/sh -c "exit 1" null_resource.example: Destruction complete Destroy complete! Resources: 1 destroyed.
  65. Destroy Provisioners • Useful for resource cleanup • Can SSH

    into machine (any machine!) prior to destruction • Recommend resource cleanup live as part of the resource itself, but destroy provisioners give you another option
  66. Remote Backends

  67. Before Remote Backends (TF <= 0.8) • Awkward "remote config"

    command • Users could accidentally run Terraform without remote init • Configuration only via CLI • Local cache of state stored in .terraform/terraform.tfstate • Changed remote configuration was manual
  68. Terminal $ # TERRAFORM <= 0.8, BEFORE REMOTE BACKENDS $

    terraform remote config \ -backend=S3 \ -backend-config="bucket=<bucket>" \ -backend-config="key=<path to file>" ...
  69. Remote Backends • Subsumes "remote state", enables locking, workspaces, more

    • Configure from tf files, external configuration, or CLI • Detects configuration change • Forces new users of a TF configuration to initialize • One command to init them all: `terraform init`
  70. File terraform { backend "s3" { bucket = "<bucket>" key

    = "<path to file>" } }
  71. Terminal $ terraform init Initializing the backend... Successfully configured the

    backend "s3"! Terraform will automatically use this backend unless the backend configuration changes. Terraform has been successfully initialized!
  72. Terminal $ # New user, didn't run init $ terraform

    console Backend reinitialization required. Please run "terraform init". Reason: Initial configuration of the requested backend "s3" ...
  73. File terraform { backend "s3" { bucket = "CHANGED-THIS-SETTING" key

    = "<path to file>" } }
  74. Terminal $ terraform console Backend reinitialization required. Please run "terraform

    init". Reason: Backend configuration changed for "s3" ...
  75. Terminal $ terraform console Backend reinitialization required. Please run "terraform

    init". Reason: Unsetting the previously set backend "s3" ...
  76. Remote Backends • One command to init: `terraform init` •

    Automatic detection of backend change (set, change, unset) • No state stored locally at all • Always gitignore ".terraform" folder
  77. A Focus on Safety • Common complaint: easy to corrupt

    remote state • Remote backends add new layer of safety: detecting changes, checking "lineage", disallowing writing unsafe state, more.
  78. A New "Init" • Init has existed since Terraform 0.1

    • Used to just setup folder structure for new projects • Now the single source of init, safe to run multiple times • Initializes backend, downloads modules, creates folders • One day: downloads providers, verifies versions, more...
  79. State Locking

  80. State Locking • For supported backends, Terraform automatically locks state

    on write operations • If unlock fails, error is shown with lock ID to allow a force unlock • Doesn't lock against concurrent reads
  81. File terraform { backend "consul" {} } resource "null_resource" "example"

    { provisioner "local-exec" { command = "sleep 10" } }
  82. Terminal $ terraform apply null_resource.example: Creating... null_resource.example: Provisioning with 'local-exec'...

    null_resource.example (local-exec): Executing: /bin/sh -c "sleep 10"
  83. Terminal $ terraform apply Error loading state: failed to lock

    state in Consul: Lock Info: ID: 5c0b66d6-018f-59b4-5536-499fec947fb2 Path: foo Operation: OperationTypeApply Who: mitchellh@Mitchells-iMac.lan Version: 0.9.1 Created: 2017-04-04 16:16:59.733058195 +0000 UTC Info: $ terraform console >
  84. Terraform

  85. SEP 18-20, 2017 | AUSTIN, TEXAS One day of training

    followed by two days of talks on the future of infrastructure. Discount Code: HUG20
  86. THANKS! Q/A