What's new in Terraform 0.10 - HUG

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

@armon ARMON DADGAR

Slide 3

Slide 3 text

AGENDA HASHICORP USE CASES INTRO TO TERRAFORM NEW FEATURES Terraform 0.10 Terraform 0.9

Slide 4

Slide 4 text

PROVISION, SECURE AND RUN ANY INFRASTRUCTURE Nomad Consul Vault Vagrant Packer Terraform Consul Enterprise Terraform Enterprise Vault Enterprise PRODUCT SUITE OSS TOOL SUITE RUN Applications SECURE Application Infrastructure PROVISION Infrastructure FOR INDIVIDUALS FOR TEAMS Nomad Enterprise

Slide 5

Slide 5 text

SEP 18-20, 2017 | AUSTIN, TEXAS One day of training followed by two days of talks on the future of infrastructure. Discount Code: HUG20

Slide 6

Slide 6 text

6 HashiCorp conﬁdential do not distribute USE CASES Easily combine and build complex infrastructure Self-Serve Infrastructure Common Workﬂow for Any Infrastructure Hybrid Cloud Management Automate Infrastructure Provisioning Infrastructure as Code

Slide 7

Slide 7 text

7 HashiCorp conﬁdential do not distribute COLLABORATE on infrastructure as code using a version control system (VCS) VALIDATE with a common integrated workﬂow AUTOMATE public cloud, private cloud, and external services THE IDEAL STATE

Slide 8

Slide 8 text

Terraform

Slide 9

Slide 9 text

Terraform Goals • Uniﬁed view of infrastructure • Infrastructure as code • Compose multiple tiers (IaaS to PaaS to SaaS) • Safely change/iterate infrastructure over time • One workﬂow

Slide 10

Slide 10 text

Terraform Features • Open Source • Infrastructure as Code • Resource Providers • Plan and Apply • Collaboration, History [Enterprise]

Slide 11

Slide 11 text

Infrastructure as Code

Slide 12

Slide 12 text

File resource "google_compute_instance" "server" { name = "server" machine_type = "g1-small" zone = "us-central1-a" disk { image = "ubuntu-1404-trusty-v20160114e" } } resource "dnsimple_record" "hello" { domain = "example.com" name = "server" value = "${google_compute_instance.server.network_interface.0.address}" type = "A" }

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Infrastructure as Code • Human-readable (HCL), Machine-editable (JSON) • Declarative • Text makes it versionable and VCS-friendly • Can be split into multiple ﬁles and modules

Slide 16

Slide 16 text

Resource Providers

Slide 17

Slide 17 text

Resource Providers • Integration point • Expose resources, such as "aws_instance", etc. • CRUD API to implement • Core vs. Providers simpliﬁes logic for external contributors

Slide 18

Slide 18 text

Resource Providers Amazon BitBucket CenturyLink Cloud CloudFlare CloudStack Cobbler Consul Datadog DigitalOcean DNSMadeEasy DNSimple Docker Dyn GitHub Fastly Google Heroku Librato MicrosoB Azure MySQL OpenStack Packet PostgreSQL SoBLayer UltraDNS VMware Sphere and more...

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Plan and Apply

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Plan • Plan shows you what will happen • Plans can be saved to guarantee what will happen • Plans show reasons for certain actions (such as re-create) • Not equivalent to "noop" due to the ability to save a plan

Slide 26

Slide 26 text

Terminal $ terraform plan + google_compute_instance.server can_ip_forward: "false" disk.#: "1" disk.0.auto_delete: "true" disk.0.image: "ubuntu-1404-trusty-..." machine_type: "g1-small" metadata_fingerprint: "" name: "server" network_interface.#: "1" network_interface.0.address: "" network_interface.0.name: "" network_interface.0.network: "default" self_link: "" tags_fingerprint: "" zone: "us-central1-a" ... Plan: 3 to add, 0 to change, 0 to destroy.

Slide 27

Slide 27 text

Apply • Executes changes in order based on dependencies • Parallelizes changes when possible • Handles and recovers transient errors

Slide 28

Slide 28 text

Terminal $ terraform apply google_compute_instance.server: Creating... can_ip_forward: "" => "false" disk.#: "" => "1" disk.0.auto_delete: "" => "true" disk.0.image: "" => "ubuntu-1404-trusty..." machine_type: "" => "g1-small" metadata_fingerprint: "" => "" name: "" => "server" network_interface.#: "" => "1" network_interface.0.address: "" => "" network_interface.0.name: "" => "" network_interface.0.network: "" => "default" self_link: "" => "" tags_fingerprint: "" => "" zone: "" => "us-central1-a" ... Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Slide 29

Slide 29 text

Apply for Changes • Not only creation, but changes over time • Plan will show you what will happen • The `-target` ﬂag can be used for ﬁne-grained change

Slide 30

Slide 30 text

NEW FEATURES IN TERRAFORM 0.10 AND 0.9

Slide 31

Slide 31 text

Terraform 0.10 (August 2, 2017) • Split of Terraform Core / Providers • Kubernetes Provider • Workspaces renamed from State Environments

Slide 32

Slide 32 text

Split of Core / Providers

Slide 33

Slide 33 text

Terraform Core vs Provider Core AWS Provider Fastly Provider HCL  Files API Integration Terraform

Slide 34

Slide 34 text

Terraform Parts • Core Responsible for: • Parsing Conﬁguration • Resource Graph • Interpolation • State Management • Execution • Providers responsible for: • Resource Deﬁnition & Validation • API Integration

Slide 35

Slide 35 text

Terraform Repository • Everything lived in a single repository • Released ~2 weeks • Version Sprawl (10+ releases per major version)

Slide 36

Slide 36 text

Mono Repo Issues • Terraform Core moves slowly • Weeks to implement new features • Terraform providers move fast • 70+ Providers today • 1000+ Resources • Hours to implement new resources

Slide 37

Slide 37 text

Mono Repo Issues • Hard to triage issues • 2000+ issues • Core Bugs to Provider Feature Requests • Hard to implement Core features • Hard to scale providers • Impossible to update Provider without updating Core

Slide 38

Slide 38 text

Divide and Conquer • Grand Split proposed months ago • Split Providers into individual repositories • Release Providers on independent cadence • Download Providers as needed • Version constrain Providers • "terraform init”

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

File provider aws { version = "~> v0.1.3" region = "us-west-2" } provider fastly { # Implicit: version = “latest” api_key = “exampleapikey" }

Slide 41

Slide 41 text

Terminal $ terraform init Initializing provider plugins... - Checking for available provider plugins on https:// releases.hashicorp.com... - Downloading plugin for provider "fastly" (0.1.2)... - Downloading plugin for provider "aws" (0.1.4)... The following providers do not have any version constraints in configuration, so the latest version was installed. To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "..." constraints to the corresponding provider blocks in configuration, with the constraint strings suggested below. * provider.fastly: version = "~> 0.1" Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands

Slide 42

Slide 42 text

Terraform Core vs Provider Core 0.10 AWS Provider 0.1.4 Fastly Provider 0.1.2 Plugin Interface

Slide 43

Slide 43 text

End Goal • Independent Versioning and Release for Providers / Core • Live documentation for multiple versions • Make it easier to add and support Providers • Infrastructure as Code for all the things!

Slide 44

Slide 44 text

Kubernetes Provider

Slide 45

Slide 45 text

Kubernetes Provider • Terraform manages IaaS, SaaS, and PaaS • Kubernetes has sub-resources to manage (Pods, Controllers) • Compose other resources (load balancers, DNS, CDN) with K8S • Common workﬂow for everything

Slide 46

Slide 46 text

Kubernetes Provider • Data Source: kubernetes_service • Data Source: kubernetes_storage_class • Resource: kubernetes_conﬁg_map • Resource: kubernetes_horizontal_pod_autoscalar • Resource: kubernetes_limit_range • Resource: kubernetes_namespace • Resource: kubernetes_persistent_volume • Resource: kubernetes_persistent_volume_claim • Resource: kubernetes_pod • Resource: kubernetes_replication_controller • Resource: kubernetes_resource_quota • Resource: kubernetes_secret • Resource: kubernetes_service • Resource: kubernetes_service_account • Resource: kubernetes_storage_class • …

Slide 47

Slide 47 text

Composing Resources Provider Google google_container_cluster Provider Kubernetes kubernetes_namespace kubernetes_pod kubernetes_replication_controller consul_keys Provider Consul

Slide 48

Slide 48 text

Using Schedulers with Terraform • Kubernetes • Nomad • Docker Swarm • AWS ECS • Manage Application Lifecycle • Compose with other resources

Slide 49

Slide 49 text

Workspaces Formerly: State Environments

Slide 50

Slide 50 text

Workspaces • A state namespace • Allows single folder of TF conﬁg to manage multiple distinct sets of infrastructure resources

Slide 51

Slide 51 text

Terminal $ terraform workspace list * default $ terraform workspace new mitchellh-test Created and switched to workspace "mitchellh-test"! $ terraform workspace list default * mitchellh-test

Slide 52

Slide 52 text

File resource "aws_instance" "example" { count = “${terraform.workspace == "default" ? 5 : 1}" tags { Name = "web - ${terraform.workspace}" } # ... other fields }

Slide 53

Slide 53 text

Terraform 0.9 (March 15, 2017) • Destroy provisioners • Remote backends • State locking • Workspaces (formerly State Environments)

Slide 54

Slide 54 text

Destroy Provisioners

Slide 55

Slide 55 text

Provisioners (Terraform <= 0.8) • Run arbitrary code locally or remotely on resource creation • If provisioner fails, resource is tainted and scheduled for recreation on the next apply

Slide 56

Slide 56 text

File resource "null_resource" "example" { provisioner "local-exec" { command = "echo foo" } }

Slide 57

Slide 57 text

File resource "null_resource" "example" { provisioner "local-exec" { command = "echo foo" } provisioner "local-exec" { command = "echo destroying" when = "destroy" } }

Slide 58

Slide 58 text

Terminal $ terraform apply null_resource.example: Creating... null_resource.example: Provisioning with 'local-exec'... null_resource.example (local-exec): Executing: /bin/sh -c "echo foo" null_resource.example (local-exec): foo null_resource.example: Creation complete (ID: 1965091882910923448) Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Slide 59

Slide 59 text

Terminal $ terraform destroy -force null_resource.example: Refreshing state... (ID: 1965091882910923448) null_resource.example: Destroying... (ID: 1965091882910923448) null_resource.example: Provisioning with 'local-exec'... null_resource.example (local-exec): Executing: /bin/sh -c "echo destroying" null_resource.example (local-exec): destroying null_resource.example: Destruction complete

Slide 60

Slide 60 text

Destroy Provisioners • Conﬁgured with when = "destroy" • Run on resource destroy (not just "terraform destroy") • Failure cancels physical resource destruction by default • Can allow failure with on_failure = "continue"

Slide 61

Slide 61 text

File resource "null_resource" "example" { provisioner "local-exec" { command = "exit 1" when = "destroy" } }

Slide 62

Slide 62 text

Terminal $ terraform destroy -force null_resource.example: Refreshing state... (ID: 8665586891184105369) null_resource.example: Destroying... (ID: 8665586891184105369) null_resource.example: Provisioning with 'local-exec'... null_resource.example (local-exec): Executing: /bin/sh -c "exit 1" Error applying plan: 1 error(s) occurred: * null_resource.example (destroy): 1 error(s) occurred: * Error running command 'exit 1': exit status 1.

Slide 63

Slide 63 text

File resource "null_resource" "example" { provisioner "local-exec" { command = "exit 1" when = "destroy" on_failure = "continue" } }

Slide 64

Slide 64 text

Terminal $ terraform destroy -force null_resource.example: Refreshing state... (ID: 8665586891184105369) null_resource.example: Destroying... (ID: 8665586891184105369) null_resource.example: Provisioning with 'local-exec'... null_resource.example (local-exec): Executing: /bin/sh -c "exit 1" null_resource.example: Destruction complete Destroy complete! Resources: 1 destroyed.

Slide 65

Slide 65 text

Destroy Provisioners • Useful for resource cleanup • Can SSH into machine (any machine!) prior to destruction • Recommend resource cleanup live as part of the resource itself, but destroy provisioners give you another option

Slide 66

Slide 66 text

Remote Backends

Slide 67

Slide 67 text

Before Remote Backends (TF <= 0.8) • Awkward "remote config" command • Users could accidentally run Terraform without remote init • Configuration only via CLI • Local cache of state stored in .terraform/terraform.tfstate • Changed remote configuration was manual

Slide 68

Slide 68 text

Terminal $ # TERRAFORM <= 0.8, BEFORE REMOTE BACKENDS $ terraform remote config \ -backend=S3 \ -backend-config="bucket=" \ -backend-config="key=" ...

Slide 69

Slide 69 text

Remote Backends • Subsumes "remote state", enables locking, workspaces, more • Configure from tf files, external configuration, or CLI • Detects configuration change • Forces new users of a TF configuration to initialize • One command to init them all: `terraform init`

Slide 70

Slide 70 text

File terraform { backend "s3" { bucket = "" key = "" } }

Slide 71

Slide 71 text

Terminal $ terraform init Initializing the backend... Successfully configured the backend "s3"! Terraform will automatically use this backend unless the backend configuration changes. Terraform has been successfully initialized!

Slide 72

Slide 72 text

Terminal $ # New user, didn't run init $ terraform console Backend reinitialization required. Please run "terraform init". Reason: Initial configuration of the requested backend "s3" ...

Slide 73

Slide 73 text

File terraform { backend "s3" { bucket = "CHANGED-THIS-SETTING" key = "" } }

Slide 74

Slide 74 text

Terminal $ terraform console Backend reinitialization required. Please run "terraform init". Reason: Backend configuration changed for "s3" ...

Slide 75

Slide 75 text

Terminal $ terraform console Backend reinitialization required. Please run "terraform init". Reason: Unsetting the previously set backend "s3" ...

Slide 76

Slide 76 text

Remote Backends • One command to init: `terraform init` • Automatic detection of backend change (set, change, unset) • No state stored locally at all • Always gitignore ".terraform" folder

Slide 77

Slide 77 text

A Focus on Safety • Common complaint: easy to corrupt remote state • Remote backends add new layer of safety: detecting changes, checking "lineage", disallowing writing unsafe state, more.

Slide 78

Slide 78 text

A New "Init" • Init has existed since Terraform 0.1 • Used to just setup folder structure for new projects • Now the single source of init, safe to run multiple times • Initializes backend, downloads modules, creates folders • One day: downloads providers, veriﬁes versions, more...

Slide 79

Slide 79 text

State Locking

Slide 80

Slide 80 text

State Locking • For supported backends, Terraform automatically locks state on write operations • If unlock fails, error is shown with lock ID to allow a force unlock • Doesn't lock against concurrent reads

Slide 81

Slide 81 text

File terraform { backend "consul" {} } resource "null_resource" "example" { provisioner "local-exec" { command = "sleep 10" } }

Slide 82

Slide 82 text

Terminal $ terraform apply null_resource.example: Creating... null_resource.example: Provisioning with 'local-exec'... null_resource.example (local-exec): Executing: /bin/sh -c "sleep 10"

Slide 83

Slide 83 text

Terminal $ terraform apply Error loading state: failed to lock state in Consul: Lock Info: ID: 5c0b66d6-018f-59b4-5536-499fec947fb2 Path: foo Operation: OperationTypeApply Who: mitchellh@Mitchells-iMac.lan Version: 0.9.1 Created: 2017-04-04 16:16:59.733058195 +0000 UTC Info: $ terraform console >

Slide 84

Slide 84 text

Terraform

Slide 85

Slide 85 text

SEP 18-20, 2017 | AUSTIN, TEXAS One day of training followed by two days of talks on the future of infrastructure. Discount Code: HUG20

Slide 86

Slide 86 text

THANKS! Q/A