Meet Terraform - Longhorn PHP

We have a ton of options for provisioning machines once we have an SSH connection available, but how do we get those machines in the first place?

Meet Terraform from Hashicorp, your infrastructure automation engine. Providing a declarative interface for your infrastructure, you can define what you need and let Terraform take care of the rest.

In this talk, we’ll take a look at what Terraform can do for you, walking through your first Terraform configuration all the way to writing your own modules to encapsulate your infrastructure across multiple deployments.

Transcript

1. MEET TERRAFORM
2. DEVELOPER ADVOCATE @NEXMO MICHAEL @MHEAP
3. IN A PREVIOUS LIFE…
4. IN A PREVIOUS LIFE… I RAN AN OPERATIONS TEAM
5. WHEN I JOINED No Infrastructure
6. THEN I STARTED ADDING THINGS
7. THEN I STARTED ADDING THINGS
8. THEN I STARTED ADDING THINGS
9. THEN I STARTED ADDING THINGS
10. THEN I STARTED ADDING THINGS
11. THEN I STARTED ADDING THINGS
12. THERE HAS TO BE A BETTER WAY
16. WHAT IS TERRAFORM?
17. INFRASTRUCTURE AS CODE
18. INFRASTRUCTURE AS CODE resource "aws_instance" "web" { ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "HelloWorld" } }
19. 1207 CONTRIBUTORS, 103 RELEASES 11,797 STARS, 3817 FORKS CREATED 22ND MAY 2014
20. TERRAFORM ALLOWS US TO DEFINE WHAT WE NEED IN A DECLARATIVE MANNER
22. Alicloud Archive Arukas AWS Bitbucket CenturyLinkCloud Chef Circonus Cloudflare CloudStack Cobbler Consul Datadog DigitalOcean DNS DNSMadeEasy DNSimple Docker Dyn External Fastly GitHub Gitlab Google Cloud Grafana Heroku HTTP Icinga2 Ignition InfluxDB Kubernetes Librato Local Logentries Mailgun New Relic Nomad NS1 Microsoft Azure Microsoft Azure (Legacy ASM) MySQL 1&1 Oracle Public Cloud OpenStack OpsGenie OVH Packet PagerDuty PostgreSQL PowerDNS ProfitBricks RabbitMQ Rancher Random Rundeck Scaleway SoftLayer StatusCake Spotinst Template Terraform Terraform Enterprise TLS Triton UltraDNS Vault VMware vCloud Director VMware vSphere
23. TERRAFORM IS NOT AN API ABSTRACTION LAYER. EACH CLOUD HAS IT’S OWN RESOURCES
24. TERRAFORM ALLOWS US TO DEFINE WHAT WE NEED IN A DECLARATIVE MANNER
25. TERRAFORM ALLOWS US TO DEFINE WHAT WE NEED IN A DECLARATIVE MANNER
26. WHAT’S THE PLAN?
27. THE TERRAFORM CLI YOUR FIRST PLAN INSTALLING TERRAFORM
28. MODULES PROVISIONING VARIABLES
29. LIFECYCLE MANAGEMENT IMMUTABLE INFRASTRUCTURE USING IT AS A TEAM
30. INSTALLING TERRAFORM
31. RUN IT THERE IS NO STEP 3 DOWNLOAD THE BINARY
32. THE TERRAFORM CLI
33. $ terraform Usage: terraform [--version] [--help] <command> [args] The available commands for execution are listed below. The most common, useful commands are shown first, followed by less common or more advanced commands. If you're just getting started with Terraform, stick with the common commands. For the other commands, please read the help and docs before usage. Common commands: apply Builds or changes infrastructure console Interactive console for Terraform interpolations destroy Destroy Terraform-managed infrastructure env Workspace management fmt Rewrites config files to canonical format get Download and install modules for the configuration graph Create a visual graph of Terraform resources import Import existing infrastructure into Terraform init Initialize a Terraform working directory output Read an output from a state file plan Generate and show an execution plan providers Prints a tree of the providers used in the configuration push Upload this Terraform module to Atlas to run refresh Update local state file against real resources show Inspect Terraform state or plan taint Manually mark a resource for recreation untaint Manually unmark a resource as tainted validate Validates the Terraform files version Prints the Terraform version workspace Workspace management All other commands: debug Debug output management (experimental) force-unlock Manually unlock the terraform state state Advanced state management
34. $ terraform Usage: terraform [--version] [--help] <command> [args] The available commands for execution are listed below. The most common, useful commands are shown first, followed by less common or more advanced commands. If you're just getting started with Terraform, stick with the common commands. For the other commands, please read the help and docs before usage. Common commands: apply Builds or changes infrastructure console Interactive console for Terraform interpolations destroy Destroy Terraform-managed infrastructure env Workspace management fmt Rewrites config files to canonical format get Download and install modules for the configuration graph Create a visual graph of Terraform resources import Import existing infrastructure into Terraform init Initialize a Terraform working directory output Read an output from a state file plan Generate and show an execution plan providers Prints a tree of the providers used in the configuration push Upload this Terraform module to Atlas to run refresh Update local state file against real resources show Inspect Terraform state or plan taint Manually mark a resource for recreation untaint Manually unmark a resource as tainted validate Validates the Terraform files version Prints the Terraform version workspace Workspace management All other commands: debug Debug output management (experimental) force-unlock Manually unlock the terraform state state Advanced state management
35. $ terraform Usage: terraform [--version] [--help] <command> [args] The available commands for execution are listed below. The most common, useful commands are shown first, followed by less common or more advanced commands. If you're just getting started with Terraform, stick with the common commands. For the other commands, please read the help and docs before usage. Common commands: apply Builds or changes infrastructure console Interactive console for Terraform interpolations destroy Destroy Terraform-managed infrastructure env Workspace management fmt Rewrites config files to canonical format get Download and install modules for the configuration graph Create a visual graph of Terraform resources import Import existing infrastructure into Terraform init Initialize a Terraform working directory output Read an output from a state file plan Generate and show an execution plan providers Prints a tree of the providers used in the configuration push Upload this Terraform module to Atlas to run refresh Update local state file against real resources show Inspect Terraform state or plan taint Manually mark a resource for recreation untaint Manually unmark a resource as tainted validate Validates the Terraform files version Prints the Terraform version workspace Workspace management All other commands: debug Debug output management (experimental) force-unlock Manually unlock the terraform state state Advanced state management
36. TERRAFORM APPLY TERRAFORM DESTROY TERRAFORM PLAN
37. TERRAFORM APPLY TERRAFORM DESTROY
38. YOUR FIRST PLAN
39. YOUR FIRST PLAN $ export AWS_ACCESS_KEY_ID="anaccesskey" $ export AWS_SECRET_ACCESS_KEY="asecretkey" $ mkdir terraform-example $ cd terraform-example $ touch main.tf
40. VARIABLES.TF OUTPUTS.TF MAIN.TF
41. LONGHORN.TF PHP.TF BANANAS.TF
42. DEFINE A PROVIDER (MAIN.TF) provider "aws" { region = "us-west-2" }
43. DEFINE A PROVIDER (MAIN.TF) provider "aws" { access_key = "KEY" secret_key = "SEKRIT" region = "us-west-2" }
44. CREATE AN INSTANCE (MAIN.TF) resource "aws_instance" "web" { ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "HelloWorld" } }
45. CREATE AN INSTANCE resource "aws_instance" "web" { ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "HelloWorld" } }
46. CREATE AN INSTANCE resource "aws_instance" "web" { ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "HelloWorld" } }
47. CREATE AN INSTANCE resource "aws_instance" "web" { ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "HelloWorld" } }
48. CREATE AN INSTANCE resource "aws_instance" "web" { ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "HelloWorld" } }
49. CREATE AN INSTANCE resource "aws_instance" "web" { ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "HelloWorld" } }
50. TERRAFORM PLAN $ terraform plan -out tfplan Plugin reinitialization required. Please run "terraform init". Reason: Could not satisfy plugin requirements. Plugins are external binaries that Terraform uses to access and manipulate resources. The configuration provided requires plugins which can't be located, don't satisfy the version constraints, or are otherwise incompatible. 1 error(s) occurred: * provider.aws: no suitable version installed version requirements: "(any version)" versions installed: none Terraform automatically discovers provider requirements from your configuration, including providers used in child modules. To see the requirements and constraints from each module, run "terraform providers".
51. TERRAFORM INIT $ terraform init Initializing provider plugins... - Checking for available provider plugins on https://releases.hashicorp.com... - Downloading plugin for provider "aws" (1.0.0)... The following providers do not have any version constraints in configuration, so the latest version was installed. To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "..." constraints to the corresponding provider blocks in configuration, with the constraint strings suggested below. * provider.aws: version = "~> 1.0" Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work.
52. TERRAFORM PLAN An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: + aws_instance.web id: <computed> ami: "ami-70728c08" associate_public_ip_address: <computed> source_dest_check: "true" subnet_id: <computed> tags.%: "1" tags.Name: "HelloWorld" Plan: 1 to add, 0 to change, 0 to destroy.
53. TERRAFORM APPLY $ terraform apply tfplan aws_instance.web: Creating... ami: "" => "ami-70728c08" instance_type: "" => "t2.micro" source_dest_check: "" => "true" subnet_id: "" => "<computed>" tags.%: "" => "1" tags.Name: "" => "HelloWorld" tenancy: "" => "<computed>" volume_tags.%: "" => "<computed>" vpc_security_group_ids.#: "" => "<computed>" aws_instance.web: Still creating... (10s elapsed) aws_instance.web: Still creating... (20s elapsed) aws_instance.web: Creation complete after 22s (ID: i-0c0b4a732311b8bce) Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
54. INSTANCE CREATED
55. TERRAFORM PLAN $ terraform plan Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. aws_instance.web: Refreshing state... (ID: i-0c0b4a732311b8bce) ------------------------------------------------------------------------ No changes. Infrastructure is up-to-date. This means that Terraform did not detect any differences between your configuration and real physical resources that exist. As a result, no actions need to be performed.
56. TERRAFORM.TFSTATE { "version": 3, "terraform_version": "0.10.4", "serial": 2, "lineage": "348d2909-5e43-4421-bd65-4f7ab6b11632", "modules": [ { "path": [ "root" ], "outputs": {}, "resources": { "aws_instance.web": { "type": "aws_instance", "depends_on": [], "primary": { "id": "i-0fc83946a097d8e27", "attributes": { "ami": "ami-70728c08", "associate_public_ip_address": "true", "availability_zone": "us-west-2a", "disable_api_termination": "false", "ebs_block_device.#": "0", "ebs_optimized": "false", "ephemeral_block_device.#": "0", "iam_instance_profile": "", "id": "i-0fc83946a097d8e27", "instance_state": "running", "instance_type": "t2.small", "ipv6_addresses.#": "0", "key_name": "", "monitoring": "false", "network_interface.#": "0", "network_interface_id": "eni-de1229f0", "primary_network_interface_id": "eni-de1229f0", "private_dns": "ip-172-31-21-4.us-west-2.compute.internal", "private_ip": "172.31.21.4", "public_dns": "ec2-34-215-64-201.us-west-2.compute.amazonaws.com", "public_ip": "34.215.64.201", "root_block_device.#": "1", "root_block_device.0.delete_on_termination": "true", "root_block_device.0.iops": "100", "root_block_device.0.volume_size": "8", "root_block_device.0.volume_type": "gp2", "security_groups.#": "1", "security_groups.3814588639": "default", "source_dest_check": "true", "subnet_id": "subnet-3e5ef758", "tags.%": "1", "tags.Name": "HelloWorld", "tenancy": "default", "volume_tags.%": "0", "vpc_security_group_ids.#": "0" }, "meta": { "e2bfb730-ecaa-11e6-8f88-34363bc7c4c0": { "create": 600000000000, "delete": 600000000000, "update": 600000000000 }, "schema_version": "1" }, "tainted": false }, "deposed": [], "provider": "" } }, "depends_on": [] } ] } "aws_instance.web": { "type": "aws_instance", "depends_on": [], "primary": { "id": "i-0fc83946a097d8e27", "attributes": { "ami": "ami-70728c08", "associate_public_ip_address": "true", "availability_zone": "us-west-2a", "disable_api_termination": "false", "ebs_block_device.#": "0", "ebs_optimized": "false", "ephemeral_block_device.#": "0", "iam_instance_profile": "", "id": "i-0fc83946a097d8e27",
57. SCALING UP resource "aws_instance" "web" { count = "3" ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "HelloWorld" } }
58. SCALING UP resource "aws_instance" "web" { count = "3" ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "HelloWorld" } }
59. TERRAFORM PLAN An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: + aws_instance.web[1] <snip> ami: "ami-70728c08" tags.Name: "HelloWorld" + aws_instance.web[2] <snip> ami: "ami-70728c08" tags.Name: “HelloWorld" Plan: 2 to add, 0 to change, 0 to destroy.
60. ADDING A LOAD BALANCER resource "aws_elb" "lb" { name = "example-lb" availability_zones = ["us-west-2a"] instances = ["${aws_instance.web.*.id}"] listener { instance_port = "8000" instance_protocol = "http" lb_port = "80" lb_protocol = "http" } }
61. ADDING A LOAD BALANCER resource "aws_elb" "lb" { name = "example-lb" availability_zones = ["us-west-2a"] instances = ["${aws_instance.web.*.id}"] listener { instance_port = "8000" instance_protocol = "http" lb_port = "80" lb_protocol = "http" } }
62. TERRAFORM APPLY PROFIT TERRAFORM PLAN
63. OUTPUTS.TF output "lb_dns" { value = "${aws_elb.lb.dns_name}" }
64. OUTPUTS.TF $ terraform refresh aws_instance.web[0]: Refreshing state... (ID: i-0c0b4a732311b8bce) aws_instance.web[1]: Refreshing state... (ID: i-0b8bde17992d4dc0c) aws_instance.web[2]: Refreshing state... (ID: i-06e74e8dc04eb1843) aws_elb.lb: Refreshing state... (ID: example-elb) Outputs: lb_dns = example-elb-1640688516.us-west-2.elb.amazonaws.com
65. OUTPUTS.TF $ terraform output -json { "lb_dns": { "sensitive": false, "type": "string", "value": "example-elb-1640688516.us-west-2.elb.amazonaws.com" } }
66. TERRAFORM DESTROY $ terraform destroy An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: - destroy Terraform will perform the following actions: - aws_elb.lb - aws_instance.web[0] - aws_instance.web[1] - aws_instance.web[2] Plan: 0 to add, 0 to change, 4 to destroy. Do you really want to destroy? Terraform will destroy all your managed infrastructure, as shown above. There is no undo. Only 'yes' will be accepted to confirm. Enter a value:
68. CONGRATULATIONS! YOU JUST CREATED YOUR FIRST PLAN
69. VARIABLES
70. SCALING UP resource "aws_instance" "web" { count = "3" ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "HelloWorld" } }
71. SCALING UP resource "aws_instance" "web" { count = "3" ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "web" } }
72. SCALING UP resource "aws_instance" "web" { count = "3" ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "web-${count.index}" } }
73. SCALING UP resource "aws_instance" "web" { count = "3" ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "web-${count.index}" Environment = "PirateNinjas" } }
74. SCALING UP resource "aws_elb" "lb" { name = "example-lb" …snip… listener { …snip… } tags { Environment = "PirateNinjas" } }
75. VARIABLES.TF variable "environment" { type = "string" }
76. SCALING UP resource "aws_instance" "web" { count = "3" ami = "ami-70728c08" instance_type = "t2.micro" tags { Name = "web-${count.index}" Environment = "${var.environment}" } }
77. SCALING UP resource "aws_elb" "lb" { name = "lb-${var.environment}" …snip… listener { …snip… } tags { Environment = "${var.environment}" } }
78. TERRAFORM -VAR ENVIRONMENT=NAME TERRAFORM -VAR-FILE MY.TFVARS EXPORT TF_VAR_ENVIRONMENT=NAME
79. CONGRATULATIONS! YOUR PLAN FILE IS NICE AND DRY
80. MODULES
81. CREATING A MODULE $ tree . !"" main.tf !"" output.tf #"" variables.tf
82. CREATING A MODULE $ tree . !"" main.tf !"" output.tf #"" variables.tf $ tree . #"" demo !"" main.tf !"" output.tf #"" variables.tf
83. USING A MODULE (LOCAL) module "pirate" { source = "./demo" environment = "Pirate" }
84. USING A MODULE (LOCAL) module "pirate" { source = "./demo" environment = "Pirate" } module "ninja" { source = "./demo" environment = "Ninja" }
85. USING A MODULE (REGISTRY) module "pirate" { source = "mheap/aws/full-env" environment = "Pirate" } module "ninja" { source = "mheap/aws/full-env" environment = "Ninja" } https://registry.terraform.io/
86. USING A MODULE (GIT) module "pirate" { source = "git::https://hashicorp.com/consul.git? ref=1.0.3" environment = "Pirate" } module "ninja" { source = “git::https://hashicorp.com/consul.git? ref=1.8.14" environment = "Ninja" }
87. LOCAL GITHUB BITBUCKET GIT MERCURIAL S3 HTTP
88. MODULE OUTPUTS output "pirate_lb" { value = "${module.pirate.lb_dns}" } output "ninja_lb" { value = "${module.ninja.lb_dns}" }
89. CONGRATULATIONS! YOU JUST CREATED A MODULE!
90. WARNING WARNING WARNING
91. BAD . !"" demo $ !"" main.tf $ !"" output.tf $ #"" variables.tf !"" main.tf !"" outputs.tf #"" terraform.tfstate
92. GOOD . !"" environments $ !"" ninja $ $ !"" main.tf $ $ !"" output.tf $ $ #"" terraform.tfstate $ #"" pirate $ !"" main.tf $ !"" output.tf $ #"" terraform.tfstate #"" modules #"" demo !"" main.tf !"" output.tf #"" variables.tf
93. PROVISIONING
94. RUNNING ANSIBLE resource "aws_instance" "web" { count = "3" ami = "ami-70728c08" instance_type = "t2.micro" tags { …snip… } provisioner "local-exec" { command = "ansible-playbook -u ubuntu --private-key ./aws-key.pem -i '$ {self.public_ip},' playbook.yml" } }
95. RUNNING APT-GET resource "aws_instance" "web" { …snip… provisioner "remote-exec" { connection { type = "ssh" user = "root" password = "${var.root_password}" } inline = [ "sudo apt-get install nginx my-app", "sudo systemctl restart nginx" ] } }
96. OTHER PROVISIONERS HABITAT SALT-MASTERLESS CHEF
97. USING IT AS A TEAM
98. CONSUL TERRAFORM ENTERPRISE AMAZON S3
99. REMOTE STATE (S3) terraform { backend "s3" { bucket = "mybucket" key = "path/to/my/key" region = “us-west-2" } }
100. REMOTE STATE (CONSUL) terraform { backend "consul" { address = "demo.consul.io" path = "example_app/ terraform_state" } }
101. REMOTE STATE (TF ENTERPRISE)
102. LIFECYCLE MANAGEMENT
103. LIFECYCLE lifecycle { create_before_destroy = true }
104. IMMUTABLE INFRASTRUCTURE
106. BONUS CONTENT
107. IMPORTING EXISTING INFRASTRUCTURE
108. MIX/MATCH PROVIDERS E.G. HEROKU + DNSIMPLE provider "heroku" { email = "ops@company.com" api_key = "${var.heroku_api_key}" } provider "dnsimple" { token = "${var.dnsimple_token}" account = "${var.dnsimple_account}" } resource "heroku_app" "default" { … } resource "dnsimple_record" "foobar" { domain = "${var.dnsimple_domain}" name = "" value = “${heroku_app.default.heroku_hostname}" type = "CNAME" ttl = 3600 }
109. TERRAFORM CONFIGURATON BLOCK REMOTE STATE / MINIMUM VERSIONS terraform { required_version = "> 0.11.0" }
110. 59 FUNCTIONS CHOMP/COALESCE/ELEMENT/FILE + MORE
111. READ ONLY DATA PROVIDERS E.G. AWS_AMI FOR SEARCHING AMIS data "aws_ami" "ubuntu" { most_recent = true filter { name = "name" values = ["ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-*"] } filter { name = "virtualization-type" values = ["hvm"] } owners = ["099720109477"] # Canonical }
112. OVERRIDE.TF MERGE CONTENT INTO A PLAN RATHER THAN APPENDING A NEW RESOURCE
113. TESTING AWSSPEC / INSPEC require "spec_helper" before do @client = Aws::EC2::Client.new @ec2 = Aws::EC2::Resource.new(client: @client) end describe "Instances" do subject { @ec2.instances.count } it { is_expected.to eq(3) } end
115. ANY QUESTIONS? @MHEAP HTTPS://JOIND.IN/TALK/2173C