Managing GCP Projects with Terraform (devfest Pisa 2018)

Managing GCP Projects with Terraform (devfest Pisa 2018)

Create and evolve simple and complex Google Cloud architectures with your text editor and Terraform.

416c04c6f0793e236381c2f5df80c9ed?s=128

Giovanni Toraldo

March 10, 2018
Tweet

Transcript

  1. Managing GCP Projects with Terraform GDG DevFest Pisa 0.1 Giovanni

    Toraldo @gionn cloudesire.com
  2. About me Giovanni Toraldo Open Source Enthusiast, Java coder, writer

    of the OpenNebula book, lead developer & co-founder at @Cloudesire, shooting 2 euro coin at 36 meters with crossbow 2
  3. Monetization & Brokering Platform for immediate SaaSification and automated distribution

    of business applications and services. 3
  4. 4

  5. Infrastructure as code? Design, implement, and deploy applications infrastructure with

    known software best practices: • Code versioning • Code reuse (modularization/abstraction) • Code sharing In order to achieve: • Repeatability • Speed • Reliability 5
  6. Cloud services are easy? Complexity is increasing Composition is a

    must Velocity is the key of success 6
  7. 7

  8. What is Terraform? Terraform is a tool for building, changing,

    and versioning infrastructure safely and efficiently. • Declarative approach ◦ Infrastructure is described using a high-level configuration syntax • Incremental changes ◦ Automatically discover the steps required from current to desired infrastructure state • Execution plan ◦ See what Terraform will do when you apply configuration • Dependency graph ◦ automatically decide the order in which action are executed • Automation 8
  9. What is NOT Terraform? • Configuration management tool ◦ You

    still need Puppet, Chef, Ansible to manage software on VM • Cloud abstraction layer ◦ Doesn’t expose any API, just a CLI mean to be used by humans ◦ Doesn’t hide the inner characteristics of each cloud provider via abstraction • A solution to your lack of cloud knowledge 9
  10. Use cases • Manage production environments by ops • Manage

    disposable test/qa environments by dev • Multi-tier (complex) infrastructures • Setup demo environments • Multi-cloud deployment 10
  11. Install Terraform Terraform is distributed as a single Go binary

    without external dependencies. Download, unpack, execute. • cd /tmp • wget <url> • unzip <file> || tar xvf <file> • sudo mv terraform /usr/local/bin • sudo chmod +x /usr/local/bin/terraform 11
  12. Create a Google Cloud Platform account 12

  13. Create a new Google Cloud project 13

  14. Navigate to Compute Engine to activate 14

  15. Create Service account key and download 15

  16. Create a new project Just create an empty folder with

    a file auth.tf: // Configure the Google Cloud provider provider "google" { credentials = "${file("account.json")}" project = "terraform-test-197317" region = "europe-west1 " } And run: $ terraform init 16
  17. Initialize terraform project And run: $ terraform init 17

  18. Apply changes to infrastructure Plan command to evaluate changes Apply

    command to modify infrastructure 18
  19. Terraform project files structure There isn’t any enforcement on how

    to arrange resources inside a terraform project: • All .tf files in the current folder are automatically sources and merged together in memory $ ls -la total 108 drwxr-xr-x 4 gionn dev 4096 mar 8 15:47 . drwxr-xr-x 3 gionn dev 4096 mar 7 18:03 .. drwxr-xr-x 7 gionn dev 4096 mar 8 15:47 .git drwxr-xr-x 3 gionn dev 4096 mar 7 18:22 .terraform -rw-r--r-- 1 gionn dev 2333 mar 7 18:18 account.json -rw-r--r-- 1 gionn dev 175 mar 7 18:19 auth.tf -rw-r--r-- 1 gionn dev 406 mar 8 15:46 vm.tf 19
  20. Create a new VM 20

  21. Plan output $ terraform plan Refreshing Terraform state in-memory prior

    to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. --------------------------------------------- An execution plan has been generated and is shown below. 21
  22. Plan output + create Terraform will perform the following actions:

    + google_compute_instance.default id: <computed> boot_disk.0.initialize_params.0.image: "debian-cloud/debian-8" cpu_platform: <computed> instance_id: <computed> label_fingerprint: <computed> machine_type: "n1-standard-1" metadata.%: "1" metadata.ssh-keys: "debian:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDnX70/3FB4j7UvewR3T... 22
  23. Plan output Plan: 1 to add, 0 to change, 0

    to destroy. ------------------------------------------------- - Note: You didn't specify an "-out" parameter to save this plan, so Terraform can't guarantee that exactly these actions will be performed if "terraform apply" is subsequently run. 23
  24. Apply output $ terraform apply An execution plan has been

    generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: + google_compute_instance.default 24
  25. Apply output Summary of the actions to perform, waiting for

    confirmation Plan: 1 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: ___ 25
  26. After an apply successful operation State is persisted to terraform.tfstate:

    26
  27. 27

  28. terraform.tfstate A state file is required in order to: •

    Map managed resources to terraform resources • Persist additional metadata • Cache, useful for large infrastructures JSON structure that can be modified (bugs happens) or inspected (custom integrations). State should be persistent and shared among developers. 28
  29. Terraform.tfstate sharing and locking For solo developers or small teams:

    just commit to GIT. For bigger teams, tfstate locking is required. Multiple backends supported: • Google cloud storage • S3 • Consul • Etcd • Terraform enterprise 29
  30. Terraform.tfstate sample output { "version": 3, "terraform_version": "0.11.3", "serial": 2,

    "lineage": "d495f1c0-3c8b-45c2-bda2-f40d28382f30", "modules": [ { "path": [ "root" ], "outputs": {}, "resources": { "google_compute_instance.default": { "type": "google_compute_instance", "depends_on": [], "primary": { "id": "test-vm", "attributes": { "attached_disk.#": "0", "boot_disk.#": "1", "boot_disk.0.auto_delete": "true", "boot_disk.0.device_name": "persistent-disk-0", "boot_disk.0.disk_encryption_key_raw": "", "boot_disk.0.disk_encryption_key_sha256": "", 30
  31. Terraform.tfstate: where is the IP address? "network_interface.#": "1", "network_interface.0.access_config.#": "1",

    "network_interface.0.access_config.0.assigned_nat_ip": "104.155.126.70", "network_interface.0.access_config.0.nat_ip": "104.155.126.70", "network_interface.0.address": "10.132.0.2", "network_interface.0.alias_ip_range.#": "0", "network_interface.0.name": "nic0", "network_interface.0.network": "https://www.googleapis.com/compute/v1/projects/terraform-test-19731 7/global/networks/default", "network_interface.0.network_ip": "10.132.0.2", "network_interface.0.subnetwork": "https://www.googleapis.com/compute/v1/projects/terraform-test-19731 7/regions/europe-west1/subnetworks/default", "network_interface.0.subnetwork_project": "terraform-test-197317", "project": "terraform-test-197317", 31
  32. Automatically print VM ip address Add to output.tf: output "ip"

    { value = "${google_compute_instance.default. network_interface.0.access_config.0.nat_ip}" } And run terraform apply: Outputs: ip = 104.155.126.70 32
  33. Change infrastructure example: VM upscale Just change the terraform attribute

    machine_type (and configure allow_stopping_for_update) Now, when terraform apply is run, terraform discover that the machine_type of the existing resource doesn’t correspond to the desiderata. 33
  34. 34

  35. 35

  36. Attach a new data disk We’ll create a new data

    disk resource, and reference it insiude the instance resource. 36
  37. 37

  38. 38

  39. 39

  40. 40 Manage multiple instances with disks • Define a variable

    resource ◦ Set a default • Define unique resource names to avoid conflicts ◦ Leverage count.index variable • Reference a difference disk for each instance ◦ Use count.index variable as disk reference • Override variable value via environment variable ◦ TF_VAR_my_counter
  41. 41

  42. 42 github.com/ gionn/terraform-example

  43. Questions? 43