Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Hands-on with Infrastructure-as-Code

Hands-on with Infrastructure-as-Code

This is an introductory workshop that covers the patterns and practices of infrastructure-as-code. We'll focus on using HashiCorp Terraform, a simple and powerful tool for safely and predictably creating, changing, and improving infrastructure. If you have never used Terraform, this is the place to start.

Be8b596c46f4c9a1aec6a7586af33134?s=128

Rosemary Wang

August 01, 2019
Tweet

More Decks by Rosemary Wang

Other Decks in Technology

Transcript

  1. Copyright © 2019 HashiCorp Hands-On with Infrastructure-as-Code A Workshop on

    Terraform & Infrastructure !1
  2. Presenter Rosemary Wang Developer Advocate at HashiCorp she/her @joatmon08 joatmon08

    linkedin.com/in/rosemarywang/ !2
  3. Teaching Assistants Allee Clark SRE at Oscar Health Abbas Syed

    Technical Account Manager at HashiCorp !3
  4. Agenda Evolution of Infrastructure How did we get to Infrastructure-as-Code?

    Terraform Concepts How do we use it? Scaling Infrastructure-as-Code What do we do as a team? !4
  5. Evolution of Infrastructure How did we get to Infrastructure-as-Code? !5

  6. Infrastructure 
 is anything that supports an application ecosystem. !6

  7. Maturity 7

  8. Manual Everything 8

  9. Basic Automation #!/bin/bash 9

  10. Machine Virtualization 10

  11. *aaS 11

  12. Datacenter-as-Computer 12

  13. Maturity 13

  14. 14

  15. Capability Complexity 15

  16. 16

  17. 17

  18. 475 def update 476 return update_api if api_request? 477 478

    if authorized_action(@account, @current_user, :manage_account_settings) 479 respond_to do |format| 480 481 custom_help_links = params[:account].delete :custom_help_links 482 if custom_help_links 483 @account.settings[:custom_help_links] = custom_help_links.select{|k, h| h['state'] != 'delete 484 hash = index_with_hash[1] 485 hash.delete('state') 486 hash.assert_valid_keys ["text", "subtext", "url", "available_to"] 487 hash 488 end 489 end 490 491 params[:account][:turnitin_host] = validated_turnitin_host(params[:account][:turnitin_host]) 492 enable_user_notes = params[:account].delete :enable_user_notes 493 allow_sis_import = params[:account].delete :allow_sis_import 494 params[:account].delete :default_user_storage_quota_mb unless @account.root_account? && !@accou 495 unless @account.grants_right? @current_user, :manage_storage_quotas 496 [:storage_quota, :default_storage_quota, :default_storage_quota_mb, 497 :default_user_storage_quota, :default_user_storage_quota_mb, 498 :default_group_storage_quota, :default_group_storage_quota_mb].each { |key| params[:account] 499 end 500 if params[:account][:services] 501 params[:account][:services].slice(*Account.services_exposed_to_ui_hash(nil, @current_user, @a 502 @account.set_service_availability(key, value == '1') 503 end 504 params[:account].delete :services 505 end 506 if @account.grants_right?(@current_user, :manage_site_settings) 507 # If the setting is present (update is called from 2 different settings forms, one for notifi 508 if params[:account][:settings] && params[:account][:settings][:outgoing_email_default_name_op 509 # If set to default, remove the custom name so it doesn't get saved 510 params[:account][:settings][:outgoing_email_default_name] = '' if params[:account][:setting 511 end 512 513 google_docs_domain = params[:account][:settings].try(:delete, :google_docs_domain) 514 if @account.feature_enabled?(:google_docs_domain_restriction) && 515 @account.root_account? && 516 !@account.site_admin? 517 @account.settings[:google_docs_domain] = google_docs_domain.present? ? google_docs_domain : 518 end 519 520 @account.enable_user_notes = enable_user_notes if enable_user_notes 521 @account.allow_sis_import = allow_sis_import if allow_sis_import && @account.root_account? 522 if @account.site_admin? && params[:account][:settings] 523 # these shouldn't get set for the site admin account 524 params[:account][:settings].delete(:enable_alerts) 525 params[:account][:settings].delete(:enable_eportfolios) 526 end 527 else 528 # must have :manage_site_settings to update these 529 [ :admins_can_change_passwords, 530 :admins_can_view_notifications, 531 :enable_alerts, 532 :enable_eportfolios, 533 :enable_profiles, 534 :show_scheduler, 535 :global_includes, 536 :gmail_domain 537 ].each do |key| 538 params[:account][:settings].try(:delete, key) 18
  19. 475 def update 476 return update_api if api_request? 477 478

    if authorized_action(@account, @current_user, :manage_account_settings) 479 respond_to do |format| 480 481 custom_help_links = params[:account].delete :custom_help_links 482 if custom_help_links 483 @account.settings[:custom_help_links] = custom_help_links.select{|k, h| h['state'] != 'delete 484 hash = index_with_hash[1] 485 hash.delete('state') 486 hash.assert_valid_keys ["text", "subtext", "url", "available_to"] 487 hash 488 end 489 end 490 491 params[:account][:turnitin_host] = validated_turnitin_host(params[:account][:turnitin_host]) 492 enable_user_notes = params[:account].delete :enable_user_notes 493 allow_sis_import = params[:account].delete :allow_sis_import 494 params[:account].delete :default_user_storage_quota_mb unless @account.root_account? && !@accou 495 unless @account.grants_right? @current_user, :manage_storage_quotas 496 [:storage_quota, :default_storage_quota, :default_storage_quota_mb, 497 :default_user_storage_quota, :default_user_storage_quota_mb, 498 :default_group_storage_quota, :default_group_storage_quota_mb].each { |key| params[:account] 499 end 500 if params[:account][:services] 501 params[:account][:services].slice(*Account.services_exposed_to_ui_hash(nil, @current_user, @a 502 @account.set_service_availability(key, value == '1') 503 end 504 params[:account].delete :services 505 end 506 if @account.grants_right?(@current_user, :manage_site_settings) 507 # If the setting is present (update is called from 2 different settings forms, one for notifi 508 if params[:account][:settings] && params[:account][:settings][:outgoing_email_default_name_op 509 # If set to default, remove the custom name so it doesn't get saved 510 params[:account][:settings][:outgoing_email_default_name] = '' if params[:account][:setting 511 end 512 513 google_docs_domain = params[:account][:settings].try(:delete, :google_docs_domain) 514 if @account.feature_enabled?(:google_docs_domain_restriction) && 515 @account.root_account? && 516 !@account.site_admin? 517 @account.settings[:google_docs_domain] = google_docs_domain.present? ? google_docs_domain : 518 end 519 520 @account.enable_user_notes = enable_user_notes if enable_user_notes 521 @account.allow_sis_import = allow_sis_import if allow_sis_import && @account.root_account? 522 if @account.site_admin? && params[:account][:settings] 523 # these shouldn't get set for the site admin account 524 params[:account][:settings].delete(:enable_alerts) 525 params[:account][:settings].delete(:enable_eportfolios) 526 end 527 else 528 # must have :manage_site_settings to update these 529 [ :admins_can_change_passwords, 530 :admins_can_view_notifications, 531 :enable_alerts, 532 :enable_eportfolios, 533 :enable_profiles, 534 :show_scheduler, 535 :global_includes, 536 :gmail_domain 537 ].each do |key| 538 params[:account][:settings].try(:delete, key) 19 Infrastructure-as-Code
  20. Infrastructure-as-Code Goals ▪ Unify the view of resources ▪ Support

    the modern data center (IaaS, PaaS, SaaS) ▪ Expose a way for individuals and teams to safely and predictably change infrastructure ▪ Provide a workflow that is technology agnostic ▪ Manage anything with an API !20
  21. AWS CloudFormation AWS, YAML/JSON !21 Azure Resource Manager Azure, JSON

    Google Cloud Deployment Manager Google Cloud, YAML/JSON Ansible, Puppet, Chef Configuration Management Tools, YAML/JSON Terraform Most Infrastructure & More, HashiCorp Configuration Language Pulumi Most Public Clouds, 
 Javascript/Typescript/Python/Golang
  22. “I don’t know how to code. But I do want

    to automate how I configure my infrastructure.” - An Infrastructure Engineer Who Started Using Google Cloud !22
  23. “It was hard enough for me to learn infrastructure terms

    but writing code for it too? Difficult.” - A Developer Who Started Using Google Cloud !23
  24. Terraform HashiCorp Configuration Language CODE EDITOR resource "google_compute_instance" "default" {

    name = "test" machine_type = "n1-standard-1" zone = "us-central1-a" tags = ["foo", "bar"] boot_disk { initialize_params { image = "debian-cloud/debian-9" } } // Local SSD disk scratch_disk { } network_interface { network = "default" access_config { // Ephemeral IP } } metadata = { foo = "bar" } metadata_startup_script = "echo hi > /test.txt" service_account { scopes = ["userinfo-email", "compute-ro", "storage-ro"] } } !24
  25. Extensibility Configure Beyond Public Cloud Infrastructure CODE EDITOR # Create

    a new Datadog monitor resource "datadog_monitor" "foo" { name = "Name for monitor foo" type = "metric alert" message = "Monitor triggered. Notify: @hipchat-channel" escalation_message = "Escalation message @pagerduty" query = "avg(last_1h):avg:aws.ec2.cpu{environment:foo,host:foo} by {host} > 4" thresholds = { ok = 0 warning = 2 warning_recovery = 1 critical = 4 critical_recovery = 3 } notify_no_data = false renotify_interval = 60 notify_audit = false timeout_h = 60 include_tags = true silenced = { "*" = 0 } tags = ["foo:bar", "baz"] } !25
  26. Terraform Concepts How do we use it? !26

  27. Get Started ▪ Open Chrome. ▪ Go to hashi.co/ wwcode-nyc

    ▪ Click Start. !27 hashi.co/wwcode-nyc
  28. !28

  29. TERMINAL > terraform help Usage: terraform [-version] [-help] <command> [args]

    The available commands for execution are listed below. The most common, useful commands are shown first, followed by less common or more advanced commands. If you're just getting started with Terraform, stick with the common commands. For the other commands, please read the help and docs before usage. Common commands: apply Builds or changes infrastructure console Interactive console for Terraform interpolations destroy Destroy Terraform-managed infrastructure env Workspace management fmt Rewrites config files to canonical format get Download and install modules for the configuration graph Create a visual graph of Terraform resources import Import existing infrastructure into Terraform init Initialize a Terraform working directory output Read an output from a state file plan Generate and show an execution plan providers Prints a tree of the providers used in the configuration refresh Update local state file against real resources show Inspect Terraform state or plan taint Manually mark a resource for recreation untaint Manually unmark a resource as tainted validate Validates the Terraform files version Prints the Terraform version workspace Workspace management !29
  30. Concept: Providers “Plug-in” to the Terraform core codebase ▪ Interface

    between Terraform core and any API ▪ Create, read, update, and delete resources Develop your own! !30
  31. !31 terraform.io/docs/providers/

  32. !32

  33. Command: terraform init Run: ▪ To initialize a new Terraform

    working directory ▪ After changing provider configuration
 You do not need to run terraform init before each command.
 Similar to git init! !33
  34. !34

  35. !35

  36. Command: terraform plan Shows: ▪ What will happen ▪ Why

    it will happen Don’t need to guess changes or their effects. Plans can be saved. !36
  37. TERMINAL ! + resource will be created
 - resource will

    be destroyed
 ~ resource will be updated in-place
 -/+ resources will be destroyed and re-created !37
  38. !38

  39. !39

  40. !40

  41. !41 terraform.io/docs/providers/kubernetes/

  42. What should be in our terraform plan? TERMINAL > cd

    /root/terraform > terraform plan -out=nginx.tfplan !42
  43. TERMINAL Terraform will perform the following actions: # kubernetes_deployment.application will

    be created + resource "kubernetes_deployment" "application" { + id = (known after apply) # . . . } # kubernetes_service.application will be created + resource "kubernetes_service" "application" { + id = (known after apply) + load_balancer_ingress = (known after apply) + metadata { + generation = (known after apply) + labels = { + "app" = "nginx" } + name = "nginx" + namespace = "default" + resource_version = (known after apply) + self_link = (known after apply) + uid = (known after apply) } # . . . } Plan: 2 to add, 0 to change, 0 to destroy. ------------------------------------------------------------------------ This plan was saved to: nginx.tfplan To perform exactly these actions, run the following command to apply: terraform apply "nginx.tfplan" !43
  44. Concept: Resource Graph Helps determine: ▪ What gets created first

    ▪ How modifications affect other resources Based on graph theory !44
  45. !45 terraform.io/docs/internals/graph.html

  46. TERMINAL > terraform graph digraph { compound = "true" newrank

    = "true" subgraph "root" { "[root] data.google_compute_zones.available" [label = "data.google_compute_zones.available", sh ape = "box"] "[root] google_compute_network.vpc" [label = "google_compute_network.vpc", shape = "box"] "[root] google_compute_subnetwork.subnet" [label = "google_compute_subnetwork.subnet", shape = "box"] "[root] google_container_cluster.kubernetes" [label = "google_container_cluster.kubernetes", sh ape = "box"] "[root] local.labels" [label = "local.labels", shape = "note"] "[root] provider.google" [label = "provider.google", shape = "diamond"] "[root] var.cluster_name" [label = "var.cluster_name", shape = "note"] "[root] var.credentials" [label = "var.credentials", shape = "note"] "[root] var.kubernetes_version" [label = "var.kubernetes_version", shape = "note"] "[root] var.machine_type" [label = "var.machine_type", shape = "note"] !46
  47. !47

  48. Command: terraform apply Executes: ▪ Changes in order based on

    resource graph ▪ Parallel changes, if possible ▪ Error handling and recovery ▪ Updates in-place if possible or re-creates
 Produces a state file. !48
  49. TERMINAL > cd /root/terraform > terraform apply An execution plan

    has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # . . . Plan: 2 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes kubernetes_service.application: Creating... kubernetes_service.application: Creation complete after 0s [id=default/nginx] kubernetes_deployment.application: Creating... kubernetes_deployment.application: Still creating... [10s elapsed] kubernetes_deployment.application: Creation complete after 16s [id=default/nginx] Apply complete! Resources: 2 added, 0 changed, 0 destroyed. !49
  50. !50

  51. TERMINAL > cat terraform.tfstate { "version": 4, "terraform_version": "0.12.5", "serial":

    7, "lineage": "1d460dfb-d28d-04d7-bebb-867ab2389221", "outputs": {}, "resources": [ { "mode": "managed", "type": "kubernetes_deployment", "name": "application", "provider": "provider.kubernetes", "instances": [ { "schema_version": 0, "attributes": { "id": "default/nginx", "metadata": [ { "annotations": null, "generate_name": "", "generation": 1, "labels": { "app": "nginx" }, "name": "nginx", "namespace": "default", "resource_version": "793", "self_link": "/apis/apps/v1/namespaces/default/deployments/nginx", "uid": "72961252-acb6-11e9-b03e-42010a840047" } ], !51
  52. Concept: State Before Infrastructure-as-Code: ▪ What’s the configuration? ▪ Manual

    changes = 
 State must be maintained appropriately. ▪ Like configuration backup. ▪ Separate state file per “environment”. !52
  53. !53 terraform.io/docs/backends/types/local.html

  54. Change State File Location From /root/terraform/ to / tmp/ CODE

    EDITOR # versions.tf terraform { required_version = "~> 0.12" backend "local" {} } provider "kubernetes" { host = "https://kubernetes:6443" version = "~> 1.7" } !54
  55. Tip: Use separate files for different backend parameters! CODE EDITOR

    # backend.conf path = "/tmp/terraform.tfstate" !55
  56. TERMINAL > terraform init -backend-config=backend.conf Initializing the backend... Initializing provider

    plugins... Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary. !56
  57. TERMINAL > terraform apply An execution plan has been generated

    and is shown below. Resource actions are indicated with the following symbols: + create # . . . Plan: 2 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes kubernetes_service.application: Creating... kubernetes_deployment.application: Creating... kubernetes_service.application: Creation complete after 5s [id=default/nginx] kubernetes_deployment.application: Creation complete after 8s [id=default/nginx] Apply complete! Resources: 2 added, 0 changed, 0 destroyed. The state of your infrastructure has been saved to the path below. This state is required to modify and destroy your infrastructure, so keep it safe. To inspect the complete state use the `terraform show` command. State path: /tmp/terraform.tfstate !57
  58. TERMINAL > cat /tmp/terraform.tfstate { "version": 4, "terraform_version": "0.12.5", "serial":

    3, "lineage": "a8d16ab1-f30d-d21c-97e5-5e3bee628081", "outputs": {}, "resources": [ { "mode": "managed", "type": "kubernetes_deployment", "name": "application", "provider": "provider.kubernetes", "instances": [ !58
  59. Concept: Variables Parametrize when possible. ▪ Can have defaults ▪

    Can be overridden by CLI or variables file For passwords or secrets, do not leave them in any files! Pass them via command line or environment variables. !59
  60. Variable Types Valid types: string, number, bool Complex types: list,

    map If omitted, the type is inferred from the type of default. If no default, type is string. !60
  61. CODE EDITOR # /root/terraform/variables.tf variable "name" { type = string

    default = “nginx” } variable "namespace" { default = "default" } variable "replicas" { default = 1 } !61
  62. !62 terraform.io/docs/configuration/variables.html

  63. TERMINAL > ls -al /root/terraform total 40 drwxr-xr-x 3 root

    root 4096 Jul 23 14:41 . drwx------ 10 root root 4096 Jul 23 14:41 .. drwxr-xr-x 3 root root 4096 Jul 23 14:40 .terraform -rw-r--r-- 1 root root 32 Jul 23 14:40 backend.conf -rw-r--r-- 1 root root 723 Jul 23 14:21 deployment.tf -rw-r--r-- 1 root root 3148 Jul 23 14:28 nginx.tfplan -rw-r--r-- 1 root root 305 Jul 23 14:21 service.tf -rw-r--r-- 1 root root 24 Jul 23 14:41 terraform.tfvars -rw-r--r-- 1 root root 335 Jul 23 14:21 variables.tf -rw-r--r-- 1 root root 152 Jul 23 14:41 versions.tf > cat terraform.tfvars replicas= name= !63
  64. Update the Variables !64

  65. !65

  66. TERMINAL > terraform apply # . . . Plan: 2

    to add, 0 to change, 2 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes kubernetes_service.application: Destroying... [id=default/nginx] kubernetes_service.application: Destruction complete after 0s kubernetes_service.application: Creating... kubernetes_service.application: Creation complete after 1s [id=default/myapp] kubernetes_deployment.application: Destroying... [id=default/nginx] kubernetes_deployment.application: Destruction complete after 0s kubernetes_deployment.application: Creating… kubernetes_deployment.application: Creation complete after 4s [id=default/myapp] Apply complete! Resources: 2 added, 0 changed, 2 destroyed. The state of your infrastructure has been saved to the path below. This state is required to modify and destroy your infrastructure, so keep it safe. To inspect the complete state use the `terraform show` command. State path: /tmp/terraform.tfstate !66
  67. Scaling Infrastructure-as- Code What do we do as a team?

    !67
  68. Modules For others to use. !68 terraform.io/docs/modules/

  69. CODE EDITOR data "google_service_account" "vault_server" { account_id = "vault-server" }

    module "cluster" { # version control, local directory, or Terraform module registry source = "./kubernetes-cluster" project = var.project region = var.region subnet_cidr = var.subnet_cidr cluster_name = var.cluster_name kubernetes_version = "1.13.7-gke.8" service_account = data.google_service_account.vault_server.email num_clusters = 16 } !69
  70. !70 registry.terraform.io/

  71. State per “Environment” Version Control Continuous Integration Testing !71

  72. Software Development Patterns? !72

  73. Workflows Software Development 1. Fix code. 2. (Maybe) test. 3.git

    commit 4. Work a little more. 5.git push 6. Test everything. Infrastructure-as-Code 1. Fix configuration. 2. Test syntax. 3. git commit 4. Work a little more. 5. git push 6. Check differences. 7. Apply new configuration. 8. Test everything. !73
  74. General Terraform Repository Structure !74

  75. Collaboration Demo !75 github.com/joatmon08/terraform0.12-cloud/tree/remote-state

  76. Many more nuances…! (Click for links below.) 1. Migrating Terraform

    Resources 2. Upgrading Terraform 3. Workspaces, Terraform Cloud’s way of separating state 4. Secrets in infrastructure configuration !76
  77. learn.hashicorp.com/terraform/ terraform.io/downloads.html terraform.io/docs/ !77

  78. Rosemary Wang Developer Advocate at HashiCorp she/her @joatmon08 joatmon08 linkedin.com/in/rosemarywang/

    !78