Best Practices of Infrastructure-as-Code with Terraform

Join this webinar to learn why Infrastructure as Code is the answer to managing large scale, distributed systems and service-oriented architectures. We will cover key use cases, a demo of how to use Infrastructure as Code to provision your infrastructure and more.

Agenda:

Intro to Infrastructure as Code: Challenges & Use cases
Writing Infrastructure as Code with Terraform
Collaborating with Teams on Infrastructure

Transcript

1. Copyright © 2019 HashiCorp Best Practices of Infrastructure-as- Code with Terraform DevOps.com | December 13, 2019 1
2. Presenter Rosemary Wang Developer Advocate at HashiCorp she/her @joatmon08 joatmon08 linkedin.com/in/rosemarywang/ 2
3. The shift to provisioning dynamic infrastructure ⁄ USING TERRAFORM IN DYNAMIC INFRASTRUCTURE Copyright © 2018 HashiCorp ⁄ 3 Static Homogeneous, Private Dynamic Heterogeneous, Distributed
4. ⁄ USING TERRAFORM IN DYNAMIC INFRASTRUCTURE Copyright © 2018 HashiCorp ⁄ 4 Dynamic Heterogeneous, Distributed Static Homogeneous, Private The shift to provisioning dynamic infrastructure
5. 475 def update 476 return update_api if api_request? 477 478 if authorized_action(@account, @current_user, :manage_account_settings) 479 respond_to do |format| 480 481 custom_help_links = params[:account].delete :custom_help_links 482 if custom_help_links 483 @account.settings[:custom_help_links] = custom_help_links.select{|k, h| h['state'] != 'delete 484 hash = index_with_hash[1] 485 hash.delete('state') 486 hash.assert_valid_keys ["text", "subtext", "url", "available_to"] 487 hash 488 end 489 end 490 491 params[:account][:turnitin_host] = validated_turnitin_host(params[:account][:turnitin_host]) 492 enable_user_notes = params[:account].delete :enable_user_notes 493 allow_sis_import = params[:account].delete :allow_sis_import 494 params[:account].delete :default_user_storage_quota_mb unless @account.root_account? && !@accou 495 unless @account.grants_right? @current_user, :manage_storage_quotas 496 [:storage_quota, :default_storage_quota, :default_storage_quota_mb, 497 :default_user_storage_quota, :default_user_storage_quota_mb, 498 :default_group_storage_quota, :default_group_storage_quota_mb].each { |key| params[:account] 499 end 500 if params[:account][:services] 501 params[:account][:services].slice(*Account.services_exposed_to_ui_hash(nil, @current_user, @a 502 @account.set_service_availability(key, value == '1') 503 end 504 params[:account].delete :services 505 end 506 if @account.grants_right?(@current_user, :manage_site_settings) 507 # If the setting is present (update is called from 2 different settings forms, one for notifi 508 if params[:account][:settings] && params[:account][:settings][:outgoing_email_default_name_op 509 # If set to default, remove the custom name so it doesn't get saved 510 params[:account][:settings][:outgoing_email_default_name] = '' if params[:account][:setting 511 end 512 513 google_docs_domain = params[:account][:settings].try(:delete, :google_docs_domain) 514 if @account.feature_enabled?(:google_docs_domain_restriction) && 515 @account.root_account? && 516 !@account.site_admin? 517 @account.settings[:google_docs_domain] = google_docs_domain.present? ? google_docs_domain : 518 end 519 520 @account.enable_user_notes = enable_user_notes if enable_user_notes 521 @account.allow_sis_import = allow_sis_import if allow_sis_import && @account.root_account? 522 if @account.site_admin? && params[:account][:settings] 523 # these shouldn't get set for the site admin account 524 params[:account][:settings].delete(:enable_alerts) 525 params[:account][:settings].delete(:enable_eportfolios) 526 end 527 else 528 # must have :manage_site_settings to update these 529 [ :admins_can_change_passwords, 530 :admins_can_view_notifications, 531 :enable_alerts, 532 :enable_eportfolios, 533 :enable_profiles, 534 :show_scheduler, 535 :global_includes, 536 :gmail_domain 537 ].each do |key| 538 params[:account][:settings].try(:delete, key) 5 Infrastructure-as-Code
6. Agenda Infrastructure-as-Code Challenges Solving Challenges with Terraform Collaboration & Scaling 6
7. ⁄ Infrastructure-as-Code Challenges 7
8. Goals ▪ Unify the view of resources ▪ Support the modern data center (IaaS, PaaS, SaaS) ▪ Expose a way for individuals and teams to safely and predictably change infrastructure ▪ Provide a workflow that is technology agnostic ▪ Manage anything with an API 8
9. Initial Challenges ▪ Need to learn to code ▪ Can’t automate a resource ▪ Can’t track changes ▪ Don’t know change impact ▪ Need to revert a change 9
10. Scaling Challenges ▪ Multiple environments for infrastructure ▪ Duplicate code ▪ “Ball of Mud” configuration ▪ Too many working on code ▪ Dry run doesn’t reflect change impact ▪ Upgrades are disruptive 10
11. ⁄ Solving Challenges with Terraform 11
12. Initial Challenges ▪ Need to learn to code ▪ Can’t automate a resource ▪ Can’t track changes ▪ Don’t know change impact ▪ Need to revert a change 12
13. Need to learn to code? CODE EDITOR resource "google_compute_instance" "default" { name = "test" machine_type = "n1-standard-1" zone = "us-central1-a" tags = ["foo", "bar"] boot_disk { initialize_params { image = "debian-cloud/debian-9" } } // omitted for clarity } 13
14. Need to learn to code? ▪ HashiCorp Configuration Language ▪ Language describes intent ▪ Declarative (I declare, therefore I am.) ▪ Handles logic of calling APIs in proper order 14 terraform.io/docs/configuration/syntax.html
15. Can’t automate a resource? 15
16. 16 terraform.io/docs/providers/
17. ▪ Many providers community- maintained ▪ Write your own with the Terraform Plugin SDK! CODE EDITOR # Create a new Datadog monitor resource "datadog_monitor" "foo" { name = "Name for monitor foo" type = "metric alert" message = "Monitor triggered." // omitted for clarity thresholds = { ok = 0 warning = 2 warning_recovery = 1 critical = 4 critical_recovery = 3 } // omitted for clarity } 17 hashicorp.com/resources/everything-as-code-with-terraform
18. Can't track changes? 18
19. Can't track changes? ▪ Track state of existing infrastructure resources ▪ State updates when changes applied IMPORTANT NOTE ▪ Non-Terraform resources not automatically added ▪ Configuration not automatically generated ▪ Manual changes get overwritten 19 terraform.io/docs/state/index.html
20. Don't know change impact? TERMINAL > terraform plan Terraform will perform the following actions: # aws_vpc.app_vpc will be created + resource "aws_vpc" "app_vpc" { + arn = (known after apply) + cidr_block = “10.128.0.0/25" // omitted for clarity } Plan: 1 to add, 0 to change, 0 to destroy. 20
21. 21 terraform.io/docs/internals/graph.html
22. TERMINAL + resource will be created - resource will be destroyed ~ resource will be updated in-place -/+ resources will be destroyed and re-created 22
23. Need to revert a change? CODE EDITOR terraform { backend "remote" { organization = “<tf cloud org>" workspaces { name = “<tf cloud workspace>” } } } 23
24. Need to revert a change? ▪ Version control working configuration ▪ Remote state and if possible, versioned ▪ Update to previous working version ▪ Add toggle for easier revert IMPORTANT NOTE ▪ More like “roll forward” 24 terraform.io/docs/backends/index.html
25. ⁄ Collaborating & Scaling 25
26. Scaling Challenges ▪ Multiple environments for infrastructure ▪ Duplicate code ▪ “Ball of Mud” configuration ▪ Too many working on code ▪ Dry run doesn’t reflect change impact ▪ Upgrades are disruptive 26
27. Multiple environ- ments? TERMINAL > terraform workspace list default dev * prod > tree terraform.tfstate.d terraform.tfstate.d ├── dev └── prod 27
28. Workspaces ▪ Each workspace isolates state ▪ Map environment to workspace prevents state contamination IMPORTANT NOTE ▪ More functionality for Terraform Cloud ▪ Manages state, access control, runs, etc. 28 terraform.io/docs/state/workspaces.html
29. TERMINAL > cd dev > terraform workspace dev > terraform init > terraform plan > terraform apply 29
30. Duplicate code? TERMINAL hello_world ├── dev │ ├── network.tf │ ├── kubernetes.tf │ ├── app.tf │ └── database.tf └── prod ├── network.tf ├── kubernetes.tf ├── app.tf └── database.tf 30 Evolving Your Infrastructure with Terraform (Nicki Watts)
31. ▪ Use modules ▪ Divide resource types into different files ▪ Other sources – Version Control (submodules) – Module registry TERMINAL hello_world ├── base // can be separately maintained │ ├── network │ │ ├── subnets.tf │ │ └── vpc.tf │ ├── kubernetes │ │ └── cluster.tf │ ├── database │ │ └── database.tf │ └── app │ └── app.tf ├── dev │ └── main.tf └── prod └── main.tf 31
32. When building modules… ▪ Set provider version in consumer ▪ Version with tagging CODE EDITOR provider "aws" { region = var.region version = "~> 2.41" } module "elb" { source = "terraform-aws-modules/elb/aws" version = "2.3.0" health_check = var.health_check listener = var.listener // omitted for clarity } output "dns" { value = module.elb.this_elb_dns_name } 32 terraform.io/docs/configuration/modules.html
33. “Ball of Mud” Config? TERMINAL > terraform plan Terraform will perform the following actions: // omitted for clarity Plan: 300 to add, 0 to change, 0 to destroy. 33
34. ▪ Decouple with data sources ▪ Run separately CODE EDITOR data "aws_vpc" "selected" { filter { name = "owner" values = [var.owner] } } resource "aws_subnet" "example" { vpc_id = data.aws_vpc.selected.id availability_zone = "us-west-2a" cidr_block = cidrsubnet(data.aws_vpc.selected.cidr_block, 4, 1) } 34 sysadvent.blogspot.com/2019/12/day-5-break-up-your-terraform-project.html
35. Too many working on code? 35
36. Software Development Patterns 36
37. Establish Collaboration Patterns ▪ Adopt a software development pattern ▪ Put it in a CI pipeline ▪ Apply and audit changes based on code push ▪ Lock state during changes to prevent overrides 37 terraform.io/docs/state/locking.html
38. Dry run doesn’t reflect change impact? TERMINAL > kitchen test -----> Starting Kitchen (v2.3.3) … Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds 38
39. Integration Tests Contract Tests Unit Tests Infrastructure Testing Manual Testing Cost (Time, $$$) End-to-End Tests hashicorp.com/resources/test-driven-development-tdd-for-infrastructure
41. Upgrades are disruptive? TERMINAL > terraform-0.7.13 apply Terraform doesn't allow running any operations against a state that was written by a future Terraform version. The state is reporting it is written by Terraform '0.8.8'. Please run at least that version of Terraform to continue 41
42. 42 0.8 0.9 0.10 0.11 0.12 CHANGELOG Upgrade Guide Template files & string interpolation changes AWS provider attribute deprecations CHANGELOG Upgrade Guide Migrating to Backends Deprecate Remote for Backend Configuration State Locking AWS provider changes may trigger recreation Providers separated as plugins from core repository & versioned Interactive approval for apply (breaks pipelines, add -auto- approve flag) CHANGELOG Upgrade Guide Changes to module inheritance of providers Always use splat (*) operator for count references CHANGELOG Upgrade Guide CHANGELOG Upgrade Guide Adds rich type system to a previously string-typed system Includes automated upgrade tool (with caveats) AWS Provider CHANGELOG AWS v2 Upgrade Guide speakerdeck.com/joatmon08/the-semi-ultimate-terraform-upgrade-guide
43. Ease Upgrade Path by… ▪ Pinning provider versions ▪ Using known functions and not creative hacks ▪ Decoupling configuration across providers (i.e., separate Kubernetes from GCP) ▪ Avoid provisioners or complicated lifecycle customizations 43 hashicorp.com/resources/closing-keynote-terraform-at-google
44. Resources ▪ Terraform Cloud | app.terraform.io/signup/account ▪ Learn Terraform | learn.hashicorp.com/terraform ▪ Community Forum | discuss.hashicorp.com 44
45. Rosemary Wang Developer Advocate at HashiCorp she/her @joatmon08 joatmon08 linkedin.com/in/rosemarywang/ 45 joatmon08.github.io