© 2018 Google LLC. All rights reserved. Why, How and Best Practices GDG Cloud Paris - October 2019 Terraform & Google Cloud Platform Aurélien Legrand ([email protected])

© 2018 Google LLC. All rights reserved. Infrastructure as Code Terraform 101 1 Terraform Best Practices 2 3 Agenda Cloud Foundation Toolkit 4

© 2019 Google LLC. All rights reserved. ● Aurélien Legrand - [email protected] ● SCE Infrastructure/ Networking @Google Paris My experience with Terraform: ● Working with multiple French & UK customers on Designing & implementing their architecture in GCP ● Contributing to Cloud Foundation Toolkit whoami

01 Infrastructure as Code

© 2018 Google LLC. All rights reserved. Infrastructure Management Before Hey does anybody know who deployed DSS-012 or what it does? Or why there are 19 instances in running production?

Infrastructure Management As Code ● Store infrastructure definition in source control ● Code serves as documentation ● Approval process ● Know who made changes and why ● Roll back infrastructure changes ● Only specify “desired state”

02 Terraform 101

© 2018 Google LLC. All rights reserved. Open Source core developed by Hashicorp in 2014, with enterprise version available Open Core Focus on provisioning infrastructure, not configuring it (cf. Chef/Puppet) Provisioning Support for dozens of different providers via pluggable design, notably GCP, AWS, VMWare, DNS, and Kubernetes Multi-cloud Resources can be shared using modules, and independent providers can be developed in Go Shareability Hashicorp Terraform

© 2018 Google LLC. All rights reserved. Written in HCL or JSON Resources and Data sources Reference other resources (interpolation) Native functions Variables and Outputs Terraform Configs

© 2018 Google LLC. All rights reserved. Terraform Plan ● Plan output can be reviewed ● Output can also be saved and replayed ● Many teams embed this into their CI process

© 2018 Google LLC. All rights reserved. Terraform Apply

© 2018 Google LLC. All rights reserved. Terraform State Refresh

© 2018 Google LLC. All rights reserved. Package of multiple resources Community or Private Configurable Terraform Reusable Modules

© 2018 Google LLC. All rights reserved. ● Stores Terraform’s view of your infrastructure ● Maps your code to the actual resources ● By default, stored locally in terraform.tfstate ● Recommended to store with a GCS backend ● Reference remote states Remote state terraform { backend "gcs" { bucket = "gcp-foundation-tfstate" prefix = "terraform/state/base" } } data "terraform_remote_state" "base" { backend = "gcs" config { bucket = "rnm-terraform" prefix = "terraform/state/base" } }

© 2018 Google LLC. All rights reserved. ● Primary method for code reuse in Terraform ● Can be loaded from local references, Git(Hub), and the Terraform Module Registry ● Google maintains some (primary development target for Cloud Foundation Toolkit) ● Can have variables and outputs Modules module "project-factory" { source = "../../modules/project-factory" name = "factory-simple-app" org_id = "${var.organization_id}" folder_id = "${google_folder.projects_folder.na me}" }

© 2018 Google LLC. All rights reserved. Small Root Configs Instead of using one massive root config, separate logical components into separate “deployments” Don’t switch on *.tfvars Don’t reuse the exact same config for different environments, instead use modules for reuse Parameterize Intelligently Only parameterize values which actually need to be, emphasis should be on ease of understanding Use built-in functions Terraform has lots of built-in functions, know them and use them Plan first Always run plan and review the output before running apply Use terraform fmt Automatically maintains consistent formatting for you Terraform Principles

03 Terraform Best Practices

© 2018 Google LLC. All rights reserved. Run from the CLI initially Baked into CI/CD pipelines State files are stored remotely, for example in GCS => IAM, encryption, versioning, state locking, ... Bundled with Puppet/Chef/Ansible Terraform in the Real World

© 2019 Google LLC. All rights reserved. Automation Pipeline The toolkit encourages customers to collaborate on infrastructure through GitOps. Collaborate in source control Reduce manual effort and errors Enforce policies proactively Ensure consistency Developer submits Pull Request CI runs Validation Administrator reviews for Policy Compliance Administrator merges the New Config CD updates Deployed Infrastructure

© 2018 Google LLC. All rights reserved. |___ bootstrap # <- Organization creation, logging, CI/CD... |___ modules # <- Reusable modules |___ application1 # <- Configuration of modules for your application |____ README.md # <- Usage, owners... |____ main.tf # <- Provider configuration, state configuration |____ variables.tf # <- Local variables for this application |____ networking.tf # <- Networking resources (modules configuration) |____ vms.tf # <- Compute resources (modules configuration) Project/Repo structure for Terraform

© 2018 Google LLC. All rights reserved. 1. Directories per environments (+) : Each environment can be quite different, small blast radius (-) : Code duplication 2. Workspace per environments: Same Terraform code except the variables (+) : Identical code between environments (-) : Lot of lists, maps… Complex if environments diverge too much. 3. Solutions like Terragrunt (Terraform wrapper with some nice features) How to manage different environments in Terraform? (PROD/DEV…)

© 2018 Google LLC. All rights reserved. Terragrunt example Terragrunt can brings better folder hierarchy, backend & remote state configuration reutilization, “apply-all” for multiple folders, dependencies... You have your Terraform repositories for your applications + an “infrastructure repository”. For each application & environment, you have a HCL file: # infrastructure-live/prod/app/terragrunt.hcl terraform { source = "github.com:foo/infrastructure-modules.git//app?ref=v0.0.1" # Your app code here! } inputs = { # Variables here! instance_count = 10 instance_type = "n1.standard" }

© 2019 Google LLC. All rights reserved. Running as a user Only viable for individual testing or Proof of Concept. Should use a service account key file, or gcloud’s default application credentials. Running as a service Viable for production installations. When running in GCP, should use the GCE/GKE service account. When running in another provider, use a dedicated service account. Operation model

© 2018 Google LLC. All rights reserved. Best Practices Service accounts tf-org-projects tf-org-network tf-org-security tf-apps-example1 tf-apps-example2 Jenkins Build Server tf-org-base-svc@ tf-org-network-svc@ tf-org-security-svc@ tf-apps-example1-svc@ tf-apps-example2-svc@ uses tf-org-foundation Split by per team/BU and/or environment to reduce blast radius!

© 2018 Google LLC. All rights reserved. Best Practices Service accounts roles Service account Role name Role id (roles/) Role level tf-org-base-svc@ Organization Admin Project Creator Folder Creator Service Account Admin resourcemanager.organizationAdmin resourcemanager.projectCreator resourcemanager.folderCreator iam.serviceAccountAdmin Organization tf-org-network-svc@ Compute Network Administrator Compute Shared VPC Admin Owner compute.networkAdmin compute.xpnAdmin owner Organization Network project tf-org-security-svc@ Compute Security Admin Owner compute.securityAdmin owner Organization Security project tf-apps-example1@ Owner owner App project Easier but riskier...

© 2018 Google LLC. All rights reserved. ● Split your Terraform code into different modules and different state files to avoid one big monolith that is slow and too complex. ● You can use “terraform import” to import existing resources in the Terraform state… But you need to explicitly code all resources and it is often harder than to start from scratch! ● Don’t modify Terraform state manually, use “terraform state” instead. ● Encrypt the Terraform state because some secrets are stored in plain text, such as service account keys. ● Terraform can call custom scripts but this should be avoided as it can break in many ways. “terraform destroy” becomes pretty hard to do right. Other best practices… (1/2)

© 2018 Google LLC. All rights reserved. ● Use Terraform 0.12 for your new Terraform code (syntax is not compatible with previous version but migration is fairly easy) ● Pin the version of Terraform and the modules you use: Other best practices… (2/2) terraform { required_version = "~> 0.11.10" } module "nat_gateway" { source = “GoogleCloudPlatform/nat-gateway/google” version = “1.2.2” […] }

04 Cloud Foundation Toolkit Overview

© 2018 Google LLC. All rights reserved. Provide a common baseline of GCP best practices implemented in code. Project mission

© 2018 Google LLC. All rights reserved. Modularized Adoption Cloud Foundation Toolkit Pillars The scope of modules are designed to fit with enterprise grade adoption of GCP Common practices across vertical / industries Customers should be able to easily adopt and modify the toolkit to match their needs The toolkit is designed as a series of independent modules which can be adopted individually. The individual modules are easier to understand, test, and develop independently. The modules are designed to work well together as a cohesive toolkit. Make infrastructure easier and more efficient to manage—not harder. Embrace best practices like immutable infrastructure. Provide an easy on-ramp from manual deployments to automated infrastructure. Enterprise Ready Simplifying IAC

© 2019 Google LLC. All rights reserved. Project Factory VPN Network Factory IAM Org Policy Terraform / Deployment Manager Application VM Bastion Host GKE Cluster ILB Foundational Modules Baseline modules for deploying services in GCP using Infrastructure as Code; used for repeatable, iterative cloud operations that lay the baseline setup CloudSQL Pipeline Tooling Project / App Modules Spinnaker Jenkins Cloud Datastore Cloud Storage Forseti Cloud Native Modules Log Sink Instance Groups Stackdriver OS Login Container Deployment Functional tooling to get started with Infrastructure as Code and the Cloud Foundation Toolkit Deploying platform services and higher-level workloads; isolated management of resources Platform level modules to support foundational usage of GCP Firewall Rules Cloud Foundation Toolkit modules Terraform Module Available Deployment Manager Module Available

© 2019 Google LLC. All rights reserved. Overview This module provides an automated tool for creating new projects in GCP which are bootstrapped with foundations. Scope 1. Creates a new project (with unique suffix, if needed) in a folder 2. Attaches to shared VPC (with subnet support) 3. Deletes default network 4. Deletes default service account and creates a custom one 5. Creates a G Suite group and gives it IAM role on project 6. Associates billing account 7. Places service accounts in groups 8. Sets up usage export and GCS bucket for Terraform 9. Enables APIs Resources ● Terraform implementation (code repository) ● How to use this module Use cases / tags #operations, #projectmanagement Terraform Module: Available Deployment Manager Module: Available Project Factory Module

© 2019 Google LLC. All rights reserved. Overview This module provides a simple way to create a custom VPC network on GCP, including subnets and shared VPC. Scope 1. Creates custom subnets in multiple regions 2. Adds optional secondary ranges to subnets for Alias IP support 3. Enables Shared VPC support Resources ● Terraform implementation (code repository) ● How to use this module Use cases / tags Terraform Module: Available Deployment Manager Module: Available Network Module

© 2019 Google LLC. All rights reserved. Putting it all together! Using Project Factory + Network modules. Only configuring the modules, the complexity is defined in the modules code.

© 2019 Google LLC. All rights reserved. Overview This module provides a simple way to configure IAM roles for organizations, projects, and other resources. Scope 1. Enables setting multiple roles at once 2. Supports an additive mode which doesn’t wipe out existing roles/memberships 3. Supports an authoritative mode which will remove all users not managed via the module 4. Includes optional standard roles 5. Can apply IAM policies to multiple projects/folders at once Resources ● Terraform implementation (code repository) ● How to use this module Use cases / tags #projectmanagement Terraform Module: Available Deployment Manager Module: Available Identity & Access Management (IAM) Module

© 2019 Google LLC. All rights reserved. Overview This module provides an automated tool for exporting Stackdriver logs to other GCP services, on the project, folder, or organization level (aggregated logs). Scope 1. Creates a log export object and associated service account. 2. Creates a destination (GCS bucket, Pub/Sub topic, BigQuery dataset). 3. Grants the service account proper IAM permissions to write to the destination. Resources ● Terraform implementation (code repository) ● How to use this module Use cases / tags #operations, #logging, #auditing Terraform Module: Available Deployment Manager Module: Available Log Export Module

© 2019 Google LLC. All rights reserved. Overview This is a collection of opinionated submodules that can be used as building blocks to provision VMs. Scope 1. Create instance templates 2. Create managed instance groups from templates with optional autoscaling 3. Create unmanaged instance groups from templates Resources ● Terraform implementation (code repository) ● How to use this module Use cases / tags Terraform Module: Available Deployment Manager Module: Unavailable VM

© 2019 Google LLC. All rights reserved. Overview This module enables management of service accounts. Scope 1. Creates service accounts in a project 2. Assigns roles to the service accounts at the organization, billing account, and project levels 3. Create keys for the service accounts Resources ● Terraform implementation (code repository) ● How to use this module Use cases / tags Terraform Module: Available Deployment Manager Module: Unavailable Service Accounts

© 2018 Google LLC. All rights reserved. Have a look! https://github.com/terraform-google-modules/terraform-google-project-factory

© 2018 Google LLC. All rights reserved. Thank you