A Terraform Journey

Transcript

1. A Terraform Journey From early adopters to power users over a two year period
2. # whoami • Currently contracted to FNB to assist with modernisation of Puppet code base • Will be implementing Terraform for FNB in Q3 • Moved back to SA from UK “recently” 
 (3 years ago) • Owner/Director of Ascendant Systems
3. What is Terraform? • Hashicorp product • The folks who gave us Vagrant and Packer • Fully open source • Written in Golang • Single binary • Multiple platforms • Frequent stable releases • Supported version: Terraform Enterprise “Terraform enables you to safely and predictably create, change, and improve production infrastructure”
4. Why Terraform? • Infrastructure as code • Configuration stored in SCM • Testable • Can be integrated into CI pipelines
 • Declarative DSL using HCL • All resources declared once • Downside: Rigid ‘code’ structure • Limited conditionals
 • Safety first • Separate Plan and apply phase
5. Multiple Providers • Provider agnostic • Not targeted at any specific service • Same syntax/structure for any provider • Can be used for on-premises infra • Extensive AWS support • New providers/resources added continuously
6. Current Providers • Alicloud • Archive • Arukas • AWS • Bitbucket • CenturyLinkCloud • Chef • Circonus • Cloudflare • CloudStack • Cobbler • Consul • Datadog • DigitalOcean • DNS • DNSMadeEasy • DNSimple • Docker • Dyn • External • Fastly • GitHub • Gitlab • Google Cloud • Grafana • Heroku • HTTP • Icinga2 • Ignition • InfluxDB • Kubernetes • Librato • Local • Logentries • Mailgun • New Relic • Nomad • NS1 • Microsoft Azure • Microsoft Azure (Legacy ASM) • MySQL • 1&1 • Oracle Public Cloud • OpenStack • OpsGenie • OVH • Packet • PagerDuty • PostgreSQL • PowerDNS • ProfitBricks • RabbitMQ • Rancher • Random • Rundeck • Scaleway • SoftLayer • StatusCake • Spotinst • Template • Terraform • Terraform Enterprise • TLS • Triton • UltraDNS • Vault • VMware vCloud Director • VMware vSphere
7. Hows does Terraform fit the Devops model? • Both Developers and Operations can work with the same toolset.
 • Moves Operations teams closer to the developer mindset: • Testing changes • Run Operational changes in CI • Allows Developers a means of applying operational changes.
8. For the security conscious • That’s all of us, right? 
 • No built-in secret management • Must be handled separately • We use eyaml
 • Integrates with Hashicorp’s Vault
 • TLS provider to manage certs
 • Auditing: SCM and CI are crucial
9. Code example
10. Two phases - Plan / Apply • Plan phase • create/update HCL config • run terraform plan • statefile created/read in • planfile is created • confirm output
11. Plan example
12. CI example - plan
13. Two phases - Plan / Apply • Apply phase • terraform apply • planfile is implemented • output shows the plan being applied • statefile is updated
14. CI example - apply
15. Terraform in CI (Jenkins)
16. Our journey at ITV • Early adopters • first public release - 0.5.0 • Some rough edges • Required a wrapper • domed city • Initial experiences: • Far too much duplication of config • Use a nested directory structure • Ecosystem directories 
 - one per ecosystem • Nested environment directories 
 within each ecosystem
17. Why a wrapper ? • Domed city • Written in ruby • Allows access to eyaml keys • Manage multiple AWS credentials • Facilitates locking with simpleDB • Install modules using Terrafile • Alternatives • Terragrunt • Various shell scripts in Github
18. Growth and Maturity • Shared code: • Modularise the config - more DRY • Moving building blocks into SCM
 • Wider implementation • More teams, better growth
 • Adding to CI pipelines • Visibility within organisation • Slack / chat notifications / monitoring • Auditing and Change management

19. Terraform Modules • Building blocks of Terraform config • Allows code to be easily re-used • Code can be stored in public repos - Github • Nested modules means even more reduction of duplication • “local” modules allows code to be tested locally before being moved into their own repo • One main drawback - no “count” parameter
20. Terrafile • No built in module version management in Terraform • Terrafile - Versions defined in yaml • Automated module install via Rake • Track github modules via tags • Semantic versioning FTW
21. Module example
22. Module example
23. Here be dragons… • State management: • Local vs remote • Local • error prone • must be checked into SCM
 in order to be shared • Secrets in plain text in SCM • Remote • Easily shared • S3 bucket to share state • No native locking (TF < 0.9.0)
24. Experiences of “terror” with Terraform • Always check the output of the plan phase! • Exercise caution - you can remove all of your infrastructure in a single run • Use “prevent_destroy” if needed - for
 example, when creating database instances • ALWAYS check the output of the plan phase! especially in CI pipelines
25. Questions? Twitter: @madAndroid Github: madAndroid andrew@ascendant-systems.co.za