Terraforming the Cloud

Dirk and I talked about our usage of HashiCorp's Terraform at the first AWS Meetup in Cologne on 2015-07-23.

Transcript

1. Terraforming the Cloud Terraforming the Cloud
2. Sebastian StormForger @tisba Dirk Freelancer @railsbros_dirk Who?
3. Challenges • Increasing complexity • Multiple target environments • Multi/hybrid cloud • Evolving infrastructure • Communication & Documentation
4. Opportunities • Infrastructure (not just servers) on demand • Flexible APIs for all infrastructure components • Affordable but powerful hardware and services • Virtualization
5. Solution SCRIPTING SCRIPTING
6. #!/bin/bash ec2-authorize \ sg-${SECURITY_GROUP_EC2} \ -P tcp \ -p 22 \ -s 203.0.113.25/32 ec2-run-instances \ ami-${ID} \ -t ${INSTANCE_TYPE} \ -s subnet-${SUBNET_ID} \ -k my-key-pair \ -g sg-${SECURITY_GROUP_EC2} \ --associate-public-ip-address true
7. Issues with Scripting • Not declarative • Dynamic runtime • Hard to reason about prior to execution • How to deal with API errors, timeouts, …? • What about idempotency, versioning and rollbacks?
8. Wishlist • Describe entire infrastructure in a declarative way • Keep track of changes to the infrastructure • Changing infrastructure is accessible to entire team • Rollback your infrastructure to a previous point
9. Solutions? Tools? This time for real…
10. – @bascht “For any (dev)ops related problem HashiCorp most likely already has a tool.”
12. – terraform.io “Terraform provides a common configuration to launch infrastructure [...]. Once launched, Terraform safely and efficiently changes infrastructure as the configuration is evolved.” Terraform
13. Setup Staging Environment $ terraform plan -out staging.plan terraform/staging $ terraform apply staging.plan
14. Terraform • Describe entire infrastructure in a declarative way • Keep track of changes to the infrastructure • Changing infrastructure is accessible to entire team • Rollback your infrastructure to a previous point
15. Terraform Providers*… Atlas AWS Azure CloudFlare CloudStack Consul DigitalOcean DNSMadeEasy DNSimple Docker Google Cloud Heroku Mailgun OpenStack *currently
16. Terraform AWS Resources aws_app_cookie_stickiness_policy aws_autoscaling_group aws_autoscaling_notification aws_autoscaling_policy aws_cloudwatch_metric_alarm aws_customer_gateway aws_db_instance aws_db_parameter_group aws_db_security_group aws_db_subnet_group aws_dynamodb_table aws_ebs_volume aws_ecs_cluster aws_ecs_service aws_ecs_task_definition aws_eip aws_elasticache_cluster
 aws_elasticache_parameter_group aws_elasticache_security_group aws_elasticache_subnet_group aws_elb aws_flow_log aws_iam_access_key aws_iam_group aws_iam_group_policy aws_iam_group_membership aws_iam_instance_profile aws_iam_policy aws_iam_policy_attachment aws_iam_role aws_iam_role_policy aws_iam_server_certificate aws_iam_user aws_iam_user_policy
 aws_instance aws_internet_gateway aws_key_pair aws_kinesis_stream aws_lambda_function aws_launch_configuration aws_lb_cookie_stickiness_policy aws_main_route_table_association aws_network_acl aws_network_interface aws_proxy_protocol_policy aws_route53_delegation_set aws_route53_health_check aws_route53_record aws_route53_zone aws_route53_zone_association aws_route_table aws_s3_bucket aws_security_group aws_security_group_rule aws_sns_topic aws_sns_topic_subscription aws_spot_instance_request aws_sqs_queue aws_subnet aws_volume_attachment aws_vpc aws_vpc_dhcp_options aws_vpc_dhcp_options_association aws_vpc_endpoint aws_vpc_peering aws_vpn_connection aws_vpn_connection_route aws_vpn_gateway
17. Example Infrastructure
18. Enterprise Cloud Architecture Plan™ ELB +TLS Certificate +DNS Record EC2 Instance RDS Instance (with PostgreSQL) Security Group Security Group
19. What we need… DNS Record aws_route53_record Load Balancer aws_elb TLS Certificate aws_iam_server_certificate EC2 Instance aws_instance PostgreSQL Instance aws_db_instance Security Groups aws_security_group ELB +TLS Certificate +DNS Record EC2 Instance RDS Instance (with PostgreSQL)
20. Configuration Layout resource TYPE NAME { CONFIG ... [count = COUNT] [depends_on = [RESOURCE NAME, ...]] [LIFECYCLE] }
21. resource "aws_iam_server_certificate" "demo" { name = "demo" certificate_body = "${file("certs/demo.cert.pem")}" certificate_chain = "${file("certs/demo.ca-bundle.pem")}" private_key = "${file("secrets/demo.pem")}" }
22. resource "aws_elb" "staging_elb" { depends_on = ["aws_iam_server_certificate.demo"] name = "staging-elb" availability_zones = ["eu-central-1a", "eu-central-1b"] listener { instance_port = 80 instance_protocol = "http" lb_port = 443 lb_protocol = "https" ssl_certificate_id = "${aws_iam_server_certificate.demo.arn}" } health_check { target = "HTTP:80/status_check" } security_groups = ["${aws_security_group.staging_elb.id}"] instances = ["${aws_instance.staging_web.id}"] }
23. resource "aws_instance" "staging_web" { ami = "${lookup(var.staging_amis, var.region)}" instance_type = "t2.micro" security_groups = [ "default", "${aws_security_group.staging_web.name}" ] iam_instance_profile = "staging-ec2" tags { Name = "staging_web" Roles = "web,db" Stages = "staging" } }
24. Q&A Sebastian Cohnen stormforger.com @tisba Dirk Breuer tfcl.de @railsbros_dirk