Pro Yearly is on sale from $80 to $50! »

Build, provision & deploy in the Cloud with Packer, Ansible & Terraform – PHPUK 2018

Ca901ddcea38854b9783781c91fc87c9?s=47 Thijs Feryn
February 15, 2018

Build, provision & deploy in the Cloud with Packer, Ansible & Terraform – PHPUK 2018

More information about this Cloud automation talk I did at PHPUK Conference 2018 in London can be found on https://feryn.eu/speaking/build-provision-deploy-cloud-packer-ansible-phpuk-2018/

Ca901ddcea38854b9783781c91fc87c9?s=128

Thijs Feryn

February 15, 2018
Tweet

Transcript

  1. By Thijs Feryn Build, provision & deploy in the Cloud

    with Packer, Ansible & Terraform PHP UK CONFERENCE
  2. I’m so sick of Cloud!

  3. None
  4. Automation

  5. Many platforms

  6. None
  7. None
  8. ✓ Less lock-in ✓ Faster deployments ✓ Better scalability ✓

    Better reproducibility ✓ Less human error ✓ Lower cost ✓ Operational stability Goals
  9. Hi, I’m Thijs

  10. I’m @ThijsFeryn on Twitter

  11. I’m an Evangelist At

  12. I’m an Evangelist At

  13. None
  14. 185 17 11 4

  15. https://joind.in/talk/ae063

  16. Build Provision Orchestrate

  17. App code Bake image Build

  18. None
  19. App code Provision Bake image Pour “sauce” Build

  20. None
  21. App code Provision Orchestrate Bake image Pour “sauce” Serve Build

  22. None
  23. Infrastructure as code

  24. Disposable stacks

  25. Reproducible environments

  26. None
  27. ✓ Alicloud ECS ✓ Amazon EC2 ✓ Azure ✓ CloudStack

    ✓ DigitalOcean ✓ Docker ✓ File ✓ Google Cloud ✓ Hyper-V ✓ LXC ✓ LXD ✓ Null ✓ 1&1 ✓ OpenStack ✓ Oracle OCI ✓ Parallels ✓ ProfitBricks ✓ QEMU ✓ Triton ✓ VirtualBox ✓ VMware ✓ Custom Packer builders
  28. None
  29. { "builders": [ { "type": "openstack", "identity_endpoint": "https://osp.combell.com:5000/v3", "tenant_name": "",

    "username": "", "password": "", "image_name": "thijsferyn_{{isotime \"2006_01_02__03_04_05\"}}", "source_image": "1b9fe43e-4cf2-4986-b7f2-51f5706b120b", "ssh_username": "debian", "domain_name" : "default", "flavor": "m1.medium", "networks": ["a1175c54-cb12-436d-ac7b-d327499dc39b"], "floating_ip_pool" : "public" } ], "provisioners": [ { "type": "shell", "inline": ["apt-get update","apt-get install -y nginx"] } ] }
  30. None
  31. { "builders": [ { "type": "amazon-ebs", "access_key": "", "secret_key": "",

    "region": "eu-west-2", "instance_type": "t2.micro", "ssh_username": "admin", "associate_public_ip_address": true, "subnet_id": “subnet-11d3d56a", "source_ami": "ami-e1e8f085", "ami_name": "thijsferyn_{{isotime \"2006_01_02__03_04_05\"}}" } ], "provisioners": [ { "type": "shell", "inline": ["apt-get update","apt-get install -y nginx"] } ] }
  32. $ packer build packer.json

  33. Packer build Cloud API Boot VM using source image Take

    VM snapshot Image registry Store VM snapshot as new image Shell provisioning run Tear down VM
  34. $ openstack image list +--------------------------------------+----------------------------------+--------+ | ID | Name |

    Status | +--------------------------------------+----------------------------------+--------+ | ef90ad1c-ed64-4ae0-8283-e446e95c33e6 | thijsferyn_2018_01_31__02_26_27 | active | | aa0a5244-0d57-477d-b985-653d6b881292 | Windows-2012-R2 | active | | 31e84bbb-abbb-4d5a-b620-ecebf4697d16 | centos-6-64bit | active | | 9eac74f9-6e6c-45a9-bd93-915d3e390687 | centos-7-64bit | active | | 3a69a110-775f-41ad-9d63-32079db57203 | cirros-0.3.4-64bit | active | | 244cbbde-77b6-4f1e-837f-9250520ee78b | coreos-stable | active | | 1b9fe43e-4cf2-4986-b7f2-51f5706b120b | debian-jessie-64bit | active | | 690b063b-f239-4032-ad23-9ae6337f248f | debian-stretch-64bit | active | | d8034a10-9b15-46b1-9c68-bd98f92d0ffb | ubuntu-server-precise-lts-64-bit | active | | 040f9d1d-bb18-4466-abd7-9f2172e6db70 | ubuntu-server-trusty-lts-64-bit | active | | 5be2b652-a965-4ecb-a2b4-b9f83d7779e6 | ubuntu-server-xenial-lts-64-bit | active | +--------------------------------------+----------------------------------+--------+
  35. $ aws ec2 describe-images --owners=self --query "Images[*]. {ID:ImageId,Name:Name,Status:State}" --output=table +--------------+-----------------------------------+-------------+

    | ID | Name | Status | +--------------+-----------------------------------+-------------+ | ami-82ccd6e6| thijsferyn_2018_01_31__02_26_27 | available | +--------------+-----------------------------------+-------------+
  36. None
  37. Configuration management system

  38. None
  39. ✓ Written in Python ✓ Agentless ✓ Standalone ✓ SSH-based

    ✓ Runs playbook (yml files) ✓ Groups playbook in roles ✓ Install software ✓ Configure your server Ansible
  40. --- - hosts: all vars: code_folders: - "web" become: true

    tasks: - name: update apt cache apt: update_cache=yes cache_valid_time=86400 - name: install required packages apt: name={{ item }} state=present with_items: - vim - curl - nginx - php7.0-fpm - rsync - name: create /var/www/html/ folder file: path: /var/www/html/ state: directory mode: 0755 owner: www-data playbook.yml
  41. - name: create /var/www/html/ folder file: path: /var/www/html/ state: directory

    mode: 0755 owner: www-data - name: Copy code synchronize: src: ../{{ item }} dest: /var/www/html delete: yes recursive: yes group: no owner: no perms: no with_items: "{{code_folders}}" - name: Chown code to www-data file: path: /var/www/html/{{ item }} group: www-data owner: www-data mode: u=rwx,g=r,o=r recurse: yes playbook.yml
  42. Organize your playbooks

  43. . !"" files # $"" db.sql !"" host_vars # $""

    default.yml !"" group_vars # $"" all.yml !"" inventory !"" playbook.yml !"" roles # $"" my-role # !"" defaults # # $"" main.yml # !"" handlers # # $"" main.yml # !"" meta # # $"" main.yml # !"" tasks # # $"" main.yml # !"" templates # # $"" template.j2 # $"" vars # $"" vars.yml !"" tasks # !"" task1.yml # !"" task2.yml $"" templates $"" template.j2
  44. --- - hosts: all tasks: - include_tasks: task1.yml - include_tasks:

    task2.yml roles: - my-role
  45. $ ansible-playbook playbook.yml

  46. { "builders": [ { "type": "openstack", "identity_endpoint": "https://osp.combell.com:5000/v3", "tenant_name": "18324-1",

    "username": "18324", "password": "", "image_name": "MyBuild-{{isotime \"2006-01-02 03:04:05\"}}", "source_image": "1b9fe43e-4cf2-4986-b7f2-51f5706b120b", "ssh_username": "debian", "domain_name" : "default", "flavor": "m1.medium", "networks": ["a1175c54-cb12-436d-ac7b-d327499dc39b"], "floating_ip_pool" : "public" } ], "provisioners": [ { "type": "ansible", "playbook_file": "./ansible/playbook.yml" } ] }
  47. $ ansible-playbook --extra-vars packer_build_name=openstack packer_builder_type=openstack -i /var/folders/j7/ kpxjpxq92vs_t97gg6tqtttm0000gn/T/packer-provisioner- ansible972172074 /Users/thijsferyn/Dropbox/Sites/openstack/

    packer-ansible-terraform/ansible/playbook.yml --private-key / var/folders/j7/kpxjpxq92vs_t97gg6tqtttm0000gn/T/ansible- key333617975
  48. Packer build Cloud API Boot VM using source image Take

    VM snapshot Image registry Store VM snapshot as new image Ansible provisioning run Tear down VM
  49. "image_name": "MyBuild-{{isotime \"2006-01-02 03:04:05\"}}", "source_image": "1b9fe43e-4cf2-4986-b7f2-51f5706b120b", "ssh_username": "debian", "domain_name" :

    "default", "flavor": "m1.medium", "networks": ["a1175c54-cb12-436d-ac7b-d327499dc39b"], "floating_ip_pool" : "public" } ], "provisioners": [ { "type": "ansible", "playbook_file": "./ansible/playbook.yml" } ], "post-processors": [ { "type": "manifest", "output": "manifest.json" } ] }
  50. Store build results Packer build Cloud API Boot VM using

    source image Take VM snapshot Image registry Store VM snapshot as new image Ansible provisioning run Tear down VM manifest.json
  51. { "builds": [ { "name": "amazon-ebs", "builder_type": "amazon-ebs", "build_time": 1517408990,

    "files": null, "artifact_id": "eu-west-2:ami-82ccd6e6", "packer_run_uuid": "ba98283c-08ca-a9fc-ab10-e49502ce4ab6" }, { "name": "openstack", "builder_type": "openstack", "build_time": 1509053610, "files": null, "artifact_id": "ef90ad1c-ed64-4ae0-8283-e446e95c33e6", "packer_run_uuid": "b6000539-f3e6-85dd-2733-9fb3ae443e9f" } ], "last_run_uuid": "b6000539-f3e6-85dd-2733-9fb3ae443e9f" }
  52. $ cat manifest.json | \ jq '.builds[-1].artifact_id'

  53. $ openstack image list +--------------------------------------+----------------------------------+--------+ | ID | Name |

    Status | +--------------------------------------+----------------------------------+--------+ | ef90ad1c-ed64-4ae0-8283-e446e95c33e6 | thijsferyn_2018_01_31__02_26_27 | active | | aa0a5244-0d57-477d-b985-653d6b881292 | Windows-2012-R2 | active | | 31e84bbb-abbb-4d5a-b620-ecebf4697d16 | centos-6-64bit | active | | 9eac74f9-6e6c-45a9-bd93-915d3e390687 | centos-7-64bit | active | | 3a69a110-775f-41ad-9d63-32079db57203 | cirros-0.3.4-64bit | active | | 244cbbde-77b6-4f1e-837f-9250520ee78b | coreos-stable | active | | 1b9fe43e-4cf2-4986-b7f2-51f5706b120b | debian-jessie-64bit | active | | 690b063b-f239-4032-ad23-9ae6337f248f | debian-stretch-64bit | active | | d8034a10-9b15-46b1-9c68-bd98f92d0ffb | ubuntu-server-precise-lts-64-bit | active | | 040f9d1d-bb18-4466-abd7-9f2172e6db70 | ubuntu-server-trusty-lts-64-bit | active | | 5be2b652-a965-4ecb-a2b4-b9f83d7779e6 | ubuntu-server-xenial-lts-64-bit | active | +--------------------------------------+----------------------------------+--------+
  54. None
  55. Deploy the VM image

  56. Orchestration

  57. ✓ Launch virtual network ✓ Boot up webservers ✓ Configure

    firewalls ✓ Assign loadbalancers ✓ Create DNS-records ✓ Assign SSH keys ✓ Create autoscaling group Orchestration
  58. ✓ CloudFormation (AWS) ✓ Heat (OpenStack) ✓ Resource manager (Azure)

    Vendor-specific orchestration tools
  59. None
  60. ✓ Infrastructure As Code ✓ Plan, graph, execute ✓ State

    management ✓ Multiple providers ✓ Incremental changes ✓ Interacts with Cloud vendor APIs Terraform
  61. ✓ Alicloud ✓ Archive ✓ Arukas ✓ AWS ✓ Bitbucket

    ✓ CenturyLinkCloud ✓ Chef ✓ Circonus ✓ Cloudflare ✓ CloudStack ✓ Cobbler ✓ Consul ✓ Datadog ✓ DigitalOcean ✓ DNS ✓ DNSMadeEasy ✓ DNSimple ✓ Docker ✓ Dyn ✓ External ✓ Fastly ✓ GitHub ✓ Gitlab ✓ Google Cloud ✓ Grafana ✓ Heroku ✓ HTTP ✓ Icinga2 ✓ Ignition ✓ InfluxDB ✓ Kubernetes ✓ Librato ✓ Local ✓ Logentries ✓ LogicMonitor ✓ Mailgun ✓ Microsoft Azure ✓ Microsoft Azure (Legacy ASM) ✓ MySQL ✓ New Relic ✓ Nomad ✓ NS1 ✓ 1&1 ✓ Oracle Public Cloud ✓ OpenStack ✓ OpsGenie ✓ OVH ✓ Packet ✓ PagerDuty ✓ PostgreSQL ✓ PowerDNS ✓ ProfitBricks ✓ RabbitMQ ✓ Rancher ✓ Random ✓ Rundeck ✓ Scaleway ✓ SoftLayer ✓ StatusCake ✓ Spotinst ✓ Template ✓ Terraform ✓ Terraform Enterprise ✓ TLS ✓ Triton ✓ UltraDNS ✓ Vault ✓ VMware vCloud Director ✓ VMware vSphere Terraform providers
  62. main.tf

  63. variable "ami" { type = "string" default = "ami-e1e8f085" #Stock

    Debian Stretch } resource "aws_instance" "web" { ami = "${var.ami}" instance_type = "t2.micro" security_groups = ["Web"] tags { Name = "web" } } data "aws_route53_zone" "web" { name = "aws.combell.com." } resource "aws_route53_record" "web" { zone_id = "${data.aws_route53_zone.web.zone_id}" name = "thijsferyn-web.${data.aws_route53_zone.web.name}" type = "A" ttl = "60" records = ["${aws_instance.web.public_ip}"] } output "public_ip" { value = "${aws_instance.web.public_ip}" } output "hostname" { value = "${aws_route53_record.web.name}" }
  64. $ terraform init

  65. $ terraform init Initializing provider plugins... - Checking for available

    provider plugins on https://releases.hashicorp.com... - Downloading plugin for provider "aws" (1.8.0)... The following providers do not have any version constraints in configuration, so the latest version was installed. To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "..." constraints to the corresponding provider blocks in configuration, with the constraint strings suggested below. * provider.aws: version = "~> 1.8" Terraform has been successfully initialized!
  66. $ terraform plan

  67. $ terraform plan Refreshing Terraform state in-memory prior to plan...

    The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage. data.aws_route53_zone.web: Refreshing state... ------------------------------------------------------------------------ An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols:
  68. An execution plan has been generated and is shown below.

    Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: + aws_instance.web id: <computed> ami: "ami-e1e8f085" associate_public_ip_address: <computed> availability_zone: <computed> ebs_block_device.#: <computed> ephemeral_block_device.#: <computed> instance_state: <computed> instance_type: "t2.micro" ipv6_address_count: <computed> ipv6_addresses.#: <computed> key_name: <computed> network_interface.#: <computed> network_interface_id: <computed> placement_group: <computed> primary_network_interface_id: <computed> private_dns: <computed> private_ip: <computed> public_dns: <computed> public_ip: <computed> root_block_device.#: <computed> security_groups.#: "1" security_groups.2661672386: "Web" source_dest_check: "true" subnet_id: <computed> tags.%: "1" tags.Name: "web" tenancy: <computed> volume_tags.%: <computed> vpc_security_group_ids.#: <computed>
  69. + aws_route53_record.web id: <computed> fqdn: <computed> name: "thijsferyn-web.aws.combell.com" records.#: <computed>

    ttl: "60" type: "A" zone_id: "Z3K2HG3W48B0MR" Plan: 2 to add, 0 to change, 0 to destroy.
  70. $ terraform apply

  71. $ terraform plan -out=bla $ terraform apply bla

  72. None
  73. $ terraform output hostname = thijsferyn-web.aws.combell.com public_ip = 35.178.38.182

  74. $ terraform output -json { "hostname": { "sensitive": false, "type":

    "string", "value": "thijsferyn-web.aws.combell.com" }, "public_ip": { "sensitive": false, "type": "string", "value": "35.178.38.182" } }
  75. $ terraform show

  76. $ terraform show aws_instance.web: id = i-0333e97ea6bdbc5ae ami = ami-e1e8f085

    associate_public_ip_address = true availability_zone = eu-west-2a disable_api_termination = false ebs_block_device.# = 0 ebs_optimized = false ephemeral_block_device.# = 0 iam_instance_profile = instance_state = running instance_type = t2.micro ipv6_addresses.# = 0 key_name = monitoring = false network_interface.# = 0 network_interface_id = eni-c62e7193 placement_group = primary_network_interface_id = eni-c62e7193 private_dns = ip-172-31-19-7.eu-west-2.compute.internal
  77. network_interface_id = eni-c62e7193 placement_group = primary_network_interface_id = eni-c62e7193 private_dns =

    ip-172-31-19-7.eu-west-2.compute.internal private_ip = 172.31.19.7 public_dns = ec2-35-178-38-182.eu-west-2.compute.amazonaws.com public_ip = 35.178.38.182 root_block_device.# = 1 root_block_device.0.delete_on_termination = true root_block_device.0.iops = 100 root_block_device.0.volume_id = vol-0b9364527ec09549e root_block_device.0.volume_size = 8 root_block_device.0.volume_type = gp2 security_groups.# = 1 security_groups.763657905 = Web source_dest_check = true subnet_id = subnet-11d3d56a tags.% = 1 tags.Name = web tenancy = default volume_tags.% = 0 vpc_security_group_ids.# = 0
  78. aws_route53_record.web: id = Z3K2HG3W48B0MR_thijsferyn-web.aws.combell.com._A fqdn = thijsferyn-web.aws.combell.com health_check_id = name

    = thijsferyn-web.aws.combell.com records.# = 1 records.1088376871 = 35.178.38.182 set_identifier = ttl = 60 type = A zone_id = Z3K2HG3W48B0MR data.aws_route53_zone.web: id = Z3K2HG3W48B0MR caller_reference = F7596F57-9ADE-9B0D-8F53-53DF42E03DB3 comment = name = aws.combell.com. private_zone = false resource_record_set_count = 3 zone_id = Z3K2HG3W48B0MR Outputs: hostname = thijsferyn-web.aws.combell.com public_ip = 35.178.38.182
  79. State

  80. State backend Terraform Cloud API

  81. terraform.tfstate terraform.tfstate.backup

  82. JSON data

  83. ✓ Artifactory ✓ Azurerm ✓ Consul ✓ Etcd ✓ Gcs

    ✓ HTTP ✓ Manta ✓ S3 ✓ Swift ✓ Terraform Enterprise Remote state
  84. Changes

  85. $ terraform plan -var 'ami=ami-20ccd644'

  86. ~ update in-place -/+ destroy and then create replacement Terraform

    will perform the following actions: -/+ aws_instance.web (new resource required) id: "i-0333e97ea6bdbc5ae" => <computed> (forces new resource) ami: "ami-e1e8f085" => "ami-20ccd644" (forces new resource) associate_public_ip_address: "true" => <computed> availability_zone: "eu-west-2a" => <computed> ebs_block_device.#: "0" => <computed> ephemeral_block_device.#: "0" => <computed> instance_state: "running" => <computed> instance_type: "t2.micro" => "t2.micro" ipv6_address_count: "" => <computed> ipv6_addresses.#: "0" => <computed> key_name: "" => <computed> network_interface.#: "0" => <computed> network_interface_id: "eni-c62e7193" => <computed> placement_group: "" => <computed> primary_network_interface_id: "eni-c62e7193" => <computed> private_dns: "ip-172-31-19-7.eu-west-2.compute.internal" => <computed> private_ip: "172.31.19.7" => <computed>
  87. key_name: "" => <computed> network_interface.#: "0" => <computed> network_interface_id: "eni-c62e7193"

    => <computed> placement_group: "" => <computed> primary_network_interface_id: "eni-c62e7193" => <computed> private_dns: "ip-172-31-19-7.eu-west-2.compute.internal" => <computed> private_ip: "172.31.19.7" => <computed> public_dns: "ec2-35-178-38-182.eu- west-2.compute.amazonaws.com" => <computed> public_ip: "35.178.38.182" => <computed> root_block_device.#: "1" => <computed> security_groups.#: "1" => "1" security_groups.763657905: "Web" => "Web" source_dest_check: "true" => "true" subnet_id: "subnet-11d3d56a" => <computed> tags.%: "1" => "1" tags.Name: "web" => "web" tenancy: "default" => <computed> volume_tags.%: "0" => <computed> vpc_security_group_ids.#: "0" => <computed> ~ aws_route53_record.web records.#: "" => <computed> Plan: 1 to add, 1 to change, 1 to destroy.
  88. $ terraform apply -var 'ami=ami-20ccd644'

  89. Downtime

  90. High availability plan

  91. Workspaces

  92. One Terraform project Multiple workspaces State per workspace Multiple environments

  93. Blue Green

  94. Workspace 1 Workspace 2 Internet Entry point BLUE GREEN

  95. $ terraform init $ terraform workspace new wsp1 $ terraform

    workspace new wsp2 $ terraform workspace select wsp1 $ terraform plan $ terraform apply $ terraform workspace select wsp2 $ terraform plan $ terraform apply
  96. . !"" main.tf !"" terraform.tfstate !"" terraform.tfstate.backup $"" terraform.tfstate.d !""

    wsp1 $"" wsp2
  97. variable "ami" { type = "string" default = "ami-e1e8f085" #Stock

    Debian Stretch } resource "aws_instance" "web" { ami = "${var.ami}" instance_type = "t2.micro" security_groups = ["Web"] tags { Name = "web-${terraform.workspace}" } } output "public_ip" { value = "${aws_instance.web.public_ip}" } Remove DNS resource Interpolate workspace info
  98. variable "ip" { type = "string" } data "aws_route53_zone" "web"

    { name = "aws.combell.com." } resource "aws_route53_record" "web" { zone_id = "${data.aws_route53_zone.web.zone_id}" name = "thijsferyn-web.${data.aws_route53_zone.web.name}" type = "A" ttl = "60" records = ["${var.ip}"] } output "hostname" { value = "${aws_route53_record.web.name}" } Separate Terraform project
  99. $ terraform apply -var 'ip="35.176.171.168"'

  100. DNS round robin

  101. variable "ami" { default = "ami-e1e8f085" #Stock Debian Stretch }

    variable "instances" { default = "2" } resource "aws_instance" "web" { ami = "${var.ami}" instance_type = "t2.micro" security_groups = ["Web"] count = "${var.instances}" tags { Name = "web-${format("%02d", count.index+1)}-${terraform.workspace}" } } output "public_ip" { value = ["${aws_instance.web.*.public_ip}"] } Create multiple instances at once
  102. variable "ip" { type = "list" } data "aws_route53_zone" "web"

    { name = "aws.combell.com." } resource "aws_route53_record" "web" { zone_id = "${data.aws_route53_zone.web.zone_id}" name = "thijsferyn-web.${data.aws_route53_zone.web.name}" type = "A" ttl = "60" records = ["${var.ip}"] } output "hostname" { value = "${aws_route53_record.web.name}" }
  103. $ terraform apply -var 'ip=["35.176.171.168","35.177.208.38"]'

  104. Rolling updates

  105. Deployment B Internet Entry point Drain connections and gradually switch

    from deployment A to deployment B Deployment A ROLLING UPDATES
  106. Autoscaling group

  107. Custom AMI V P C ALB Private subnet 1 Private

    subnet 2 Public subnet 1 Public subnet 2 Internet gateway Route table Target group Internet Launch config Autoscaling group Metrics Security group
  108. resource "aws_vpc" "thijsferyn_terraform" { cidr_block = "10.0.0.0/16" tags { Name

    = "thijsferyn_${terraform.workspace}_terraform" } } resource "aws_subnet" "thijsferyn_terraform_public" { availability_zone = "eu-west-2a" vpc_id = "${aws_vpc.thijsferyn_terraform.id}" cidr_block = "10.0.0.0/24" tags { Name = "thijsferyn_terraform_${terraform.workspace}_public" } } resource "aws_subnet" "thijsferyn_terraform_private" { availability_zone = "eu-west-2a" vpc_id = "${aws_vpc.thijsferyn_terraform.id}" cidr_block = "10.0.1.0/24" tags { Name = "thijsferyn_terraform_${terraform.workspace}_private" } } Virtual network
  109. resource "aws_security_group" "thijsferyn_terraform" { name = "thijsferyn-terraform-${terraform.workspace}" description = "Allow

    HTTP(S) & SSH" vpc_id = "${aws_vpc.thijsferyn_terraform.id}" ingress { from_port = 80 to_port = 80 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] } ingress { from_port = 443 to_port = 443 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] } ingress { from_port = 22 to_port = 22 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] } egress { from_port = 0 to_port = 0 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] } tags { Name = "thijsferyn_terraform-${terraform.workspace}" } } Firewall
  110. data "aws_ami" "thijsferyn_terraform" { most_recent = true filter { name

    = "name" values = ["thijsferyn_*"] } filter { name = "virtualization-type" values = ["hvm"] } owners = ["826481595599"] } resource "aws_launch_configuration" "thijsferyn_terraform" { image_id = "${data.aws_ami.thijsferyn_terraform.id}" instance_type = "t2.micro" security_groups = ["${aws_security_group.thijsferyn_terraform.id}"] lifecycle { create_before_destroy = true } } Prepare VMs
  111. resource "aws_lb" "thijsferyn_terraform" { name = "thijsferyn-terraform-${terraform.workspace}" internal = false

    security_groups = ["${aws_security_group.thijsferyn_terraform.id}"] subnets = ["${aws_subnet.thijsferyn_terraform_public.*.id}","$ {aws_subnet.thijsferyn_terraform_public2.*.id}"] } resource "aws_lb_listener" "thijsferyn_terraform" { load_balancer_arn = "${aws_lb.thijsferyn_terraform.arn}" port = "80" protocol = "HTTP" default_action { target_group_arn = "${aws_lb_target_group.thijsferyn_terraform.arn}" type = "forward" } } resource "aws_lb_target_group" "thijsferyn_terraform" { name = "thijsferyn-terraform-${terraform.workspace}" port = 80 protocol = "HTTP" vpc_id = "${aws_vpc.thijsferyn_terraform.id}" } Load balancing
  112. resource "aws_autoscaling_group" "thijsferyn_terraform" { availability_zones = ["eu-west-2a"] name = "thijsferyn-terraform-${terraform.workspace}-$

    {aws_launch_configuration.thijsferyn_terraform.name}" max_size = 10 min_size = 3 launch_configuration = "${aws_launch_configuration.thijsferyn_terraform.name}" target_group_arns = ["${aws_lb_target_group.thijsferyn_terraform.id}"] health_check_grace_period = 10 health_check_type = "ELB" vpc_zone_identifier = ["${aws_subnet.thijsferyn_terraform_private.id}"] default_cooldown = 60 depends_on = ["aws_launch_configuration.thijsferyn_terraform"] lifecycle { create_before_destroy = true } tag { key = "Name" value = "thijsferyn-terraform-${terraform.workspace}-$ {aws_launch_configuration.thijsferyn_terraform.name}" propagate_at_launch = true } } Autoscaling group
  113. resource "aws_autoscaling_policy" "thijsferyn_terraform_high" { name = "thijsferyn-terraform-${terraform.workspace}-high" scaling_adjustment = 1

    adjustment_type = "ChangeInCapacity" cooldown = 10 autoscaling_group_name = "${aws_autoscaling_group.thijsferyn_terraform.name}" } resource "aws_autoscaling_policy" "thijsferyn_terraform_low" { name = "thijsferyn-terraform-${terraform.workspace}low" scaling_adjustment = -1 adjustment_type = "ChangeInCapacity" cooldown = 10 autoscaling_group_name = "${aws_autoscaling_group.thijsferyn_terraform.name}" } Scale up scale down
  114. resource "aws_cloudwatch_metric_alarm" "thijsferyn_terraform_high" { alarm_name = "thijsferyn-terraform-${terraform.workspace}-high" comparison_operator = "GreaterThanOrEqualToThreshold"

    evaluation_periods = "1" metric_name = "RequestCountPerTarget" namespace = "AWS/ApplicationELB" period = "60" statistic = "Sum" threshold = "20" dimensions { TargetGroup = "${aws_lb_target_group.thijsferyn_terraform.name}" LoadBalancer = "${aws_lb.thijsferyn_terraform.name}" } alarm_description = "More than 20 requests per target causes scaleout" alarm_actions = ["${aws_autoscaling_policy.thijsferyn_terraform_high.arn}"] } High water mark
  115. $ terraform plan $ terraform apply

  116. $ terraform graph | dot -Tpng > graph.png

  117. None
  118. None
  119. ✓ vars.tf ✓ output.tf ✓ secgroup.tf ✓ keypair.tf ✓ compute.tf

    ✓ network.tf ✓ provider.tf Organize Terraform files
  120. Modules

  121. !"" autoscaling # !"" README.MD # !"" main.tf # !""

    outputs.tf # $"" variables.tf !"" dns # !"" README.MD # !"" main.tf # !"" outputs.tf # $"" variables.tf !"" launch # !"" README.MD # !"" main.tf # !"" outputs.tf # $"" variables.tf !"" loadbalancing # !"" README.MD # !"" main.tf # !"" outputs.tf # $"" variables.tf !"" main.tf !"" networking # !"" README.MD # !"" main.tf # !"" outputs.tf # $"" variables.tf
  122. variable "zone_name" { default = "aws.combell.com." } variable "record_name" {

    default = "" } variable "records" { type = "list" } output "dns_endpoint" { value = "${aws_route53_record.thijsferyn_terraform_aws_combell_com.fqdn}" } data "aws_route53_zone" "thijsferyn_terraform_aws_combell_com" { name = "${var.zone_name}" } resource "aws_route53_record" "thijsferyn_terraform_aws_combell_com" { zone_id = "${data.aws_route53_zone.thijsferyn_terraform_aws_combell_com.zone_id}" name = "${var.record_name == "" ? "thijsferyn-terraform${terraform.workspace == "default" ? "" : "-${terraform.workspace}"}": var.record_name}.$ {data.aws_route53_zone.thijsferyn_terraform_aws_combell_com.name}" type = "CNAME" ttl = "60" records = ["${var.records}"] } Variables.tf module input args Output.tf module output Main.tf processing
  123. module "dns" { source = "./dns" record_name = "www" records

    = ["bla.domain.com"] } output "dns_endpoint" { value = "${module.dns.dns_endpoint}" } Root Terraform file
  124. module "networking" { source = "./networking" } module "loadbalancing" {

    source = "./loadbalancing" security_groups = ["${module.networking.security_group_id}"] vpc_id = "${module.networking.vpc_id}" subnets = ["${module.networking.subnet_public1_id}","${module.networking.subnet_public2_id}"] } module "launch" { source = "./launch" instance_type = "t2.small" security_groups = ["${module.networking.security_group_id}"] } module "autoscaling" { source = "./autoscaling" target_group = "${module.loadbalancing.target_group_name}" loadbalancer = "${module.loadbalancing.loadbalancer_name}" min_size = "4" subnets = ["${module.networking.subnet_private1_id}","${module.networking.subnet_private2_id}"] launch_configuration = "${module.launch.launch_configuration_name}" target_group_arn = "${module.loadbalancing.target_group_id}" } module "dns" { source = "./dns" records = ["${module.loadbalancing.dns_name}"] } output "dns_endpoint" { value = "${module.dns.dns_endpoint}" }
  125. https://registry.terraform.io

  126. None
  127. module "consul" { source = "hashicorp/consul/aws" }

  128. $ terraform init

  129. Downloading modules... Initializing provider plugins... - Checking for available provider

    plugins on https://releases.hashicorp.com... - Downloading plugin for provider "aws" (1.9.0)... - Downloading plugin for provider "template" (1.0.0)... The following providers do not have any version constraints in configuration, so the latest version was installed. To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "..." constraints to the corresponding provider blocks in configuration, with the constraint strings suggested below. * provider.aws: version = "~> 1.9" * provider.template: version = "~> 1.0" Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.
  130. module “example" { source = "github.com/hashicorp/example" } module "consul" {

    source = "git::https://hashicorp.com/consul.git" } module "ami" { source = "git::ssh://git@github.com/owner/repo.git" }
  131. ✓Local files ✓Terraform Registry ✓GitHub ✓Bitbucket ✓Generic Git Repo ✓Mercurial

    ✓HTTP ✓S3 buckets
  132. Remote state

  133. None
  134. $ consul agent -data-dir=/tmp/consul \ -server -ui -dev

  135. terraform { backend "consul" { address = "127.0.0.1:8500" path =

    "example/terraform_state" } }
  136. $ terraform apply Error locking state: Error acquiring the state

    lock: resource temporarily unavailable Lock Info: ID: d2c36deb-32c0-ecc7-b1f1-10068c2ed93b Path: terraform.tfstate Operation: OperationTypeApply Who: thijsferyn@MacBook-Pro-van-Thijs.local Version: 0.10.7 Created: 2018-02-02 16:03:17.005327423 +0000 UTC Info: Terraform acquires a state lock to protect the state from being written by multiple users at the same time. Please resolve the issue above and try again. For most commands, you can disable locking with the "-lock=false" flag, but this is not recommended.
  137. data "terraform_remote_state" "web" { backend = "consul" config { address

    = "127.0.0.1:8500" path = "thijsferyn/terraform/production" } } data "aws_route53_zone" "web" { name = "aws.combell.com." } resource "aws_route53_record" "web" { zone_id = "${data.aws_route53_zone.web.zone_id}" name = "thijsferyn-web.${data.aws_route53_zone.web.name}" type = "A" ttl = "60" records = ["${data.terraform_remote_state.web.public_ip}"] } output "hostname" { value = "${aws_route53_record.web.name}" }
  138. Instead of using CLI variables, we can use remote state

    to communicate between Terraform projects
  139. Hybrid Cloud?

  140. https://feryn.eu https://twitter.com/ThijsFeryn https://instagram.com/ThijsFeryn

  141. https://joind.in/talk/ae063

  142. None