Test-Driven Infrastructure 101

Test-Driven Infrastructure 101

Creating better Infrastructure with Terraform, Test Kitchen, AWS & awspec.

374ccc5bcc36b5a07fa84af039e0cb95?s=128

GetCloudnative

August 04, 2018
Tweet

Transcript

  1. 1.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Version: 20180804 Creating Better

    Infrastructure with Terraform, Test Kitchen, AWS & awspec Test-Driven Infrastructure 101
  2. 5.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 5 „Our highest

    priority is to satisfy the customer through early and continuous delivery of valuable software.“ Agile Manifesto Principle No. 1
  3. 6.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 6 The Continuous

    Delivery Deployment Pipeline Overview Developer Version Control Production Environment Code check-in Application release query check-out CI/CD Server (Pipeline Executor) Release Test Deploy Build working and deployable software a build advances along the pipeline in stages (in case of error, the pipeline is stopped)
  4. 8.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 8 „Make it

    work. Make it right. Make it fast.“ Kent Beck, Creator of Extreme Programming
  5. 9.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 9 „Make it

    work. Make it right. Make it fast.“ Kent Beck, Creator of Extreme Programming get the code to operate correctly
  6. 10.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 10 „Make it

    work. Make it right. Make it fast.“ Kent Beck, Creator of Extreme Programming make the code clear and enforce good design
  7. 11.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 11 „Make it

    work. Make it right. Make it fast.“ Kent Beck, Creator of Extreme Programming optimize
  8. 12.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 12 The Red-Green-Refactor

    Cycle of Test-Driven Development Write a Failing Test
  9. 13.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 13 The Red-Green-Refactor

    Cycle of Test-Driven Development Write a Failing Test Make the Test Pass
  10. 14.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 14 The Red-Green-Refactor

    Cycle of Test-Driven Development Clean up Your Code Write a Failing Test Make the Test Pass
  11. 15.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 15 The Red-Green-Refactor

    Cycle of Test-Driven Development Write a Failing Test Make the Test Pass Clean up Your Code
  12. 16.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 16 The Red-Green-Refactor

    Cycle of Test-Driven Development cycle iterates in small increments Write a Failing Test Make the Test Pass Clean up Your Code
  13. 17.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 17 The Red-Green-Refactor

    Cycle of Test-Driven Development code is designed and tested cycle iterates in small increments Write a Failing Test Make the Test Pass Clean up Your Code
  14. 18.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 18 The Red-Green-Refactor

    Cycle of Test-Driven Development code is designed and tested cycle iterates in small increments code is of higher quality Write a Failing Test Make the Test Pass Clean up Your Code
  15. 19.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 19 The Red-Green-Refactor

    Cycle of Test-Driven Development code is designed and tested cycle iterates in small increments code is of higher quality Write a Failing Test Make the Test Pass Clean up Your Code cycle creates regression safety
  16. 20.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 20 You Want

    to Trust but Verify You want to verify that your automation tool does the job right. You Want to Refactor your Code You want to simplify existing code or integrate a third-party module without compromising quality. You Want to Migrate to a Different Platform You want to migrate to a different operating system or cloud provider. You Want to Migrate to a Different Automation Tool You want to migrate from Bash or Perl to Ansible, Chef, Puppet or Terraform. Test-Driven Infrastructure Practical Reasons
  17. 21.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 21 „With TDD,

    you express your intentions twice; once as a test, and once as production code. If the two approaches don’t match, your tests fail, and you’ve caught a bug.“ Kent Beck, Creator of Extreme Programming
  18. 24.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 24 1. Drivers

    Let you run infrastructure code on various cloud providers and virtualization technologies. § Amazon EC2 § Azure RM § DigitalOcean Test Kitchen Basic Concepts § Docker § Google GCE § Microsoft Hyper-V § Rackspace § Vagrant § VMware vSphere
  19. 25.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 25 1. Drivers

    Let you run infrastructure code on various cloud providers and virtualization technologies. § Amazon EC2 § Azure RM § DigitalOcean 2. Platforms Represent a configuration of an operating system. § Linux § Windows Test Kitchen Basic Concepts § Docker § Google GCE § Microsoft Hyper-V § Rackspace § Vagrant § VMware vSphere
  20. 26.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 26 3. Provisioners

    These are the tools needed to converge the enrionment. § Ansible § Chef § Puppet Test Kitchen Basic Concepts § Terraform § SaltStack
  21. 27.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 27 3. Provisioners

    These are the tools needed to converge the enrionment. § Ansible § Chef § Puppet 4. Test Suites Define the tests you want to run against an instance using a verifier. § Bash § Bats § Cucumber Test Kitchen Basic Concepts § Terraform § SaltStack § InSpec § RSpec § Serverspec
  22. 28.

    Slide 28 MARTIN ETMAJER Founder | GetCloudnative e.U. How to

    Test-Driven Infrastructure? 1. Create a Terraform Module $ mkdir –p terraform/modules/my $ cd terraform/modules/my $ touch README.md \ main.tf \ variables.tf \ outputs.tf
  23. 34.

    Slide 34 File 1 2 3 4 5 6 7

    8 9 10 11 MARTIN ETMAJER Founder | GetCloudnative e.U. RSpec Behaviour-Driven Development for Ruby File foo_spec.rb 1 require ‘foo’ 2 3 describe Foo do 4 before Foo do 5 @foo = Foo.new 6 end 7 8 it “method #bar does something useful” do 9 @foo.bar should eq “something useful” 10 end 11 end
  24. 36.

    Slide 36 File 1 12 2 13 3 14 4

    15 5 16 6 17 7 18 8 19 9 20 10 21 11 22 MARTIN ETMAJER Founder | GetCloudnative e.U. Serverspec RSpec Tests for Your Servers File server_spec.rb 1 require “serverspec” 12 describe service(“bar”) do 2 describe user(“foo”) do 13 it { should be_enabled } 3 it { should exist } 14 it { should be_running } 4 it { should belong_to_group “foo” } 15 end 5 end 16 describe port(8080) do 6 describe file(“/opt/bar”) do 17 it { should be_listening } 7 it { should be_directory } 18 end 8 it { should be_mode 777 } 19 describe command(“apachectl –M”) do 9 it { should be_owned_by “foo” } 20 its(:stdout) { should contain(“proxy_module”) } 10 it { should be_grouped_into “foo” } 21 end 11 end
  25. 38.

    Slide 38 File 1 2 3 4 5 6 7

    8 9 10 11 MARTIN ETMAJER Founder | GetCloudnative e.U. awspec RSpec Tests for Your AWS Resources File ec2_spec.rb 1 require “awspec” 2 describe ec2(“my-ec2-tag-name”) do 3 it { should be_running } 4 its(:instance_id) { should eq “i-ec12345a” } 5 its(:image_id) { should eq “ami-abc12def” } 6 its(:public_ip_address) { should eq “123.0.456.789” } 7 it { should have_security_group(“my-security-group-name”) } 8 it { should belong_to_vpc(“my-vpc”) } 9 it { should belong_to_subnet(“subnet-1234a567”) } 10 it { should have_eip(“123.0.456.789”) } 11 end
  26. 39.

    Slide 39 MARTIN ETMAJER Founder | GetCloudnative e.U. How to

    Test-Driven Infrastructure? 2. Install Test Kitchen Dependencies $ gem install \ test-kitchen \ kitchen-terraform \ kitchen-verifier-awspec $ kitchen version Test Kitchen version 1.21.2
  27. 40.

    Slide 40 MARTIN ETMAJER Founder | GetCloudnative e.U. How to

    Test-Driven Infrastructure? 3. Configure Test Kitchen $ kitchen init --driver terraform --provisioner terraform create kitchen.yml create test/integration/default
  28. 41.

    Slide 41 File 1 12 2 13 3 14 4

    15 5 16 6 17 7 18 8 19 9 20 10 21 11 22 MARTIN ETMAJER Founder | GetCloudnative e.U. How to Test-Driven Infrastructure? 3. Configure Test Kitchen File kitchen.yml 1 driver: 8 suites: 2 name: terraform 9 - name: default 10 run_list: 11 attributes: 3 provisioner: 4 name: terraform 5 platforms: 6 - name: ubuntu-16.04 7 - name: centos-7
  29. 42.

    Slide 42 File 1 12 2 13 3 14 4

    15 5 16 6 17 7 18 8 19 9 20 10 21 11 22 MARTIN ETMAJER Founder | GetCloudnative e.U. How to Test-Driven Infrastructure? 3. Configure Test Kitchen with kitchen-terraform and kitchen-verifier-awspec File kitchen.yml 1 driver: 11 suites: 2 name: terraform 12 - name: default 3 variable_files: 13 verifier: 4 - terraform-testing.tfvars.json 14 patterns: 5 provisioner: 15 - test/integration/default/*_spec.rb 6 name: terraform 7 platforms: 8 - name: aws 9 verifier: 10 name: awspec
  30. 43.

    Slide 43 MARTIN ETMAJER Founder | GetCloudnative e.U. How to

    Test-Driven Infrastructure? 3. Configure Test Kitchen $ kitchen list Instance Driver Provisioner Verifier Transport Last Action default-aws Terraform Terraform Awspec Ssh <Not Created>
  31. 44.

    Slide 44 File 1 2 3 4 5 6 7

    8 9 10 11 MARTIN ETMAJER Founder | GetCloudnative e.U. How to Test-Driven Infrastructure? 5. Configure AWS Credentials File ~/.aws/credentials 1 [staging] 2 aws_access_key_id=[...] 3 aws_secret_access_key=[...]
  32. 45.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 45 How to

    Test-Driven Infrastructure? 5. Write a Failing Test Write a Failing Test
  33. 46.

    Slide 46 File 1 12 2 13 3 14 4

    15 5 16 6 17 7 18 8 19 9 20 10 21 11 22 MARTIN ETMAJER Founder | GetCloudnative e.U. How to Test-Driven Infrastructure? 5. Write a Failing Test File kitchen.yml 1 driver: 11 suites: 2 name: terraform 12 - name: default 3 variable_files: 13 verifier: 4 - terraform-testing.tfvars.json 14 patterns: 5 provisioner: 15 - test/integration/default/*_spec.rb 6 name: terraform 7 platforms: 8 - name: aws 9 verifier: 10 name: awspec
  34. 47.

    Slide 47 File 1 2 3 4 5 6 7

    8 9 10 11 MARTIN ETMAJER Founder | GetCloudnative e.U. How to Test-Driven Infrastructure? 5. Write a Failing Test File test/integration/default/default_spec.rb 1 require ‘awspec’ 2 require_relative ‘./spec_helper’ 3 4 describe “My AWS Deployment” do 5 describe ec2(EC2Helper.get_ec2_instance_id_from_tag_name(“my-aws-ec2-instance”)) do 6 it { should exist } 7 it { should be_running } 8 end 9 end
  35. 48.

    Slide 48 MARTIN ETMAJER Founder | GetCloudnative e.U. How to

    Test-Driven Infrastructure? 5. Write a Failing Test $ kitchen verify -----> Starting Kitchen (v1.21.2) [...] $$$$$$ Running command `terraform apply [...]` Apply complete! Resources: 0 added, 0 changed, 0 destroyed. -----> Verifying <default-aws>... Running Awspec Error: 'my-aws-ec2-instance' Instance not found Finished in 0.02451 seconds (files took 3.37 seconds to load) 2 examples, 2 failures
  36. 49.

    Slide 49 MARTIN ETMAJER Founder | GetCloudnative e.U. How to

    Test-Driven Infrastructure? 5. Write a Failing Test $ kitchen list Instance Driver Provisioner Verifier Transport Last Action default-aws Terraform Terraform Awspec Ssh Kitchen:ActionFailed
  37. 50.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 50 How to

    Test-Driven Infrastructure? 6. Make the Test Pass Write a Failing Test Make the Test Pass
  38. 51.

    Slide 51 File 1 2 3 4 5 6 7

    8 9 10 11 MARTIN ETMAJER Founder | GetCloudnative e.U. How to Test-Driven Infrastructure? 6. Make the Test Pass File main.tf 1 provider “aws” { 2 region = “us-east-1” 3 } 4 resource “aws_instance” “compute” { 5 ami = “ami-97785bed” 6 instance_type = “t2.micro” 7 tags { 8 Name = “my-aws-ec2-instance” 9 } 10 }
  39. 52.

    How to Test-Driven Infrastructure? 6. Make the Test Pass $

    kitchen verify -----> Starting Kitchen (v1.21.2) [...] $$$$$$ Running command `terraform apply [...]` Apply complete! Resources: 0 added, 0 changed, 0 destroyed. -----> Verifying <default-aws>... Running Awspec [...] Deployment ec2 'i-02a6a01b0509c55de' should exist should be running Finished in 0.16529 seconds (files took 3.63 seconds to load) 2 examples, 0 failures
  40. 53.

    Slide 53 MARTIN ETMAJER Founder | GetCloudnative e.U. How to

    Test-Driven Infrastructure? 6. Make the Test Pass $ kitchen list Instance Driver Provisioner Verifier Transport Last Action default-aws Terraform Terraform Awspec Ssh Verified
  41. 54.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 54 How to

    Test-Driven Infrastructure? 6. Make the Test Pass Write a Failing Test Make the Test Pass Clean up Your Code
  42. 55.

    MARTIN ETMAJER Founder | GetCloudnative e.U. Slide 55 How to

    Test-Driven Infrastructure? 7. Rinse and Repeat Write a Failing Test Make the Test Pass Clean up Your Code