DevOpsDays Cuba 2017: Using TDD For Infrastructure as Code. Confidence when moving to the cloud

Author: Alina Murphy & Pat Dale
Summary: We believe that in today’s fast moving world, it’s more important than ever that infrastructure and software changes are repeatable and can be done with confidence. To support a scalable platform for applications, teams must be able to make changes quickly and reliably. This isn’t limited to applications, but potentially even more important for infrastructure changes that historically have had a less formal testing process. To ensure this, teams must both keep their code and infrastructure definitions in source control, use a pipeline for deployment, and have tests to validate their changes. We will walk participants through the basics of version control for infrastructure changes, using a variety of options, and show how these changes can be pushed through a pipeline that includes testing.

Transcript

1. 1 Using TDD For Infrastructure as Code Confidence when moving to the cloud Alina Murphy & Pat Dale
2. 2 Who are we?!
3. WHO ARE WE?! 3 Pat Dale Consultant at ThoughtWorks Pairs well with whiskey - Likes climbing rocks - San Franciscan Alina Murphy Consultant at ThoughtWorks Sweets monster - General bear expert - New Yorker
4. WHAT BROUGHT US HERE? A client case study… 4
5. 5 Tribal knowledge
6. 6 Deconstructed pipelines
7. 7 No tests :(
8. HOW DID WE HELP SOLVE THIS? A test driven approach to infrastructure as code 8
9. A TEST DRIVEN APPROACH TO INFRA AS CODE What do we mean by infrastructure as code? 9 To sum it up in two words: “Automate everything” We want to treat our computing, networking, and infrastructure code the same as any other piece of software. This will allow us to more easily test, build, and deploy changes at a faster and more incremental pace as well as keep our existing services readily available, even during deployments or upgrades.
10. TDD AND INFRASTRUCTURE AS CODE What is Test Driven Development? (TDD) 10 The mantra goes, “Red, Green, Refactor” TDD helps us write production code incrementally which in turn reduces the complexity of our systems. Our tests also help describe the intent of our production code — you can think of them as living and breathing documentation. And last but not least, it ensures all of our production code is sufficiently tested.
11. WHAT DOES THIS LOOK LIKE IN PRACTICE? An example of testing driven infrastructure as code 11
12. AN EXAMPLE OF TESTING INFRA AS CODE Securing a build pipeline 12 Story: As a developer I want to ensure that my build server can only be accessed internally So that my company continues to practice proper security standards
13. AN EXAMPLE OF TESTING INFRA AS CODE Securing a build pipeline 13 Our source code repository is modeled in a similar way to any application source code repo: we have a production code folder, terraform/concourse, and a test code folder, tests/concourse. Our tests use a tool called aws_spec which dictates the structure of our test folder
14. AN EXAMPLE OF TESTING INFRA AS CODE Securing a build pipeline 14 Our main.tf file currently provisions a Concourse CI (pipeline) instance as well as a Postgres database to store our build history. As we can see, we currently do not force our pipeline to be a part of any security group so we could be open to vulnerabilities.
15. AN EXAMPLE OF TESTING INFRA AS CODE Securing a build pipeline 15 Luckily we have teammates who have introduced a testing harness already. W00t! Our aws_spec.rb file describes how any piece of infrastructure that we provision will behave. Let’s begin our story following the TDD pattern of Red, Green, Refactor.
16. 16 STEP 1 Write a failing test Let’s write a failing test to introduce a specific security group to our build pipeline server. The aws_spec testing framework gives us a nice interface to describe our infrastructure, and easily add tests to new or existing pieces of infrastructure. Our new test dictates that our build pipeline server should belong to a specific security group.
17. 17 STEP 2 Make the test pass We now need to provision our build pipeline server with a specific security group in order for our test to pass. Easy enough…
18. 18 STEP 3 Refactor Our teammate who we are working with points out that we can improve our code by making it more modular.
19. HOW DOES THIS HELP US ACHIEVE OUR ORIGINAL GOALS? 19
20. 20 Tests dictate the intent of our infrastructure, and will protect this intent when modifying our IaC in the future
21. 21 No more snowflake environments!
22. 22 IaC reduces tribal knowledge and enables wholistic ownership over app environments
23. 23 THANK YOU! Questions? Alina Murphy amurphy@thoughtworks.com Pat Dale pdale@thoughtworks.com