DevOpsDays Cuba 2017: Using TDD For Infrastructure as Code. Confidence when moving to the cloud

Author: Alina Murphy & Pat Dale
Summary: We believe that in today’s fast moving world, it’s more important than ever that infrastructure and software changes are repeatable and can be done with confidence. To support a scalable platform for applications, teams must be able to make changes quickly and reliably. This isn’t limited to applications, but potentially even more important for infrastructure changes that historically have had a less formal testing process. To ensure this, teams must both keep their code and infrastructure definitions in source control, use a pipeline for deployment, and have tests to validate their changes. We will walk participants through the basics of version control for infrastructure changes, using a variety of options, and show how these changes can be pushed through a pipeline that includes testing.

Transcript

1. 1
   Using TDD For Infrastructure
   as Code
   Confidence when moving to the cloud
   Alina Murphy & Pat Dale
   

   View Slide

2. 2
   Who are we?!
   

   View Slide

3. WHO ARE WE?!
   3
   Pat Dale
   Consultant at ThoughtWorks
   Pairs well with whiskey - Likes
   climbing rocks - San Franciscan
   Alina Murphy
   Consultant at ThoughtWorks
   Sweets monster - General bear
   expert - New Yorker
   

   View Slide

4. WHAT BROUGHT US
   HERE?
   A client case study…
   4
   

   View Slide

5. 5
   Tribal knowledge
   

   View Slide

6. 6
   Deconstructed
   pipelines
   

   View Slide

7. 7
   No tests :(
   

   View Slide

8. HOW DID WE HELP
   SOLVE THIS?
   A test driven approach to infrastructure as code
   8
   

   View Slide

9. A TEST DRIVEN APPROACH TO INFRA AS CODE
   What do we mean by infrastructure as code?
   9
   To sum it up in two words: “Automate everything”
   We want to treat our computing, networking, and
   infrastructure code the same as any other piece of
   software. This will allow us to more easily test, build,
   and deploy changes at a faster and more
   incremental pace as well as keep our existing
   services readily available, even during deployments
   or upgrades.
   

   View Slide

10. TDD AND INFRASTRUCTURE AS CODE
    What is Test Driven Development? (TDD)
    10
    The mantra goes, “Red, Green, Refactor”
    TDD helps us write production code
    incrementally which in turn reduces the
    complexity of our systems. Our tests also help
    describe the intent of our production code — you
    can think of them as living and breathing
    documentation. And last but not least, it ensures
    all of our production code is sufficiently tested.
    

    View Slide

11. WHAT DOES THIS
    LOOK LIKE IN
    PRACTICE?
    An example of testing driven infrastructure as code
    11
    

    View Slide

12. AN EXAMPLE OF TESTING INFRA AS CODE
    Securing a build pipeline
    12
    Story:
    As a developer
    I want to ensure that my build server can only be accessed
    internally
    So that my company continues to practice proper security
    standards
    

    View Slide

13. AN EXAMPLE OF TESTING INFRA AS CODE
    Securing a build pipeline
    13
    Our source code repository is modeled in a
    similar way to any application source code repo:
    we have a production code folder,
    terraform/concourse, and a test code folder,
    tests/concourse.
    Our tests use a tool called aws_spec which
    dictates the structure of our test folder
    

    View Slide

14. AN EXAMPLE OF TESTING INFRA AS CODE
    Securing a build pipeline
    14
    Our main.tf file currently provisions a
    Concourse CI (pipeline) instance as well as a
    Postgres database to store our build history.
    As we can see, we currently do not force our
    pipeline to be a part of any security group so we
    could be open to vulnerabilities.
    

    View Slide

15. AN EXAMPLE OF TESTING INFRA AS CODE
    Securing a build pipeline
    15
    Luckily we have teammates who have introduced
    a testing harness already. W00t!
    Our aws_spec.rb file describes how any piece
    of infrastructure that we provision will behave.
    Let’s begin our story following the TDD pattern of
    Red, Green, Refactor.
    

    View Slide

16. 16
    STEP 1 Write a failing test
    Let’s write a failing test to introduce a specific
    security group to our build pipeline server.
    The aws_spec testing framework gives us a nice
    interface to describe our infrastructure, and easily
    add tests to new or existing pieces of
    infrastructure.
    Our new test dictates that our build pipeline
    server should belong to a specific security group.
    

    View Slide

17. 17
    STEP 2 Make the test pass
    We now need to provision our build pipeline
    server with a specific security group in order for
    our test to pass. Easy enough…
    

    View Slide

18. 18
    STEP 3 Refactor
    Our teammate who we are working with points
    out that we can improve our code by making it
    more modular.
    

    View Slide

19. HOW DOES THIS HELP
    US ACHIEVE OUR
    ORIGINAL GOALS?
    19
    

    View Slide

20. 20
    Tests dictate the intent
    of our infrastructure,
    and will protect this
    intent when modifying
    our IaC in the future
    

    View Slide

21. 21
    No more snowflake
    environments!
    

    View Slide

22. 22
    IaC reduces tribal
    knowledge and enables
    wholistic ownership
    over app environments
    

    View Slide

23. 23
    THANK YOU!
    Questions?
    Alina Murphy
    amurphy@thoughtworks.com
    Pat Dale
    pdale@thoughtworks.com
    

    View Slide