Applying TDD to Infrastructure Code

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Test-driven Infrastructure with Chef September 12th, 2014 ! Stephen Nelson-Smith [email protected] @LordCope

Slide 3

Slide 3 text

Is your production code tested?

Slide 4

Slide 4 text

Why do you bother?

Slide 5

Slide 5 text

Why do you bother? Catch regressions? Meet requirements? Build confidence? Maintain quality? Maintain stability?

Slide 6

Slide 6 text

Infrastructure as Code

Slide 7

Slide 7 text

Is your infrastructure code tested?

Slide 8

Slide 8 text

Erm…. about that…

Slide 9

Slide 9 text

Why not?

Slide 10

Slide 10 text

A TDD Primer

Slide 11

Slide 11 text

Software Engineering is Learning

Slide 12

Slide 12 text

Software Engineering != Civil Engineering

Slide 13

Slide 13 text

Unexpected changes are guaranteed

Slide 14

Slide 14 text

Effective learning requires empirical feedback

Slide 15

Slide 15 text

Deploy Often

Slide 16

Slide 16 text

Demonstrate Regularly

Slide 17

Slide 17 text

Test Constantly

Slide 18

Slide 18 text

Optimize for reading

Slide 19

Slide 19 text

Keep feedback loops short

Slide 20

Slide 20 text

Test…. …. FIRST

Slide 21

Slide 21 text

Red - Green - Refactor

Slide 22

Slide 22 text

Never write new functionality without a failing test

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

Clarify acceptance criteria

Slide 25

Slide 25 text

Encourages Loose Coupling

Slide 26

Slide 26 text

Provides Executable Documentation

Slide 27

Slide 27 text

Grows regression suite

Slide 28

Slide 28 text

Detect errors when context fresh

Slide 29

Slide 29 text

Avoid Gold Plating

Slide 30

Slide 30 text

Design & Implementation •Does our code work? •Is our code well-structured?

Slide 31

Slide 31 text

Steps to write a unit test •Ensure the object can operate in isolation •Create the object •Provide any dependencies •Interact with the object •Verify the object behaved as expiated

Slide 32

Slide 32 text

Listening to the tests If the test is hard to write: The design is probably wrong.

Slide 33

Slide 33 text

“We’ve seen projects with high-quality, well unit-tested code that turned out not to be called from anywhere, or that could not be integrated with the rest of the system and had to be rewritten.” ! Steve Freeman & Nat Pryce, Growing Object-Oriented Software, Guided by Tests ! Doing the wrong thing right

Slide 34

Slide 34 text

Gojko Adžić’s Useless Crap Diagram Build the right thing Business Failure Useless Crap Success Maintenance Nightmare Build the thing right

Slide 35

Slide 35 text

Gojko Adžić’s Useless Crap Diagram Build the right thing Business Failure Useless Crap Success Maintenance Nightmare Build the thing right

Slide 36

Slide 36 text

Antidote: ! Write Acceptance Tests First

Slide 37

Slide 37 text

BDD Outside-In Cycle

Slide 38

Slide 38 text

Test-driven Infrastructure

Slide 39

Slide 39 text

Test-driven Infrastructure

Slide 40

Slide 40 text

Code: AUTHD 50% off print books 40% off eBooks

Slide 41

Slide 41 text

Brian Marick’s Testing Quadrant Business-facing Technology-facing Support development Critique Project Acceptance Tests Unit Tests Integration Tests Usability Tests Exploratory Tests Load Tests Penetration Tests

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Nelson-Smith’s TDI Workﬂow

Slide 44

Slide 44 text

Testing Tools • ChefSpec • Test-Kitchen • ServerSpec • Cucumber • Leibniz

Slide 45

Slide 45 text

Testing Tools • ChefSpec • Test-Kitchen • ServerSpec • Cucumber • Leibniz

Slide 46

Slide 46 text

Internal and External Quality

Slide 47

Slide 47 text

Internal and External Quality

Slide 48

Slide 48 text

Time/Cost vs Value Assessment

Slide 49

Slide 49 text

Time/Cost vs Value Assessment

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

• Pluggable • Framework • Harness • Interface

Slide 52

Slide 52 text

Converge Create Verify Setup Destroy } Test

Slide 53

Slide 53 text

Write & Run Tests Rspec? MiniTest? BATS? nose? cucumber? shunit2? serverspec? BBC BASIC?

Slide 54

Slide 54 text

Slide 55

Slide 55 text

+ How shall we install these tools?

Slide 56

Slide 56 text

+ How shall we install these tools?

Slide 57

Slide 57 text

Problem: ! Write a cookbook, test-first, that builds a test-kitchen environment suitable for TDD…

Slide 58

Slide 58 text

Thankfully…. kitchen@tk00:~$ chef --version! Chef Development Kit Version: 0.2.1! kitchen@tk00:~$ kitchen --version! Test Kitchen version 1.2.1! kitchen@tk00:~$ VBoxManage --version! 4.3.16r95972! kitchen@tk00:~$ vagrant --version! Vagrant 1.6.5

Slide 59

Slide 59 text

$ chef --help! Usage:! chef -h/--help! chef -v/--version! chef command [arguments...] [options...]! ! ! Available Commands:! exec Runs the command in context of the embedded ruby! gem Runs the `gem` command in context of the embedded ruby! generate Generate a new app, cookbook, or component! shell-init Initialize your shell to use ChefDK as your primary ruby! verify Test the embedded ChefDK applications Chef CLI

Slide 60

Slide 60 text

What’s in the box? • Foodcritic • Test-Kitchen (KitchenCI) • Chefspec • Berkshelf • Chef CLI • Knife • Chef Client

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Create a wrapper cookbook

Slide 63

Slide 63 text

Create a wrapper cookbook

Slide 64

Slide 64 text

Update .kitchen.yml

Slide 65

Slide 65 text

Run kitchen test •Obtains Vagrant boxes •Creates instances (ubuntu & centos) •Converges node (no runlist) •Skips setup (no tests) •Skips verify (no tests) •Destroys instances •Exits zero

Slide 66

Slide 66 text

Run kitchen test

Slide 67

Slide 67 text

Test Layout

Slide 68

Slide 68 text

Create a spec_helper.rb

Slide 69

Slide 69 text

Write our ﬁrst test

Slide 70

Slide 70 text

Create our test instances •Run kitchen create •Creates instances (ubuntu & centos) •Can keep them around for a while (saves time) •Beware long-lived instances

Slide 71

Slide 71 text

Run our test •Run kitchen verify •Installs Chef •Runs kitchen setup (installs testing tools) •Copies tests to instance •Runs tests •Reports results

Slide 72

Slide 72 text

Watch the test fail

Slide 73

Slide 73 text

Write the code to make it pass

Slide 74

Slide 74 text

Reconverge the ubuntu node •Run kitchen converge ubuntu •Runs Chef to converge the node

Slide 75

Slide 75 text

Run the test again •Run kitchen verify ubuntu

Slide 76

Slide 76 text

Test the CentOS node •Run kitchen verify centos

Slide 77

Slide 77 text

Ensure Ruby is in PATH

Slide 78

Slide 78 text

Upload to Chef Server (Or use chef-zero, and berks vendor)

Slide 79

Slide 79 text

Bootstrap a machine knife digital_ocean droplet create \! --server-name tk \! --image 1234 --size 62 --location 4 \ -- ssh-keys 4321 --ssh-port 22 \ ! --bootstrap \ ! --identity-file ~/.ssh/lordcope

Slide 80

Slide 80 text

Log onto machine and look!

Slide 81

Slide 81 text

Log onto machine and look!

Slide 82

Slide 82 text

Never write new functionality without a failing test

Slide 83

Slide 83 text

Write a failing test

Slide 84

Slide 84 text

Test CentOS

Slide 85

Slide 85 text

Test Ubuntu

Slide 86

Slide 86 text

Fix the problem (this is a known gotcha on Ubuntu vs CentOS)

Slide 87

Slide 87 text

Converge the node & run the tests…

Slide 88

Slide 88 text

Observe idempotence on CentOS

Slide 89

Slide 89 text

Tests now pass…

Slide 90

Slide 90 text

Bump version, release, deploy

Slide 91

Slide 91 text

Never write new functionality without a failing test

Slide 92

Slide 92 text

Add another test… and watch it fail…

Slide 93

Slide 93 text

Write the code to make it pass… (see the power of Chef / Berkshelf) In recipes/default.rb: In metadata.rb:

Slide 94

Slide 94 text

We want to use a newer vagrant cookbook… In Berksﬁle:

Slide 95

Slide 95 text

Converge the node… •Berkshelf recursively solves dependencies •Dependencies made available to Chef

Slide 96

Slide 96 text

Tests now pass…

Slide 97

Slide 97 text

Bump the version, release, deploy… See http://chef-community.github.io/cvp/ for semver/chef

Slide 98

Slide 98 text

We need ChefDK and Git

Slide 99

Slide 99 text

Watch the tests fail…

Slide 100

Slide 100 text

Adding Git is simple

Slide 101

Slide 101 text

ChefDK needs some work

Slide 102

Slide 102 text

Relies on cookbook attributes In attributes/default.rb:

Slide 103

Slide 103 text

Converge the node…

Slide 104

Slide 104 text

Tests now pass…

Slide 105

Slide 105 text

SUCCESS!!

Slide 106

Slide 106 text

“Code without tests is bad code. It doesn't matter how well written it is; it doesn't matter how pretty or object-oriented or well-encapsulated it is. With tests, we can change the behavior of our code quickly and verifiably. Without them, we really don't know if our code is getting better or worse.”! ― Michael C. Feathers, Working Effectively with Legacy Code" Conclusion

Slide 107

Slide 107 text

No content

Slide 108

Slide 108 text

! “Tests that take too long to run end up not being run.”! ― Michael C. Feathers, Working Effectively with Legacy Code Speeding up tests