Applying TDD to Infrastructure Code

Talk given at engineers4engineers, Waltham, MA

Transcript

2. Test-driven Infrastructure with Chef September 12th, 2014 ! Stephen Nelson-Smith stephen@atalanta-systems.com @LordCope
3. Is your production code tested?
4. Why do you bother?
5. Why do you bother? Catch regressions? Meet requirements? Build confidence? Maintain quality? Maintain stability?
6. Infrastructure as Code
7. Is your infrastructure code tested?
8. Erm…. about that…
9. Why not?
10. A TDD Primer
11. Software Engineering is Learning
12. Software Engineering != Civil Engineering
13. Unexpected changes are guaranteed
14. Effective learning requires empirical feedback
15. Deploy Often
16. Demonstrate Regularly
17. Test Constantly
18. Optimize for reading
19. Keep feedback loops short
20. Test…. …. FIRST
21. Red - Green - Refactor
22. Never write new functionality without a failing test
24. Clarify acceptance criteria
25. Encourages Loose Coupling
26. Provides Executable Documentation
27. Grows regression suite
28. Detect errors when context fresh
29. Avoid Gold Plating
30. Design & Implementation •Does our code work? •Is our code well-structured?
31. Steps to write a unit test •Ensure the object can operate in isolation •Create the object •Provide any dependencies •Interact with the object •Verify the object behaved as expiated
32. Listening to the tests If the test is hard to write: The design is probably wrong.
33. “We’ve seen projects with high-quality, well unit-tested code that turned out not to be called from anywhere, or that could not be integrated with the rest of the system and had to be rewritten.” ! Steve Freeman & Nat Pryce, Growing Object-Oriented Software, Guided by Tests ! Doing the wrong thing right
34. Gojko Adžić’s Useless Crap Diagram Build the right thing Business Failure Useless Crap Success Maintenance Nightmare Build the thing right
35. Gojko Adžić’s Useless Crap Diagram Build the right thing Business Failure Useless Crap Success Maintenance Nightmare Build the thing right
36. Antidote: ! Write Acceptance Tests First
37. BDD Outside-In Cycle
38. Test-driven Infrastructure
39. Test-driven Infrastructure
40. Code: AUTHD 50% off print books 40% off eBooks
41. Brian Marick’s Testing Quadrant Business-facing Technology-facing Support development Critique Project Acceptance Tests Unit Tests Integration Tests Usability Tests Exploratory Tests Load Tests Penetration Tests
42. Brian Marick’s Testing Quadrant Business-facing Technology-facing Support development Critique Project Acceptance Tests Unit Tests Integration Tests Usability Tests Exploratory Tests Load Tests Penetration Tests
43. Nelson-Smith’s TDI Workflow
44. Testing Tools • ChefSpec • Test-Kitchen • ServerSpec • Cucumber • Leibniz
45. Testing Tools • ChefSpec • Test-Kitchen • ServerSpec • Cucumber • Leibniz
46. Internal and External Quality
47. Internal and External Quality
48. Time/Cost vs Value Assessment
49. Time/Cost vs Value Assessment
51. • Pluggable • Framework • Harness • Interface
52. Converge Create Verify Setup Destroy } Test
53. Write & Run Tests Rspec? MiniTest? BATS? nose? cucumber? shunit2? serverspec? BBC BASIC?
54. +
55. + How shall we install these tools?
56. + How shall we install these tools?
57. Problem: ! Write a cookbook, test-first, that builds a test-kitchen environment suitable for TDD…
58. Thankfully…. kitchen@tk00:~$ chef --version! Chef Development Kit Version: 0.2.1! kitchen@tk00:~$ kitchen --version! Test Kitchen version 1.2.1! kitchen@tk00:~$ VBoxManage --version! 4.3.16r95972! kitchen@tk00:~$ vagrant --version! Vagrant 1.6.5
59. $ chef --help! Usage:! chef -h/--help! chef -v/--version! chef command [arguments...] [options...]! ! ! Available Commands:! exec Runs the command in context of the embedded ruby! gem Runs the `gem` command in context of the embedded ruby! generate Generate a new app, cookbook, or component! shell-init Initialize your shell to use ChefDK as your primary ruby! verify Test the embedded ChefDK applications Chef CLI
60. What’s in the box? • Foodcritic • Test-Kitchen (KitchenCI) • Chefspec • Berkshelf • Chef CLI • Knife • Chef Client
61. $ chef --help! Usage:! chef -h/--help! chef -v/--version! chef command [arguments...] [options...]! ! ! Available Commands:! exec Runs the command in context of the embedded ruby! gem Runs the `gem` command in context of the embedded ruby! generate Generate a new app, cookbook, or component! shell-init Initialize your shell to use ChefDK as your primary ruby! verify Test the embedded ChefDK applications Chef CLI
62. Create a wrapper cookbook
63. Create a wrapper cookbook
64. Update .kitchen.yml
65. Run kitchen test •Obtains Vagrant boxes •Creates instances (ubuntu & centos) •Converges node (no runlist) •Skips setup (no tests) •Skips verify (no tests) •Destroys instances •Exits zero
66. Run kitchen test
67. Test Layout
68. Create a spec_helper.rb
69. Write our first test
70. Create our test instances •Run kitchen create •Creates instances (ubuntu & centos) •Can keep them around for a while (saves time) •Beware long-lived instances
71. Run our test •Run kitchen verify •Installs Chef •Runs kitchen setup (installs testing tools) •Copies tests to instance •Runs tests •Reports results
72. Watch the test fail
73. Write the code to make it pass
74. Reconverge the ubuntu node •Run kitchen converge ubuntu •Runs Chef to converge the node
75. Run the test again •Run kitchen verify ubuntu
76. Test the CentOS node •Run kitchen verify centos
77. Ensure Ruby is in PATH
78. Upload to Chef Server (Or use chef-zero, and berks vendor)
79. Bootstrap a machine knife digital_ocean droplet create \! --server-name tk \! --image 1234 --size 62 --location 4 \ -- ssh-keys 4321 --ssh-port 22 \ ! --bootstrap \ ! --identity-file ~/.ssh/lordcope
80. Log onto machine and look!
81. Log onto machine and look!
82. Never write new functionality without a failing test
83. Write a failing test
84. Test CentOS
85. Test Ubuntu
86. Fix the problem (this is a known gotcha on Ubuntu vs CentOS)
87. Converge the node & run the tests…
88. Observe idempotence on CentOS
89. Tests now pass…
90. Bump version, release, deploy
91. Never write new functionality without a failing test
92. Add another test… and watch it fail…
93. Write the code to make it pass… (see the power of Chef / Berkshelf) In recipes/default.rb: In metadata.rb:
94. We want to use a newer vagrant cookbook… In Berksfile:
95. Converge the node… •Berkshelf recursively solves dependencies •Dependencies made available to Chef
96. Tests now pass…
97. Bump the version, release, deploy… See http://chef-community.github.io/cvp/ for semver/chef
98. We need ChefDK and Git
99. Watch the tests fail…
100. Adding Git is simple
101. ChefDK needs some work
102. Relies on cookbook attributes In attributes/default.rb:
103. Converge the node…
104. Tests now pass…
105. SUCCESS!!
106. “Code without tests is bad code. It doesn't matter how well written it is; it doesn't matter how pretty or object-oriented or well-encapsulated it is. With tests, we can change the behavior of our code quickly and verifiably. Without them, we really don't know if our code is getting better or worse.”! ― Michael C. Feathers, Working Effectively with Legacy Code" Conclusion
108. ! “Tests that take too long to run end up not being run.”! ― Michael C. Feathers, Working Effectively with Legacy Code Speeding up tests
111. Thank you… ! @LordCope ! stephen@atalanta-systems.com