Test-Driven Development (TDD) for Infrastructure

Slide 1

Slide 1 text

Slide 2

Slide 2 text

“TDD is dead.” https://dhh.dk/2014/tdd-is-dead-long-live-testing.html

Slide 3

Slide 3 text

What’s Test- Driven Development (TDD)? ▪ Write tests firsts, then the feature. ▪ “Red, green, refactor” ▪ Benefits: – Only build defined features – Modular, testable code

Slide 4

Slide 4 text

Integration Tests Contract Tests Unit Tests “Ideal” Testing Pyramid Manual Testing Cost (Time, $$$) End-to-End Tests

Slide 5

Slide 5 text

Infrastructure Testing Signpost End-to-End Tests Manual Testing Integration Tests Contract Tests Unit Tests Cost (Time, $$$)

Slide 6

Slide 6 text

Why is it a signpost? ▪ #YOLO-driven development ▪ Lack of education ▪ Lack of automation / interface ▪ Lack of tooling

Slide 7

Slide 7 text

Let’s try TDD for Infrastructure. ▪ Infrastructure configuration still requires good code practices ▪ Infrastructure architecture has definition of “feature complete” ▪ terraform yolo isn’t a thing (or insert deploy command here).

Slide 8

Slide 8 text

Target State

Slide 9

Slide 9 text

Integration Tests Contract Tests Unit Tests TDD Up the Pyramid Manual Testing Cost (Time, $$$) End-to-End Tests

Slide 10

Slide 10 text

Warning, Some Terraform VS Code Live Share at: hashi.co/tdd-velocity

Slide 11

Slide 11 text

⁄ Unit Testing

Slide 12

Slide 12 text

Unit Tests ▪ Test individual component ▪ Can be run without dependencies ▪ Infrastructure: check speciﬁc conﬁguration & syntax

Slide 13

Slide 13 text

Live Code ▪ We’ll use conftest (Open Policy Agent) & terraform validate. ▪ Terraform-speciﬁc tools, compiled by Peter Souter: – clarity – terraform-compliance – terraform_validate ▪ Other infrastructure: native testing frameworks

Slide 14

Slide 14 text

Target State

Slide 15

Slide 15 text

Unit TDD Function logic? Execution? Resource dependencies? Fast feedback Architectural Conformance Lint and check syntax

Slide 16

Slide 16 text

⁄ Contract Testing

Slide 17

Slide 17 text

Contract Tests ▪ Validate interactions between 2 components (input & output) ▪ “Real” resources not required ▪ Compare infrastructure state

Slide 18

Slide 18 text

Live Code ▪ We’ll use conftest (Open Policy Agent) & terraform plan. ▪ Terraform-speciﬁc tools: kitchen-terraform ▪ Can also use native testing frameworks – Example for network switches with Python – Example for S3 Bucket Policy with Golang – If Kubernetes, check Kubernetes to external components (i.e., DNS)

Slide 19

Slide 19 text

Target State

Slide 20

Slide 20 text

Contract TDD Resource dependencies? Interactions? Fast feedback Check functional logic Catch our input errors More holistic

Slide 21

Slide 21 text

⁄ Integration Testing

Slide 22

Slide 22 text

Integration Tests ▪ Conﬁrms interactions between 2+ components ▪ Real resources required ▪ May also include acceptance tests

Slide 23

Slide 23 text

IMPORTANT ▪ Integration tests are about interactions. ▪ When using IaC tool, testing deployment is redundant. – In Terraform, covered by Provider Acceptance Tests ▪ Many types of integration tests, including Functional, Policy, & Security

Slide 24

Slide 24 text

Testing with Kitchen

Slide 25

Slide 25 text

Tools ▪ Terraform-speciﬁc tools: terratest, kitchen-terraform, Sentinel ▪ Other tools: – AWS localstack / mocking framework (USE WITH CAUTION) – Kubernetes local environments (KIND, Minikube, etc.) – Various Inspec resource packs

Slide 26

Slide 26 text

Fails! TERMINAL > kitchen test -----> Starting Kitchen (v2.3.3) … Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds Waiting for SSH service on 54.93.35.169:22, retrying in 3 seconds

Slide 27

Slide 27 text

Passing the Test

Slide 28

Slide 28 text

Pass! TERMINAL > kitchen test -----> Starting Kitchen (v2.3.3) … Profile: Integration Tests for Application (default) ✔ db: Database: check routing from public to private subnet ✔ Host 10.128.0.43 port 27017 proto tcp should be reachable ✔ Host 10.128.0.43 port 27017 proto tcp should be resolvable ✔ Host 10.128.0.43 port 80 proto tcp should not be reachable ✔ outbound: Public Subnet: check routing out to public internet ✔ HTTP GET on https://hashicorp.com status should cmp == 301 Profile Summary: 2 successful controls, 0 control failures, 0 controls skipped Test Summary: 3 successful, 0 failures, 0 skipped

Slide 29

Slide 29 text

Integration TDD Not that fast Requires real resources Convert theory to reality Caught a missing component Isolate sections of system

Slide 30

Slide 30 text

⁄ End-to-End Testing

Slide 31

Slide 31 text

End-to-End Tests ▪ “The Real Deal” ▪ Can be manual, fully automated, or smoke tests ▪ If it starts smoking, it doesn’t work.

Slide 32

Slide 32 text

TDD it! Did it fail?

Slide 33

Slide 33 text

Tools ▪ Terraform-speciﬁc tools: terratest, Sentinel ▪ Other: – Behavior-Driven Development (BDD) Frameworks – Can also use native testing frameworks – Kubernetes: Sonobuoy conformance tests, local environments (KIND, Minikube, etc.)

Slide 34

Slide 34 text

⁄ Final Thoughts

Slide 35

Slide 35 text

Ugh, so much effort. ▪ (Yes, it can be.) ▪ Unit testing is useless here! ▪ I don’t ﬁnd testing tools for this! ▪ Stop with pedantic practices!

Slide 36

Slide 36 text

[Preaching] TDD is dead. So is #YOLO-driven development.

Slide 37

Slide 37 text

Use TDD to Learn Infrastructure ▪ Conﬁdence in infrastructure-as-code ▪ Develop knowledge of infrastructure ▪ More useful integration & end-to-end tests ▪ Tacit knowledge of a change’s blast radius

Slide 38

Slide 38 text

Eventually, YDD & TDD of infrastructure are dead …to you.

Slide 39

Slide 39 text

joatmon08.github.io/#slides Please rate the session! Rosemary Wang she/her Developer Advocate at HashiCorp [email protected] @joatmon08 joatmon08 linkedin.com/in/rosemarywang/ 39