LET'S GET SERIOUS • Testing comes from software/developers (first mention in 1979) • Many levels : Unit, Integration, End-to-end, ... • Many approaches : Smoke, Regression, Functional, ...
WHY DOES IT MATTER ? • We moved from "racking stuff" to "software engineers" (even of we still do rack servers) • We write code to do our job • We also write bugs, and they can have huge effects
WHAT CAN WE TEST ? • Fortunately, many things • Unfortunately, way less than we would like to • But this area is quite young in our particular field of application
A BIT OF FEEDBACK ON CHEF • We focus our efforts on integration tests • All PRs are tested, using CircleCI • Our AWS base images are automatically built upon this code • We have to support multiple platforms (focus is done on Amazon Linux 2 and Ubuntu)
A BIT OF FEEDBACK ON CHEF • Allows us to iterate quite fast • We know that small changes won't blow up the whole stack • Avoid "refucktoring" and enables us to refactor code with confidence
INFRASTRUCTURE AS CODE • We will focus on terraform here. YMMV • How to know that the things we wrote to provision via code actually did ? Note : all things here are not real-production feedback, but experiments I've made (in an actual sandbox)
INFRASTRUCTURE AS CODE • Kitchen-terraform (yes, I like kitchen) • Quite limited to EC2 testing, but simple to use • Allows to test the "content" of the host, using inspec • Terratest • Many terraform providers supported (weee) • Need Golang knowledge, hence benefit its power
INFRASTRUCTURE AS CODE • Kubernetes users in the room ? • Ever got bitten by a bad YAML ? • Unexpected results in resource creation ? • Multiple approaches here • Simple rendering + yamllint : ~1h work • Or Terratest, again. Heavier investment, bigger outcome
INFRASTRUCTURE AS CODE • All these require a dedicated testing environment • This has a cost, so do outages and bugs • Testbed must be destroyed (use cloud-nuke) • Components need standalone testing, but also as a whole, complex, assembly
CAN IT BE DONE WITHOUT HYPE ? • Of course • Example : patterndb has integrated unit tests (seen at sysadmindays #8 with faxm0dem) • It can be used for single things that are "standalone"