Even Chris sometimes writes…
Smelly Tests Chris Hartjes
Pacific Northwest PHP
September 12, 2015
Slide 2
Slide 2 text
No content
Slide 3
Slide 3 text
– talk description
“You're now writing unit tests for your code...but you can't help
but wonder if you are writing them the Right Way(tm). Given
that learning to write good tests is no different from learning to
write good code, it's inevitable that we make mistakes as we learn
to create effective unit test suites.”
Slide 4
Slide 4 text
This isn’t about shaming people
Slide 5
Slide 5 text
By writing tests you are doing more than
the vast majority of PHP developers
Slide 6
Slide 6 text
Testing has been around almost as
long as computer science itself
Slide 7
Slide 7 text
It pre-dates the rise of the internet
Slide 8
Slide 8 text
No content
Slide 9
Slide 9 text
It’s never too late to care about
automated tests for your application
Slide 10
Slide 10 text
So you’ve decided to write tests
❖ Is your code ready to test?
❖ Do you understand the testing tools?
❖ Do you understand common testing patterns?
Slide 11
Slide 11 text
No content
Slide 12
Slide 12 text
What Do I Like To See?
❖ Dependency injection in use
❖ Small objects with few methods
❖ Bootstrapping that allows for easy overriding
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
No content
Slide 15
Slide 15 text
Don’t Lose Control
❖ Dependencies you cannot control means tests you
cannot write
❖ Untested code can lead to weird bugs and unanticipated
behaviour
Slide 16
Slide 16 text
How To Control Dependencies
❖ Think of your program flow as “message passing”
❖ Refactor code to create required dependencies outside
of where they are used
❖ No shame in refactoring towards using globally-
available containers
Slide 17
Slide 17 text
Dependency Managermane As “Message Passing”
❖ Architect your application so results and dependencies
flow through it
❖ Keep “side effects” to a minimum
❖ Give opportunities for tests to create doubles of
dependencies that need to be in specific states
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
No content
Slide 20
Slide 20 text
Taming Side Effects
❖ in-memory databases using SQLite
❖ in-memory file systems using vfsStream
❖ “dependency overloading” using Mockery
Slide 21
Slide 21 text
In-Memory Databases
❖ Eliminates the side effect of modifying common
databases
❖ Much better control over initial data sets
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
In-Memory Filesystems
❖ great for tests that need to read and/or write files
❖ no need to write code to clean up files after testing
Slide 24
Slide 24 text
No content
Slide 25
Slide 25 text
No content
Slide 26
Slide 26 text
No content
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
Small objects with few methods
Slide 29
Slide 29 text
No content
Slide 30
Slide 30 text
– some grumpy developer speaking to you
“Using Test-Driven Development tends to result in
large numbers of objects with small numbers of
methods. Single responsibility principle in action!”
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
– me
“Again, your tests smell because your code smells.
No. Really.”
Slide 34
Slide 34 text
How To Detect Smelly Code
❖ Your tests require extensive setup steps
❖ Your code to set dependencies fills your editor screen
❖ It’s extremely difficult to tell if you’re getting the
expected results
Slide 35
Slide 35 text
Extensive Setup Steps
❖ Does your app have complicated bootstrapping?
❖ Does it rely on hard-coded values for configuration
options?
❖ How hard is it to swap out dependencies?
Slide 36
Slide 36 text
No content
Slide 37
Slide 37 text
No content
Slide 38
Slide 38 text
No content
Slide 39
Slide 39 text
For those counting along:
24 lines of setup
5 lines of tests
Slide 40
Slide 40 text
No content
Slide 41
Slide 41 text
For those counting along:
24 lines of setup
5 lines of tests
for 7 lines of code
Slide 42
Slide 42 text
Not all smelly code is wrong
Slide 43
Slide 43 text
Other things I see people doing
Slide 44
Slide 44 text
No content
Slide 45
Slide 45 text
No content
Slide 46
Slide 46 text
No content
Slide 47
Slide 47 text
– a parent who hates repeating himself to his children
“DO YOU NOT NOTICE HOW SIMILAR ALL
THESE TESTS ARE?!?.”
Slide 48
Slide 48 text
No content
Slide 49
Slide 49 text
– guy who never wishes for clones of himself
“Just like duplicated code can be bad, duplicated
tests can be bad.”
Slide 50
Slide 50 text
What else do I see that I don’t like?
Slide 51
Slide 51 text
Don’t Do These
❖ conditional statements in your tests!
❖ loops in your tests!
❖ creating a test double of the thing you are testing just so
“the damn thing works”
Slide 52
Slide 52 text
– developer who shouldn’t manage people
“I’m sure I missed your favourite underused
technique that is actually just lazy.”
Slide 53
Slide 53 text
Not all smelly tests are wrong
Slide 54
Slide 54 text
They represent things to keep an
eye on going forward
Slide 55
Slide 55 text
Do You Understand The Tools?
❖ Do you know how to use the testing framework
❖ Do you know how test doubles work?
Slide 56
Slide 56 text
“The tools are hard to use”
Slide 57
Slide 57 text
Personal Opinions Ahead
Slide 58
Slide 58 text
The problem isn’t the tools
Slide 59
Slide 59 text
The problem is
unrealistic expectations
Slide 60
Slide 60 text
Tests are code you write
to prove your other code is right
Slide 61
Slide 61 text
Grumpy’s 4 Steps
To Test Mastery
Slide 62
Slide 62 text
Step The First:
Figure out your dependencies
Slide 63
Slide 63 text
Step The Second:
Figure out you expected outcome
Slide 64
Slide 64 text
Step The Third:
Write the test like everything already works
Slide 65
Slide 65 text
Step The Last:
Write code until the test passes
Slide 66
Slide 66 text
Want to learn more?
Search Github
Slide 67
Slide 67 text
Really.
Best place ever to find tests.
Slide 68
Slide 68 text
https://github.com/opencfp/opencfp
Slide 69
Slide 69 text
–Chris Hartjes
“There are too many examples of well-written tests
and clear instructions for people to claim
ignorance of how to write tests or use the tools to
execute them.”
Slide 70
Slide 70 text
No content
Slide 71
Slide 71 text
Handling The
Weird Stuff
Slide 72
Slide 72 text
Understanding Test Doubles
Slide 73
Slide 73 text
(people call them mocks)
Slide 74
Slide 74 text
Test Doubles
– husband of the most patient woman in the world
“Use them when you have a dependency that is
not under your control that needs to be in a
specific state.”
Slide 75
Slide 75 text
Creating doubles ONLY WHEN
REQUIRED is a good practice
Slide 76
Slide 76 text
So What Should We Double?
❖ database connections
❖ code that calls 3rd party API’s
❖ code that has side effects
Slide 77
Slide 77 text
Database Connections?
❖ does your unit test REALLY need to make sure the
database is working
❖ lets you control the expected responses in terms of
result sets or record ID’s
Slide 78
Slide 78 text
3rd Party API’s?
❖ API use could be restricted (rate-limited, sandboxed for
tests, pay-per-use)
❖ Again, are we in the business of testing their API or
testing our code?
Slide 79
Slide 79 text
– some guy who knows a thing or two about testing
“As an aside, consider the use of contract-style
tests as part of your integration test suite”
Slide 80
Slide 80 text
Weird Stuff To Test
❖ dependencies that you cannot inject without risky
refactoring
❖ getters and setters
❖ how to verify execution of specific code
Slide 81
Slide 81 text
No content
Slide 82
Slide 82 text
No content
Slide 83
Slide 83 text
Using “Overloading”
❖ can override almost anything
❖ the overrides are globally available…
❖ …so annotate tests to run in separate process
Slide 84
Slide 84 text
No content
Slide 85
Slide 85 text
Mockery is awesome
Slide 86
Slide 86 text
Getters and Setters
Considered Harmful?
Slide 87
Slide 87 text
Having to test that a non-public
method works is a testing smell
Slide 88
Slide 88 text
Having to test the contents and/or state
of a protected attribute is a test smell