100% Code Coverage via automated tests in Symfony applications

100% Code Coverage via automated tests
in Symfony applications
Symfony User Group Cologne
2017-04-26

View Slide

Wir entwickeln weiter

View Slide

Once upon a time
*** *** Shipment Tracking
 Only a few deployments slots per year
 Deployment -> $$$
 Redeployment?
 wait for next slot
 pay more $$$
 -> Quality or die!

View Slide

Automated Testing & Metrics

View Slide

Why automated testing?
“I can‘t believe we‘re still talking about this“ – Sebastian Bergmann @
SymfonyLive 2017
Cf. Sebastians talk @ Symfony Live 2017
Some selected topics:
 Confidence by repeatable cause & effect
 Best case: Agility… or, at least: maintainability
 Testability can lead to better code design…
 … bad design yields code that is hard to test
 Reducing complexity to merely binary
information: Red -> Green cycle

View Slide

Reduce complexity to merely binary information
”Complexity” by Mark Skipper, License: CC BY 2.0

View Slide

© mensemedia Gesellschaft für Neue Medien mbH 7
www.mensemedia.net

View Slide

www.mensemedia.net

View Slide

www.mensemedia.net
Code Coverage

View Slide

Why Code Coverage?
”So, ummmm, how many undiscovered ruins are there?“
– Dean Leffingwell & Don Widrig
We want to find out what we don‘t know.
 Blind spots
 Dead/unused code
 Design flaws
Is my code hard to test because of its design?
 Produces kind of a checklist:
What must still be done before we ship?
 Example:
 Airplane pilot is happy he started 1 engine…
 …but plane has 3 more
“Managing Software Requirements”, page 83, Dean Leffingwell, Don Widrig, Addison-Wesley 2000

View Slide

Systems Engineering: Software Lifecycle
”Be nice to future-you“
– unknown
How many people will have worked on software after…
 … 2 years?
 … 5 years?
 … 10 years?
 -> There will be more people new to the project than initial team members
 Future “generations“ of developers must be prepared for…
 … changes, enhancements etc
 … patches, updates, upgrades, replacements (AngularJS 1, anyone?) etc
 -> Team needs to know about cause and effect, needs confidence in the code
 -> Tests help a lot!
1 2 5 10

View Slide

But why the fudge 100%???

View Slide

Why 100% Code Coverage?
 -> Tests help a lot!
 … for managing the code that the tests cover
 … for code they don‘t cover there‘s no help available
 So, why not 100% to help future generations?
 What makes the uncovered code special
that it deserves to be excluded?
 Will the future team still know the reason?
 After the usual and inevitable drifts in the code base…
 …will the reason still be valid in the future?
”U Turn“ by André-Pierre du Plessis, License: CC BY 2.0

View Slide

 Problem distribution:
 Bug density is often not distributed homogeneously
 “90% of problems are caused by 10% of the code“
 What if these 10% are in your uncovered code?
 No? R u sure? Really?
 How can you be sure when you don‘t have tests?
 … after all, tests make you durably confident
 Or, which code will likely cause more problems:
 highly tested code?
 untested, potentially even untestable code?
”Detective with pipe and magnifzing glass“ by Juhele, https://openclipart.org/detail/254342/detective-with-pipe-and-magnifying-glass
”Confidence is ignorance. If you're feeling cocky, it's because there's
something you don't know“ – Eoin Colfer, Artemis Fowl

View Slide

”It wasn’t me. It was that way already.“ – my children
Prevent decay by the “Broken Windows“ excuse:
 If some code allows for coding less well
then it will soon accrete more dirty code
 … and developers can blame it
on the darned legacy code
 Instead, you should embrace a
“No code left behind“ strategy
Cf. also
https://blog.codinghorror.com/the-broken-window-theory/
“Broken windows, Northampton State Hospital” (https://commons.wikimedia.org/w/index.php?curid=22625973 ) by Karan Jain, License: CC BY-SA 2.0

View Slide

Use the power of metrics
 It‘s rules that everyone needs to stick to
 … objectively measured by a 3rd party
 -> No more discussions about the relevance of some code
 … that only waste time and energy
 Exceptions to the rules will add even MORE complexity
 Instead…
 … let the metric “reward“ the team when at 100%
and boost it with extra motivation
 … use the metric to reduce complexity to…
… Red and Green
“Complexity kills. It sucks the life out of developers […]“ – Ray Ozzie
Source: Geek & Poke http://geek-and-poke.com/geekandpoke/2017/2/18/50-shades-part-2

View Slide

www.mensemedia.net

View Slide

Let‘s talk about accountability

View Slide

www.mensemedia.net
Uncovered Code can have impact on
quality,
maintenance, training and onboarding cost,
eventually leading to technical debt.

View Slide

www.mensemedia.net
Quality, cost and task planning
are BUSINESS issues,
and only the PO can do a trade off.

View Slide

www.mensemedia.net
For professional developers*,
automated testing is an obligation.
Uncovered Code does not comply
to that obligation.
* especially in interpreted or dynamically typed langages

View Slide

But wait, that‘s not feasible!
!

View Slide

Or is it?
?

View Slide

Advice #1:
Start @ 100%, Stay @ 100%

View Slide

TDD – Test Driven/First Development
Start @ 100%, Stay @ 100%
“The three rules of TDD“ by Uncle Bob Martin: http://butunclebob.com/ArticleS.UncleBob.TheThreeRulesOfTdd
Image: http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2014/ch1b_28_cg by user “Jgu7“
”The three rules of TDD“:
 1: No production code unless to make a test pass
 2: Only so much test code to make the test fail
 3: Only so much production code to make the
failing test pass
 What should be the expected Code Coverage?
 -> 100%!
 Then, if needed: refactor (test or production code)
-> can still yield 100% Coverage
 -> run the tests with Code Coverage to find it out
and “fill the gaps” as part of the refactoring

View Slide

TDD DON‘Ts
Start @ 100%, Stay @ 100%
“Hexagonal Architecture” a.k.a. “Ports and Adapters” by Alistair Cockburn, http://alistair.cockburn.us/Hexagonal+architecture
For this to work you should not start on the
“outside“, i.e. do NOT write any of these first:
 Controller
 Command
 View
 Data Access Object / Query
 File System Accessor
 …
 Symfony supports this approach:
 There is no need to write the Controller first

View Slide

www.mensemedia.net
Also, it‘s a psych thing
If you start on the outside you will likely experience:
 the need for non-Unit tests (i.e. slow tests)
 the need to mock stuff (mocks are evil)
 slow progress
 a Code Coverage below 100%
 less motivation because you‘re always “in debt“,
i.e. < 100%
https://de.wikipedia.org/wiki/Datei:Sigmund_Freud_LIFE.jpg

View Slide

www.mensemedia.net
Also, it‘s a psych thing
If you start on the outside you will likely experience:
 a tendency to do the wrong things…
 … a.k.a.“Coder‘s Stockholm-Syndrome“
 the urge to put more into the class, e.g. the
Controller, than you should…
 … to kind of make the effort spent on the
Controller “pay off“
 … potentially leading to
higher coupling and less cohesion
https://de.wikipedia.org/wiki/Datei:Sigmund_Freud_LIFE.jpg

View Slide

TDD – DOs
Start @ 100%, Stay @ 100%
“The Clean Architecture” by Uncle Bob Martin, https://8thlight.com/blog/uncle-bob/2012/08/13/the-clean-architecture.html
Instead start at the core: The DOMAIN
 Understand your domain (it‘s there, trust me)
 Create the DOMAIN using TDD
 … while ignoring the outside (DB, Controller etc)
 Code against interfaces (=ports) and stubs
 Avoid the Stockholm-Syndrome:
Integrate “real” outside layers later
 Let the Code Coverage metric show you where
to add missing tests to fill the gaps
-> Symfony‘s good design
supports this approach (usually) pretty well “source code dependencies can only point inwards”

View Slide

There is no reason why you
SHOULDN‘T WANT 100% Code Coverage.
(There may be reasons why you
CAN‘T get to 100%, though…)

View Slide

Code Coverage is a fallacy just a tool.
lie
lie fallacy
It depends on what you use it for.

View Slide

How to trick Code Coverage
“Ohh, I‘m a liar. Yeah.“ – Henry Rollins
 When you run a Test Case
it will always cover some code
 For example, calling assertNotNull(new MyClass())
for each class should yield 5-20% coverage
without any real value (@covers might help…)
 However, you can also fake tests, e.g. no/fake
asserts, redundant checks, (some) mocks, …
 -> So, we shouldn‘t write tests anymore?
 … and you can fake code as well, e.g. if you require
“no if/else, switch/case etc” and developers get
creative
 -> So, we shouldn‘t write code anymore?
while ($status === "done") {
echo "Status: Done";
break;
}

View Slide

Just because Code Coverage can be faked
does not mean it is irrelevant.

View Slide

Code Coverage != Test Coverage
Code Coverage means…
 … your existing code is covered by tests
 … but you still can have
– pretty obvious –
bugs
Code Coverage does NOT mean…
 … that all possible scenarios are covered
 … that would be Test Coverage …
public static function divide($a, $b)
{
return $a / $b;
}
public function test_divide()
{
$actual = MyCalculator::divide(6, 2);
$this->assertEquals(3, $actual);
}
// no type check, no range validation,
// division by zero is possible!
// ...

View Slide

Where Code Coverage cannot be measured
“I don't think anybody tests enough of anything.“ – James Gosling
 Views
 Twig and others that allow for
markup + conditional code
 There is no tool (that I know of) to
measure Code Coverage in Twig
 Configuration
 How can we spot we tested ALL configuration settings?
 How can we spot obsolete configuration?
“Game over” by porpom, CC BY 2.0, https://www.flickr.com/photos/poirpom/4894850703/
{% if temp > 18 and temp < 27 %}
Let‘s take a walk.
{% endif %}

View Slide

Where Code Coverage breaks:
Aliens, Trade Offs & Lack of Knowledge

View Slide

Alien Code
Generally: full coverage can be impossible
if you cannot change the production code
 3rd party code
 -> avoid mixing “alien“ code with your precious own code
 -> consider moving it to a separate source dir, e.g. via composer
 -> beware of alien code that looks like yours (by scaffolding etc)
 Own legacy code…
 … static calls, global calls, global state, tight coupling etc …
 Symfony supports you nicely…
 … no static calls needed, interfaces available,
Dependency Injection container
https://www.amazon.de/dp/0131177052/

View Slide

Conscious Trade Offs
Proofs of Concept (PoC)
 … basically: “I don‘t really know…
 … what I want yet.“
 … how to do it yet.“
 -> TDD and Code Coverage don‘t make much sense
when you don‘t know what to assert
 However, professionals should create PoCs only as throw-away code

View Slide

Lack of Knowledge
 Things you don‘t know how to test (yet)
 Sometimes you need to master the technology first
 It may take some time to find solutions for testing
 -> Don‘t write testing off for good, keep looking for solutions

View Slide

Lack of Knowledge
 Insufficiently designed code that…
 … you haven‘t realized to be poorly designed yet
 … you don‘t know how to fix yet
 … don‘t blame the test for not covering badly designed code
 … don‘t declare badly designed code as a “special case“
that may be excluded from the rule
 -> trust the test

View Slide

Advice #2: When in doubt,
trust the test

View Slide

Trust the test
”Tests are for logic. My code does not contain logic “
 You might not see any logic…
 …but, usually, it‘s there
 If your
Controller or Data Access Object
are larger than your Service
then, usually, logic is there
but you put it in the wrong place
BIG
Controller
BIG
DAO
tiny
Service
“Weight Training Crossfit Fitness Models” by ThoroughlyReviewed, https://thoroughlyreviewed.com/,
found at https://www.flickr.com/photos/poirpom/4894850703/, License: CC BY 2.0

View Slide

Trust the test
”Tests are for logic. My code does not contain logic“
If the code has NO conditional statements
and you think of it as “configuration as code“…
 … why don‘t you turn it into real
“configuration as configuration“?
Speaking of configuration…
hey SensioLabs…
 …why do I need to create a “*Bundle“ class?
Can‘t that just be
configuration or convention?
namespace AppBundle;
use Symfony\Component\HttpKernel\Bundle\Bundle;
class AppBundle extends Bundle
{
}

View Slide

www.mensemedia.net
Trust the test:
If (your own) code is hard to test
then it likely has a design problem.

View Slide

Reality Check
 Our experience shows: it is possible to achieve
100% Code Coverage in Symfony Applications
 … so don‘t blame it on Symfony ;-)
 However, you might need to change the way you
code and design
 … but those changes will eventually lead you to
Clean Code and Clean Architecture, cf.
 Clean Code Developer (German)
 Clean Code Principles Wiki (English)
 Clean Code book
 Clean Architecture book (2017)
“Big_Data_Higgs” by KamiPhuc, License: CC BY 2.0

View Slide

I know…
… it‘s a
controversial topic.
Can we still
be friends?

View Slide

THANKS!
Questions?
Feedback?
-> Please contact us!
mensemedia Gesellschaft für Neue Medien mbH
Neumannstraße 10
D- 40235 Düsseldorf
[email protected]
www.mensemedia.net
Andreas Czakaj
Managing Director IT / CTO
Tel +49.211.55 04 78-25
[email protected]
Twitter: AndreasCzakaj

View Slide

100% Code Coverage via automated tests in Symfony applications

100% Code Coverage via automated tests in Symfony applications

mensemedia

Other Decks in Programming

Featured

Transcript