Upgrade to Pro — share decks privately, control downloads, hide ads and more …

100% Code Coverage via automated tests in Symfony applications

100% Code Coverage via automated tests in Symfony applications

If you create software that is planned for continuous enhancements and maintenance over several years then you'll need a sustainable strategy for code quality.
Code coverage by automated tests is one important metric, and its value should equal 100% at all times.
This talk will show why this is so and how it can be achieved in PHP / Symfony based applications.

mensemedia

April 26, 2017
Tweet

Other Decks in Programming

Transcript

  1. 100% Code Coverage via automated tests
    in Symfony applications
    Symfony User Group Cologne
    2017-04-26

    View Slide

  2. Wir entwickeln weiter

    View Slide

  3. Once upon a time
    *** *** Shipment Tracking
     Only a few deployments slots per year
     Deployment -> $$$
     Redeployment?
     wait for next slot
     pay more $$$
     -> Quality or die!

    View Slide

  4. Automated Testing & Metrics

    View Slide

  5. Why automated testing?
    “I can‘t believe we‘re still talking about this“ – Sebastian Bergmann @
    SymfonyLive 2017
    Cf. Sebastians talk @ Symfony Live 2017
    Some selected topics:
     Confidence by repeatable cause & effect
     Best case: Agility… or, at least: maintainability
     Testability can lead to better code design…
     … bad design yields code that is hard to test
     Reducing complexity to merely binary
    information: Red -> Green cycle

    View Slide

  6. Reduce complexity to merely binary information
    ”Complexity” by Mark Skipper, License: CC BY 2.0

    View Slide

  7. © mensemedia Gesellschaft für Neue Medien mbH 7
    www.mensemedia.net

    View Slide

  8. © mensemedia Gesellschaft für Neue Medien mbH 8
    www.mensemedia.net

    View Slide

  9. © mensemedia Gesellschaft für Neue Medien mbH 9
    www.mensemedia.net
    Code Coverage

    View Slide

  10. Why Code Coverage?
    ”So, ummmm, how many undiscovered ruins are there?“
    – Dean Leffingwell & Don Widrig
    We want to find out what we don‘t know.
     Blind spots
     Dead/unused code
     Design flaws
    Is my code hard to test because of its design?
     Produces kind of a checklist:
    What must still be done before we ship?
     Example:
     Airplane pilot is happy he started 1 engine…
     …but plane has 3 more
    “Managing Software Requirements”, page 83, Dean Leffingwell, Don Widrig, Addison-Wesley 2000

    View Slide

  11. Systems Engineering: Software Lifecycle
    ”Be nice to future-you“
    – unknown
    How many people will have worked on software after…
     … 2 years?
     … 5 years?
     … 10 years?
     -> There will be more people new to the project than initial team members
     Future “generations“ of developers must be prepared for…
     … changes, enhancements etc
     … patches, updates, upgrades, replacements (AngularJS 1, anyone?) etc
     -> Team needs to know about cause and effect, needs confidence in the code
     -> Tests help a lot!
    1 2 5 10

    View Slide

  12. But why the fudge 100%???

    View Slide

  13. Why 100% Code Coverage?
     -> Tests help a lot!
     … for managing the code that the tests cover
     … for code they don‘t cover there‘s no help available
     So, why not 100% to help future generations?
     What makes the uncovered code special
    that it deserves to be excluded?
     Will the future team still know the reason?
     After the usual and inevitable drifts in the code base…
     …will the reason still be valid in the future?
    ”U Turn“ by André-Pierre du Plessis, License: CC BY 2.0

    View Slide

  14. Why 100% Code Coverage?
     Problem distribution:
     Bug density is often not distributed homogeneously
     “90% of problems are caused by 10% of the code“
     What if these 10% are in your uncovered code?
     No? R u sure? Really?
     How can you be sure when you don‘t have tests?
     … after all, tests make you durably confident
     Or, which code will likely cause more problems:
     highly tested code?
     untested, potentially even untestable code?
    ”Detective with pipe and magnifzing glass“ by Juhele, https://openclipart.org/detail/254342/detective-with-pipe-and-magnifying-glass
    ”Confidence is ignorance. If you're feeling cocky, it's because there's
    something you don't know“ – Eoin Colfer, Artemis Fowl

    View Slide

  15. Why 100% Code Coverage?
    ”It wasn’t me. It was that way already.“ – my children
    Prevent decay by the “Broken Windows“ excuse:
     If some code allows for coding less well
    then it will soon accrete more dirty code
     … and developers can blame it
    on the darned legacy code
     Instead, you should embrace a
    “No code left behind“ strategy
    Cf. also
    https://blog.codinghorror.com/the-broken-window-theory/
    “Broken windows, Northampton State Hospital” (https://commons.wikimedia.org/w/index.php?curid=22625973 ) by Karan Jain, License: CC BY-SA 2.0

    View Slide

  16. Why 100% Code Coverage?
    Use the power of metrics
     It‘s rules that everyone needs to stick to
     … objectively measured by a 3rd party
     -> No more discussions about the relevance of some code
     … that only waste time and energy
     Exceptions to the rules will add even MORE complexity
     Instead…
     … let the metric “reward“ the team when at 100%
    and boost it with extra motivation
     … use the metric to reduce complexity to…
    … Red and Green
    “Complexity kills. It sucks the life out of developers […]“ – Ray Ozzie
    Source: Geek & Poke http://geek-and-poke.com/geekandpoke/2017/2/18/50-shades-part-2

    View Slide

  17. © mensemedia Gesellschaft für Neue Medien mbH 17
    www.mensemedia.net

    View Slide

  18. © mensemedia Gesellschaft für Neue Medien mbH 18
    www.mensemedia.net

    View Slide

  19. Let‘s talk about accountability

    View Slide

  20. © mensemedia Gesellschaft für Neue Medien mbH 21
    www.mensemedia.net
    Uncovered Code can have impact on
    quality,
    maintenance, training and onboarding cost,
    eventually leading to technical debt.

    View Slide

  21. © mensemedia Gesellschaft für Neue Medien mbH 22
    www.mensemedia.net
    Quality, cost and task planning
    are BUSINESS issues,
    and only the PO can do a trade off.

    View Slide

  22. © mensemedia Gesellschaft für Neue Medien mbH 23
    www.mensemedia.net
    For professional developers*,
    automated testing is an obligation.
    Uncovered Code does not comply
    to that obligation.
    * especially in interpreted or dynamically typed langages

    View Slide

  23. But wait, that‘s not feasible!
    !

    View Slide

  24. Or is it?
    ?

    View Slide

  25. Advice #1:
    Start @ 100%, Stay @ 100%

    View Slide

  26. TDD – Test Driven/First Development
    Start @ 100%, Stay @ 100%
    “The three rules of TDD“ by Uncle Bob Martin: http://butunclebob.com/ArticleS.UncleBob.TheThreeRulesOfTdd
    Image: http://wiki.expertiza.ncsu.edu/index.php/CSC/ECE_517_Fall_2014/ch1b_28_cg by user “Jgu7“
    ”The three rules of TDD“:
     1: No production code unless to make a test pass
     2: Only so much test code to make the test fail
     3: Only so much production code to make the
    failing test pass
     What should be the expected Code Coverage?
     -> 100%!
     Then, if needed: refactor (test or production code)
    -> can still yield 100% Coverage
     -> run the tests with Code Coverage to find it out
    and “fill the gaps” as part of the refactoring

    View Slide

  27. TDD DON‘Ts
    Start @ 100%, Stay @ 100%
    “Hexagonal Architecture” a.k.a. “Ports and Adapters” by Alistair Cockburn, http://alistair.cockburn.us/Hexagonal+architecture
    For this to work you should not start on the
    “outside“, i.e. do NOT write any of these first:
     Controller
     Command
     View
     Data Access Object / Query
     File System Accessor
     …
     Symfony supports this approach:
     There is no need to write the Controller first

    View Slide

  28. © mensemedia Gesellschaft für Neue Medien mbH 29
    www.mensemedia.net
    Also, it‘s a psych thing
    If you start on the outside you will likely experience:
     the need for non-Unit tests (i.e. slow tests)
     the need to mock stuff (mocks are evil)
     slow progress
     a Code Coverage below 100%
     less motivation because you‘re always “in debt“,
    i.e. < 100%
    https://de.wikipedia.org/wiki/Datei:Sigmund_Freud_LIFE.jpg

    View Slide

  29. © mensemedia Gesellschaft für Neue Medien mbH 30
    www.mensemedia.net
    Also, it‘s a psych thing
    If you start on the outside you will likely experience:
     a tendency to do the wrong things…
     … a.k.a.“Coder‘s Stockholm-Syndrome“
     the urge to put more into the class, e.g. the
    Controller, than you should…
     … to kind of make the effort spent on the
    Controller “pay off“
     … potentially leading to
    higher coupling and less cohesion
    https://de.wikipedia.org/wiki/Datei:Sigmund_Freud_LIFE.jpg

    View Slide

  30. TDD – DOs
    Start @ 100%, Stay @ 100%
    “The Clean Architecture” by Uncle Bob Martin, https://8thlight.com/blog/uncle-bob/2012/08/13/the-clean-architecture.html
    Instead start at the core: The DOMAIN
     Understand your domain (it‘s there, trust me)
     Create the DOMAIN using TDD
     … while ignoring the outside (DB, Controller etc)
     Code against interfaces (=ports) and stubs
     Avoid the Stockholm-Syndrome:
    Integrate “real” outside layers later
     Let the Code Coverage metric show you where
    to add missing tests to fill the gaps
    -> Symfony‘s good design
    supports this approach (usually) pretty well “source code dependencies can only point inwards”

    View Slide

  31. There is no reason why you
    SHOULDN‘T WANT 100% Code Coverage.
    (There may be reasons why you
    CAN‘T get to 100%, though…)

    View Slide

  32. Code Coverage is a fallacy just a tool.
    lie
    lie fallacy
    It depends on what you use it for.

    View Slide

  33. How to trick Code Coverage
    “Ohh, I‘m a liar. Yeah.“ – Henry Rollins
     When you run a Test Case
    it will always cover some code
     For example, calling assertNotNull(new MyClass())
    for each class should yield 5-20% coverage
    without any real value (@covers might help…)
     However, you can also fake tests, e.g. no/fake
    asserts, redundant checks, (some) mocks, …
     -> So, we shouldn‘t write tests anymore?
     … and you can fake code as well, e.g. if you require
    “no if/else, switch/case etc” and developers get
    creative
     -> So, we shouldn‘t write code anymore?
    while ($status === "done") {
    echo "Status: Done";
    break;
    }

    View Slide

  34. Just because Code Coverage can be faked
    does not mean it is irrelevant.

    View Slide

  35. Code Coverage != Test Coverage
    Code Coverage means…
     … your existing code is covered by tests
     … but you still can have
    – pretty obvious –
    bugs
    Code Coverage does NOT mean…
     … that all possible scenarios are covered
     … that would be Test Coverage …
    public static function divide($a, $b)
    {
    return $a / $b;
    }
    public function test_divide()
    {
    $actual = MyCalculator::divide(6, 2);
    $this->assertEquals(3, $actual);
    }
    // no type check, no range validation,
    // division by zero is possible!
    // ...

    View Slide

  36. Where Code Coverage cannot be measured
    “I don't think anybody tests enough of anything.“ – James Gosling
     Views
     Twig and others that allow for
    markup + conditional code
     There is no tool (that I know of) to
    measure Code Coverage in Twig
     Configuration
     How can we spot we tested ALL configuration settings?
     How can we spot obsolete configuration?
    “Game over” by porpom, CC BY 2.0, https://www.flickr.com/photos/poirpom/4894850703/
    {% if temp > 18 and temp < 27 %}
    Let‘s take a walk.
    {% endif %}

    View Slide

  37. Where Code Coverage breaks:
    Aliens, Trade Offs & Lack of Knowledge

    View Slide

  38. Alien Code
    Generally: full coverage can be impossible
    if you cannot change the production code
     3rd party code
     -> avoid mixing “alien“ code with your precious own code
     -> consider moving it to a separate source dir, e.g. via composer
     -> beware of alien code that looks like yours (by scaffolding etc)
     Own legacy code…
     … static calls, global calls, global state, tight coupling etc …
     Symfony supports you nicely…
     … no static calls needed, interfaces available,
    Dependency Injection container
    https://www.amazon.de/dp/0131177052/

    View Slide

  39. Conscious Trade Offs
    Proofs of Concept (PoC)
     … basically: “I don‘t really know…
     … what I want yet.“
     … how to do it yet.“
     -> TDD and Code Coverage don‘t make much sense
    when you don‘t know what to assert
     However, professionals should create PoCs only as throw-away code

    View Slide

  40. Lack of Knowledge
     Things you don‘t know how to test (yet)
     Sometimes you need to master the technology first
     It may take some time to find solutions for testing
     -> Don‘t write testing off for good, keep looking for solutions

    View Slide

  41. Lack of Knowledge
     Insufficiently designed code that…
     … you haven‘t realized to be poorly designed yet
     … you don‘t know how to fix yet
     … don‘t blame the test for not covering badly designed code
     … don‘t declare badly designed code as a “special case“
    that may be excluded from the rule
     -> trust the test

    View Slide

  42. Advice #2: When in doubt,
    trust the test

    View Slide

  43. Trust the test
    ”Tests are for logic. My code does not contain logic “
     You might not see any logic…
     …but, usually, it‘s there
     If your
    Controller or Data Access Object
    are larger than your Service
    then, usually, logic is there
    but you put it in the wrong place
    BIG
    Controller
    BIG
    DAO
    tiny
    Service
    “Weight Training Crossfit Fitness Models” by ThoroughlyReviewed, https://thoroughlyreviewed.com/,
    found at https://www.flickr.com/photos/poirpom/4894850703/, License: CC BY 2.0

    View Slide

  44. Trust the test
    ”Tests are for logic. My code does not contain logic“
    If the code has NO conditional statements
    and you think of it as “configuration as code“…
     … why don‘t you turn it into real
    “configuration as configuration“?
    Speaking of configuration…
    hey SensioLabs…
     …why do I need to create a “*Bundle“ class?
    Can‘t that just be
    configuration or convention?
    namespace AppBundle;
    use Symfony\Component\HttpKernel\Bundle\Bundle;
    class AppBundle extends Bundle
    {
    }

    View Slide

  45. © mensemedia Gesellschaft für Neue Medien mbH 46
    www.mensemedia.net
    Trust the test:
    If (your own) code is hard to test
    then it likely has a design problem.

    View Slide

  46. Reality Check
     Our experience shows: it is possible to achieve
    100% Code Coverage in Symfony Applications
     … so don‘t blame it on Symfony ;-)
     However, you might need to change the way you
    code and design
     … but those changes will eventually lead you to
    Clean Code and Clean Architecture, cf.
     Clean Code Developer (German)
     Clean Code Principles Wiki (English)
     Clean Code book
     Clean Architecture book (2017)
    “Big_Data_Higgs” by KamiPhuc, License: CC BY 2.0

    View Slide

  47. I know…
    … it‘s a
    controversial topic.
    Can we still
    be friends?

    View Slide

  48. THANKS!
    Questions?
    Feedback?
    -> Please contact us!
    mensemedia Gesellschaft für Neue Medien mbH
    Neumannstraße 10
    D- 40235 Düsseldorf
    [email protected]
    www.mensemedia.net
    Andreas Czakaj
    Managing Director IT / CTO
    Tel +49.211.55 04 78-25
    [email protected]
    Twitter: AndreasCzakaj

    View Slide