99% is not enough

Everyone agrees that unit tests are essential for building robust and maintainable software components. But how much code coverage should a project have? We claim that even 99% coverage is not enough.

We will show how we write tests, what we needed to do in order to make our code testable and why reaching the magical goal of 100% code coverage is so important to us.

Transcript

1. 99% IS NOT ENOUGH
2. Sebastian Thoß Kruno Knego Sebastian Heuer Backend Lead Backend Developer Developer Advocate
8. Goods and Bads • fully object-oriented • easy refactoring thanks to 100% code coverage • not enough automated acceptance tests • dependencies to legacy software (database changes, API calls)
11. Code Coverage 100% != 100%
12. public function execute() { $orders = $this->orderService->getOrders($this->orderNumber); $json = $this->orderJsonMapper->map($orders); return new CommandResult($json); } Example
13. public function testExecute() { $service = $this->createMock(OrderService::class); $mapper = $this->createMock(OrderJsonMapper::class); $order1 = $this->createMock(Order::class); $order2 = $this->createMock(Order::class); $command = new GetOrdersCommand('123', $service, $mapper); $service->expects($this->once()) ->method('getOrders') ->with('123') ->willReturn([$order1, $order2]); $mapper->expects($this->once()) ->method('map') ->with([$order1, $order2]); $command->execute(); }
14. 100%!
15. public function execute() { $orders = $this->orderService->getOrders($this->orderNumber); $json = $this->orderJsonMapper->map($orders); return new CommandResult(json_encode('{"foo":"bar"}')); } Breaking Stuff
16. Still 100%!
17. Test Pyramid UNIT TESTS
18. Unit Testing leads to SOLID(ish) code • Single Responsibility • Open/Closed • Liskov Substitution • Interface Segregation • Dependency Inversion
20. Test Pyramid INTEGRATION TESTS UNIT TESTS
21. Class 1 Class 2 Class 3 Mock Integration Test
22. Test Pyramid ACCEPTANCE TESTS INTEGRATION TESTS UNIT TESTS
23. Class 1 Class 2 Class 3 Acceptance Test
24. Test Pyramid ACCEPTANCE TESTS INTEGRATION TESTS UNIT TESTS ? $$$ ¢
26. <?php namespace Kartenmacherei\Phpughh; use PHPUnit_Framework_TestCase; class SmokeTest extends PHPUnit_Framework_TestCase { /** * @dataProvider furyUrlProvider * * @param Url $url */ public function testFuryUrl(Url $url) { $result = $this->sendGetRequest($url); $this->assertSame(200, $result->getStatusCode()); $this->assertNotEmpty($result->getBody()); $this->assertLessThanOrEqual(100, $result->getTimeToFirstByteInMilliseconds()); }
27. Test Pyramid ACCEPTANCE TESTS INTEGRATION TESTS UNIT TESTS SMOKE TESTS $$$ ¢
28. Test Pyramid ACCEPTANCE TESTS INTEGRATION TESTS SMOKE TESTS 92% COVERAGE UNIT TESTS
29. Test Pyramid ACCEPTANCE TESTS INTEGRATION TESTS SMOKE TESTS UNIT TESTS 100% COVERAGE
30. Why 100%?
31. Why 100%? • Robustness • Feeling of security when refactoring • Fixing bugs is cheaper • Reducing amount of major bugs found in production • Broken Windows Theory
32. "Factory Denver5" by dliban licensed under CC BY-NC-ND 2 - https://flic.kr/p/73y9dg
33. "Factory Denver5" by dliban licensed under CC BY-NC-ND 2 - https://flic.kr/p/73y9dg
34. But what about… • Setters? • Factories? • Trivial Classes?
35. Tips & Tricks
36. Strict PHPUnit Config <?xml version="1.0" encoding="UTF-8"?> <phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/5.6/phpunit.xsd" bootstrap="bootstrap.php" backupGlobals="false" backupStaticAttributes="false" beStrictAboutChangesToGlobalState="true" beStrictAboutOutputDuringTests="true" beStrictAboutTestsThatDoNotTestAnything="true" forceCoversAnnotation="true" beStrictAboutCoversAnnotation="true">
37. Mocking public function testCalculatorMultiplies() { $multiplier = $this->getMultiplierMock(); $calculator = new Calculator($multiplier); // (…) } /** * @return PHPUnit_Framework_MockObject_MockObject|Multiplier */ private function getMultiplierMock() { return $this->createMock(Multiplier::class); }
38. @codeCoverageIgnore • errorHandlers • shutdownHandlers Be very strict about excluding code from coverage!
39. @codeCoverageIgnore Be very strict about excluding code from coverage! /** * @param mixed $targetNode * * @throws WrongInstanceException */ private function ensureIsElement($targetNode) { if (!$targetNode instanceof fDOMElement) { // @codeCoverageIgnoreStart throw new WrongInstanceException('target node is not instance of fDOMElement'); // @codeCoverageIgnoreEnd } }
40. Don't test your privates! • Uncovered private methods are probably obsolete • "If covering private methods by only testing public methods is hard, there might be another class in there struggling to get out"
41. "Who's going to pay for all that?" Relative Cost of a Bugfix 40x 80x 120x 160x Requirements Design Code Developer Tests Acceptance Tests Operations 1x 5x 10x 20x 50x 150x Source: Barry Boehm, Keynote Address, EQUITY 2007.
42. https://www.instagram.com/kartenmacherei/ Q&A https://www.facebook.com/kartenmacherei/ +49 40 468996861 jobs@kartenmacherei.de http://www.kartenmacherei.de/recruiting