• Database Coupling • Network Services • Do I Need CI • Code Coverage • Ice Cream Cones and BDD • Intermittent Failure To Launch • My Worst UI Test • Sticky Bits • Its Not Me, Its the Environment
= isset($this->pageDataForView->books) ? $this->pageDataForView- >books : null ; } /** * The first method I want to test */ private function sortBooksByAuthor() { //usort is going to modify $books by reference, //so lets copy it to prevent side effects $sortedBooks = $this->books; usort($sortedBooks, function($a, $b) { /** * sort by author last name, so first get the last name * or last segment of the name. Will not work with JR/III suffixes */ $authorA = explode(" ", trim($a->author)); $authorB = explode(" ", trim($b->author)); return strcmp($authorA[count($authorA)-1], $authorB[count($authorB)-1]); }); return $sortedBooks; } /** * @param $books * @return mixed * The second method I want to test Private Method I want to test
database • Add code die(print_r($result,1)) in my app • Look at the code • Comment out the code • Never test it again • Hope no one ever breaks the working function • Hope I never accidentally check in debug code
• Have PHP make a copy of the class • Have PHP make a copy of the function • Change the copied function to public • Call the function with my data • Verify results
PHPUnit_Framework_TestCase; /** * Created by IntelliJ IDEA. * User: mland * Date: 4/17/15 * Time: 11:25 AM */ date_default_timezone_set('America/Chicago'); class RenderTest extends PHPUnit_Framework_Testcase { protected $jsonBody; /** * this is to simulate the json data we feed to the template */ public function setUp() { $this->jsonBody = json_decode( file_get_contents(__DIR__ . '/pageDataForView.json') ); } public function _testRenderer() { Setup $this->jsonBody for all tests
= $renderObj->toHtml(); $this->fail('This is complex: ' . PHP_EOL . $rendering . " Lets test the methods instead."); } /** * reflecting a method, where we have to set the value in the class first */ public function testSortBooksByAuthor() { $reflectedClass = new ReflectionClass('\\Samples\\Render'); $realClass = $reflectedClass->newInstance(); $booksProperty = $reflectedClass->getProperty('books'); $booksProperty->setAccessible(1); $booksProperty->setValue($realClass, $this->jsonBody->books); $sortByAuthorMethod = $reflectedClass->getMethod('sortBooksByAuthor'); $sortByAuthorMethod->setAccessible(true); $sortedBooks = $sortByAuthorMethod->invoke($realClass); $this->assertEquals('Kwame Alexander', $sortedBooks[0]->author); $this->assertEquals('William Ritter', $sortedBooks[4]->author); } /** * this time written statically/no object oriented dependency * Much easier */ Test that uses reflection 10 LOC
by reference, //so lets copy it to prevent side effects $sortedBooks = $this->books; usort($sortedBooks, function ($a, $b) { /** * sort by author last name, so first get the last name * or last segment of the name. Will not work with JR/III suffixes */ $authorA = explode(" ", trim($a->author)); $authorB = explode(" ", trim($b->author)); return strcmp($authorA[count($authorA) - 1], $authorB[count($authorB) - 1]); }); return $sortedBooks; } public function testSortBooksByAuthor() { $reflectedClass = new ReflectionClass('\\Samples\\Render'); $realClass = $reflectedClass->newInstance(); $booksProperty = $reflectedClass->getProperty('books'); $booksProperty->setAccessible(1); $booksProperty->setValue($realClass, $this->jsonBody->books); $sortByAuthorMethod = $reflectedClass->getMethod('sortBooksByAuthor'); $sortByAuthorMethod->setAccessible(true); $sortedBooks = $sortByAuthorMethod->invoke($realClass); $this->assertEquals('Kwame Alexander', $sortedBooks[0]->author); $this->assertEquals('William Ritter', $sortedBooks[4]->author); }
method I want to test */ private static function sortBooksByPrice($books) { //usort is going to modify $books by reference, //so lets copy it to prevent side effects $sortedBooks = $books; usort($sortedBooks, function($a, $b) { /** * compare decimal price */ return $a->price >= $b->price; }); return $sortedBooks; } public function testSortBooksByPrice() { $realClass = new Render(); Second Method I want to test Changed to static Parameter is passed in Gave up $this, but still have self::
going to modify $books by reference, //so lets copy it to prevent side effects $sortedBooks = $books; usort($sortedBooks, function($a, $b) { /** * sort by ISBN string which might being with zero */ return strcmp($a->isbn, $b->isbn); }); return $sortedBooks; } public function testSortBooksByISBN() { $sortedBooks = Render::sortBooksByIsbn($this->jsonBody- >books); $this->assertEquals('9780544107717', $sortedBooks[0]->isbn); $this->assertEquals('9781616203535', $sortedBooks[4]->isbn); Third Method I want to test Changed to public No side effect risk
$books; usort($sortedBooks, function($a, $b) { /** * sort by ISBN string which might being with zero */ return strcmp($a->isbn, $b->isbn); }); return $sortedBooks; } public function testSortBooksByISBN() { $sortedBooks = Render::sortBooksByIsbn($this->jsonBody->books); $this->assertEquals('9780544107717', $sortedBooks[0]->isbn); $this->assertEquals('9781616203535', $sortedBooks[4]->isbn); } } Testing Third Method without Reflection 3 LOC
$books by reference, //so lets copy it to prevent side effects $sortedBooks = $books; usort($sortedBooks, function($a, $b) { /** * sort by ISBN string which might being with zero */ return strcmp($a->isbn, $b->isbn); }); return $sortedBooks; } public function testSortBooksByISBN() { $sortedBooks = Render::sortBooksByIsbn($this->jsonBody->books); $this->assertEquals('9780544107717', $sortedBooks[0]->isbn); $this->assertEquals('9781616203535', $sortedBooks[4]->isbn); } }
for self in method • Not hitting the database • No side effects or • Calculated side effects • Add object type hints • Can’t do it another way Reflection Use Both
have an empty DB: • Refresh from DB from master as often as ‘reasonable’ • unique() will prevent column restrictions and hash collisions • Hacks are OK, at the level “some tests > no tests”
• Build factories that chain mock object relationships together • Test complex things, start small User Payment Method Account Location Booking Statement Batch Statement Sales Order Invoice
[]; $this->pdo->insert('some record'); $insertIds[] = $this->pdo->getLastInsertId(); $this->pdo->insert('some record2'); $insertIds[] = $this->pdo->getLastInsertId(); ... $records = My\Service::findRecord('my report'); $found = false; foreach ($records as $record) { if ($record->someCondition() !== false) { $found = true; } } if (! $found) { $this->fail('did not find the record we expected'); } foreach ($insertIds as $id) { $this->pdo->delete($id); } } Test will leave records any time it fails
[]; $this->pdo->insert('some record'); $insertIds[] = $this->pdo->getLastInsertId(); $this->pdo->insert('some record2'); $insertIds[] = $this->pdo->getLastInsertId(); ... $records = My\Service::findRecord('my report'); $found = false; foreach ($records as $record) { if ($record->someCondition() !== false) { $found = true; } } foreach ($insertIds as $id) { $this->pdo->delete($id); } if (! $found) { $this->fail('did not find the record we expected'); } } This test will clean up before failing
the network layer like Man-in-middle attack • Intercepts service requests • Returns mock data or routes to a service provider, based on your pre-defined logic • http://wiremock.org/ • https://github.com/rowanhill/wiremock-php
between the feature and the maintainers of the application. It says, ‘This feature was important enough to have a test written for it. This contract forbids you from (accidentally) breaking said feature.’ Regressions
• Manipulate the UI • Bounce the UI, since first time was just jQuery • Verify the side effects in persistence Tech Debt The test is doing too much, or test the service separately Because our logic is everywhere Because the DB is coupled Because our logic is everywhere Typical UI Test
• Manipulate the UI • Bounce the UI, since first time was just jQuery • Verify the side effects in persistence Tech Debt The test is doing too much, or test the service separately Because our logic is everywhere Because the DB is coupled Because our logic is everywhere The result is each test is very slow. Typical UI Test
• Manipulate the UI • Bounce the UI, since first time was just jQuery • Verify the side effects in persistence Tech Debt The test is doing too much, or test the service separately Because our logic is everywhere Because the DB is coupled Because our logic is everywhere Its OK! Tests > No Tests, and tech debt is fixable. But its only OK up to a point. Typical UI Test
notice build hasn’t passed yet today • 2:10pm - bounce selenium grid for luck • 2:30pm - look in hip chat debug room for screenshots • 2:40pm - deploy time is coming. Last chance to go green in time • 2:55pm - build failure. curse violently. Devbot mocks me • 3:10pm - bow out of the deployment • 4:05pm - finally a green build. !Freeze the build, indefinitely • 6:45pm - DevOps is back from kid’s swim practice • Successfully Deploy!
the CI environment • Blame CI environment performance • Blame the JavaScript plugins (jQuery whatever) • Your MVC code is doing too damn much • Code actually fails a large percentage of the time in prod. You just hadn’t noticed.
of difficulty • Test on hotel Wifi, see if you can pass • Remove side effects from controllers • Sending Emails • Uploads to S3 • Calls to push data to other SaaS’s
and in host OS (interactive) Both are super useful to my workflow: • In the host OS for writing and debugging a broken condition • In the VM for running a suite before opening my pull request
regression) balance • speed of quick feedback • full regression coverage • (don’t trust anyone) Code coverage plug-ins are invalid in the scope for integration level tests.
Write a dedicated testsuite API for Datepicker() • Expected, Actual, Whoops-Message • Explicitly wait for _everything_ • Disable tests just like C++ by underscore prefixing • Fail as fast as possible (Temporarily disable long tests higher in the class)
with 404 • add controller/action/behavior ids to the dom • Use identical IDs when FF’ing so the test can follow • Request/Factory/Response Mock Factory • Watch for _test nerving in pull requests