Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing the untestable

Testing the untestable

When you need to fix or extend code that is 'far down the rabbit hole', most of the work becomes setting up the complex conditions to push the code to a certain point in the application flow. Instead, open classes up directly using testing harnesses, reflection classes and functional programming. Phpunit will be used to demonstrate code coverage before and after.

973e0861866767b3940a6dbb5aac3642?s=128

Matt Land

April 18, 2015
Tweet

More Decks by Matt Land

Other Decks in Programming

Transcript

  1. Testing the untestable Lone Star PHP 2015

  2. We help people get storage. Move in and move on.

  3. Matt Land Code Samples: github.com/matt-land Slides: speakerdeck.com/mattland/ linkedin.com/in/matthewland CERTIFIED A

    R C H I T E CT 2 https://joind.in/13567
  4. Testing the Untestable Create the World, and Poke it With

    a Stick Do I Need CI? Real Fake Data Ice Cream Cones and BDD Intermittent Failure To Launch My Worst UI Test Sticky Bits Its Not Me, Its the Environment
  5. (Day 17, 2:45 at SpareFoot) “It doesn’t matter if it

    works or not. Just make the tests pass so we can deploy.”
  6. Poke it with a stick

  7. The First Test Noise: • PSR Autoloaders • Composer •

    Namespaces • Coding Standards • Continuous Integration • Git & Repositories • Best Practices • Mocks • … Not Noise: • Writing Tests • Passing Tests
  8. None
  9. PROTECTED PRIVATE PRIVATE PRIVATE PRIVATE PROTECTED PRIVATE PRIVATE PROTECTED PRIVATE

  10. None
  11. • Sample application: • http://localhost:63342/apps/test-samples/index.php? sort=author • https://github.com/matt-land/test-samples

  12. <?php /** * Created by IntelliJ IDEA. * User: mland

    * Date: 4/17/15 * Time: 4:33 PM */ require_once __DIR__ . '/vendor/autoload.php'; use Samples\Render; $dsnBooks = json_encode([ 'category' => 'Young Adult', 'records' => 5, 'meta' => [ 'next' => '/whatever?q=345w5', 'prev' => '/whatever?q=92342' ], 'books' => [ [ 'title' => 'The Crossover' , 'author' => 'Kwame Alexander', 'publisher' => 'Houghton', 'price' => '16.99', 'isbn' => '9780544107717' ], [ 'title' => 'The Carnival at Bray' , 'author' => 'Jessie Ann Foley', 'publisher' => 'Elephant Rock', 'price' => '12.95', 'isbn' => '97809895115597' ], [ 'title' => 'I\'ll Give You the Sun' , 'author' => 'Jandy Nelson', 'publisher' => 'Dial', 'price' => '17.99', 'publisher' => 'Houghton', 'price' => '16.99', 'isbn' => '9780544107717' ], [ 'title' => 'The Carnival at Bray' , 'author' => 'Jessie Ann Foley', 'publisher' => 'Elephant Rock', 'price' => '12.95', 'isbn' => '97809895115597' ], [ 'title' => 'I\'ll Give You the Sun' , 'author' => 'Jandy Nelson', 'publisher' => 'Dial', 'price' => '17.99', 'isbn' => '9780803734968' ], [ 'title' => 'Jackaby' , 'author' => 'William Ritter', 'publisher' => 'William Ritter', 'price' => '16.95', 'isbn' => '9781616203535' ], [ 'title' => 'The Gospel of Winter' , 'author' => 'Brendan Kiely', 'publisher' => 'Simon & Schuster/Margaret K. McElderry', 'price' => '17.99', 'isbn' => '9781442484894' ], ] ]); $render = new Render($dsnBooks); $sortOrder = isset($_REQUEST['sort']) ? $_REQUEST['sort'] : null; echo $render->toHtml($sortOrder); Continues
  13. <h3>Total Records: ' . $this->data->records . '</h3> <a href="' .

    $this->data->meta->prev . '">Prev</a> <a href="' . $this->data->meta->next . '">Next</a>'; return $html; } private function sortBooksByAuthor() { //usort is going to modify $books by reference, //so lets copy it to prevent side effects $sortedBooks = $this->books; usort($sortedBooks, function($a, $b) { /** * sort by author last name, so first get the last name * or last segment of the name. Will not work with JR/III suffixes */ $authorA = explode(" ", trim($a->author)); $authorB = explode(" ", trim($b->author)); return strcmp($authorA[count($authorA)-1], $authorB[count($authorB)-1]); }); return $sortedBooks; } private static function sortBooksByPrice($books) { //usort is going to modify $books by reference, //so lets copy it to prevent side effects
  14. { $renderObj = new Render(json_encode($this->jsonBody)); $rendering = $renderObj->toHtml(); $this->fail('This is

    complex: ' . PHP_EOL . $rendering . " Lets test the methods in } /** * reflecting a method, where we have to set the value in the class first */ public function testSortBooksByAuthor() { $reflectedClass = new ReflectionClass('\\Samples\\Render'); $realClass = $reflectedClass->newInstance(); $booksProperty = $reflectedClass->getProperty('books'); // $rea $booksProperty->setAccessible(1); // $rea $booksProperty->setValue($realClass, $this->jsonBody->books); // $rea $sortBooksByAuthorMethod = $reflectedClass->getMethod('sortBooksByAuthor'); //$real $sortBooksByAuthorMethod->setAccessible(true); //$real $sortedBooks = $sortBooksByAuthorMethod->invoke($realClass); //$real $this->assertEquals('Kwame Alexander', $sortedBooks[0]->author); $this->assertEquals('William Ritter', $sortedBooks[4]->author); } /** * this time written statically/no object oriented dependency
  15. None
  16. return strcmp($authorA[count($authorA)-1], $authorB[count($authorB)- }); return $sortedBooks; } private static function

    sortBooksByPrice($books) { //usort is going to modify $books by reference, //so lets copy it to prevent side effects $sortedBooks = $books; usort($sortedBooks, function($a, $b) { /** * compare decimal price */ return $a->price >= $b->price; }); return $sortedBooks; } public static function sortBooksByIsbn($books) { //usort is going to modify $books by reference, //so lets copy it to prevent side effects $sortedBooks = $books; usort($sortedBooks, function($a, $b) { /** * sort by ISBN string which might being with zero Changed to static Parameter is passed in Gave up $this, but still have self::
  17. $sortBooksByAuthorMethod->setAccessible(true); //$realC $sortedBooks = $sortBooksByAuthorMethod->invoke($realClass); //$realC $this->assertEquals('Kwame Alexander', $sortedBooks[0]->author); $this->assertEquals('William

    Ritter', $sortedBooks[4]->author); } /** * this time written statically/no object oriented dependency * Much easier */ public function testSortBooksByPrice() { $realClass = new Render(); $sortBooksByPriceMethod = new ReflectionMethod($realClass, 'sortBooksByPrice'); $sortBooksByPriceMethod->setAccessible(true); $sortedBooks = $sortBooksByPriceMethod->invoke($realClass, $this->jsonBody->books); $this->assertEquals(12.95, $sortedBooks[0]->price); $this->assertEquals(17.99, $sortedBooks[4]->price); } /** * the easiest yet. now that its static and public, we can just use it */ public function testSortBooksByISBN()
  18. None
  19. */ return strcmp($a->isbn, $b->isbn); }); return $sortedBooks; } public static

    function sortBooksByTitle($books) { //usort is going to modify $books by reference, //so lets copy it to prevent side effects $sortedBooks = $books; usort($sortedBooks, function($a, $b) { /** * remove articles and sort titles */ return strcmp( str_ireplace(["a ", "an ", "the "], '', $a->title), str_ireplace(["a ", "an ", "the "], '', $b->title) ); }); return $sortedBooks; } } Changed to public No side effect risk
  20. { $sortedBooks = Render::sortBooksByIsbn($this->jsonBody->books); $this->assertEquals('9780544107717', $sortedBooks[0]->isbn); $this->assertEquals('9781616203535', $sortedBooks[4]->isbn); } /**

    * just throw this one in too */ public function testSortBooksByTitle() { $sortedBooks = Render::sortBooksByTitle($this->jsonBody->books); $this->assertEquals('The Carnival at Bray', $sortedBooks[0]->title); $this->assertEquals('Jackaby', $sortedBooks[4]->title); } }
  21. None
  22. Do I need CI?

  23. With the addition of CI, each test becomes a contract,

    between the feature and the maintainers of the application. It says, ‘This feature was important enough to have a test written for it. This contract forbids you from breaking said feature.’
  24. • One person: I have self discipline. • Two people:

    I know who to blame. • Three people: Things were fine until the snake came.
  25. Earn Sweet GitHub README badges!

  26. None
  27. None
  28. Dot the i’s and cross the t’s • Best practice

    really helps with CI setup • Now make the investment in Composer, PRS namespaces, and a phpunit.xml file • Build tool like ant helps glue together with vagrant (shell exec)
  29. <?xml version="1.0" encoding="UTF-8"?> <phpunit colors="true" stopOnFailure="false" syntaxCheck="false" verbose="true" convertErrorsToExceptions="true" convertNoticesToExceptions="true"

    convertWarningsToExceptions="true" processIsolation="false" bootstrap="tests/Sparefoot/bootstrap.php"> <logging> <log type="junit" target="junit.xml"/> <log type="json" target="results.json"/> <log type="coverage-html" target="codeCoverage" charset="UTF-8" yui="true" highlight="true" lowUpperBound="50" highLowerBound="80"/> </logging> <testsuites> <testsuite name="MyFoot_Test_Suite"> <directory>tests</directory> </testsuite> <testsuite name="MyFoot_SmokeTest"> <file>tests/Sparefoot/Myfoot/LoginTest.php</file> <file>tests/Sparefoot/Myfoot/Facility/DetailsTest.php</file> <file>tests/Sparefoot/Myfoot/Signup/ResidualSignupTest.php</file> <file>tests/Sparefoot/Myfoot/Settings/PaymentTest.php</file> <file>tests/Sparefoot/Myfoot/Statement/CpaDefaultsTest.php</file> <file>tests/Sparefoot/Myfoot/FacilityTest.php</file> <file>tests/Sparefoot/Myfoot/UsersTest.php</file>
  30. Game of Life code coverage https://github.com/matt-land/Game-of-Life http://localhost:63342/apps/Game-of-Life/codeCoverage/index.html

  31. Real Fake Data

  32. public function setUp() { $this->customer = self::mockCustomer(); } public static

    function mockCustomer() { //build up a test object $customer = new \stdClass(); $customer->name = uniqid('John Test'); $customer->email = uniqid('test').'@test.com'; $customer->address = rand(1,5000).' test street'; $cities = ['Roswell', 'Austin', 'Memphis', 'San Jose']; $customer->city = $cities[array_rand($cities)]; $states = ['NC', 'TX', 'VA', 'CT']; $customer->state = $states[array_rand($states)]; $customer->zip = rand(10000,70000); return $customer; } public function testUpdateCustomer() { return ; Pass & set FK relationships for related models
  33. */ public static function generateValidCreditCardNumber($leader = 4) { //set to

    amex 15 digit if prefix is 3. otherwise its 16 $totalDigits = (substr($leader, 0, 1) == 3) ? 14 : 15; $forwardCardDigits = str_split($leader); // build out the rest of the digits, except for checksum (last digit) for ($i = 0; $i < $totalDigits - strlen($leader); $i++) { $forwardCardDigits[] = rand(0,9); } $reverseCardDigits = array_reverse($forwardCardDigits); $checksum = 0; foreach ($reverseCardDigits as $key => $value) { if ($key % 2 === 0) { $value = 2 * $value; } if ($value > 9) { $checksum += $value - 9; } else { $checksum += $value; } } //get the least significant digit for the checksum $tmp = str_split($checksum); $lsd = (int) end($tmp); $checksumCompliment = $lsd === 0 ? 0 : 10 - $lsd; return implode(null, $forwardCardDigits) . $checksumCompliment; }
  34. Ice Cream Cones & Behavior Driven Development

  35. None
  36. None
  37. • Setup the environment in persistence • Load the UI

    • Manipulate the UI • Bounce the UI, because last time was just jQuery • Verify the side effects in persistence
  38. • Setup the environment in persistence • Load the UI

    • Manipulate the UI • Bounce the UI, because the first time was just jQuery • Verify the side effects in persistence Tech Debt The test is doing too much or test the service separately Because our logic is everywhere
  39. Intermittent Failure To Launch

  40. • 1:45pm - click build again • 2:00pm - notice

    haven’t passed yet today • 2:10pm - bounce selenium grid for luck • 2:30pm - in hip chat debug room for screenshots • 2:40pm - last chance to build and go green on time • 2:55pm - curse violently. Devbot mocks you • 3:10pm - bow out so other teams can deploy • 4:05pm - finally a green build. !Freeze • 6:45pm - DevOps back from kid’s swim practice. Successfully Deploy!
  41. Intermittent test failure smell • Blame the test • Blame

    the CI environment performance • Blame the JavaScript plugins (jQuery whatever) • Realize your code is doing too damn much. • Actually fails a large percentage of the time in prod.
  42. My Worst UI Test

  43. Watch a robot sign a contract

  44. All the brittle things • Send some emails on account

    creation • Step 3 in a sign up flow • Create an envelope via API (docusign) • Redirect via JS • Simulate Abandoning signup and coming back • ‘Sign’ using a third party service • Upload a PDF copy to a different API (salesforce) • Send a ticket to sales via API (zendesk) • Continue signup flow for 3 more steps
  45. All the things I learned • Selenium environment is brittle.

    Use a launcher script to minimize debug time. • Run and expose mysql on 3307 in host and vagrant. • Keep VM selenium ‘trapped’ • No manual config changes • No dependency on injected Envars
  46. Sticky Bits

  47. Sticky Bits I run tests both ways. In Vagrant (headless)

    and in host OS (interactive) Both are super useful to my workflow: • In the host OS for writing and debugging a broken condition • In the VM for running a suite before opening my pull request
  48. Sticky Bits (2) Multiple Ant targets (smoke vs full vs

    regression) balance • speed of quick feedback • full regression coverage • (don’t trust anyone) Code coverage plug-ins are invalid in the scope for integration level tests.
  49. • IFRAME->IFRAME is one way trip • Write a dedicated

    testsuite API for Datepicker() • Expected, Actual, Whoops-Message • Explicitly wait for _everything_ • Disable tests just like C++ by underscore prefixing • Fail as fast as possible (Temporarily disable long tests higher in the class) Sticky Bits (3)
  50. • Feature Toggle Iteration • Cookie up with 404 •

    add controller/action/behavior ids to the dom • Use identical IDs when FF’ing so the test can follow • Request/Factory/Response Mock Factory • Watch for _test nerving in pull requests Sticky Bits (4)
  51. Our launcher… https://github.com/matt-land/upstart

  52. Its not me, its the environment

  53. It Starts With One Test Code Samples: github.com/matt-land Slides: speakerdeck.com/mattland/

    linkedin.com/in/matthewland https://joind.in/13567