Testing the untestable Lone Star PHP 2015 

We help people get storage. Move in and move on. 

Matt Land Code Samples: github.com/matt-land Slides: speakerdeck.com/mattland/ linkedin.com/in/matthewland CERTIFIED A R C H I T E CT 2 https://joind.in/13567 

Testing the Untestable Create the World, and Poke it With a Stick Do I Need CI? Real Fake Data Ice Cream Cones and BDD Intermittent Failure To Launch My Worst UI Test Sticky Bits Its Not Me, Its the Environment 

(Day 17, 2:45 at SpareFoot) “It doesn’t matter if it works or not. Just make the tests pass so we can deploy.” 

Poke it with a stick 

The First Test Noise: • PSR Autoloaders • Composer • Namespaces • Coding Standards • Continuous Integration • Git & Repositories • Best Practices • Mocks • … Not Noise: • Writing Tests • Passing Tests 

No content

PROTECTED PRIVATE PRIVATE PRIVATE PRIVATE PROTECTED PRIVATE PRIVATE PROTECTED PRIVATE 

No content

• Sample application: • http://localhost:63342/apps/test-samples/index.php? sort=author • https://github.com/matt-land/test-samples 

'Young Adult', 'records' => 5, 'meta' => [ 'next' => '/whatever?q=345w5', 'prev' => '/whatever?q=92342' ], 'books' => [ [ 'title' => 'The Crossover' , 'author' => 'Kwame Alexander', 'publisher' => 'Houghton', 'price' => '16.99', 'isbn' => '9780544107717' ], [ 'title' => 'The Carnival at Bray' , 'author' => 'Jessie Ann Foley', 'publisher' => 'Elephant Rock', 'price' => '12.95', 'isbn' => '97809895115597' ], [ 'title' => 'I\'ll Give You the Sun' , 'author' => 'Jandy Nelson', 'publisher' => 'Dial', 'price' => '17.99', 'publisher' => 'Houghton', 'price' => '16.99', 'isbn' => '9780544107717' ], [ 'title' => 'The Carnival at Bray' , 'author' => 'Jessie Ann Foley', 'publisher' => 'Elephant Rock', 'price' => '12.95', 'isbn' => '97809895115597' ], [ 'title' => 'I\'ll Give You the Sun' , 'author' => 'Jandy Nelson', 'publisher' => 'Dial', 'price' => '17.99', 'isbn' => '9780803734968' ], [ 'title' => 'Jackaby' , 'author' => 'William Ritter', 'publisher' => 'William Ritter', 'price' => '16.95', 'isbn' => '9781616203535' ], [ 'title' => 'The Gospel of Winter' , 'author' => 'Brendan Kiely', 'publisher' => 'Simon & Schuster/Margaret K. McElderry', 'price' => '17.99', 'isbn' => '9781442484894' ], ] ]); $render = new Render($dsnBooks); $sortOrder = isset($_REQUEST['sort']) ? $_REQUEST['sort'] : null; echo $render->toHtml($sortOrder); Continues 

Total Records: ' . $this->data->records . '

Prev Next'; return $html; } private function sortBooksByAuthor() { //usort is going to modify $books by reference, //so lets copy it to prevent side effects $sortedBooks = $this->books; usort($sortedBooks, function($a, $b) { /** * sort by author last name, so first get the last name * or last segment of the name. Will not work with JR/III suffixes */ $authorA = explode(" ", trim($a->author)); $authorB = explode(" ", trim($b->author)); return strcmp($authorA[count($authorA)-1], $authorB[count($authorB)-1]); }); return $sortedBooks; } private static function sortBooksByPrice($books) { //usort is going to modify $books by reference, //so lets copy it to prevent side effects 

{ $renderObj = new Render(json_encode($this->jsonBody)); $rendering = $renderObj->toHtml(); $this->fail('This is complex: ' . PHP_EOL . $rendering . " Lets test the methods in } /** * reflecting a method, where we have to set the value in the class first */ public function testSortBooksByAuthor() { $reflectedClass = new ReflectionClass('\\Samples\\Render'); $realClass = $reflectedClass->newInstance(); $booksProperty = $reflectedClass->getProperty('books'); // $rea $booksProperty->setAccessible(1); // $rea $booksProperty->setValue($realClass, $this->jsonBody->books); // $rea $sortBooksByAuthorMethod = $reflectedClass->getMethod('sortBooksByAuthor'); //$real $sortBooksByAuthorMethod->setAccessible(true); //$real $sortedBooks = $sortBooksByAuthorMethod->invoke($realClass); //$real $this->assertEquals('Kwame Alexander', $sortedBooks[0]->author); $this->assertEquals('William Ritter', $sortedBooks[4]->author); } /** * this time written statically/no object oriented dependency 

No content

return strcmp($authorA[count($authorA)-1], $authorB[count($authorB)- }); return $sortedBooks; } private static function sortBooksByPrice($books) { //usort is going to modify $books by reference, //so lets copy it to prevent side effects $sortedBooks = $books; usort($sortedBooks, function($a, $b) { /** * compare decimal price */ return $a->price >= $b->price; }); return $sortedBooks; } public static function sortBooksByIsbn($books) { //usort is going to modify $books by reference, //so lets copy it to prevent side effects $sortedBooks = $books; usort($sortedBooks, function($a, $b) { /** * sort by ISBN string which might being with zero Changed to static Parameter is passed in Gave up $this, but still have self:: 

$sortBooksByAuthorMethod->setAccessible(true); //$realC $sortedBooks = $sortBooksByAuthorMethod->invoke($realClass); //$realC $this->assertEquals('Kwame Alexander', $sortedBooks[0]->author); $this->assertEquals('William Ritter', $sortedBooks[4]->author); } /** * this time written statically/no object oriented dependency * Much easier */ public function testSortBooksByPrice() { $realClass = new Render(); $sortBooksByPriceMethod = new ReflectionMethod($realClass, 'sortBooksByPrice'); $sortBooksByPriceMethod->setAccessible(true); $sortedBooks = $sortBooksByPriceMethod->invoke($realClass, $this->jsonBody->books); $this->assertEquals(12.95, $sortedBooks[0]->price); $this->assertEquals(17.99, $sortedBooks[4]->price); } /** * the easiest yet. now that its static and public, we can just use it */ public function testSortBooksByISBN() 

No content

*/ return strcmp($a->isbn, $b->isbn); }); return $sortedBooks; } public static function sortBooksByTitle($books) { //usort is going to modify $books by reference, //so lets copy it to prevent side effects $sortedBooks = $books; usort($sortedBooks, function($a, $b) { /** * remove articles and sort titles */ return strcmp( str_ireplace(["a ", "an ", "the "], '', $a->title), str_ireplace(["a ", "an ", "the "], '', $b->title) ); }); return $sortedBooks; } } Changed to public No side effect risk 

{ $sortedBooks = Render::sortBooksByIsbn($this->jsonBody->books); $this->assertEquals('9780544107717', $sortedBooks[0]->isbn); $this->assertEquals('9781616203535', $sortedBooks[4]->isbn); } /** * just throw this one in too */ public function testSortBooksByTitle() { $sortedBooks = Render::sortBooksByTitle($this->jsonBody->books); $this->assertEquals('The Carnival at Bray', $sortedBooks[0]->title); $this->assertEquals('Jackaby', $sortedBooks[4]->title); } } 

No content

Do I need CI? 

With the addition of CI, each test becomes a contract, between the feature and the maintainers of the application. It says, ‘This feature was important enough to have a test written for it. This contract forbids you from breaking said feature.’ 

• One person: I have self discipline. • Two people: I know who to blame. • Three people: Things were fine until the snake came. 

Earn Sweet GitHub README badges! 

No content

No content

Dot the i’s and cross the t’s • Best practice really helps with CI setup • Now make the investment in Composer, PRS namespaces, and a phpunit.xml file • Build tool like ant helps glue together with vagrant (shell exec) 

tests tests/Sparefoot/Myfoot/LoginTest.php tests/Sparefoot/Myfoot/Facility/DetailsTest.php tests/Sparefoot/Myfoot/Signup/ResidualSignupTest.php tests/Sparefoot/Myfoot/Settings/PaymentTest.php tests/Sparefoot/Myfoot/Statement/CpaDefaultsTest.php tests/Sparefoot/Myfoot/FacilityTest.php tests/Sparefoot/Myfoot/UsersTest.php 

Game of Life code coverage https://github.com/matt-land/Game-of-Life http://localhost:63342/apps/Game-of-Life/codeCoverage/index.html 

Real Fake Data 

public function setUp() { $this->customer = self::mockCustomer(); } public static function mockCustomer() { //build up a test object $customer = new \stdClass(); $customer->name = uniqid('John Test'); $customer->email = uniqid('test').'@test.com'; $customer->address = rand(1,5000).' test street'; $cities = ['Roswell', 'Austin', 'Memphis', 'San Jose']; $customer->city = $cities[array_rand($cities)]; $states = ['NC', 'TX', 'VA', 'CT']; $customer->state = $states[array_rand($states)]; $customer->zip = rand(10000,70000); return $customer; } public function testUpdateCustomer() { return ; Pass & set FK relationships for related models 

*/ public static function generateValidCreditCardNumber($leader = 4) { //set to amex 15 digit if prefix is 3. otherwise its 16 $totalDigits = (substr($leader, 0, 1) == 3) ? 14 : 15; $forwardCardDigits = str_split($leader); // build out the rest of the digits, except for checksum (last digit) for ($i = 0; $i < $totalDigits - strlen($leader); $i++) { $forwardCardDigits[] = rand(0,9); } $reverseCardDigits = array_reverse($forwardCardDigits); $checksum = 0; foreach ($reverseCardDigits as $key => $value) { if ($key % 2 === 0) { $value = 2 * $value; } if ($value > 9) { $checksum += $value - 9; } else { $checksum += $value; } } //get the least significant digit for the checksum $tmp = str_split($checksum); $lsd = (int) end($tmp); $checksumCompliment = $lsd === 0 ? 0 : 10 - $lsd; return implode(null, $forwardCardDigits) . $checksumCompliment; } 

Ice Cream Cones & Behavior Driven Development 

No content

No content

• Setup the environment in persistence • Load the UI • Manipulate the UI • Bounce the UI, because last time was just jQuery • Verify the side effects in persistence 

• Setup the environment in persistence • Load the UI • Manipulate the UI • Bounce the UI, because the first time was just jQuery • Verify the side effects in persistence Tech Debt The test is doing too much or test the service separately Because our logic is everywhere 

Intermittent Failure To Launch 

• 1:45pm - click build again • 2:00pm - notice haven’t passed yet today • 2:10pm - bounce selenium grid for luck • 2:30pm - in hip chat debug room for screenshots • 2:40pm - last chance to build and go green on time • 2:55pm - curse violently. Devbot mocks you • 3:10pm - bow out so other teams can deploy • 4:05pm - finally a green build. !Freeze • 6:45pm - DevOps back from kid’s swim practice. Successfully Deploy! 

Intermittent test failure smell • Blame the test • Blame the CI environment performance • Blame the JavaScript plugins (jQuery whatever) • Realize your code is doing too damn much. • Actually fails a large percentage of the time in prod. 

My Worst UI Test 

Watch a robot sign a contract 

All the brittle things • Send some emails on account creation • Step 3 in a sign up flow • Create an envelope via API (docusign) • Redirect via JS • Simulate Abandoning signup and coming back • ‘Sign’ using a third party service • Upload a PDF copy to a different API (salesforce) • Send a ticket to sales via API (zendesk) • Continue signup flow for 3 more steps 

All the things I learned • Selenium environment is brittle. Use a launcher script to minimize debug time. • Run and expose mysql on 3307 in host and vagrant. • Keep VM selenium ‘trapped’ • No manual config changes • No dependency on injected Envars 

Sticky Bits 

Sticky Bits I run tests both ways. In Vagrant (headless) and in host OS (interactive) Both are super useful to my workflow: • In the host OS for writing and debugging a broken condition • In the VM for running a suite before opening my pull request 

Sticky Bits (2) Multiple Ant targets (smoke vs full vs regression) balance • speed of quick feedback • full regression coverage • (don’t trust anyone) Code coverage plug-ins are invalid in the scope for integration level tests. 

• IFRAME->IFRAME is one way trip • Write a dedicated testsuite API for Datepicker() • Expected, Actual, Whoops-Message • Explicitly wait for _everything_ • Disable tests just like C++ by underscore prefixing • Fail as fast as possible (Temporarily disable long tests higher in the class) Sticky Bits (3) 

• Feature Toggle Iteration • Cookie up with 404 • add controller/action/behavior ids to the dom • Use identical IDs when FF’ing so the test can follow • Request/Factory/Response Mock Factory • Watch for _test nerving in pull requests Sticky Bits (4) 

Our launcher… https://github.com/matt-land/upstart 

Its not me, its the environment 

It Starts With One Test Code Samples: github.com/matt-land Slides: speakerdeck.com/mattland/ linkedin.com/in/matthewland https://joind.in/13567