Advanced UI Testing Code Samples: github.com/matt-land/uitest Slides: speakerdeck.com/mattland/advanced-ui-testing 

We help people get storage. Move in and move on. 

Matt Land CERTIFIED A R C H I T E CT 2 linkedin.com/in/matthewland 

No content

I Write Tests 

Testing Pyramid* manual tests Exploratory Feature * close enough Cost Time 

Exploratory Testing • Finding finding out our software actually works • Spending our time where we have the most to gain • Critical Path (sign in, sign up, payment settings) • Where users spend the bulk of time • Rewards creativity of the developer • Excellent for existing projects with no tests • Excellent for ‘new to you’ products 

Feature Testing • Behavior Driven Development • Test Driven Development • Test as contract: This feature was important enough to have a test written for it. This contract forbids you from breaking said feature. 

Some of Each • Exploring a site before creating a test plan • Exploring how things actually work • Become more Feature Test driven as a test suite matures 

Developing a New Test • What am I testing? • How can I capture information from UI or DB? • What is the current application flow in pseudo code? • Wait for everything. • $this->fail() at each step to see what was captured. 

* Time: 11:49 PM */ use \PHPUnit_Extensions_Selenium2TestCase_Keys as Keys; class SearchTest extends MyAbstract { public function testSearch() { //go to the google home page //wait for it to load //enter search terms in the query box //click search //verify i get results } } What I am testing Pseudo code *This would be the only test in this file. Each test gets its own. 

Standard Recipe Setup the environment in persistence Load the UI Manipulate the UI Bounce the UI, because last time was just jQuery Verify the side effects in persistence 

Process Overview PDO Driver :3306 phpunit php-1 runs a reflection Selenium :4444 Firefox :7055 Apache :80 :443 php-1 runs the app paratest junit.xml terminal output return code Ant/Maven Hudson/ Bamboo bash 

Setup the Environment • Inject directly into the storage engine OR use the UI • Use the factory pattern to setup complex conditions on each test run OR make permanent data • Shell accounts with semi randomized objects for your app 

Test & Interact • Assert things are on the page with ::byId, ::byTag, ::byXpath, ::byCssSelector, ::by ClassName, ::byName • Interact with ->click(), ->clear(), ->submit(), - >value, ::keys($keys) • Wait for something to happen (another assertion) • Continue with the next command 

Bouncing • Verify both PHP and JavaScript set the correct page state after an event • Perform the click to test JS first • Wait and assert • Reload the current url with self::url() • Make the same assertions 

Side Effects • Objects have changed since creation at the beginning of test • Query the storage engine at the end of the test for states not revealed in the UI • Should be moved to a (much faster) service test • But ‘Bad’ Tests > No Tests. Balance technical debt. 

*/ use \PHPUnit_Extensions_Selenium2TestCase_Keys as Keys; class SearchTest extends MyAbstract { protected function waitForText($string) { $maxTime = time() + 15; //wait 15 sec max while (stripos(self::source(), ' $maxTime) { throw new \Exception('Timeout waiting for page to load'); } }; while (stripos(self::byTag('body')->text(), $string) === false) { usleep(333333); if (time() > $maxTime) { throw new \Exception('Timeout waiting for string ' . $string); } } return true; } public function testSearch() { self::url('/'); self::waitForText('Privacy'); self::byName('q')->value('ninjas or monkeys'); self::keys(Keys::ENTER); self::waitForText('results'); } } waiting waiting https://github.com/matt-land/uitest 

class SignupTest extends MyAbstract { public function testSignup() { //signup values $user = []; $names = ['elephant', 'chains', 'climbing', 'purplish', 'keen', 'exhaled', 'packe $user['email'] = uniqid('seleium').'@nowhere.net'; $user['password'] = sha1(uniqid('password')); $user['name'] = $names[array_rand($names)].$names[array_rand($names)].uniqid(); //register account self::url('https://www.tumblr.com/'); self::waitForText('Explore Tumblr'); self::byName("user[email]")->value($user['email']); self::byName("user[password]")->value($user['password']); self::byName("tumblelog[name]")->value($user['name']); self::byId('signup_forms_submit')->click(); //say we are of age self::waitForText('How old are you?'); self::byId("signup_age")->value(rand(18,69)); self::byId('signup_tos')->click(); self::byId('signup_forms_submit')->click(); //stop at captcha step self::waitForText('I am not a robot'); } } https://github.com/matt-land/uitest 

class ResidualSignupTest extends AbstractSignup { public function testResidualSignup() { $this->_doCode('R3X3'); $this->byId('submit')->click(); $email = $this->_doUserInfo('testResidual'); $this->byId('submit')->click(); if ( Genesis_Service_Feature::isActive(Genesis_Entity_Feature::MYFOOT_RESIDUAL_ESIGNATURE)) { $this->_doDocuSign(); $this->byId('submit')->click(); } $this->_doTerms(); $this->byId('submit')->click(); $this->_doBilling($email); // skip billing info while netsuite is down if (\Genesis_Service_Feature::isActive(\Genesis_Entity_Feature::MYFOOT_TEST_NETSUITE_SANDBOX) $this->byId('submit')->click(); } else { $this->byId('do_this_later')->click(); } $this->_assertWeReachedWelcome(); } } 

Intro Over 

jQuery Exploring 

…speeds Test Development • ‘Fail’ faster • Compliments Selenium selectors almost 1:1 • Translates simply back and forth • Can be loaded into any page from the console • Allows running a test to the fail point, call sleep(), break (cntl-c). Switch to the browser console and continue. 

Loading into the Dom Open the console and paste: var jq = document.createElement('script');
 jq.src = "https://ajax.googleapis.com/ajax/libs/jquery/1/ jquery.min.js";
 doument.getElementsByTagName('head') [0].appendChild(jq);
 Test with version number $().jquery; https://github.com/matt-land/uitest 

Selector Compliments self::byId(‘idName’); self::byTag(‘body’); self::byName(‘email’); self::byCssSelector(‘.nav .link’); $(‘#idName’); $(‘body’); $(‘input[name=email]’); $(‘.nav .link’); 

Manipulator Compliments $element.click(); $element.val(‘new val’); $().trigger({type: ‘keypress’, which: $key, keyCode: $key}); $object->click(); $object->value(‘new val’); self::keys($keyPressed); 

Attribute Compliments $object->displayed(); $object->text(); $object->value(); $object->attribute(‘checked’); $element.is(‘:visible’); $element.text(); $element.val(); $element.attr(‘checked’); 

XPath Selectors //*[@id = 'bid-locale-picker']//*[contains(@class, ‘bid-by-zip')]/a //div[contains(@class, 'right') and contains(@class, 'menu')]//div[contains(@class, 'ui') and contains(@class, 'dropdown') and contains(@class, ‘item')] //tr[@id={$unitNumber}]/td[contains(@class, 'availability-checkbox')]/ div[contains(@class, 'checkbox') and contains(@class, ‘ui')] //td[@id='action-$confirmationCode']//*[contains(@class,'confirm-button') and contains(@class, 'active')] 

Method Chaining self::byXPath( “//*[@id=‘site-fac-dropdown’]//*[@class='search']"); ") $(‘#site-fac-dropdown’).find(‘.search’); self::byId(‘menu’)->byTag(‘a’)->click(); $(‘#menu’).find(‘a’)->click(); 

Notable Difference • jQuery returns an array of all matches on selector • Selenium returns first match. • Call the same Selenium selector again to iterate. 

Crazy While Loops 

*/ use PHPUnit_Extensions_Selenium2TestCase; class WhileLoops extends MyAbstract { public function waitForElementDisplay($id) { $time = time(); while(! $this->byId($id)->displayed()) { usleep(10000); if (time() > $time + 15) { throw new \Exception('element ' . $id . ' failed to display in time'); } } } public function testSomething() { self::url('/myPage'); self::waitForText('Please enter how much rent the customer paid you during'); $value = round(rand(100,9999)/100, 2, PHP_ROUND_HALF_DOWN); self::byId('rent-tenant-other')->value($value); self::byId('edit-tenant-rent-submit')->click(); //wait for client side jquery event to finish while(self::byId('edit-tenant-rent-collected-title')->displayed()) { sleep(1); } $this->assertEquals('$' . $value, self::byId('rent')); } } https://github.com/matt-land/uitest 

Feature Toggles 

Feature Toggles Your new feature is crap if: • the toggle for it is off in the CI environment • CI tests never ran against it before it was flipped on in Production Surrender control of Feature Toggles in the testing environment to the tests. Deliberately consider what toggles a test should be setting for itself. Iterate over the variants in CI 

class ExampleFeatureToggleTest extends MyAbstract { public function testMyFeature() { try { ToggleService::setFeature('featureName', $enabled = 0); $this->_testMyFeature(); ToggleService::setFeature('featureName', $enabled = 1); $this->_testMyFeature(); } catch (\Exception $e) { $this->fail( "With featureName set to " . $enabled . PHP_EOL . $e->getMessage() . PHP_EOL . "Line:".$e->getLine() . PHP_EOL } } private function _testMyFeature() { /** * Do some UI test here */ } } https://github.com/matt-land/uitest 

Paratest • A wrapper to run multiple threads of phpunit parallel • Cut times by 50% - ? • add to you composer.json and spend a half hour grokking command line args • Must break all singleton dependencies in your app • Must create separate shell accounts to test auth areas 

Race Conditions self::byId(‘20340ff’)->value(); PHPUnit_Extensions_Selenium2TestCase_WebDriverException Element is no longer attached to the DOM For documentation on this error, please visit: http://seleniumhq.org/exceptions/ stale_element_reference.html PHPUnit_Extensions_Selenium2TestCase_WebDriverException Element not found in the cache - perhaps the page has changed since it was looked up For documentation on this error, please visit: http://seleniumhq.org/exceptions/ stale_element_reference.htmlt 

'$("#' . $inputId . '").datepicker("show"); console.log("datepick 'args' => array() ]); } public function testInjectJavascript() { self::url('/'); self::waitForText('privacy'); self::_testFireJqueryUiDatepicker('mypicker'); } } Headless Considerations https://github.com/matt-land/uitest 

IFrames • Dealing with iframes means more than just one • One way trip down on traversal • Set root node null to exit 

class IframeExampleTest extends MyAbstract { public function testTraverseIframe() { //switch to the iframe self::frame($this->byId('docusign-iframe')); self::waitForText('Please review the documents below.'); //switch to child iframe self::frame($this->byId('nav-buttons')); // Do STUFF HERE //go back to the parent self::frame(null); //back into the outer iframe self::frame($this->byId('docusign-iframe')); //go back to the parent self::frame(null); } } https://github.com/matt-land/uitest 

UI Tests Are Expensive! 

Push Everything Down manual tests 

Put MVC controllers on a diet and move to MVCS. Automate testing even locally, especially as team grows. You have a node app with microservice endpoints? Its still MVC. Test the services endpoints separately. Push Everything Down 

• Don’t test everything. • Test whats valuable. • test > no test Code Samples: github.com/matt-land/uitest Slides: speakerdeck.com/mattland/advanced-ui-testing 

Bonus Round 

Metainfo • One test per class/file pattern • Define separate test suites in phpunit.xml • Fail on error, or don’t • Code coverage is useless • Smoke tests • This is NOT performance testing 

Headless Vagrant Strategy • Run on same ports in both OS • Including mysql • PDO connect by IP, not localhost • Vagrant virtual frame buffer and custom firefox