Share
Embed
Advanced UI Testing
Dive deep into your application with Advanced UI Testing! After a brief background on exploratory and feature testing, lift the fog on how selenium testing actually works at the network layer, and discuss sample code from real automation tests. Topics include leveraging jquery to speed iteration in development, demystifying xPath, strategies for validating features behind multi-arm bandits, running headless, identifying race conditions, and work with iFrames.
Matt Land
May 14, 2015
Transcript
47 slides
-
Matt Land CERTIFIED A R C H I T E
CT 2 linkedin.com/in/matthewland -
Exploratory Testing • Finding finding out our software actually works
• Spending our time where we have the most to gain • Critical Path (sign in, sign up, payment settings) • Where users spend the bulk of time • Rewards creativity of the developer • Excellent for existing projects with no tests • Excellent for ‘new to you’ products -
Feature Testing • Behavior Driven Development • Test Driven Development
• Test as contract: This feature was important enough to have a test written for it. This contract forbids you from breaking said feature. -
Some of Each • Exploring a site before creating a
test plan • Exploring how things actually work • Become more Feature Test driven as a test suite matures -
Developing a New Test • What am I testing? •
How can I capture information from UI or DB? • What is the current application flow in pseudo code? • Wait for everything. • $this->fail() at each step to see what was captured. -
* Time: 11:49 PM */ use \PHPUnit_Extensions_Selenium2TestCase_Keys as Keys; class
SearchTest extends MyAbstract { public function testSearch() { //go to the google home page //wait for it to load //enter search terms in the query box //click search //verify i get results } } What I am testing Pseudo code *This would be the only test in this file. Each test gets its own. -
Standard Recipe Setup the environment in persistence Load the UI
Manipulate the UI Bounce the UI, because last time was just jQuery Verify the side effects in persistence -
Process Overview PDO Driver :3306 phpunit php-1 runs a reflection
Selenium :4444 Firefox :7055 Apache :80 :443 php-1 runs the app paratest junit.xml terminal output return code Ant/Maven Hudson/ Bamboo bash -
Setup the Environment • Inject directly into the storage engine
OR use the UI • Use the factory pattern to setup complex conditions on each test run OR make permanent data • Shell accounts with semi randomized objects for your app -
Test & Interact • Assert things are on the page
with ::byId, ::byTag, ::byXpath, ::byCssSelector, ::by ClassName, ::byName • Interact with ->click(), ->clear(), ->submit(), - >value, ::keys($keys) • Wait for something to happen (another assertion) • Continue with the next command -
Bouncing • Verify both PHP and JavaScript set the correct
page state after an event • Perform the click to test JS first • Wait and assert • Reload the current url with self::url() • Make the same assertions -
Side Effects • Objects have changed since creation at the
beginning of test • Query the storage engine at the end of the test for states not revealed in the UI • Should be moved to a (much faster) service test • But ‘Bad’ Tests > No Tests. Balance technical debt. -
*/ use \PHPUnit_Extensions_Selenium2TestCase_Keys as Keys; class SearchTest extends MyAbstract {
protected function waitForText($string) { $maxTime = time() + 15; //wait 15 sec max while (stripos(self::source(), '<body') === false) { usleep(333333); if (time() > $maxTime) { throw new \Exception('Timeout waiting for page to load'); } }; while (stripos(self::byTag('body')->text(), $string) === false) { usleep(333333); if (time() > $maxTime) { throw new \Exception('Timeout waiting for string ' . $string); } } return true; } public function testSearch() { self::url('/'); self::waitForText('Privacy'); self::byName('q')->value('ninjas or monkeys'); self::keys(Keys::ENTER); self::waitForText('results'); } } waiting waiting https://github.com/matt-land/uitest -
class SignupTest extends MyAbstract { public function testSignup() { //signup
values $user = []; $names = ['elephant', 'chains', 'climbing', 'purplish', 'keen', 'exhaled', 'packe $user['email'] = uniqid('seleium').'@nowhere.net'; $user['password'] = sha1(uniqid('password')); $user['name'] = $names[array_rand($names)].$names[array_rand($names)].uniqid(); //register account self::url('https://www.tumblr.com/'); self::waitForText('Explore Tumblr'); self::byName("user[email]")->value($user['email']); self::byName("user[password]")->value($user['password']); self::byName("tumblelog[name]")->value($user['name']); self::byId('signup_forms_submit')->click(); //say we are of age self::waitForText('How old are you?'); self::byId("signup_age")->value(rand(18,69)); self::byId('signup_tos')->click(); self::byId('signup_forms_submit')->click(); //stop at captcha step self::waitForText('I am not a robot'); } } https://github.com/matt-land/uitest -
class ResidualSignupTest extends AbstractSignup { public function testResidualSignup() { $this->_doCode('R3X3');
$this->byId('submit')->click(); $email = $this->_doUserInfo('testResidual'); $this->byId('submit')->click(); if ( Genesis_Service_Feature::isActive(Genesis_Entity_Feature::MYFOOT_RESIDUAL_ESIGNATURE)) { $this->_doDocuSign(); $this->byId('submit')->click(); } $this->_doTerms(); $this->byId('submit')->click(); $this->_doBilling($email); // skip billing info while netsuite is down if (\Genesis_Service_Feature::isActive(\Genesis_Entity_Feature::MYFOOT_TEST_NETSUITE_SANDBOX) $this->byId('submit')->click(); } else { $this->byId('do_this_later')->click(); } $this->_assertWeReachedWelcome(); } } -
…speeds Test Development • ‘Fail’ faster • Compliments Selenium selectors
almost 1:1 • Translates simply back and forth • Can be loaded into any page from the console • Allows running a test to the fail point, call sleep(), break (cntl-c). Switch to the browser console and continue. -
Loading into the Dom Open the console and paste: var
jq = document.createElement('script'); jq.src = "https://ajax.googleapis.com/ajax/libs/jquery/1/ jquery.min.js"; doument.getElementsByTagName('head') [0].appendChild(jq); Test with version number $().jquery; https://github.com/matt-land/uitest -
Manipulator Compliments $element.click(); $element.val(‘new val’); $().trigger({type: ‘keypress’, which: $key, keyCode:
$key}); $object->click(); $object->value(‘new val’); self::keys($keyPressed); -
XPath Selectors //*[@id = 'bid-locale-picker']//*[contains(@class, ‘bid-by-zip')]/a //div[contains(@class, 'right') and contains(@class,
'menu')]//div[contains(@class, 'ui') and contains(@class, 'dropdown') and contains(@class, ‘item')] //tr[@id={$unitNumber}]/td[contains(@class, 'availability-checkbox')]/ div[contains(@class, 'checkbox') and contains(@class, ‘ui')] //td[@id='action-$confirmationCode']//*[contains(@class,'confirm-button') and contains(@class, 'active')] -
Notable Difference • jQuery returns an array of all matches
on selector • Selenium returns first match. • Call the same Selenium selector again to iterate. -
*/ use PHPUnit_Extensions_Selenium2TestCase; class WhileLoops extends MyAbstract { public function
waitForElementDisplay($id) { $time = time(); while(! $this->byId($id)->displayed()) { usleep(10000); if (time() > $time + 15) { throw new \Exception('element ' . $id . ' failed to display in time'); } } } public function testSomething() { self::url('/myPage'); self::waitForText('Please enter how much rent the customer paid you during'); $value = round(rand(100,9999)/100, 2, PHP_ROUND_HALF_DOWN); self::byId('rent-tenant-other')->value($value); self::byId('edit-tenant-rent-submit')->click(); //wait for client side jquery event to finish while(self::byId('edit-tenant-rent-collected-title')->displayed()) { sleep(1); } $this->assertEquals('$' . $value, self::byId('rent')); } } https://github.com/matt-land/uitest -
Feature Toggles Your new feature is crap if: • the
toggle for it is off in the CI environment • CI tests never ran against it before it was flipped on in Production Surrender control of Feature Toggles in the testing environment to the tests. Deliberately consider what toggles a test should be setting for itself. Iterate over the variants in CI -
class ExampleFeatureToggleTest extends MyAbstract { public function testMyFeature() { try
{ ToggleService::setFeature('featureName', $enabled = 0); $this->_testMyFeature(); ToggleService::setFeature('featureName', $enabled = 1); $this->_testMyFeature(); } catch (\Exception $e) { $this->fail( "With featureName set to " . $enabled . PHP_EOL . $e->getMessage() . PHP_EOL . "Line:".$e->getLine() . PHP_EOL } } private function _testMyFeature() { /** * Do some UI test here */ } } https://github.com/matt-land/uitest -
Paratest • A wrapper to run multiple threads of phpunit
parallel • Cut times by 50% - ? • add to you composer.json and spend a half hour grokking command line args • Must break all singleton dependencies in your app • Must create separate shell accounts to test auth areas -
Race Conditions self::byId(‘20340ff’)->value(); PHPUnit_Extensions_Selenium2TestCase_WebDriverException Element is no longer attached to
the DOM For documentation on this error, please visit: http://seleniumhq.org/exceptions/ stale_element_reference.html PHPUnit_Extensions_Selenium2TestCase_WebDriverException Element not found in the cache - perhaps the page has changed since it was looked up For documentation on this error, please visit: http://seleniumhq.org/exceptions/ stale_element_reference.htmlt -
<?php namespace UiTest; /** * Created by IntelliJ IDEA. *
User: blitzcat * Date: 5/14/15 * Time: 2:53 PM */ class InjectJavascriptTest extends MyAbstract { protected function _testFireJqueryUiDatepicker($inputId) { self::execute([ 'script' => '$("#' . $inputId . '").datepicker("show"); console.log("datepick 'args' => array() ]); } public function testInjectJavascript() { self::url('/'); self::waitForText('privacy'); self::_testFireJqueryUiDatepicker('mypicker'); } } Headless Considerations https://github.com/matt-land/uitest -
IFrames • Dealing with iframes means more than just one
• One way trip down on traversal • Set root node null to exit -
class IframeExampleTest extends MyAbstract { public function testTraverseIframe() { //switch
to the iframe self::frame($this->byId('docusign-iframe')); self::waitForText('Please review the documents below.'); //switch to child iframe self::frame($this->byId('nav-buttons')); // Do STUFF HERE //go back to the parent self::frame(null); //back into the outer iframe self::frame($this->byId('docusign-iframe')); //go back to the parent self::frame(null); } } https://github.com/matt-land/uitest -
Put MVC controllers on a diet and move to MVCS.
Automate testing even locally, especially as team grows. You have a node app with microservice endpoints? Its still MVC. Test the services endpoints separately. Push Everything Down -
• Don’t test everything. • Test whats valuable. • test
> no test Code Samples: github.com/matt-land/uitest Slides: speakerdeck.com/mattland/advanced-ui-testing -
Metainfo • One test per class/file pattern • Define separate
test suites in phpunit.xml • Fail on error, or don’t • Code coverage is useless • Smoke tests • This is NOT performance testing -
Headless Vagrant Strategy • Run on same ports in both
OS • Including mysql • PDO connect by IP, not localhost • Vagrant virtual frame buffer and custom firefox