Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Advanced UI Testing

Advanced UI Testing

Dive deep into your application with Advanced UI Testing! After a brief background on exploratory and feature testing, lift the fog on how selenium testing actually works at the network layer, and discuss sample code from real automation tests. Topics include leveraging jquery to speed iteration in development, demystifying xPath, strategies for validating features behind multi-arm bandits, running headless, identifying race conditions, and work with iFrames.


Matt Land

May 14, 2015

More Decks by Matt Land

Other Decks in Technology


  1. Advanced UI Testing Code Samples: github.com/matt-land/uitest Slides: speakerdeck.com/mattland/advanced-ui-testing

  2. We help people get storage. Move in and move on.

  3. Matt Land CERTIFIED A R C H I T E

    CT 2 linkedin.com/in/matthewland
  4. None
  5. I Write Tests

  6. Testing Pyramid* manual tests Exploratory Feature * close enough Cost

  7. Exploratory Testing • Finding finding out our software actually works

    • Spending our time where we have the most to gain • Critical Path (sign in, sign up, payment settings) • Where users spend the bulk of time • Rewards creativity of the developer • Excellent for existing projects with no tests • Excellent for ‘new to you’ products
  8. Feature Testing • Behavior Driven Development • Test Driven Development

    • Test as contract: This feature was important enough to have a test written for it. This contract forbids you from breaking said feature.
  9. Some of Each • Exploring a site before creating a

    test plan • Exploring how things actually work • Become more Feature Test driven as a test suite matures
  10. Developing a New Test • What am I testing? •

    How can I capture information from UI or DB? • What is the current application flow in pseudo code? • Wait for everything. • $this->fail() at each step to see what was captured.
  11. * Time: 11:49 PM */ use \PHPUnit_Extensions_Selenium2TestCase_Keys as Keys; class

    SearchTest extends MyAbstract { public function testSearch() { //go to the google home page //wait for it to load //enter search terms in the query box //click search //verify i get results } } What I am testing Pseudo code *This would be the only test in this file. Each test gets its own.
  12. Standard Recipe Setup the environment in persistence Load the UI

    Manipulate the UI Bounce the UI, because last time was just jQuery Verify the side effects in persistence
  13. Process Overview PDO Driver :3306 phpunit php-1 runs a reflection

    Selenium :4444 Firefox :7055 Apache :80 :443 php-1 runs the app paratest junit.xml terminal output return code Ant/Maven Hudson/ Bamboo bash
  14. Setup the Environment • Inject directly into the storage engine

    OR use the UI • Use the factory pattern to setup complex conditions on each test run OR make permanent data • Shell accounts with semi randomized objects for your app
  15. Test & Interact • Assert things are on the page

    with ::byId, ::byTag, ::byXpath, ::byCssSelector, ::by ClassName, ::byName • Interact with ->click(), ->clear(), ->submit(), - >value, ::keys($keys) • Wait for something to happen (another assertion) • Continue with the next command
  16. Bouncing • Verify both PHP and JavaScript set the correct

    page state after an event • Perform the click to test JS first • Wait and assert • Reload the current url with self::url() • Make the same assertions
  17. Side Effects • Objects have changed since creation at the

    beginning of test • Query the storage engine at the end of the test for states not revealed in the UI • Should be moved to a (much faster) service test • But ‘Bad’ Tests > No Tests. Balance technical debt.
  18. */ use \PHPUnit_Extensions_Selenium2TestCase_Keys as Keys; class SearchTest extends MyAbstract {

    protected function waitForText($string) { $maxTime = time() + 15; //wait 15 sec max while (stripos(self::source(), '<body') === false) { usleep(333333); if (time() > $maxTime) { throw new \Exception('Timeout waiting for page to load'); } }; while (stripos(self::byTag('body')->text(), $string) === false) { usleep(333333); if (time() > $maxTime) { throw new \Exception('Timeout waiting for string ' . $string); } } return true; } public function testSearch() { self::url('/'); self::waitForText('Privacy'); self::byName('q')->value('ninjas or monkeys'); self::keys(Keys::ENTER); self::waitForText('results'); } } waiting waiting https://github.com/matt-land/uitest
  19. class SignupTest extends MyAbstract { public function testSignup() { //signup

    values $user = []; $names = ['elephant', 'chains', 'climbing', 'purplish', 'keen', 'exhaled', 'packe $user['email'] = uniqid('seleium').'@nowhere.net'; $user['password'] = sha1(uniqid('password')); $user['name'] = $names[array_rand($names)].$names[array_rand($names)].uniqid(); //register account self::url('https://www.tumblr.com/'); self::waitForText('Explore Tumblr'); self::byName("user[email]")->value($user['email']); self::byName("user[password]")->value($user['password']); self::byName("tumblelog[name]")->value($user['name']); self::byId('signup_forms_submit')->click(); //say we are of age self::waitForText('How old are you?'); self::byId("signup_age")->value(rand(18,69)); self::byId('signup_tos')->click(); self::byId('signup_forms_submit')->click(); //stop at captcha step self::waitForText('I am not a robot'); } } https://github.com/matt-land/uitest
  20. class ResidualSignupTest extends AbstractSignup { public function testResidualSignup() { $this->_doCode('R3X3');

    $this->byId('submit')->click(); $email = $this->_doUserInfo('testResidual'); $this->byId('submit')->click(); if ( Genesis_Service_Feature::isActive(Genesis_Entity_Feature::MYFOOT_RESIDUAL_ESIGNATURE)) { $this->_doDocuSign(); $this->byId('submit')->click(); } $this->_doTerms(); $this->byId('submit')->click(); $this->_doBilling($email); // skip billing info while netsuite is down if (\Genesis_Service_Feature::isActive(\Genesis_Entity_Feature::MYFOOT_TEST_NETSUITE_SANDBOX) $this->byId('submit')->click(); } else { $this->byId('do_this_later')->click(); } $this->_assertWeReachedWelcome(); } }
  21. Intro Over

  22. jQuery Exploring

  23. …speeds Test Development • ‘Fail’ faster • Compliments Selenium selectors

    almost 1:1 • Translates simply back and forth • Can be loaded into any page from the console • Allows running a test to the fail point, call sleep(), break (cntl-c). Switch to the browser console and continue.
  24. Loading into the Dom Open the console and paste: var

    jq = document.createElement('script');
 jq.src = "https://ajax.googleapis.com/ajax/libs/jquery/1/ jquery.min.js";
 doument.getElementsByTagName('head') [0].appendChild(jq);
 Test with version number $().jquery; https://github.com/matt-land/uitest
  25. Selector Compliments self::byId(‘idName’); self::byTag(‘body’); self::byName(‘email’); self::byCssSelector(‘.nav .link’); $(‘#idName’); $(‘body’); $(‘input[name=email]’);

    $(‘.nav .link’);
  26. Manipulator Compliments $element.click(); $element.val(‘new val’); $().trigger({type: ‘keypress’, which: $key, keyCode:

    $key}); $object->click(); $object->value(‘new val’); self::keys($keyPressed);
  27. Attribute Compliments $object->displayed(); $object->text(); $object->value(); $object->attribute(‘checked’); $element.is(‘:visible’); $element.text(); $element.val(); $element.attr(‘checked’);

  28. XPath Selectors //*[@id = 'bid-locale-picker']//*[contains(@class, ‘bid-by-zip')]/a //div[contains(@class, 'right') and contains(@class,

    'menu')]//div[contains(@class, 'ui') and contains(@class, 'dropdown') and contains(@class, ‘item')] //tr[@id={$unitNumber}]/td[contains(@class, 'availability-checkbox')]/ div[contains(@class, 'checkbox') and contains(@class, ‘ui')] //td[@id='action-$confirmationCode']//*[contains(@class,'confirm-button') and contains(@class, 'active')]
  29. Method Chaining self::byXPath( “//*[@id=‘site-fac-dropdown’]//*[@class='search']"); ") $(‘#site-fac-dropdown’).find(‘.search’); self::byId(‘menu’)->byTag(‘a’)->click(); $(‘#menu’).find(‘a’)->click();

  30. Notable Difference • jQuery returns an array of all matches

    on selector • Selenium returns first match. • Call the same Selenium selector again to iterate.
  31. Crazy While Loops

  32. */ use PHPUnit_Extensions_Selenium2TestCase; class WhileLoops extends MyAbstract { public function

    waitForElementDisplay($id) { $time = time(); while(! $this->byId($id)->displayed()) { usleep(10000); if (time() > $time + 15) { throw new \Exception('element ' . $id . ' failed to display in time'); } } } public function testSomething() { self::url('/myPage'); self::waitForText('Please enter how much rent the customer paid you during'); $value = round(rand(100,9999)/100, 2, PHP_ROUND_HALF_DOWN); self::byId('rent-tenant-other')->value($value); self::byId('edit-tenant-rent-submit')->click(); //wait for client side jquery event to finish while(self::byId('edit-tenant-rent-collected-title')->displayed()) { sleep(1); } $this->assertEquals('$' . $value, self::byId('rent')); } } https://github.com/matt-land/uitest
  33. Feature Toggles

  34. Feature Toggles Your new feature is crap if: • the

    toggle for it is off in the CI environment • CI tests never ran against it before it was flipped on in Production Surrender control of Feature Toggles in the testing environment to the tests. Deliberately consider what toggles a test should be setting for itself. Iterate over the variants in CI
  35. class ExampleFeatureToggleTest extends MyAbstract { public function testMyFeature() { try

    { ToggleService::setFeature('featureName', $enabled = 0); $this->_testMyFeature(); ToggleService::setFeature('featureName', $enabled = 1); $this->_testMyFeature(); } catch (\Exception $e) { $this->fail( "With featureName set to " . $enabled . PHP_EOL . $e->getMessage() . PHP_EOL . "Line:".$e->getLine() . PHP_EOL } } private function _testMyFeature() { /** * Do some UI test here */ } } https://github.com/matt-land/uitest
  36. Paratest • A wrapper to run multiple threads of phpunit

    parallel • Cut times by 50% - ? • add to you composer.json and spend a half hour grokking command line args • Must break all singleton dependencies in your app • Must create separate shell accounts to test auth areas
  37. Race Conditions self::byId(‘20340ff’)->value(); PHPUnit_Extensions_Selenium2TestCase_WebDriverException Element is no longer attached to

    the DOM For documentation on this error, please visit: http://seleniumhq.org/exceptions/ stale_element_reference.html PHPUnit_Extensions_Selenium2TestCase_WebDriverException Element not found in the cache - perhaps the page has changed since it was looked up For documentation on this error, please visit: http://seleniumhq.org/exceptions/ stale_element_reference.htmlt
  38. <?php namespace UiTest; /** * Created by IntelliJ IDEA. *

    User: blitzcat * Date: 5/14/15 * Time: 2:53 PM */ class InjectJavascriptTest extends MyAbstract { protected function _testFireJqueryUiDatepicker($inputId) { self::execute([ 'script' => '$("#' . $inputId . '").datepicker("show"); console.log("datepick 'args' => array() ]); } public function testInjectJavascript() { self::url('/'); self::waitForText('privacy'); self::_testFireJqueryUiDatepicker('mypicker'); } } Headless Considerations https://github.com/matt-land/uitest
  39. IFrames • Dealing with iframes means more than just one

    • One way trip down on traversal • Set root node null to exit
  40. class IframeExampleTest extends MyAbstract { public function testTraverseIframe() { //switch

    to the iframe self::frame($this->byId('docusign-iframe')); self::waitForText('Please review the documents below.'); //switch to child iframe self::frame($this->byId('nav-buttons')); // Do STUFF HERE //go back to the parent self::frame(null); //back into the outer iframe self::frame($this->byId('docusign-iframe')); //go back to the parent self::frame(null); } } https://github.com/matt-land/uitest
  41. UI Tests Are Expensive!

  42. Push Everything Down manual tests

  43. Put MVC controllers on a diet and move to MVCS.

    Automate testing even locally, especially as team grows. You have a node app with microservice endpoints? Its still MVC. Test the services endpoints separately. Push Everything Down
  44. • Don’t test everything. • Test whats valuable. • test

    > no test Code Samples: github.com/matt-land/uitest Slides: speakerdeck.com/mattland/advanced-ui-testing
  45. Bonus Round

  46. Metainfo • One test per class/file pattern • Define separate

    test suites in phpunit.xml • Fail on error, or don’t • Code coverage is useless • Smoke tests • This is NOT performance testing
  47. Headless Vagrant Strategy • Run on same ports in both

    OS • Including mysql • PDO connect by IP, not localhost • Vagrant virtual frame buffer and custom firefox