Simplifying Test Automation of Web Applications Using Visual Inspection and Page Objects

Delivered at StarWest conference in Anaheim, CA on Wednesday October 4th, 2017.

- Locate elements on page and create meaningful locators
- Dynamically create page objects to create robust and maintainable tests
- Create end-to-end test automation by integrating with Jenkins and JIRA

Transcript

1. Simplifying Test Automation of Web Applications Using Visual Inspection and Page Objects Daniel Gallo Senior Solutions Architect Sencha, Inc. dan@sencha.com
2. Overview • The problems and complexities of testing dynamic web apps • Overview of Sencha Test and how it helps • Locating elements on the page and generating meaningful locators • Creating page objects for robust and maintainable tests • Writing end-to-end tests • Automating tests by integrating with Jenkins and JIRA
3. Problems and Complexities of Testing Web Apps
4. Permissive Open Source Test Frameworks • A typical web testing project would utilize multiple solutions • Wikipedia: • 40+ listed on the ‘List of Unit Testing Frameworks’ • Selenium • Automates browsers, and works in tandem with all test frameworks • Limited on its own though
5. Open Source Testing Pain Points • Customers have to spend a lot of time to get the barebones structure in place • Reports are available only in HTML files at best. Customers have to do the heavy lifting to render it in a more readable format • Don’t have capabilities such as visual screen comparison • Lots of choices • Dependency on 3rd party contributors
6. Dynamic Web Apps • JavaScript web app frameworks can abstract the HTML layer away from the developer • This can mean the rendered HTML differs across browsers and form factors • Because of these dynamic web apps, IDs of elements may change between each test run • There needs to be a more stable way of referencing components on a page
7. How Sencha Test can help
8. Sencha Test: Cross-Browser Unit and End-to-End Functional Testing for Ext JS Apps Accelerate the testing of complex web applications for improved quality and enhanced developer productivity • Unit test and end-to-end functional testing • Author and execute tests • Record and playback tests • Review and archive results • Review and report on code coverage
9. Sencha Test Components Test Studio Test Authoring Test Runner Test Execution Test CLI Test Automation Test Archiver Results Reporting And Archiving
10. Test Authoring: Sencha Test Studio • Write unit and end-to-end functional tests in JavaScript (Jasmine) by using built-in test editor, or in a separate IDE • Store tests in your SCM • Quickly and easily write and execute tests while coding • See all test results in a grid/matrix • Access the Inspect utility, to generate reliable locators
11. Test Execution: Test Runner • Run select tests on any or all browsers on local machine or browser farm • Dramatically reduce testing time by executing tests simultaneously on multiple browsers through integration with leading browser farms such as Sauce Labs and BrowserStack
12. Test Automation: Sencha Test CLI (Command Line Interface) • Maximize the efficiency of your testing through automated test runs • Launch tests within the CI system as soon as application changes and tests are checked into the source control repository • Out of the box integration with TeamCity and Jenkins code test review
13. Test Execution: Test Archiver • Review results from automated and manual test runs • Summary level results & drill into the detail of failed tests • Out of the box integration with Istanbul enables users to identify and rectify code coverage gaps • View % coverage at the Statements, Branches, Functions and Lines levels
14. Visual Screen Comparison: Test Archiver • Minimize the time spent verifying images on hundreds of screens • Screenshot tests are flagged as failures if there’s a difference between baseline and latest screenshot • Review images that are captured from previous test runs and compare to each subsequent test run
15. Out Of The Box Integrations Jasmine Test Framework Browser Farms Jenkins / TeamCity Istanbul Code Coverage Istanbul
16. Demo Sencha Test Studio Sample App
17. How Tests Are Run Two Methods
18. Sencha Test: How tests are run • Two ways of executing tests: • In-browser mode – web app is proxied through Sencha Test • WebDriver mode – browser directly accesses app
19. Sencha Test: How tests are run (Proxy) • Sencha Test Studio will include the Jasmine library inside of a customized index.html page, along with references to your app’s code files and test files • Tests can then be executed in the context of that page • Sencha Test hosts the app using its own built-in reverse proxy to facilitate all requests to the app • So this app URL: • Will show as this URL in the browser: https://local.sencha.com/MyApp/ http://127.0.0.1:8700/MyApp/?orionAgentId=1
20. Sencha Test: How tests are run (WebDriver) • Sencha Test Studio will leverage WebDriver to establish a connection to the browser • Browser launches the application directly • Tests run separately from the browser • Requires configuration of WebDriver add-on for each browser • This approach supports multi-page app testing (navigability between pages during test runs)
21. Demo WebDriver scenario
22. Futures API
23. Make asynchronous tests simple and reliable (like synchronous tests)
24. Sencha Test - Futures API • Testing dynamic web apps is difficult, because tests may try and reference elements in the page before they’re fully rendered, or have data • Many Integration and App tests are asynchronous. They wait on: • Loading data • Rendering • Animations • Layouts • Asynchronous tests • Harder to write • Harder to maintain • Sometimes fail sporadically (race conditions)
25. A future is an object used to describe a sequence of actions to take in the future.
26. Futures API – Sencha Ext JS Example ST.grid('#mygrid') // locate the grid .rowAt(1000) // interact with a row .cell('firstName') // now with a cell .reveal() // scroll into view .click(10, 10); // click at offset
27. Futures API – Generic HTML Table Example ST.table('@mytable') // locate the table .rowAt(1000) // interact with a row .cellAt(2) // now with a cell .reveal() // scroll into view .click(); // click the cell
28. Sencha Test - Futures API • Futures API enables chainable methods • These methods are added to an internal queue, and processed in order • Won’t proceed until the previous method succeeds • If any of the methods fails, it causes the containing test/spec to fail, along with an appropriate error ST.grid('#mygrid') // locate the grid .rowAt(1000) // interact with a row .cell('firstName') // now with a cell .reveal() // scroll into view .click(10, 10); // click at offset
29. Futures: Locator • Futures are fetched using ST.Locator selector syntax • At-path • Xpath • DOM Query • Component Query • Composite query (component query plus a DOM selector) • You can also run ST.Locator.find() directly, in which case a DOM element is returned
30. Futures: Locator // At-path ST.Locator.find('@businessdetail-1025'); // Xpath ST.Locator.find('//div[@id="businessdetail-1025"]'); // DOM query ST.Locator.find('>>#businessdetail-1025 img'); // Component query -- returns the component's <div> ST.Locator.find('businessdetail'); // Combined component query plus DOM query ST.Locator.find('businessdetail => img');
31. Inspect Elements and Create Page Objects
32. Sencha Test – Inspect • Sencha Test features an Inspect tool, to help inspect elements in the page, and generate stable locators for them • These locators can either be inserted directly in to the test, along with the applicable API, or added to a reusable Page Object
33. Sencha Test – Page Objects • A Page Object is simply an object that contains a central list of all the locators required by the test suites • Rather than duplicate usage of locators in multiple tests, across multiple test suites, a Page Object acts as a central lookup for these locators • Page Objects can either be defined directly in the test suite as a simple JavaScript object, or added as a separate JS file for use across all test suites in the scenario
34. Demo Inspect Elements Page Objects Run Tests
35. Automating Tests
36. Automating Tests - Jenkins • Sencha Test features a Command Line utility • This helps to automate test runs, by enabling integration with Jenkins and TeamCity, or any other CI environment that allows custom shell scripts • Sencha Test CLI outputs the results to Text and JUnit formats that can be read by the CI environment
37. Automating Tests - JIRA • When tests fail, these failures can be easily pushed to JIRA as new issues • After a test run, the results can be reviewed, and all issues can be pushed to JIRA – you choose which project to publish the issues • For screenshot failures, the screenshots associated with the failure are attached to the JIRA ticket • Browser/OS/Version details are also logged against the ticket • If needed, certain issues can be excluded from the push to JIRA
38. Demo Automating Tests - Jenkins and JIRA Integration
39. Tip: Browser Farms
40. Browser Farms • If you don’t have the infrastructure internally, an external browser farm can be a great choice • Sauce Labs is a good example • Can select multiple combinations of Operating System, Browser, and Browser Version • Instances are quickly started on Sauce Labs infrastructure when you commence a test run • No need to maintain lots of different internal VMs
41. Demo Browser Farms
42. Tip: Dealing with data
43. Dealing with data • For any test, you need a known starting state • For screenshot comparison to work effectively, the same base data is required on each test run • In the case of end-to-end testing, you often need the database to be in a known initial state • You could have a backend service that initializes the database, and the test suite would call that before running any tests • Another technique is to define hardcoded data
44. Dealing with data: Ext.ux.ajax.SimManager • Sencha Ext JS provides a SimManager class that enables a way to serve hardcoded data for the URLs used by your proxies • Data is provided by Ext.ux.ajax.Simlet objects • Each defined URL maps to a Simlet config object containing the hardcoded data • Requires the ux package (require this in app.json) Ext.ux.ajax.SimManager.register({ "some/url/": { type: 'json', // The simlet alias data: [] // The data -- it is a hard-coded version of what would be in the feed }, "https://itunes.apple.com/us/rss/topmovies/limit=5/json": { type: 'json', data: {feed: {entry: [{},{}]}} } });
45. Questions?
46. Daniel Gallo Senior Solutions Architect Sencha, Inc. dan@sencha.com