UI Testing - What we learnt while breaking Expedia’s UI Toolkit

A presentation given at an Expedia Tech Know How event in London about how front-end developers can write automated tests to test their work efficiently.

How do you ensure your JavaScript continues to function the way you intended?

How to you ensure every click the user makes does what it should?

How do you ensure your UI looks correct with every release?

How do you ensure your work is accessible to all?

Expedia's Client Side Engineering team have had to answer all these questions while building (and occasionally breaking) Expedia's UI Toolkit - a library of 30+ UI components built from the ground up and relied upon by multiple brands' sites including Expedia, Travelocity and Orbitz.

Transcript

1. UI Testing What we learnt while breaking Expedia’s UI Toolkit David Strugnell | @struggyone | #expediaeng
2. What will we cover • What is Expedia’s UI Toolkit? • How do we test it ◦ JavaScript Unit Tests ◦ Functional Tests ◦ Visual Regression Tests ◦ Accessibility Tests • Questions?
3. What is Expedia’s UI Toolkit?
4. Expedia’s UI Toolkit
6. How do we test it...
7. JavaScript Unit Tests • Used to test individual JavaScript functions • Very quick to run • Frameworks available: ◦ Jasmine ◦ qUnit ◦ Mocha
8. Example var addNumbers = function(num1, num2){ return num1 + num2; };
9. > addNumbers(5, 6); > 11 > addNumbers('10', '7'); > "107"
10. Example var addNumbers = function(num1, num2){ return num1 + num2; };
11. Example var addNumbers = function(num1, num2){ return parseFloat(num1) + parseFloat(num2); };
12. > addNumbers(5, 6); > 11 > addNumbers('10', '7'); > 17
13. Example: Jasmine Test describe(‘addNumbers’, function(){ });
14. Example: Jasmine Test describe(‘addNumbers’, function(){ it(‘should add together two integers’, function(){ }); });
15. Example: Jasmine Test describe(‘addNumbers’, function() { it(‘should add together two integers’, function() { expect(addNumbers(1,5)).toBe(6); }); });
16. Example: Jasmine Test describe(‘addNumbers’, function() { it(‘should add together two integers’, function() { expect(addNumbers(1, 5)).toBe(6); }); it(‘should add together two number strings’, function() { expect(addNumbers(‘2’, ‘8’)).toBe(10); }); });
17. Demo Time
18. Functional Tests • Used to test user interactions • Frameworks available ◦ Cucumber ◦ Nightwatch JS ◦ FuncUnit
19. Demo Time
20. Example: Nightwatch JS 'Add numbers' : function (browser) { browser .url('http://localhost:3000/addNumbers.html') .waitForElementVisible('body', 1000) .setValue('#numInput1', '10') .setValue('#numInput2', '20') .click('#submitBtn) .getText('#results', function(result) { this.assert.equal(result.value, 'Total: 17'); }) .end(); }
21. Example: Nightwatch JS 'Add numbers' : function (browser) { browser .url('http://localhost:3000/addNumbers.html') .waitForElementVisible('body', 1000) .setValue('#numInput1', '10') .setValue('#numInput2', '20') .click('#submitBtn) ..getText('#results', function(result) { this.assert.equal(result.value, 'Total: 17'); }) .end(); }
22. Example: Nightwatch JS 'Add numbers' : function (browser) { browser .url('http://localhost:3000/addNumbers.html') .waitForElementVisible('body', 1000) .setValue('#numInput1', '10') .setValue('#numInput2', '20') .click('#submitBtn) ..getText('#results', function(result) { this.assert.equal(result.value, 'Total: 17'); }) .end(); }
23. Example: Nightwatch JS 'Add numbers' : function (browser) { browser .url('http://localhost:3000/addNumbers.html') .waitForElementVisible('body', 1000) .setValue('#numInput1', '10') .setValue('#numInput2', '20') .click('#submitBtn) .getText('#results', function(result) { this.assert.equal(result.value, 'Total: 17'); }) .end(); }
24. Example: Nightwatch JS 'Add numbers' : function (browser) { browser .url('http://localhost:3000/addNumbers.html') .waitForElementVisible('body', 1000) .setValue('#numInput1', '10') .setValue('#numInput2', '20') .click('#submitBtn) .getText('#results', function(result) { this.assert.equal(result.value, 'Total: 30'); }) .end(); }
25. Example: Nightwatch JS 'Add numbers' : function (browser) { browser .url('http://localhost:3000/addNumbers.html') .waitForElementVisible('body', 1000) .setValue('#numInput1', '10') .setValue('#numInput2', '20') .click('#submitBtn) .getText('#results', function(result) { this.assert.equal(result.value, 'Total: 17'); }) .end(); }
26. Demo Time
27. Visual Regression Tests • Used to test for visual changes in the UI • Frameworks available: ◦ PhantomCSS ◦ BackstopJS ◦ WebdriverCSS ◦ Wraith
28. We had to update the button...
30. PhantomCSS PhantomJS CasperJS PhantomCSS
31. Example: PhantomCSS
32. casper .start('http://localhost:3000/addNumbers.html') .viewport(600, 800) .then(function(){ phantomcss.screenshot('#addNumbers', 'add numbers'); }) .then(function(){ phantomcss.compareAll(); }) .run(); Example: PhantomCSS
33. casper .start('http://localhost:3000/addNumbers.html') .viewport(600, 800) .then(function(){ phantomcss.screenshot('#addNumbers', 'add numbers'); }) .then(function(){ phantomcss.compareAll(); }) .run(); Example: PhantomCSS
34. casper .start('http://localhost:3000/addNumbers.html') .viewport(600, 800) .then(function(){ phantomcss.screenshot('#addNumbers', 'add numbers'); }) .then(function(){ phantomcss.compareAll(); }) .run(); Example: PhantomCSS
35. casper .start('http://localhost:3000/addNumbers.html') .viewport(600, 800) .then(function(){ phantomcss.screenshot('#addNumbers', 'add numbers'); }) .then(function(){ phantomcss.compareAll(); }) .run(); Example: PhantomCSS
36. Demo
37. • Avoid taking a single screenshot of a complete page • Avoid breaking the page up too much • Test can take quite a while to run • Tests have to be run on the same OS / machine Type Things to keep in mind
38. Accessibility Tests • Used to test the website can be accessed by as many people as possible • Frameworks available: ◦ aXe ◦ Pa11y ◦ a11y
39. What does aXe check? • Colour contrast • Form elements have labels • Images have alternative text • ARIA roles and landmarks are used correctly • Tables are correctly structured
40. <script src="/axe/axe.min.js" ></script> How to setup and run aXe 1. Chrome developers tool extension 2. Automatically
41. <script src="/axe/axe.min.js" ></script> How to setup and run aXe 1. Chrome developers tool extension 2. Automatically axe.a11yCheck(document, function (results) { // do something with results });
42. Demo
43. Conclusion Automated testing: • Allows you to catch bugs early • Saves you time manually testing • Gives you confidence to make changes
44. Conclusion How much testing should you do: • As much as you need
45. Resources Jasmine: http://jasmine.github.io/ Karma: https://karma-runner.github.io/ Nightwatch.js: http://nightwatchjs.org/ PhantomCSS: https://github.com/Huddle/PhantomCSS CasperJS: http://casperjs.org/ aXe: http://www.deque.com/products/axe/
46. Thank you Any Questions?