The State of Front-End Testing

Web Directions | Code 16
Sydney, July 29 2016
Melbourne, August 2 2016

Transcript

1. The State of Front-End Testing Web Directions | Code 16
2. #frontendtesting
3. • Web Developer @
 Society of Grownups • @aliciability on Twitter • Teacher with Girl Develop It • Run Hedgehogs & Reflogs
4. #frontendtesting
5. #frontendtesting As developers we don’t want to • Give users broken products • Break trust with QA & business teams • Waste development time
6. What is “front-end testing”, anyway? #frontendtesting
7. #frontendtesting Front-end testing is… • A collection of techniques to hold developers accountable for writing and maintaining functioning and usable code bases.
8. #frontendtesting The Beginnings of Testing • Hi, server-side developers! • Here comes the client-side! • Opportunities galore and (sometimes neglected) responsibilities
9. I’m not here to prescribe. #frontendtesting
10. A focus on outcome, not the input. #frontendtesting
11. Automation is your friend! #frontendtesting
12. Components of front-end testing #frontendtesting
13. #frontendtesting Visual
 Regression Performance Accessibility Unit Integration Acceptance
14. #frontendtesting Visual Regression Performance Accessibility Unit Integration Acceptance
15. #frontendtesting Unit tests are for ensuring small pieces of code work as expected.
16. #frontendtesting describe(“Calculator Operations”, function () { it(“Should add two numbers”, function () { Calculator.init(); var result = Calculator.addNumbers(7,3); expect(result).toBe(10); }); });
17. #frontendtesting describe(“Calculator Operations”, function () { it(“Should add two numbers”, function () { Calculator.init(); var result = Calculator.addNumbers(7,3); expect(result).toBe(10); }); });
18. #frontendtesting describe(“Calculator Operations”, function () { it(“Should add two numbers”, function () { Calculator.init(); var result = Calculator.addNumbers(7,3); expect(result).toBe(10); }); });
19. #frontendtesting describe(“Calculator Operations”, function () { it(“Should add two numbers”, function () { Calculator.init(); var result = Calculator.addNumbers(7,3); expect(result).toBe(10); }); });
20. #frontendtesting describe(“Calculator Operations”, function () { it(“Should add two numbers”, function () { Calculator.init(); var result = Calculator.addNumbers(7,3); expect(result).toBe(10); }); });
21. #frontendtesting describe(“Calculator Operations”, function () { beforeEach(function () { Calculator.init(); }); ... afterEach(function () { Calculator.teardown(); }); });
22. #frontendtesting it(“Should remember the last calculation”, function () { spyOn(Calculator, “updateCurrentValue”); Calculator.addNumbers(7,10); expect(Calculator.updateCurrentValue).toHaveBeenCalled(); expect(Calculator.updateCurrentValue) .toHaveBeenCalledWith(17); expect(Calculator.currentValue).toBe(17); });
23. #frontendtesting it(“Should remember the last calculation”, function () { spyOn(Calculator, “updateCurrentValue”); Calculator.addNumbers(7,10); expect(Calculator.updateCurrentValue).toHaveBeenCalled(); expect(Calculator.updateCurrentValue) .toHaveBeenCalledWith(17); expect(Calculator.currentValue).toBe(17); });
24. #frontendtesting it(“Should remember the last calculation”, function () { spyOn(Calculator, “updateCurrentValue”); Calculator.addNumbers(7,10); expect(Calculator.updateCurrentValue).toHaveBeenCalled(); expect(Calculator.updateCurrentValue) .toHaveBeenCalledWith(17); expect(Calculator.currentValue).toBe(17); });
25. #frontendtesting it(“Should remember the last calculation”, function () { spyOn(Calculator, “updateCurrentValue”); Calculator.addNumbers(7,10); expect(Calculator.updateCurrentValue).toHaveBeenCalled(); expect(Calculator.updateCurrentValue) .toHaveBeenCalledWith(17); expect(Calculator.currentValue).toBe(17); });
26. #frontendtesting it(“Should remember the last calculation”, function () { spyOn(Calculator, “updateCurrentValue”); Calculator.addNumbers(7,10); expect(Calculator.updateCurrentValue).toHaveBeenCalled(); expect(Calculator.updateCurrentValue) .toHaveBeenCalledWith(17); expect(Calculator.currentValue).toBe(17); });
27. #frontendtesting Some big players • Jasmine • Mocha • Chai • Webdriver.io • QUnit • Unit.js • Sinon
28. #frontendtesting Integration tests check that all the small pieces play well together in the bigger picture.
29. #frontendtesting Unit
30. #frontendtesting Unit Integration
31. #frontendtesting Unit Integration Integration
32. #frontendtesting test('it renders completed class properly', function(assert) { set(this, 'currentStepIndex', 0); set(this, 'step', { index: 1 }); this.render(hbs`{{progress-bar/progress-step this}}`); assert.notOk(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'), 'step is not marked complete'); run(() => { set(this, 'currentStepIndex', 1); }); assert.ok(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'), 'step is marked complete'); });
33. #frontendtesting test('it renders completed class properly', function(assert) { set(this, 'currentStepIndex', 0); set(this, 'step', { index: 1 }); this.render(hbs`{{progress-bar/progress-step this}}`); assert.notOk(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'), 'step is not marked complete'); run(() => { set(this, 'currentStepIndex', 1); }); assert.ok(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'), 'step is marked complete'); });
34. #frontendtesting test('it renders completed class properly', function(assert) { set(this, 'currentStepIndex', 0); set(this, 'step', { index: 1 }); this.render(hbs`{{progress-bar/progress-step this}}`); assert.notOk(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'), 'step is not marked complete'); run(() => { set(this, 'currentStepIndex', 1); }); assert.ok(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'), 'step is marked complete'); });
35. #frontendtesting test('it renders completed class properly', function(assert) { set(this, 'currentStepIndex', 0); set(this, 'step', { index: 1 }); this.render(hbs`{{progress-bar/progress-step this}}`); assert.notOk(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'), 'step is not marked complete'); run(() => { set(this, 'currentStepIndex', 1); }); assert.ok(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'), 'step is marked complete'); });
36. #frontendtesting test('it renders completed class properly', function(assert) { set(this, 'currentStepIndex', 0); set(this, 'step', { index: 1 }); this.render(hbs`{{progress-bar/progress-step this}}`); assert.notOk(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'), 'step is not marked complete'); run(() => { set(this, 'currentStepIndex', 1); }); assert.ok(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'), 'step is marked complete'); });
37. #frontendtesting Acceptance tests make sure we can accomplish major tasks.
38. #frontendtesting describe(“Acceptance tests”, function () { var page; beforeEach(function (done) { page = visit(“/home”); page.ready(done); }); describe(“Sign Up Failure state”, function () { beforeEach(function (done) { page.fill_in(“input[name='email']", “Not An Email”); page.click(“button[type=submit]”); page.onBodyChange(done); }); it(“Shouldn’t allow signup with invalid information”, function () { expect(page.find(“#signupError”) .hasClass(“hidden”)).toBeFalsy(); }); }); });
39. #frontendtesting describe(“Acceptance tests”, function () { var page; beforeEach(function (done) { page = visit(“/home”); page.ready(done); }); describe(“Sign Up Failure State ”, function () { beforeEach(function (done) { page.fill_in(“input[name='email']", “Not An Email”); page.click(“button[type=submit]”); page.onBodyChange(done); }); it(“Shouldn’t allow signup with invalid information”, function () { expect(page.find(“#signupError”) .hasClass(“hidden”)).toBeFalsy(); }); }); });
40. #frontendtesting describe(“Acceptance tests”, function () { var page; beforeEach(function (done) { page = visit(“/home”); page.ready(done); }); describe(“Sign Up Failure state”, function () { beforeEach(function (done) { page.fill_in(“input[name='email']", “Not An Email”); page.click(“button[type=submit]”); page.onBodyChange(done); }); it(“Shouldn’t allow signup with invalid information”, function () { expect(page.find(“#signupError”) .hasClass(“hidden”)).toBeFalsy(); }); }); });
41. #frontendtesting describe(“Acceptance tests”, function () { var page; beforeEach(function (done) { page = visit(“/home”); page.ready(done); }); describe(“Sign Up Failure state”, function () { beforeEach(function (done) { page.fill_in(“input[name='email']", “Not An Email”); page.click(“button[type=submit]”); page.onBodyChange(done); }); it(“Shouldn’t allow signup with invalid information”, function () { expect(page.find(“#signupError”) .hasClass(“hidden”)).toBeFalsy(); }); }); });
42. #frontendtesting More tools! • jasmine-integration • Karma • Selenium • Nightwatch • Again, many others
43. #frontendtesting Visual Regression Performance Accessibility Unit Integration Acceptance
44. #frontendtesting Visual
 Regression Performance Accessibility Unit Integration Acceptance
45. #frontendtesting The Evolution • Front-end development is more than just business logic • There are other parts of the code that can fail • Apply the tactic of testing expected outcomes
46. #frontendtesting Visual regression tests check for inconsistencies in the view.
47. #frontendtesting Before After Difference
48. #frontendtesting
49. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(“#signUpForm”, { name: “Alicia Sedlock”, email: “alicia@example.com” }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
50. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(‘#signUpForm', { name: ‘Alicia Sedlock’, email: ‘alicia@example.com’ }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
51. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(‘#signUpForm', { name: ‘Alicia Sedlock’, email: ‘alicia@example.com’ }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
52. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(‘#signUpForm', { name: ‘Alicia Sedlock’, email: ‘alicia@example.com’ }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
53. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(‘#signUpForm', { name: ‘Alicia Sedlock’, email: ‘alicia@example.com’ }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
54. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(‘#signUpForm', { name: ‘Alicia Sedlock’, email: ‘alicia@example.com’ }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
55. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(‘#signUpForm', { name: ‘Alicia Sedlock’, email: ‘alicia@example.com’ }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
56. #frontendtesting Visual regression tests full pages vs. components
57. #frontendtesting casper.start(“/styleguide”).then(function(){ phantomcss .screenshot(“a.link”, “link example”); phantomcss .screenshot(“.btn-primary”, “primary button”); }).then(function() { this.mouse.move(“.btn-primary”'); phantomcss .screenshot(“.btn-primary”, “button hover”); });
58. #frontendtesting Visual regression tests and that responsive design thing…
59. #frontendtesting phantomcss: { options: { mismatchTolerance: 0.05, screenshots: ‘tests/visual/baselines’, results: ‘tests/visual/results’, viewportSize: [1280, 800] }, src: [‘tests/visual/tests.js’] }
60. #frontendtesting phantomcss: { big_size: { options: { ... viewportSize: [1280, 800] } }, small_size: { options: { ... viewportSize: [480, 320] } } }
61. The workflow, though… #frontendtesting
62. #frontendtesting
63. #frontendtesting
64. #frontendtesting
65. #frontendtesting Visual regression tools • PhantomCSS/PhantomFlow • BackstopJS • Wraith • Webdriver.io • Percy.io
66. #frontendtesting Accessibility tests compare your site against accessibility standards
67. #frontendtesting
68. #frontendtesting Tools I’ve heard of • a11y • use grunt-a11y for build tool automation • Pa11y • ember-a11y-testing • protractor-accessibility-plugin (Angular) • react-a11y
69. #frontendtesting Performance tests keep your project honest about performance
70. #frontendtesting perfbudget: { default: { options: { url: [your site’s URL], budget: { visualComplete: '3000', SpeedIndex: ‘1500’, render: ‘500’, fullyLoaded: ‘3000’, requests: ’25’, bytesInDoc: '100000' } } } }
71. #frontendtesting Tools for performance • grunt-perfbudget • gulp size • perf.js
72. #frontendtesting Monkey testing is something I just read about yesterday morning.
73. #frontendtesting
74. #frontendtesting var horde = gremlins.createHorde(); horde.unleash();
75. #frontendtesting gremlins.createHorde() .gremlin(gremlins.species.formFiller()) .gremlin(gremlins.species.clicker().clickTypes(['click'])) .gremlin(gremlins.species.toucher()) .gremlin(gremlins.species.scroller()) .gremlin(gremlins.species.typer()) .unleash();
76. #frontendtesting gremlin formFiller input 5 in <input type="number" name="age"> gremlin formFiller input pzdohh0k9@o8cpdb7mi.r7r in <input type="email"> gremlin clicker click at 1219 301 gremlin scroller scroll to 100 25 ... mogwai fps 33.21 mogwai fps 59.45 mogwai fps 12.67 err > mogwai fps 7.54 < err
77. #frontendtesting A single library • Gremlins.js
78. What about linting? #frontendtesting
79. #frontendtesting Interested in linting? • CSSLint • JSLint/JSHint • UnCSS • HTMLLint • Framework-specific linting, too!
80. So, you’re saying I should use all these tools, right? #frontendtesting
81. #frontendtesting Assessment • How much business logic are we off-loading to the client-side? • What is our history with regressions? • Are we constantly fixing, and re-fixing, visual/UI issues? • In general, which piece of our stack is the most instable/Jenga-like? • Is there a lot of inconsistency in how we write code?
82. Yes, you can go overboard with tests. #frontendtesting
83. #frontendtesting Visual
 Regression Performance Accessibility Unit Integration Acceptance
84. #frontendtesting Visual
 Regression Performance Accessibility Acceptance
85. #frontendtesting Visual
 Regression Performance Accessibility
86. #frontendtesting Accessibility
87. #frontendtesting Accessibility Unit Integration Acceptance
88. Yes, writing good tests takes practice. #frontendtesting
89. #frontendtesting
90. The Excuse Corner
91. This is more code to write. #frontendtesting
92. This is more code to write, and my code will still have bugs. #frontendtesting
93. I’m working with legacy code, and it’ll be impossible to test old features. #frontendtesting
94. #frontendtesting Build A Test-Conscience Team through education, experimentation, and practice
95. #frontendtesting • Frontend Architecture for Design Systems by Micah Godbolt
 • FrontEndTesting.com Looking for more?
96. Thank you!