The Landscape of Front-End Testing

Given @ Render Conf 2016
======================
You may have heard developers say that testing your code is a best practice. But what kinds of testing are they talking about? From unit and acceptance testings, to code linting and visual regression testing and more, it may be difficult to sort out which kind of testing is right for your team or project. In this talk, we will cover the breadth of testing frameworks available to front-end developers, and how to determine which ones will help your team produce a stable code base.

Transcript

1. The Landscape of Front-End Testing Render Conf 2016 Alicia Sedlock | @aliciability | #frontendtesting
2. I’m not here to prescribe. #frontendtesting
3. #frontendtesting The Beginning • Origins on the server • Here comes the client-side! • Opportunity and (sometimes neglected) responsibility
4. A focus on outcome, not the input. #frontendtesting
5. Automation is your friend! #frontendtesting
6. #frontendtesting Unit tests are for ensuring small pieces of code work as expected.
7. #frontendtesting describe(“Calculator Operations”, function () { it(“Should add two numbers”, function () { Calculator.init(); var result = Calculator.addNumbers(7,3); expect(result).toBe(10); }); });
8. #frontendtesting describe(“Calculator Operations”, function () { it(“Should add two numbers”, function () { Calculator.init(); var result = Calculator.addNumbers(7,3); expect(result).toBe(10); }); });
9. #frontendtesting describe(“Calculator Operations”, function () { it(“Should add two numbers”, function () { Calculator.init(); var result = Calculator.addNumbers(7,3); expect(result).toBe(10); }); });
10. #frontendtesting describe(“Calculator Operations”, function () { it(“Should add two numbers”, function () { Calculator.init(); var result = Calculator.addNumbers(7,3); expect(result).toBe(10); }); });
11. #frontendtesting describe(“Calculator Operations”, function () { beforeEach(function () { Calculator.init(); }); ... afterEach(function () { Calculator.teardown(); }); });
12. #frontendtesting it(“Should remember the last calculation”, function () { spyOn(Calculator, “updateCurrentValue”); Calculator.addNumbers(7,10); expect(Calculator.updateCurrentValue).toHaveBeenCalled(); expect(Calculator.updateCurrentValue) .toHaveBeenCalledWith(17); expect(Calculator.currentValue).toBe(17); });
13. #frontendtesting it(“Should remember the last calculation”, function () { spyOn(Calculator, “updateCurrentValue”); Calculator.addNumbers(7,10); expect(Calculator.updateCurrentValue).toHaveBeenCalled(); expect(Calculator.updateCurrentValue) .toHaveBeenCalledWith(17); expect(Calculator.currentValue).toBe(17); });
14. #frontendtesting it(“Should remember the last calculation”, function () { spyOn(Calculator, “updateCurrentValue”); Calculator.addNumbers(7,10); expect(Calculator.updateCurrentValue).toHaveBeenCalled(); expect(Calculator.updateCurrentValue) .toHaveBeenCalledWith(17); expect(Calculator.currentValue).toBe(17); });
15. #frontendtesting it(“Should remember the last calculation”, function () { spyOn(Calculator, “updateCurrentValue”); Calculator.addNumbers(7,10); expect(Calculator.updateCurrentValue).toHaveBeenCalled(); expect(Calculator.updateCurrentValue) .toHaveBeenCalledWith(17); expect(Calculator.currentValue).toBe(17); });
16. #frontendtesting it(“Should remember the last calculation”, function () { spyOn(Calculator, “updateCurrentValue”); Calculator.addNumbers(7,10); expect(Calculator.updateCurrentValue).toHaveBeenCalled(); expect(Calculator.updateCurrentValue) .toHaveBeenCalledWith(17); expect(Calculator.currentValue).toBe(17); });
17. #frontendtesting Recap! • Test tiny pieces of code • Validations, calculations, etc. • Pairs well with functional requirements
18. #frontendtesting Some big players • Jasmine • Mocha • Chai • Webdriver.io • QUnit • Unit.js • Sinon
19. #frontendtesting Acceptance tests check that all the small pieces play well together in the bigger picture.
20. #frontendtesting describe(“Integration tests”, function () { var page; beforeEach(function (done) { page = visit(“/home”); page.ready(done); }); describe(“Sign Up Failure state”, function () { beforeEach(function (done) { page.fill_in(“input[name='email']", “Not An Email”); page.click(“button[type=submit]”); page.onBodyChange(done); }); it(“Shouldn’t allow signup with invalid information”, function () { expect(page.find(“#signupError”) .hasClass(“hidden”)).toBeFalsy(); }); }); });
21. #frontendtesting describe(“Integration tests”, function () { var page; beforeEach(function (done) { page = visit(“/home”); page.ready(done); }); describe(“Sign Up Failure State ”, function () { beforeEach(function (done) { page.fill_in(“input[name='email']", “Not An Email”); page.click(“button[type=submit]”); page.onBodyChange(done); }); it(“Shouldn’t allow signup with invalid information”, function () { expect(page.find(“#signupError”) .hasClass(“hidden”)).toBeFalsy(); }); }); });
22. #frontendtesting describe(“Integration tests”, function () { var page; beforeEach(function (done) { page = visit(“/home”); page.ready(done); }); describe(“Sign Up Failure state”, function () { beforeEach(function (done) { page.fill_in(“input[name='email']", “Not An Email”); page.click(“button[type=submit]”); page.onBodyChange(done); }); it(“Shouldn’t allow signup with invalid information”, function () { expect(page.find(“#signupError”) .hasClass(“hidden”)).toBeFalsy(); }); }); });
23. #frontendtesting describe(“Integration tests”, function () { var page; beforeEach(function (done) { page = visit(“/home”); page.ready(done); }); describe(“Sign Up Failure state”, function () { beforeEach(function (done) { page.fill_in(“input[name='email']", “Not An Email”); page.click(“button[type=submit]”); page.onBodyChange(done); }); it(“Shouldn’t allow signup with invalid information”, function () { expect(page.find(“#signupError”) .hasClass(“hidden”)).toBeFalsy(); }); }); });
24. #frontendtesting Another recap! • Do units and other pieces play well together? • Line up well with user stories
25. #frontendtesting More tools! • jasmine-integration • Karma • Selenium • Nightwatch • Again, many others
26. #frontendtesting The Evolution • Front-end development is more than just business logic • There are other parts of the code that can fail • Apply the tactic of testing expected outcomes
27. #frontendtesting Visual regression tests check for inconsistencies in the view.
28. #frontendtesting Before After Difference
29. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(“#signUpForm”, { name: “Alicia Sedlock”, email: “alicia@example.com” }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
30. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(‘#signUpForm', { name: ‘Alicia Sedlock’, email: ‘alicia@example.com’ }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
31. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(‘#signUpForm', { name: ‘Alicia Sedlock’, email: ‘alicia@example.com’ }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
32. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(‘#signUpForm', { name: ‘Alicia Sedlock’, email: ‘alicia@example.com’ }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
33. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(‘#signUpForm', { name: ‘Alicia Sedlock’, email: ‘alicia@example.com’ }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
34. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(‘#signUpForm', { name: ‘Alicia Sedlock’, email: ‘alicia@example.com’ }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
35. #frontendtesting casper.start(“/home”).then(function(){ // Initial state of form phantomcss .screenshot(“#signUpForm”, “sign up form”); // Hit the sign up button (should trigger error) casper.click(“button#signUp”); // Take a screenshot of the UI component phantomcss .screenshot(“#signUpForm”, “sign up form error”); // Fill in form by name attributes & submit casper.fill(‘#signUpForm', { name: ‘Alicia Sedlock’, email: ‘alicia@example.com’ }, true); // Take a second screenshot of success state phantomcss .screenshot(“#signUpForm”, “sign up form success”); });
36. #frontendtesting Visual regression tests full pages vs. components
37. #frontendtesting casper.start(“/styleguide”).then(function(){ phantomcss .screenshot(“a.link”, “link example”); phantomcss .screenshot(“.btn-primary”, “primary button”); }).then(function() { this.mouse.move(“.btn-primary”'); phantomcss .screenshot(“.btn-primary”, “button hover”); });
38. #frontendtesting Visual regression tests and that responsive design thing…
39. #frontendtesting phantomcss: { options: { mismatchTolerance: 0.05, screenshots: ‘tests/visual/baselines’, results: ‘tests/visual/results’, viewportSize: [1280, 800] }, src: [‘tests/visual/tests.js’] }
40. #frontendtesting phantomcss: { big_size: { options: { ... viewportSize: [1280, 800] } }, small_size: { options: { ... viewportSize: [480, 320] } } }
41. #frontendtesting Recap! • Fairly young concept, but is growing quickly • Test components, not full pages • Great for responsive testing
42. #frontendtesting Visual regression tools • PhantomCSS/PhantomFlow • BackstopJS • Wraith • Webdriver.io • Percy.io
43. #frontendtesting Accessibility tests compare your site against accessibility standards
44. #frontendtesting
45. #frontendtesting Tools I’ve heard of • a11y • use grunt-a11y for build tool automation • Pa11y • …that’s it for now?
46. #frontendtesting Performance tests keep your project honest about performance
47. #frontendtesting perfbudget: { default: { options: { url: [your site’s URL], budget: { visualComplete: '3000', SpeedIndex: ‘1500’, render: ‘500’, fullyLoaded: ‘3000’, requests: ’25’, bytesInDoc: '100000' } } } }
48. #frontendtesting Tools for performance • grunt-perfbudget • gulp size • perf.js
49. So, you’re saying I should use all these tools, right? #frontendtesting
50. #frontendtesting Assessment • How much business logic are we off-loading to the client-side? • What is our history with regressions? • Are we constantly fixing, and re-fixing, visual/UI issues? • In general, which piece of our stack is the most instable/Jenga-like?
51. Yes, you can go overboard with tests. #frontendtesting
52. Yes, writing good tests takes practice. #frontendtesting
53. The Excuse Corner
54. This is more code to write. #frontendtesting
55. This is more code to write, and my code will still have bugs. #frontendtesting
56. I’m working with legacy code, and it’ll be impossible to test old features. #frontendtesting
57. #frontendtesting Build A Test-Conscience Team through education, experimentation, and practice
58. Thanks for listening!