The Landscape of Front-End Testing

Given @ Render Conf 2016
======================
You may have heard developers say that testing your code is a best practice. But what kinds of testing are they talking about? From unit and acceptance testings, to code linting and visual regression testing and more, it may be difficult to sort out which kind of testing is right for your team or project. In this talk, we will cover the breadth of testing frameworks available to front-end developers, and how to determine which ones will help your team produce a stable code base.

Transcript

1. The Landscape of
   Front-End Testing
   Render Conf 2016
   Alicia Sedlock | @aliciability | #frontendtesting
   

   View Slide

2. I’m not here to
   prescribe.
   #frontendtesting
   

   View Slide

3. #frontendtesting
   The Beginning
   • Origins on the server
   • Here comes the client-side!
   • Opportunity and (sometimes neglected)
   responsibility
   

   View Slide

4. A focus on outcome,
   not the input.
   #frontendtesting
   

   View Slide

5. Automation is your
   friend!
   #frontendtesting
   

   View Slide

6. #frontendtesting
   Unit tests
   are for ensuring small pieces of
   code work as expected.
   

   View Slide

7. #frontendtesting
   describe(“Calculator Operations”, function () {
   it(“Should add two numbers”, function () {
   Calculator.init();
   var result = Calculator.addNumbers(7,3);
   expect(result).toBe(10);
   });
   });
   

   View Slide

8. #frontendtesting
   describe(“Calculator Operations”, function () {
   it(“Should add two numbers”, function () {
   Calculator.init();
   var result = Calculator.addNumbers(7,3);
   expect(result).toBe(10);
   });
   });
   

   View Slide

9. #frontendtesting
   describe(“Calculator Operations”, function () {
   it(“Should add two numbers”, function () {
   Calculator.init();
   var result = Calculator.addNumbers(7,3);
   expect(result).toBe(10);
   });
   });
   

   View Slide

10. #frontendtesting
    describe(“Calculator Operations”, function () {
    it(“Should add two numbers”, function () {
    Calculator.init();
    var result = Calculator.addNumbers(7,3);
    expect(result).toBe(10);
    });
    });
    

    View Slide

11. #frontendtesting
    describe(“Calculator Operations”, function () {
    beforeEach(function () {
    Calculator.init();
    });
    ...
    afterEach(function () {
    Calculator.teardown();
    });
    });
    

    View Slide

12. #frontendtesting
    it(“Should remember the last calculation”, function () {
    spyOn(Calculator, “updateCurrentValue”);
    Calculator.addNumbers(7,10);
    expect(Calculator.updateCurrentValue).toHaveBeenCalled();
    expect(Calculator.updateCurrentValue)
    .toHaveBeenCalledWith(17);
    expect(Calculator.currentValue).toBe(17);
    });
    

    View Slide

13. #frontendtesting
    it(“Should remember the last calculation”, function () {
    spyOn(Calculator, “updateCurrentValue”);
    Calculator.addNumbers(7,10);
    expect(Calculator.updateCurrentValue).toHaveBeenCalled();
    expect(Calculator.updateCurrentValue)
    .toHaveBeenCalledWith(17);
    expect(Calculator.currentValue).toBe(17);
    });
    

    View Slide

14. #frontendtesting
    it(“Should remember the last calculation”, function () {
    spyOn(Calculator, “updateCurrentValue”);
    Calculator.addNumbers(7,10);
    expect(Calculator.updateCurrentValue).toHaveBeenCalled();
    expect(Calculator.updateCurrentValue)
    .toHaveBeenCalledWith(17);
    expect(Calculator.currentValue).toBe(17);
    });
    

    View Slide

15. #frontendtesting
    it(“Should remember the last calculation”, function () {
    spyOn(Calculator, “updateCurrentValue”);
    Calculator.addNumbers(7,10);
    expect(Calculator.updateCurrentValue).toHaveBeenCalled();
    expect(Calculator.updateCurrentValue)
    .toHaveBeenCalledWith(17);
    expect(Calculator.currentValue).toBe(17);
    });
    

    View Slide

16. #frontendtesting
    it(“Should remember the last calculation”, function () {
    spyOn(Calculator, “updateCurrentValue”);
    Calculator.addNumbers(7,10);
    expect(Calculator.updateCurrentValue).toHaveBeenCalled();
    expect(Calculator.updateCurrentValue)
    .toHaveBeenCalledWith(17);
    expect(Calculator.currentValue).toBe(17);
    });
    

    View Slide

17. #frontendtesting
    Recap!
    • Test tiny pieces of code
    • Validations, calculations, etc.
    • Pairs well with functional requirements
    

    View Slide

18. #frontendtesting
    Some big players
    • Jasmine
    • Mocha
    • Chai
    • Webdriver.io
    • QUnit
    • Unit.js
    • Sinon
    

    View Slide

19. #frontendtesting
    Acceptance tests
    check that all the small pieces play
    well together in the bigger picture.
    

    View Slide

20. #frontendtesting
    describe(“Integration tests”, function () {
    var page;
    beforeEach(function (done) {
    page = visit(“/home”);
    page.ready(done);
    });
    describe(“Sign Up Failure state”, function () {
    beforeEach(function (done) {
    page.fill_in(“input[name='email']", “Not An Email”);
    page.click(“button[type=submit]”);
    page.onBodyChange(done);
    });
    it(“Shouldn’t allow signup with invalid information”, function () {
    expect(page.find(“#signupError”)
    .hasClass(“hidden”)).toBeFalsy();
    });
    });
    });
    

    View Slide

21. #frontendtesting
    describe(“Integration tests”, function () {
    var page;
    beforeEach(function (done) {
    page = visit(“/home”);
    page.ready(done);
    });
    describe(“Sign Up Failure State ”, function () {
    beforeEach(function (done) {
    page.fill_in(“input[name='email']", “Not An Email”);
    page.click(“button[type=submit]”);
    page.onBodyChange(done);
    });
    it(“Shouldn’t allow signup with invalid information”, function () {
    expect(page.find(“#signupError”)
    .hasClass(“hidden”)).toBeFalsy();
    });
    });
    });
    

    View Slide

22. #frontendtesting
    describe(“Integration tests”, function () {
    var page;
    beforeEach(function (done) {
    page = visit(“/home”);
    page.ready(done);
    });
    describe(“Sign Up Failure state”, function () {
    beforeEach(function (done) {
    page.fill_in(“input[name='email']", “Not An Email”);
    page.click(“button[type=submit]”);
    page.onBodyChange(done);
    });
    it(“Shouldn’t allow signup with invalid information”, function () {
    expect(page.find(“#signupError”)
    .hasClass(“hidden”)).toBeFalsy();
    });
    });
    });
    

    View Slide

23. #frontendtesting
    describe(“Integration tests”, function () {
    var page;
    beforeEach(function (done) {
    page = visit(“/home”);
    page.ready(done);
    });
    describe(“Sign Up Failure state”, function () {
    beforeEach(function (done) {
    page.fill_in(“input[name='email']", “Not An Email”);
    page.click(“button[type=submit]”);
    page.onBodyChange(done);
    });
    it(“Shouldn’t allow signup with invalid information”, function () {
    expect(page.find(“#signupError”)
    .hasClass(“hidden”)).toBeFalsy();
    });
    });
    });
    

    View Slide

24. #frontendtesting
    Another recap!
    • Do units and other pieces play well together?
    • Line up well with user stories
    

    View Slide

25. #frontendtesting
    More tools!
    • jasmine-integration
    • Karma
    • Selenium
    • Nightwatch
    • Again, many others
    

    View Slide

26. #frontendtesting
    The Evolution
    • Front-end development is more than just
    business logic
    • There are other parts of the code that can fail
    • Apply the tactic of testing expected outcomes
    

    View Slide

27. #frontendtesting
    Visual regression tests
    check for inconsistencies in the view.
    

    View Slide

28. #frontendtesting
    Before After Difference
    

    View Slide

29. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(“#signUpForm”, {
    name: “Alicia Sedlock”,
    email: “alicia@example.com”
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });
    

    View Slide

30. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(‘#signUpForm', {
    name: ‘Alicia Sedlock’,
    email: ‘alicia@example.com’
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });
    

    View Slide

31. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(‘#signUpForm', {
    name: ‘Alicia Sedlock’,
    email: ‘alicia@example.com’
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });
    

    View Slide

32. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(‘#signUpForm', {
    name: ‘Alicia Sedlock’,
    email: ‘alicia@example.com’
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });
    

    View Slide

33. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(‘#signUpForm', {
    name: ‘Alicia Sedlock’,
    email: ‘alicia@example.com’
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });
    

    View Slide

34. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(‘#signUpForm', {
    name: ‘Alicia Sedlock’,
    email: ‘alicia@example.com’
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });
    

    View Slide

35. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(‘#signUpForm', {
    name: ‘Alicia Sedlock’,
    email: ‘alicia@example.com’
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });
    

    View Slide

36. #frontendtesting
    Visual regression tests
    full pages vs. components
    

    View Slide

37. #frontendtesting
    casper.start(“/styleguide”).then(function(){
    phantomcss
    .screenshot(“a.link”, “link example”);
    phantomcss
    .screenshot(“.btn-primary”, “primary button”);
    }).then(function() {
    this.mouse.move(“.btn-primary”');
    phantomcss
    .screenshot(“.btn-primary”, “button hover”);
    });
    

    View Slide

38. #frontendtesting
    Visual regression tests
    and that responsive design thing…
    

    View Slide

39. #frontendtesting
    phantomcss: {
    options: {
    mismatchTolerance: 0.05,
    screenshots: ‘tests/visual/baselines’,
    results: ‘tests/visual/results’,
    viewportSize: [1280, 800]
    },
    src: [‘tests/visual/tests.js’]
    }
    

    View Slide

40. #frontendtesting
    phantomcss: {
    big_size: {
    options: {
    ...
    viewportSize: [1280, 800]
    }
    },
    small_size: {
    options: {
    ...
    viewportSize: [480, 320]
    }
    }
    }
    

    View Slide

41. #frontendtesting
    Recap!
    • Fairly young concept, but is growing quickly
    • Test components, not full pages
    • Great for responsive testing
    

    View Slide

42. #frontendtesting
    Visual regression tools
    • PhantomCSS/PhantomFlow
    • BackstopJS
    • Wraith
    • Webdriver.io
    • Percy.io
    

    View Slide

43. #frontendtesting
    Accessibility tests
    compare your site against
    accessibility standards
    

    View Slide

44. #frontendtesting
    

    View Slide

45. #frontendtesting
    Tools I’ve heard of
    • a11y
    • use grunt-a11y for build tool automation
    • Pa11y
    • …that’s it for now?
    

    View Slide

46. #frontendtesting
    Performance tests
    keep your project honest about
    performance
    

    View Slide

47. #frontendtesting
    perfbudget: {
    default: {
    options: {
    url: [your site’s URL],
    budget: {
    visualComplete: '3000',
    SpeedIndex: ‘1500’,
    render: ‘500’,
    fullyLoaded: ‘3000’,
    requests: ’25’,
    bytesInDoc: '100000'
    }
    }
    }
    }
    

    View Slide

48. #frontendtesting
    Tools for performance
    • grunt-perfbudget
    • gulp size
    • perf.js
    

    View Slide

49. So, you’re saying I should
    use all these tools, right?
    #frontendtesting
    

    View Slide

50. #frontendtesting
    Assessment
    • How much business logic are we off-loading to
    the client-side?
    • What is our history with regressions?
    • Are we constantly fixing, and re-fixing, visual/UI
    issues?
    • In general, which piece of our stack is the most
    instable/Jenga-like?
    

    View Slide

51. Yes, you can go
    overboard with tests.
    #frontendtesting
    

    View Slide

52. Yes, writing good
    tests takes practice.
    #frontendtesting
    

    View Slide

53. The Excuse Corner
    

    View Slide

54. This is more code to
    write.
    #frontendtesting
    

    View Slide

55. This is more code to
    write, and my code will
    still have bugs.
    #frontendtesting
    

    View Slide

56. I’m working with legacy
    code, and it’ll be impossible
    to test old features.
    #frontendtesting
    

    View Slide

57. #frontendtesting
    Build A Test-Conscience Team
    through education, experimentation,
    and practice
    

    View Slide

58. Thanks for listening!
    

    View Slide