Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The State of Front-End Testing

The State of Front-End Testing

Web Directions | Code 16
Sydney, July 29 2016
Melbourne, August 2 2016

Alicia Sedlock

July 29, 2016
Tweet

More Decks by Alicia Sedlock

Other Decks in Technology

Transcript

  1. The State of
    Front-End Testing
    Web Directions | Code 16

    View full-size slide

  2. #frontendtesting

    View full-size slide

  3. • Web Developer @

    Society of Grownups
    • @aliciability on Twitter
    • Teacher with Girl Develop It
    • Run Hedgehogs & Reflogs

    View full-size slide

  4. #frontendtesting

    View full-size slide

  5. #frontendtesting
    As developers we don’t
    want to
    • Give users broken products
    • Break trust with QA & business teams
    • Waste development time

    View full-size slide

  6. What is “front-end
    testing”, anyway?
    #frontendtesting

    View full-size slide

  7. #frontendtesting
    Front-end testing is…
    • A collection of techniques to hold developers
    accountable for writing and maintaining
    functioning and usable code bases.

    View full-size slide

  8. #frontendtesting
    The Beginnings of Testing
    • Hi, server-side developers!
    • Here comes the client-side!
    • Opportunities galore and (sometimes
    neglected) responsibilities

    View full-size slide

  9. I’m not here to
    prescribe.
    #frontendtesting

    View full-size slide

  10. A focus on outcome,
    not the input.
    #frontendtesting

    View full-size slide

  11. Automation is your
    friend!
    #frontendtesting

    View full-size slide

  12. Components of
    front-end testing
    #frontendtesting

    View full-size slide

  13. #frontendtesting
    Visual

    Regression
    Performance Accessibility
    Unit Integration Acceptance

    View full-size slide

  14. #frontendtesting
    Visual
    Regression
    Performance Accessibility
    Unit Integration Acceptance

    View full-size slide

  15. #frontendtesting
    Unit tests
    are for ensuring small pieces of
    code work as expected.

    View full-size slide

  16. #frontendtesting
    describe(“Calculator Operations”, function () {
    it(“Should add two numbers”, function () {
    Calculator.init();
    var result = Calculator.addNumbers(7,3);
    expect(result).toBe(10);
    });
    });

    View full-size slide

  17. #frontendtesting
    describe(“Calculator Operations”, function () {
    it(“Should add two numbers”, function () {
    Calculator.init();
    var result = Calculator.addNumbers(7,3);
    expect(result).toBe(10);
    });
    });

    View full-size slide

  18. #frontendtesting
    describe(“Calculator Operations”, function () {
    it(“Should add two numbers”, function () {
    Calculator.init();
    var result = Calculator.addNumbers(7,3);
    expect(result).toBe(10);
    });
    });

    View full-size slide

  19. #frontendtesting
    describe(“Calculator Operations”, function () {
    it(“Should add two numbers”, function () {
    Calculator.init();
    var result = Calculator.addNumbers(7,3);
    expect(result).toBe(10);
    });
    });

    View full-size slide

  20. #frontendtesting
    describe(“Calculator Operations”, function () {
    it(“Should add two numbers”, function () {
    Calculator.init();
    var result = Calculator.addNumbers(7,3);
    expect(result).toBe(10);
    });
    });

    View full-size slide

  21. #frontendtesting
    describe(“Calculator Operations”, function () {
    beforeEach(function () {
    Calculator.init();
    });
    ...
    afterEach(function () {
    Calculator.teardown();
    });
    });

    View full-size slide

  22. #frontendtesting
    it(“Should remember the last calculation”, function () {
    spyOn(Calculator, “updateCurrentValue”);
    Calculator.addNumbers(7,10);
    expect(Calculator.updateCurrentValue).toHaveBeenCalled();
    expect(Calculator.updateCurrentValue)
    .toHaveBeenCalledWith(17);
    expect(Calculator.currentValue).toBe(17);
    });

    View full-size slide

  23. #frontendtesting
    it(“Should remember the last calculation”, function () {
    spyOn(Calculator, “updateCurrentValue”);
    Calculator.addNumbers(7,10);
    expect(Calculator.updateCurrentValue).toHaveBeenCalled();
    expect(Calculator.updateCurrentValue)
    .toHaveBeenCalledWith(17);
    expect(Calculator.currentValue).toBe(17);
    });

    View full-size slide

  24. #frontendtesting
    it(“Should remember the last calculation”, function () {
    spyOn(Calculator, “updateCurrentValue”);
    Calculator.addNumbers(7,10);
    expect(Calculator.updateCurrentValue).toHaveBeenCalled();
    expect(Calculator.updateCurrentValue)
    .toHaveBeenCalledWith(17);
    expect(Calculator.currentValue).toBe(17);
    });

    View full-size slide

  25. #frontendtesting
    it(“Should remember the last calculation”, function () {
    spyOn(Calculator, “updateCurrentValue”);
    Calculator.addNumbers(7,10);
    expect(Calculator.updateCurrentValue).toHaveBeenCalled();
    expect(Calculator.updateCurrentValue)
    .toHaveBeenCalledWith(17);
    expect(Calculator.currentValue).toBe(17);
    });

    View full-size slide

  26. #frontendtesting
    it(“Should remember the last calculation”, function () {
    spyOn(Calculator, “updateCurrentValue”);
    Calculator.addNumbers(7,10);
    expect(Calculator.updateCurrentValue).toHaveBeenCalled();
    expect(Calculator.updateCurrentValue)
    .toHaveBeenCalledWith(17);
    expect(Calculator.currentValue).toBe(17);
    });

    View full-size slide

  27. #frontendtesting
    Some big players
    • Jasmine
    • Mocha
    • Chai
    • Webdriver.io
    • QUnit
    • Unit.js
    • Sinon

    View full-size slide

  28. #frontendtesting
    Integration tests
    check that all the small pieces play
    well together in the bigger picture.

    View full-size slide

  29. #frontendtesting
    Unit

    View full-size slide

  30. #frontendtesting
    Unit
    Integration

    View full-size slide

  31. #frontendtesting
    Unit
    Integration
    Integration

    View full-size slide

  32. #frontendtesting
    test('it renders completed class properly', function(assert) {
    set(this, 'currentStepIndex', 0);
    set(this, 'step', { index: 1 });
    this.render(hbs`{{progress-bar/progress-step this}}`);
    assert.notOk(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'),
    'step is not marked complete');
    run(() => {
    set(this, 'currentStepIndex', 1);
    });
    assert.ok(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'),
    'step is marked complete');
    });

    View full-size slide

  33. #frontendtesting
    test('it renders completed class properly', function(assert) {
    set(this, 'currentStepIndex', 0);
    set(this, 'step', { index: 1 });
    this.render(hbs`{{progress-bar/progress-step this}}`);
    assert.notOk(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'),
    'step is not marked complete');
    run(() => {
    set(this, 'currentStepIndex', 1);
    });
    assert.ok(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'),
    'step is marked complete');
    });

    View full-size slide

  34. #frontendtesting
    test('it renders completed class properly', function(assert) {
    set(this, 'currentStepIndex', 0);
    set(this, 'step', { index: 1 });
    this.render(hbs`{{progress-bar/progress-step this}}`);
    assert.notOk(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'),
    'step is not marked complete');
    run(() => {
    set(this, 'currentStepIndex', 1);
    });
    assert.ok(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'),
    'step is marked complete');
    });

    View full-size slide

  35. #frontendtesting
    test('it renders completed class properly', function(assert) {
    set(this, 'currentStepIndex', 0);
    set(this, 'step', { index: 1 });
    this.render(hbs`{{progress-bar/progress-step this}}`);
    assert.notOk(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'),
    'step is not marked complete');
    run(() => {
    set(this, 'currentStepIndex', 1);
    });
    assert.ok(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'),
    'step is marked complete');
    });

    View full-size slide

  36. #frontendtesting
    test('it renders completed class properly', function(assert) {
    set(this, 'currentStepIndex', 0);
    set(this, 'step', { index: 1 });
    this.render(hbs`{{progress-bar/progress-step this}}`);
    assert.notOk(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'),
    'step is not marked complete');
    run(() => {
    set(this, 'currentStepIndex', 1);
    });
    assert.ok(this.$(‘.progress-bar__step').hasClass('progress-bar__step--completed'),
    'step is marked complete');
    });

    View full-size slide

  37. #frontendtesting
    Acceptance tests
    make sure we can accomplish
    major tasks.

    View full-size slide

  38. #frontendtesting
    describe(“Acceptance tests”, function () {
    var page;
    beforeEach(function (done) {
    page = visit(“/home”);
    page.ready(done);
    });
    describe(“Sign Up Failure state”, function () {
    beforeEach(function (done) {
    page.fill_in(“input[name='email']", “Not An Email”);
    page.click(“button[type=submit]”);
    page.onBodyChange(done);
    });
    it(“Shouldn’t allow signup with invalid information”, function () {
    expect(page.find(“#signupError”)
    .hasClass(“hidden”)).toBeFalsy();
    });
    });
    });

    View full-size slide

  39. #frontendtesting
    describe(“Acceptance tests”, function () {
    var page;
    beforeEach(function (done) {
    page = visit(“/home”);
    page.ready(done);
    });
    describe(“Sign Up Failure State ”, function () {
    beforeEach(function (done) {
    page.fill_in(“input[name='email']", “Not An Email”);
    page.click(“button[type=submit]”);
    page.onBodyChange(done);
    });
    it(“Shouldn’t allow signup with invalid information”, function () {
    expect(page.find(“#signupError”)
    .hasClass(“hidden”)).toBeFalsy();
    });
    });
    });

    View full-size slide

  40. #frontendtesting
    describe(“Acceptance tests”, function () {
    var page;
    beforeEach(function (done) {
    page = visit(“/home”);
    page.ready(done);
    });
    describe(“Sign Up Failure state”, function () {
    beforeEach(function (done) {
    page.fill_in(“input[name='email']", “Not An Email”);
    page.click(“button[type=submit]”);
    page.onBodyChange(done);
    });
    it(“Shouldn’t allow signup with invalid information”, function () {
    expect(page.find(“#signupError”)
    .hasClass(“hidden”)).toBeFalsy();
    });
    });
    });

    View full-size slide

  41. #frontendtesting
    describe(“Acceptance tests”, function () {
    var page;
    beforeEach(function (done) {
    page = visit(“/home”);
    page.ready(done);
    });
    describe(“Sign Up Failure state”, function () {
    beforeEach(function (done) {
    page.fill_in(“input[name='email']", “Not An Email”);
    page.click(“button[type=submit]”);
    page.onBodyChange(done);
    });
    it(“Shouldn’t allow signup with invalid information”, function () {
    expect(page.find(“#signupError”)
    .hasClass(“hidden”)).toBeFalsy();
    });
    });
    });

    View full-size slide

  42. #frontendtesting
    More tools!
    • jasmine-integration
    • Karma
    • Selenium
    • Nightwatch
    • Again, many others

    View full-size slide

  43. #frontendtesting
    Visual
    Regression
    Performance Accessibility
    Unit Integration Acceptance

    View full-size slide

  44. #frontendtesting
    Visual

    Regression
    Performance Accessibility
    Unit Integration Acceptance

    View full-size slide

  45. #frontendtesting
    The Evolution
    • Front-end development is more than just
    business logic
    • There are other parts of the code that can fail
    • Apply the tactic of testing expected outcomes

    View full-size slide

  46. #frontendtesting
    Visual regression tests
    check for inconsistencies in the view.

    View full-size slide

  47. #frontendtesting
    Before After Difference

    View full-size slide

  48. #frontendtesting

    View full-size slide

  49. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(“#signUpForm”, {
    name: “Alicia Sedlock”,
    email: “[email protected]
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });

    View full-size slide

  50. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(‘#signUpForm', {
    name: ‘Alicia Sedlock’,
    email: ‘[email protected]
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });

    View full-size slide

  51. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(‘#signUpForm', {
    name: ‘Alicia Sedlock’,
    email: ‘[email protected]
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });

    View full-size slide

  52. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(‘#signUpForm', {
    name: ‘Alicia Sedlock’,
    email: ‘[email protected]
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });

    View full-size slide

  53. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(‘#signUpForm', {
    name: ‘Alicia Sedlock’,
    email: ‘[email protected]
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });

    View full-size slide

  54. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(‘#signUpForm', {
    name: ‘Alicia Sedlock’,
    email: ‘[email protected]
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });

    View full-size slide

  55. #frontendtesting
    casper.start(“/home”).then(function(){
    // Initial state of form
    phantomcss
    .screenshot(“#signUpForm”, “sign up form”);
    // Hit the sign up button (should trigger error)
    casper.click(“button#signUp”);
    // Take a screenshot of the UI component
    phantomcss
    .screenshot(“#signUpForm”, “sign up form error”);
    // Fill in form by name attributes & submit
    casper.fill(‘#signUpForm', {
    name: ‘Alicia Sedlock’,
    email: ‘[email protected]
    }, true);
    // Take a second screenshot of success state
    phantomcss
    .screenshot(“#signUpForm”, “sign up form success”);
    });

    View full-size slide

  56. #frontendtesting
    Visual regression tests
    full pages vs. components

    View full-size slide

  57. #frontendtesting
    casper.start(“/styleguide”).then(function(){
    phantomcss
    .screenshot(“a.link”, “link example”);
    phantomcss
    .screenshot(“.btn-primary”, “primary button”);
    }).then(function() {
    this.mouse.move(“.btn-primary”');
    phantomcss
    .screenshot(“.btn-primary”, “button hover”);
    });

    View full-size slide

  58. #frontendtesting
    Visual regression tests
    and that responsive design thing…

    View full-size slide

  59. #frontendtesting
    phantomcss: {
    options: {
    mismatchTolerance: 0.05,
    screenshots: ‘tests/visual/baselines’,
    results: ‘tests/visual/results’,
    viewportSize: [1280, 800]
    },
    src: [‘tests/visual/tests.js’]
    }

    View full-size slide

  60. #frontendtesting
    phantomcss: {
    big_size: {
    options: {
    ...
    viewportSize: [1280, 800]
    }
    },
    small_size: {
    options: {
    ...
    viewportSize: [480, 320]
    }
    }
    }

    View full-size slide

  61. The workflow, though…
    #frontendtesting

    View full-size slide

  62. #frontendtesting

    View full-size slide

  63. #frontendtesting

    View full-size slide

  64. #frontendtesting

    View full-size slide

  65. #frontendtesting
    Visual regression tools
    • PhantomCSS/PhantomFlow
    • BackstopJS
    • Wraith
    • Webdriver.io
    • Percy.io

    View full-size slide

  66. #frontendtesting
    Accessibility tests
    compare your site against
    accessibility standards

    View full-size slide

  67. #frontendtesting

    View full-size slide

  68. #frontendtesting
    Tools I’ve heard of
    • a11y
    • use grunt-a11y for build tool automation
    • Pa11y
    • ember-a11y-testing
    • protractor-accessibility-plugin (Angular)
    • react-a11y

    View full-size slide

  69. #frontendtesting
    Performance tests
    keep your project honest about
    performance

    View full-size slide

  70. #frontendtesting
    perfbudget: {
    default: {
    options: {
    url: [your site’s URL],
    budget: {
    visualComplete: '3000',
    SpeedIndex: ‘1500’,
    render: ‘500’,
    fullyLoaded: ‘3000’,
    requests: ’25’,
    bytesInDoc: '100000'
    }
    }
    }
    }

    View full-size slide

  71. #frontendtesting
    Tools for performance
    • grunt-perfbudget
    • gulp size
    • perf.js

    View full-size slide

  72. #frontendtesting
    Monkey testing
    is something I just read about
    yesterday morning.

    View full-size slide

  73. #frontendtesting

    View full-size slide

  74. #frontendtesting
    var horde = gremlins.createHorde();
    horde.unleash();

    View full-size slide

  75. #frontendtesting
    gremlins.createHorde()
    .gremlin(gremlins.species.formFiller())
    .gremlin(gremlins.species.clicker().clickTypes(['click']))
    .gremlin(gremlins.species.toucher())
    .gremlin(gremlins.species.scroller())
    .gremlin(gremlins.species.typer())
    .unleash();

    View full-size slide

  76. #frontendtesting
    gremlin formFiller input 5 in
    gremlin formFiller input [email protected] in
    gremlin clicker click at 1219 301
    gremlin scroller scroll to 100 25
    ...
    mogwai fps 33.21
    mogwai fps 59.45
    mogwai fps 12.67
    err > mogwai fps 7.54 < err

    View full-size slide

  77. #frontendtesting
    A single library
    • Gremlins.js

    View full-size slide

  78. What about linting?
    #frontendtesting

    View full-size slide

  79. #frontendtesting
    Interested in linting?
    • CSSLint
    • JSLint/JSHint
    • UnCSS
    • HTMLLint
    • Framework-specific linting, too!

    View full-size slide

  80. So, you’re saying I should
    use all these tools, right?
    #frontendtesting

    View full-size slide

  81. #frontendtesting
    Assessment
    • How much business logic are we off-loading to
    the client-side?
    • What is our history with regressions?
    • Are we constantly fixing, and re-fixing, visual/UI
    issues?
    • In general, which piece of our stack is the most
    instable/Jenga-like?
    • Is there a lot of inconsistency in how we write
    code?

    View full-size slide

  82. Yes, you can go
    overboard with tests.
    #frontendtesting

    View full-size slide

  83. #frontendtesting
    Visual

    Regression
    Performance Accessibility
    Unit Integration Acceptance

    View full-size slide

  84. #frontendtesting
    Visual

    Regression
    Performance
    Accessibility
    Acceptance

    View full-size slide

  85. #frontendtesting
    Visual

    Regression
    Performance
    Accessibility

    View full-size slide

  86. #frontendtesting
    Accessibility

    View full-size slide

  87. #frontendtesting
    Accessibility
    Unit
    Integration Acceptance

    View full-size slide

  88. Yes, writing good
    tests takes practice.
    #frontendtesting

    View full-size slide

  89. #frontendtesting

    View full-size slide

  90. The Excuse Corner

    View full-size slide

  91. This is more code to
    write.
    #frontendtesting

    View full-size slide

  92. This is more code to
    write, and my code will
    still have bugs.
    #frontendtesting

    View full-size slide

  93. I’m working with legacy
    code, and it’ll be impossible
    to test old features.
    #frontendtesting

    View full-size slide

  94. #frontendtesting
    Build A Test-Conscience Team
    through education, experimentation,
    and practice

    View full-size slide

  95. #frontendtesting
    • Frontend Architecture for
    Design Systems by Micah
    Godbolt

    • FrontEndTesting.com
    Looking for more?

    View full-size slide