Tweet
Tweet

Slide 1

Slide 1 text

Testing in JavaScript (a quick introduction) *based on real failures experience Jakub Synowiec @jakubsynowiec

Slide 2

Slide 2 text

Jakub Synowiec Software Development Manager @ Future Mind @jakubsynowiec

Slide 3

Slide 3 text

What is „testing”? Jakub Synowiec @jakubsynowiec

Slide 4

Slide 4 text

But I ran it and it didn't blow up… Jakub Synowiec @jakubsynowiec

Slide 5

Slide 5 text

Testing The process consisting of all life cycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects. Jakub Synowiec @jakubsynowiec

Slide 6

Slide 6 text

what? Jakub Synowiec @jakubsynowiec

Slide 7

Slide 7 text

Testing An investigation conducted to provide stakeholders with information about the quality of the product or service. Jakub Synowiec @jakubsynowiec

Slide 8

Slide 8 text

testing acceptance testing alpha testing beta testing safety testing accuracy testing suitability testing exploratory testing integration testing interface testing component testing negative testing load testing recovery testing failover testing stress testing data flow testing regression testing scalability testing user story testing big-bang testing top-down testing bottom-up testing Jakub Synowiec @jakubsynowiec active testing age testing all-pairs testing boundary value testing component testing compilance testing conversion testing domain testing endurance testing fuzz testing loop testing penetration testing sanity testing smoke testing static testing

Slide 9

Slide 9 text

What is „testing” for us? Jakub Synowiec @jakubsynowiec *usually

Slide 10

Slide 10 text

Unit Testing automated Automated test comparison of actual results from testing of individual software components (units) with expected results. Jakub Synowiec @jakubsynowiec

Slide 11

Slide 11 text

But, what is a unit? Jakub Synowiec @jakubsynowiec

Slide 12

Slide 12 text

Software unit Software units are software items that can't be split into sub-items (IEC 62304) Jakub Synowiec @jakubsynowiec

Slide 13

Slide 13 text

what? Jakub Synowiec @jakubsynowiec

Slide 14

Slide 14 text

Software unit • a set of procedures or functions, in a procedural or functional language, • a class and its nested classes, in an object or object-oriented language, • a set of procedures/functions/classes grouped to implement a pattern, Jakub Synowiec @jakubsynowiec

Slide 15

Slide 15 text

Software unit • a set of procedures or functions, in a procedural or functional language, • a class and its nested classes, in an object or object-oriented language, • a set of procedures/functions/classes grouped to implement a pattern, Jakub Synowiec @jakubsynowiec

Slide 16

Slide 16 text

Software unit /** * Gets the last element of `array`. * * @static * @memberOf _ * @since 0.1.0 * @category Array * @param {Array} array The array to query. * @returns {*} Returns the last element of `array`. * @example * * _.last([1, 2, 3]); * ! " 3 # function last(array) { var length = array ? array.length : 0; return length ? array[length - 1] : undefined; } module.exports = last; Jakub Synowiec @jakubsynowiec

Slide 17

Slide 17 text

Software unit At a high-level, unit testing refers to the practice of testing certain functions and areas – or units – of our code. Jakub Synowiec @jakubsynowiec

Slide 18

Slide 18 text

Unit test Jakub Synowiec @jakubsynowiec

Slide 19

Slide 19 text

Test Suite A set of several test cases for a component or system under test, where the post condition of one test is often used as the precondition for the next one. Test Condition An item or event of a component or system that could be verified by one or more test cases, e.g. a function, transaction, feature, quality attribute, or structural element. Test Case A set of input values, execution preconditions, expected results and execution postconditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement. Test Comparison The process of identifying differences between the actual results produced by the component or system under test and the expected results for a test. Jakub Synowiec @jakubsynowiec

Slide 20

Slide 20 text

Jakub Synowiec @jakubsynowiec describe('Observable', () " { ! test suite it('Should be subscribable', () " { ! test case const instance = new ko.observable(); expect(ko.isSubscribable(instance)).toEqual(true); ! test comparison }); it('Should advertise that instances are observable', () " { ! test case const instance = new ko.observable(); expect(ko.isObservable(instance)).toEqual(true); ! test comparison }); ! $ }); Test suite

Slide 21

Slide 21 text

Jakub Synowiec @jakubsynowiec describe('Observable', () " { ! test suite it('Should be subscribable', () " { ! test case const instance = new ko.observable(); expect(ko.isSubscribable(instance)).toEqual(true); ! test comparison }); it('Should advertise that instances are observable', () " { ! test case const instance = new ko.observable(); expect(ko.isObservable(instance)).toEqual(true); ! test comparison }); ! $ }); Test case

Slide 22

Slide 22 text

Jakub Synowiec @jakubsynowiec describe('Observable', () " { ! test suite it('Should be subscribable', () " { ! test case const instance = new ko.observable(); expect(ko.isSubscribable(instance)).toEqual(true); ! test comparison }); it('Should advertise that instances are observable', () " { ! test case const instance = new ko.observable(); expect(ko.isObservable(instance)).toEqual(true); ! test comparison }); ! $ }); Test comparison

Slide 23

Slide 23 text

Jakub Synowiec @jakubsynowiec describe('Observable', () " { ! test suite it('Should be subscribable', () " { ! test case const instance = new ko.observable(); expect(ko.isSubscribable(instance)).toEqual(true); ! test comparison }); it('Should advertise that instances are observable', () " { ! test case const instance = new ko.observable(); expect(ko.isObservable(instance)).toEqual(true); ! test comparison }); ! $ }); Unit test

Slide 24

Slide 24 text

Unit testing makes developer lives easier* • easier to find bugs, • easier to maintain, • easier to understand, • easier to develop. Jakub Synowiec @jakubsynowiec

Slide 25

Slide 25 text

Easier to understand Reading tests can be a great way of understanding how things work and should be used. It’s often better than the documentation. Jakub Synowiec @jakubsynowiec

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

What is needed • proper approach to tests design, • tests following the AAA system, • tests without branching or looping, • doing test reviews, Jakub Synowiec @jakubsynowiec

Slide 28

Slide 28 text

Test-First, Test-After, Test-during (TDD)? Jakub Synowiec @jakubsynowiec

Slide 29

Slide 29 text

Test-After Jakub Synowiec @jakubsynowiec • If you write your API and then you write your tests against it, you often realise that your API isn't what it should be.

Slide 30

Slide 30 text

TDD Jakub Synowiec @jakubsynowiec • TDD is OK if you don’t know the algorithm, you can use TDD for algorithm triangulation,

Slide 31

Slide 31 text

Test-First Jakub Synowiec @jakubsynowiec • You are thinking about your code from the standpoint of the consumer of your code, • Often you discover something that you didn't know that was important, that'll make the code better, • You find that if you designed something that's hard to test, it's usually because you haven't designed it very well. It has strayed from its purpose, trying to do too much.

Slide 32

Slide 32 text

Unit test should be Jakub Synowiec @jakubsynowiec • Trustworthy & consistent, • Repeatable, • Easy to write & read, • Fast & short, • Separated,

Slide 33

Slide 33 text

The don'ts Jakub Synowiec @jakubsynowiec

Slide 34

Slide 34 text

Jakub Synowiec @jakubsynowiec Test name is the first thing you read QUnit.test( "jQuery('massive html #7990')", function( assert ) { assert.expect( 3 ); var i, li = "
  • very very very very large html string %li>", html = [ "
      " ]; for ( i = 0; i < 30000; i += 1 ) { html[ html.length ] = li; } html[ html.length ] = " %ul>"; html = jQuery( html.join( "" ) )[ 0 ]; assert.equal( html.nodeName.toLowerCase(), "ul" ); assert.equal( html.firstChild.nodeName.toLowerCase(), "li" ); assert.equal( html.childNodes.length, 30000 ); } );

    • Slide 35

    Slide 35 text

    Jakub Synowiec @jakubsynowiec Loops and branches == bugs QUnit.test( "jQuery('massive html #7990')", function( assert ) { assert.expect( 3 ); var i, li = "
  • very very very very large html string %li>", html = [ "
      " ]; for ( i = 0; i < 30000; i += 1 ) { html[ html.length ] = li; } html[ html.length ] = " %ul>"; html = jQuery( html.join( "" ) )[ 0 ]; assert.equal( html.nodeName.toLowerCase(), "ul" ); assert.equal( html.firstChild.nodeName.toLowerCase(), "li" ); assert.equal( html.childNodes.length, 30000 ); } );

    • Slide 36

    Slide 36 text

    Jakub Synowiec @jakubsynowiec No commented out tests ! can actually yield more than one, when iframes are included, the window is an array as well assert.equal( jQuery( window ).length, 1, "Correct number of elements generated for jQuery(window)" ); & ! disabled since this test was doing nothing. i tried to fix it but i'm not sure ! what the expected behavior should even be. FF returns "\n" for the text node ! make sure this is handled var crlfContainer = jQuery('

    \r\n %p>'); var x = crlfContainer.contents().get(0).nodeValue; assert.equal( x, what???, "Check for 'r and 'n in jQuery()" ); # & ! Disabled until we add this functionality in var pass = true; try { jQuery("

    Testing % div>").appendTo(document.getElementById("iframe").contentDocument.body); } catch(e){ pass = false; } assert.ok( pass, "jQuery('<tag>') needs optional document parameter to ease cross-frame DOM wrangling, see #968" ); # assert.equal( code.length, 1, "Correct number of elements generated for code" );

    Slide 37

    Slide 37 text

    Jakub Synowiec @jakubsynowiec Do not use test case as test suite QUnit.test( "attr(String)", function( assert ) { assert.expect( 50 ); var extras, body, $body, select, optgroup, option, $img, styleElem, $button, $form, $a; assert.equal( jQuery( "#text1" ).attr( "type" ), "text", "Check for type attribute" ); assert.equal( jQuery( "#radio1" ).attr( "type" ), "radio", "Check for type attribute" ); assert.equal( jQuery( "#check1" ).attr( "type" ), "checkbox", "Check for type attribute" ); assert.equal( jQuery( "#simon1" ).attr( "rel" ), "bookmark", "Check for rel attribute" ); assert.equal( jQuery( "#google" ).attr( "title" ), "Google!", "Check for title attribute" ); assert.equal( jQuery( "#mark" ).attr( "hreflang" ), "en", "Check for hreflang attribute" ); assert.equal( jQuery( "#en" ).attr( "lang" ), "en", "Check for lang attribute" ); assert.equal( jQuery( "#simon" ).attr( "class" ), "blog link", "Check for class attribute" ); assert.equal( jQuery( "#name" ).attr( "name" ), "name", "Check for name attribute" ); assert.equal( jQuery( "#text1" ).attr( "name" ), "action", "Check for name attribute" ); assert.ok( jQuery( "#form" ).attr( "action" ).indexOf( "formaction" ) ) 0, "Check for action attribute" ); assert.equal( jQuery( "#text1" ).attr( "value", "t" ).attr( "value" ), "t", "Check setting the value attribute" ); assert.equal( jQuery( "#text1" ).attr( "value", "" ).attr( "value" ), "", "Check setting the value attribute to empty string" ); assert.equal( jQuery( "
    %div>" ).attr( "value" ), "t", "Check setting custom attr named 'value' on a div" ); assert.equal( jQuery( "#form" ).attr( "blah", "blah" ).attr( "blah" ), "blah", "Set non-existent attribute on a form" ); assert.equal( jQuery( "#foo" ).attr( "height" ), undefined, "Non existent height attribute should return undefined" ); ! [7472] & [3113] (form contains an input with name="action" or name="id") extras = jQuery( " %script>" ).appendTo( "#qunit-fixture" ); assert.equal( jQuery( "#tAnchor5" ).prop( "href" ), jQuery( "#scriptSrc" ).prop( "src" ), "Check for absolute src prop on a script" ); ! list attribute is readonly by default in browsers that support it jQuery( "#list-test" ).attr( "list", "datalist" ); assert.equal( jQuery( "#list-test" ).attr( "list" ), "datalist", "Check setting list attribute" ); ! Related to [5574] and [5683] body = document.body; $body = jQuery( body ); assert.strictEqual( $body.attr( "foo" ), undefined, "Make sure that a non existent attribute returns undefined" ); body.setAttribute( "foo", "baz" ); assert.equal( $body.attr( "foo" ), "baz", "Make sure the dom attribute is retrieved when no expando is found" ); $body.attr( "foo", "cool" ); assert.equal( $body.attr( "foo" ), "cool", "Make sure that setting works well when both expando and dom attribute are available" ); body.removeAttribute( "foo" ); ! Cleanup select = document.createElement( "select" ); optgroup = document.createElement( "optgroup" ); option = document.createElement( "option" ); optgroup.appendChild( option ); select.appendChild( optgroup ); assert.equal( jQuery( option ).prop( "selected" ), true, "Make sure that a single option is selected, even when in an optgroup." ); $img = jQuery( "text %button>" ).insertAfter( "#button" ); assert.strictEqual( $button.attr( "value" ), undefined, "Absence of value attribute on a button" ); assert.equal( $button.attr( "value", "foobar" ).attr( "value" ), "foobar", "Value attribute on a button does not return innerHTML" ); assert.equal( $button.attr( "value", "baz" ).html(), "text", "Setting the value attribute does not change innerHTML" ); ! Attributes with a colon on a table element (#1591) assert.equal( jQuery( "#table" ).attr( "test:attrib" ), undefined, "Retrieving a non-existent attribute on a table with a colon does not throw an error." ); assert.equal( jQuery( "#table" ).attr( "test:attrib", "foobar" ).attr( "test:attrib" ), "foobar", "Setting an attribute on a table with a colon does not throw an error." ); $form = jQuery( " %form>" ).appendTo( "#qunit-fixture" ); assert.equal( $form.attr( "class" ), "something", "Retrieve the class attribute on a form." ); $a = jQuery( "Click %a>" ).appendTo( "#qunit-fixture" ); assert.equal( $a.attr( "onclick" ), "something()", "Retrieve ^on attribute without anonymous function wrapper." ); assert.ok( jQuery( "
    %option> %select>" ).attr( "value" ), undefined, "An unset value on a select returns undefined." ); $form = jQuery( "#form" ).attr( "enctype", "multipart/form-data" ); assert.equal( $form.prop( "enctype" ), "multipart/form-data", "Set the enctype of a form (encoding in IE6/7 #6743)" ); } ); 116 lines, 50 test cases

    Slide 38

    Slide 38 text

    The dos Jakub Synowiec @jakubsynowiec

    Slide 39

    Slide 39 text

    Arrange, Act, Assert Jakub Synowiec @jakubsynowiec describe('arrayFirst', () " { var matchB; beforeEach(() " { ! arrange matchB = jasmine.createSpy('matchB').andCallFake(x " x.charAt(0) - 'b'); }); it('Should return the first matching element from the input array', () " { var result = ko.utils.arrayFirst(['a', 'b', 'c', 'b2'], matchB); ! act expect(result).toBe('b'); ! assert }); ! $ });

    Slide 40

    Slide 40 text

    Arrange Jakub Synowiec @jakubsynowiec describe('arrayFirst', () " { var matchB; beforeEach(() " { ! arrange matchB = jasmine.createSpy('matchB').andCallFake(x " x.charAt(0) - 'b'); }); it('Should return the first matching element from the input array', () " { var result = ko.utils.arrayFirst(['a', 'b', 'c', 'b2'], matchB); ! act expect(result).toBe('b'); ! assert }); ! $ });

    Slide 41

    Slide 41 text

    Act Jakub Synowiec @jakubsynowiec describe('arrayFirst', () " { var matchB; beforeEach(() " { ! arrange matchB = jasmine.createSpy('matchB').andCallFake(x " x.charAt(0) - 'b'); }); it('Should return the first matching element from the input array', () " { var result = ko.utils.arrayFirst(['a', 'b', 'c', 'b2'], matchB); ! act expect(result).toBe('b'); ! assert }); ! $ });

    Slide 42

    Slide 42 text

    Assert Jakub Synowiec @jakubsynowiec describe('arrayFirst', () " { var matchB; beforeEach(() " { ! arrange matchB = jasmine.createSpy('matchB').andCallFake(x " x.charAt(0) - 'b'); }); it('Should return the first matching element from the input array', () " { var result = ko.utils.arrayFirst(['a', 'b', 'c', 'b2'], matchB); ! act expect(result).toBe('b'); ! assert }); ! $ });

    Slide 43

    Slide 43 text

    Short Jakub Synowiec @jakubsynowiec describe('arrayFirst', () " { var matchB; beforeEach(() " { ! arrange matchB = jasmine.createSpy('matchB').andCallFake(x " x.charAt(0) - 'b'); }); it('Should return the first matching element from the input array', () " { var result = ko.utils.arrayFirst(['a', 'b', 'c', 'b2'], matchB); ! act expect(result).toBe('b'); ! assert }); ! $ });

    Slide 44

    Slide 44 text

    Test Separation Jakub Synowiec @jakubsynowiec • Isolation - sociable vs solitary tests, • Full control of dependencies, sociable test solitary test

    Slide 45

    Slide 45 text

    Jakub Synowiec @jakubsynowiec Test Doubles • Test Double is simply another object that conforms to the interface of the required Collaborator, and can be passed in in its place, • Dummy is an object that simply implements an Interface, and does nothing else, • Stub is a Dummy that returns responses in order to allow to make assertions, • Spies are Stubs that maintain state and allow to check the parameters or total number of times a method was called, • Fakes not only return values, but they also work like a real Collaborator. Usually a Fake has some sort of a shortcut unusable in production.

    Slide 46

    Slide 46 text

    Jakub Synowiec @jakubsynowiec var matchB; beforeEach(() " { matchB = jasmine .createSpy('matchB') ! spy .andCallFake(x " x.charAt(0) - 'b'); ! stub }); Test Double

    Slide 47

    Slide 47 text

    Jakub Synowiec @jakubsynowiec var matchB; beforeEach(() " { matchB = jasmine .createSpy('matchB') ! spy .andCallFake(x " x.charAt(0) - 'b'); ! stub }); Stub

    Slide 48

    Slide 48 text

    Jakub Synowiec @jakubsynowiec var matchB; beforeEach(() " { matchB = jasmine .createSpy('matchB') ! spy .andCallFake(x " x.charAt(0) - 'b'); ! stub }); Spy

    Slide 49

    Slide 49 text

    Jakub Synowiec @jakubsynowiec Mocks are not Stubs Mocks are test doubles that make assertions on behaviour. Mocks use behaviour verification, where we instead (of asserting results) check to see if the correct calls on the mocked Collaborator were made. We do this check by telling the mock what to expect during setup and asking the mock to verify itself during verification.

    Slide 50

    Slide 50 text

    Jakub Synowiec @jakubsynowiec Setup Mock it('should pass object with correct values to save only once', () " { var user = { name: 'test' }; var expectedUser = { name: user.name, nameLowercase: user.name.toLowerCase() }; ! mock setup var database = sinon.mock(Database); database.expects('save').once().withArgs(expectedUser); ! eof mock setup setupNewUser(user); ! verify mock database.verify(); database.restore(); });

    Slide 51

    Slide 51 text

    Jakub Synowiec @jakubsynowiec Verify Mock it('should pass object with correct values to save only once', () " { var user = { name: 'test' }; var expectedUser = { name: user.name, nameLowercase: user.name.toLowerCase() }; ! mock setup var database = sinon.mock(Database); database.expects('save').once().withArgs(expectedUser); ! eof mock setup setupNewUser(user); ! verify mock database.verify(); database.restore(); });

    Slide 52

    Slide 52 text

    Jakub Synowiec @jakubsynowiec

    Slide 53

    Slide 53 text

    Jakub Synowiec @jakubsynowiec Test Doubles - when to use • You should use Stubs (or Stubs + Spies) to isolate units from dependencies (internal APIs), • You can use Fakes to simplify Collaborators, e.g. in memory database, • You should only use Mocks to mock external APIs, e.g. remote HTTP calls,

    Slide 54

    Slide 54 text

    Test Review Jakub Synowiec @jakubsynowiec • Understand intent of the developer, • Verify understanding of the business value, • Many times quicker than a full code review, • You can drill-down if needed,

    Slide 55

    Slide 55 text

    Jakub Synowiec @jakubsynowiec

    Slide 56

    Slide 56 text

    How to start? Jakub Synowiec @jakubsynowiec

    Slide 57

    Slide 57 text

    Jakub Synowiec @jakubsynowiec

    Slide 58

    Slide 58 text

    4 main excuses • It costs too much time, we’d better fix these bugs first, • Our system/code is too big for unit tests, • We just ought to be more careful not to introduce bugs, that will help us a lot, • Our system is not suited for unit tests. Jakub Synowiec @jakubsynowiec

    Slide 59

    Slide 59 text

    Too much time Yes, it costs time to design and write tests, but you will win that time back, along with other benefits. Jakub Synowiec @jakubsynowiec

    Slide 60

    Slide 60 text

    System is to big If you start writing tests now, steadily your codebase will be covered more and more by tests. Jakub Synowiec @jakubsynowiec

    Slide 61

    Slide 61 text

    Should be more careful Everybody writes bugs, we all write bugs. The more complex problem, the more mistakes are made. But if you can detect it, you can fix it before it does any damage. Jakub Synowiec @jakubsynowiec

    Slide 62

    Slide 62 text

    System is not suited This one might actually be true, but “Your special case isn’t special enough”. If your units are not separated and are tightly coupled or your units are not „pure”, then your system is poorly designed and hardly testable. But if its poorly designed, it is likely to break and modifying it will be super painful. Jakub Synowiec @jakubsynowiec

    Slide 63

    Slide 63 text

    Save the world… A good test reads like a little story: I set something up, I change it, I test it. Jakub Synowiec @jakubsynowiec

    Slide 64

    Slide 64 text

    …one unit at a time. When implementing a new functionality (or refactoring an old one), first think about the desired results, necessary parameters and required interfaces. Write your tests while designing your code (Test-First). Your tests should guide you through the implementation. Jakub Synowiec @jakubsynowiec

    Slide 65

    Slide 65 text

    Write tests Jakub Synowiec @jakubsynowiec 1. Select unit, design your tests, 2. Arrange for testing: 1. declare local variables, 2. set state, 3. load fixtures, 4. stub internal APIs, 5. mock external APIs, 3. Act - execute tested unit, 4. Assert - compare actual results with expected results, 5. Repeat,

    Slide 66

    Slide 66 text

    When it’s hard to test. If something is hard to test, it's usually because you haven't designed it very well. It has strayed from its purpose, trying to do too much. Split it to smaller units, extract some functionality to modules. Jakub Synowiec @jakubsynowiec

    Slide 67

    Slide 67 text

    When it’s hard to test… return new Promise((resolve, reject) " { request ! $ .end((err, res) " { if (res . res.ok) { resolve(res.body.reduce((acc, current, userId) " { userId = decode(jwt).user.id; if (current.device . current.device.imei) { if (current.owner.id - userId / (current.state . Object.keys(current.state).length > 0)) { acc.push(current.device.imei); } } return acc; }, [])); } else { handleError(err, reject); } }); }); Jakub Synowiec @jakubsynowiec

    Slide 68

    Slide 68 text

    …extract unit return new Promise((resolve, reject) " { request ! $ .end((err, res) " { if (res . res.ok) { resolve(getUserAccessibleDevices(res.body, decode(jwt).user.id)); } else { handleError(err, reject); } }); }); Jakub Synowiec @jakubsynowiec

    Slide 69

    Slide 69 text

    Coverage, what coverage? Strive for 100%, because then you’ll hit 80-ish %, which is a good number, but if you say 80%, you're gonna end with 40%. But it depends…. Jakub Synowiec @jakubsynowiec

    Slide 70

    Slide 70 text

    Coverage, what coverage? Jakub Synowiec @jakubsynowiec

    Slide 71

    Slide 71 text

    When enough is enough Knowing when not to test is kind of the mark of experience and wisdom when it comes to testing. You should do it too much, and then learn to back off. Jakub Synowiec @jakubsynowiec

    Slide 72

    Slide 72 text

    Jakub Synowiec @jakubsynowiec

    Slide 73

    Slide 73 text

    Reference materials Jakub Synowiec @jakubsynowiec

    Slide 74

    Slide 74 text

    Reference materials • https://github.com/atinfo/awesome-test- automation/blob/master/javascript-test- automation.md • https://github.com/atinfo/awesome-test- automation/blob/master/automation-and-testing- as-service.md Jakub Synowiec @jakubsynowiec

    Slide 75

    Slide 75 text

    Thank you! Jakub Synowiec @jakubsynowiec