Tweet
Tweet

Slide 1

Slide 1 text

LNUG July 2016 Mark Wubben novemberborn.net @novemberborn Hi! My name’s Mark Wubben. I’m a humanist & technologist who loves working with the web. I’m here today to talk about AVA, the futuristic test runner. I’m currently based here in London, mixing open source development with contracting.

Slide 2

Slide 2 text

Make Testing Less Frustrating Writing unit tests can be frustrating. AVA tries to make it less so. I’d even say makes testing fun, but that’s rarely the case. Also, it’s Mocha’s slogan Let’s quickly go through some of the ways in which AVA helps make testing less frustrating.

Slide 3

Slide 3 text

ES2015 Built-In You can write your tests using ES2015, regardless of whether you’re restricting yourself to ES5 in your application code. AVA has Babel built-in so it can’t be easier.

Slide 4

Slide 4 text

Async / Await AVA lets you use async/await in your tests, even if you’re not using it in the rest of your codebase. Which is fantastic because it lets you write tests without returning promises all the time. This makes testing asynchronous code a lot easier.

Slide 5

Slide 5 text

You’ve probably struggled remembering the specific API of your assertion library. AVA comes with “power-assert” built-in. This library, by Takuto Wada, lets you write assertions using plain JavaScript expressions. If the assertion fails it’ll show you what the values were, so you can easily spot the error. power-assert: https://github.com/power-assert-js/

Slide 6

Slide 6 text

t.true(arr.indexOf(zero) === two) | | | | | | | | | 2 | -1 0 false [1,2,3] Here’s an example of when an assertion failed. You can see the assertion in question and each intermediate value. You can easily see that the array does not include zero without having to write debug statements. It’s also easier to see what it is your testing without having to look up the documentation of a particular assertion method. Now you don’t need to memorise Chai!

Slide 7

Slide 7 text

Opinionated Test Interface AVA doesn’t come with any of the test interfaces you may be used to, like BDD, TDD or an exports object. Instead you get a test function which you can call with a title and test implementation.

Slide 8

Slide 8 text

import test from 'ava' test('1 + 1 = 2', t => { t.true(1 + 1 === 2) }) AVA lets you write tests, but not test hierarchies. This lets you focus on testing what your program does, without having to mimic how it’s structured. If you have more than one test file AVA will automatically prepend the file name to the test titles, so there’s no need to even repeat it in the file itself.

Slide 9

Slide 9 text

Process Isolation AVA starts a new Node.js process for every test file. This means you can safely modify global state without accidentally affecting unrelated tests. It can also run multiple test files concurrently, each in their own process.

Slide 10

Slide 10 text

AVA is, at the time of writing*, 79 contributors (and that’s just in the main repository). It’s also got well over 5,000 stars so we’ve convinced a lot of people to press that button. * July 24th, 2016

Slide 11

Slide 11 text

The download count isn’t too shabby either, undoubtedly helped by CI installations ;-) According to https://gist.github.com/feross/e0882df2fe673d6ce064 we’re the 80th most depended on package in npm.

Slide 12

Slide 12 text

AVA was started by Sindre Sorhus and Kevin Mårtensson in November 2014. Other members of the core team are Vadim Demedes, James Talmage and Juan Soto. We’re there to help with issues and contributions from others. Whilst we’re quite familiar with AVA’s internals none of us understand all of it. Major contributions have been made by people not on the core team — it’s not about us, it’s about building a great test runner.

Slide 13

Slide 13 text

We’re not just looking for code contributions either. Updates to the documentation and recipes on how to use AVA are incredibly valuable. (Those “closed” PRs were merged manually, before GitHub released squash&merge.)

Slide 14

Slide 14 text

And then there’s the awesome people who help translate the documentation into, so far, eight languages, with 20 contributors outside of the core team!

Slide 15

Slide 15 text

And then there’s the awesome people who help translate the documentation into, so far, eight languages, with 20 contributors outside of the core team!

Slide 16

Slide 16 text

Moar Features Let’s get back to what AVA does. We’ve already talked about async/await, power-assert, the test interface and process isolation.

Slide 17

Slide 17 text

Per-Process Concurrency AVA runs each test file in its own process. But within each test file it by default starts all tests at the same time. So if you have asynchronous tests these will run concurrently, hopefully resulting in a faster test execution.

Slide 18

Slide 18 text

test('first', async t => { console.log('start first') await new Promise(resolve => setTimeout(resolve, 500)) console.log('finish first') }) test('second', async t => { console.log('start second') await new Promise(resolve => setTimeout(resolve, 100)) console.log('finish second') }) This example has two asynchronous tests. The second will finish before the first.

Slide 19

Slide 19 text

start first start second finish second finish first Which means we get this log output. This can be a problem if both tests share state. Ideally you make sure that’s not the case, but if you can’t…

Slide 20

Slide 20 text

test.serial('first', async t => { console.log('start first') await new Promise(resolve => setTimeout(resolve, 500)) console.log('finish first') }) test('second', async t => { console.log('start second') await new Promise(resolve => setTimeout(resolve, 100)) console.log('finish second') }) console.log('start first') await new Promise(resolve => setTimeout(resolve, 500)) console.log('finish first') }) test('second', async t => { console.log('start second') await new Promise(resolve => setTimeout(resolve, 100)) console.log('finish second') }) You can mark individual test as serial. AVA will execute them first, in order, one at a time, before starting the remaining tests. (There’s also a --serial command line flag which causes all tests to run serially.)

Slide 21

Slide 21 text

Test Modifiers .serial is a test modifier. There are plenty more.

Slide 22

Slide 22 text

test.before(() => {}) test.after(() => {}) These let you register hooks that run before the first test, and after the last. Note that due to process isolation you can usually get away with performing shared setup in the test file itself, and you don’t necessarily have to clean up either. (Any code that runs when the test file is loaded delays when tests actually start running, so there is an impact, but I find it nicer not to use test.before)

Slide 23

Slide 23 text

test.beforeEach(() => {}) test.afterEach(() => {}) These register hooks that are run before and after each test. Like tests, hooks run concurrently.

Slide 24

Slide 24 text

test.beforeEach(t => { t.context.foo = 'bar' }) test(t => { t.true(t.context.foo === 'bar') }) beforeEach and afterEach hooks receive the test object. This is mostly useful because of the context property that is shared with the tests. You can set up a new context for each test. (t.context can also be assigned to.)

Slide 25

Slide 25 text

test.skip('a test to ignore while developing', t => {}) test.only('a test to focus on while developing', t => {}) The .skip and .only modifiers should only be used during development. They can be useful when debugging a test, e.g. you can skip a test that distracts you, or run just those tests you’re having trouble getting right.

Slide 26

Slide 26 text

test.todo('a test you still need to write') test.failing('a test that showcases a bug', t => {}) The .todo modifier can be used when sketching out tests. They’re included in the AVA’s output so you won’t forget about them, which is better than using code comments. The .failing modifier is useful if you or a contributor knows how to reproduce a bug, but not yet how to solve it. You can check in the failing test without it breaking CI. This way a contributor gets the credit for finding the bug, and somebody else already has a test case for when they go try and fix it.

Slide 27

Slide 27 text

test.cb('doSomething() invokes the callback', t => { doSomething((err, result) => { t.true(err === null) t.true(result === 42) t.end() }) }) The .cb modifier can be used when testing (asynchronous) functions that invoke a callback. You must end these tests using t.end().

Slide 28

Slide 28 text

test.cb('doSomething() invokes the callback', t => { doSomething((err, result) => { t.true(err === null) t.true(result === 42) t.end() }) }) test.cb('doSomething() invokes the callback', t => { doSomething((err, result) => { t.true(err === null) t.true(result === 42) t.end() }) }) The .cb modifier can be used when testing (asychronous) functions that invoke a callback. You must end these tests using t.end().

Slide 29

Slide 29 text

import pify from 'pify' test('doSomething() invokes the callback', async t => { const result = await pify(doSomething)() t.true(result === 42) }) That said, you’re probably better off using async/await and promisifying the callback-taking function. AVA can’t trace uncaught exceptions back to the test. Using a promise you’re less likely to run into this problem. Here I’m using pify to promisify the function. See https://www.npmjs.com/package/pify.

Slide 30

Slide 30 text

Test Macros AVA makes it easy to reuse test implementations.

Slide 31

Slide 31 text

function macro(t, input, expected) { t.true(eval(input) === expected); } test('2 + 2 === 4', macro, '2 + 2', 4); test('2 * 3 === 6', macro, '2 * 3', 6); Here the same test implementation is used for two different tests. Additional arguments are passed to the macro itself.

Slide 32

Slide 32 text

function macro(t, input, expected) { t.true(eval(input) === expected) } macro.title = (_, input, expected) => { return `${input} === ${expected}` } test(macro, '2 + 2', 4) test(macro, '2 * 3', 6) macro.title = (_, input, expected) => { return `${input} === ${expected}` } test(macro, '2 + 2', 4) test(macro, '2 * 3', 6) Macros can even generate test titles. We encourage you to use macros, rather than generating tests. For instance it’s easier to statically analyse macros.

Slide 33

Slide 33 text

Assertions AVA has 15 built-in assertions. However with power-assert you won’t use all of them. Let’s look at the most useful ones. You can find the full list in our documentation of course: https://github.com/avajs/ava#assertions.

Slide 34

Slide 34 text

t.true(expression) t.false(expression) You’ve already seen the first one. t.false is the opposite. We recommend you use the t.true assertion as much as possible.

Slide 35

Slide 35 text

t.truthy(expression) t.falsy(expression) These are less strict than t.true and t.false. Of course you use the logical not operator to get an actual boolean.

Slide 36

Slide 36 text

t.truthy(expression) t.falsy(expression) t.truthy(expression) ! t.true(!!expression) t.falsy(expression) ! t.true(!expression) These are less strict than t.true and t.false. Of course you use the logical not operator to get an actual boolean.

Slide 37

Slide 37 text

t.deepEqual(value, expected) t.notDeepEqual(value, expected) These compare whether the actual value structurally matches the expectation. Value types (e.g. strings and numbers) are tested using strict equality.

Slide 38

Slide 38 text

t.throws(fn, TypeError) t.notThrows(fn, TypeError) These check whether the function throws the expected error. The assertion is currently a bit overloaded, I recommend you test by passing an error constructor.

Slide 39

Slide 39 text

const err = t.throws(fn, TypeError) t.true(err.message === 'Bad type') t.throws() returns the error that was thrown. So you can use power-assert to do further comparisons.

Slide 40

Slide 40 text

await t.throws(promise, TypeError) await t.notThrows(promise, TypeError) You can also use t.throws() to test promise rejections.

Slide 41

Slide 41 text

const err = await t.throws(promise, TypeError) t.true(err.message === 'Bad type') And you can get the rejection reason.

Slide 42

Slide 42 text

t.throws(fn(), TypeError) Also, if you’ve ever done this (calling the function rather than passing it to t.throws(), causing the exception to be thrown and your test to fail), AVA does a pretty good job at catching it and warning you about it.

Slide 43

Slide 43 text

t.pass() t.fail() You can use these assertions to ensure certain code paths are run, or indeed are not run.

Slide 44

Slide 44 text

t.fail('This should never happen') All assertions take a message as their last argument. For those rare cases where that’s actually useful.

Slide 45

Slide 45 text

t.plan(5) If you want you can specify how many assertions need to pass for your test to succeed. This can be useful in certain asynchronous or loop situations. If too few or too many assertions pass your test will fail. Note that AVA doesn’t automatically end your test once the expected count is reached.

Slide 46

Slide 46 text

t.skip.true(expression) You can also skip individual assertions. These are still counted so you don’t need to change the plan count.

Slide 47

Slide 47 text

⌚⌚⌚ Watch Mode AVA’s watch mode is fantastic. If you change a test file it’ll rerun just that file. If you change a source file it’ll rerun all tests that depend on it. And if can’t figure out which test to rerun it just reruns all. Demo time!

Slide 48

Slide 48 text

"scripts": { "test": "ava" }, "devDependencies": { "ava": "^0.15.2" } You should add AVA to your devDependencies. Then configure the test script to call AVA.

Slide 49

Slide 49 text

npm test -- --watch You can use npm to start AVA in watch mode.

Slide 50

Slide 50 text

"scripts": { "test": "ava", "watch:test": "ava --watch" } Or better yet, add it to your scripts.

Slide 51

Slide 51 text

npm run watch:test Now you can simply do this.

Slide 52

Slide 52 text

CLI There are some other flags that you can pass to the CLI.

Slide 53

Slide 53 text

npm test -- --serial Like the .serial test modifier, however this causes all tests in all test files to run serially. Note that AVA still runs test files concurrently. (You could also use these modifiers in your test script definition, of course, but you’re probably better off configuring them).

Slide 54

Slide 54 text

npm test -- --fail-fast Causes AVA to abruptly stop running tests once a failure occurs.

Slide 55

Slide 55 text

npm test -- --match '*end' npm test -- --match 'start*' Use match to run just those tests that match the expression. The expression syntax is purposefully limited, you can find more details in the documentation: https:// github.com/avajs/ava#running-tests-with-matching-titles.

Slide 56

Slide 56 text

npm test -- --timeout=5s AVA doesn’t support per-test timeouts. Instead you can tell it to exit if no test completes in the given timeout window.

Slide 57

Slide 57 text

npm test -- --tap npm test -- --verbose You can output test results in TAP format, or using the verbose reporter. Otherwise you get minimal output.

Slide 58

Slide 58 text

npm test -- my-test.js 'my-dir/**/*.js' You can also pass test files or glob patterns as the last arguments. Make sure to use quotes around the glob patterns to prevent your shell from trying to expand them. The same goes for the match argument!

Slide 59

Slide 59 text

Configuration CLI flags can be a bit cumbersome. Luckily AVA can also be configured in the package.json file.

Slide 60

Slide 60 text

"ava": { "failFast": true, "files": [ "my-test.js", "my-dir/**/*.js" ], "match": [ "*end", "start*" ], "serial": true, "timeout": "5s" "verbose": true } Use the AVA stanza to add your configuration. Some options can only be configured here.

Slide 61

Slide 61 text

"ava": { "source": [ "**/*.{js,jsx}", "!dist/**/*" ] } For instance this is where you’d customise the patterns used by the watcher to detect changes to your source files.

Slide 62

Slide 62 text

Purrfection So that’s it. AVA’s perfect and there are definitely no issues we need to talk about. OK so that’s not true.

Slide 63

Slide 63 text

So that’s it. AVA’s perfect and there are definitely no issues we need to talk about. OK so that’s not true.

Slide 64

Slide 64 text

Concurrency AVA starts a process for each of your test files, all at once. If you have lots of test files that can be a bit extreme, to the point where your machine freezes up and your CI build fails. We’re working on several improvements in this area. For now you should use the concurrency flag and option.

Slide 65

Slide 65 text

"ava": { "concurrency": 5 } npm test -- --concurrency=5 This limits AVA to five concurrent child processes. It’s currently an experimental feature while we figure out a good default and resolve issues with the .only modifier.

Slide 66

Slide 66 text

ES2015 Sources AVA transpiles your tests, but not your source files. We’re working on integrating the precompiler in AVA’s main process. In the meantime you have to require babel- register when you run AVA.

Slide 67

Slide 67 text

"ava": { "require": ["babel-register"] } npm test -- --require=babel-register Note that you need to make sure babel-register is installed. babel-register can be quite slow. It’ll be more efficient if you precompile your source files and test your precompiled code. This is my preferred strategy. Also, you can configure the Babel config AVA uses to compile the test files. I won’t go into those details though, please see the documentation: https://github.com/avajs/ ava/blob/master/docs/recipes/babelrc.md.

Slide 68

Slide 68 text

Browser Testing Currently AVA doesn’t support browser testing. We really want to make this work though, it’s just very difficult. That said, you can usually get pretty far by emulating the DOM. We’ve got a recipe for that: https://github.com/avajs/ava/blob/master/docs/recipes/browser-testing.md.

Slide 69

Slide 69 text

chdir Currently AVA sets the current working directory to that of the test file. This is more confusing than beneficial so we’ll be changing it. In the future the current working directory will be that of the package.json file.

Slide 70

Slide 70 text

Ecosystem There’s a fair amount that AVA does not do. Luckily there are other tools that can help.

Slide 71

Slide 71 text

Code Coverage Currently AVA does not come with any code coverage support. We recommend you use nyc,

Slide 72

Slide 72 text

We recommend you use nyc. It’s now the official command line interface for Istanbul so you can’t go wrong. Both me and James Talmage are also contributors to nyc. https://www.npmjs.com/package/nyc

Slide 73

Slide 73 text

Test Quality AVA has its own ESLint plugin. It’ll help you with AVA’s best practices.

Slide 74

Slide 74 text

We couldn’t have done eslint-plugin-ava without the help of Jeroen Engels: https://github.com/jfmengels.

Slide 75

Slide 75 text

Editor Plugins We have plugins for Sublime, Atom and VSCode. As well as integrations with Grunt, Gulp, and so on. Find the links in our documentation: https://github.com/avajs/ ava#related.

Slide 76

Slide 76 text

And if you really like BDD, there’s a third-party package that lets you use describe and it to your harts content.

Slide 77

Slide 77 text

ava.li @ava__js gitter.im/avajs/ava You can find AVA on https://ava.li. We’re also on Gitter: https://gitter.im/avajs/ava, and of course Twitter: https://twitter.com/ava__js.

Slide 78

Slide 78 text

I’ve been Mark Wubben novemberborn.net @novemberborn t.end() Thanks!