Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AVA @ LNUG

AVA @ LNUG

A talk on AVA, presented at the London Node User Group on July 27, 2016.

Mark Wubben

July 27, 2016
Tweet

More Decks by Mark Wubben

Other Decks in Programming

Transcript

  1. LNUG July 2016 Mark Wubben novemberborn.net @novemberborn Hi! My name’s

    Mark Wubben. I’m a humanist & technologist who loves working with the web. I’m here today to talk about AVA, the futuristic test runner. I’m currently based here in London, mixing open source development with contracting.
  2. Make Testing Less Frustrating Writing unit tests can be frustrating.

    AVA tries to make it less so. I’d even say makes testing fun, but that’s rarely the case. Also, it’s Mocha’s slogan Let’s quickly go through some of the ways in which AVA helps make testing less frustrating.
  3. ES2015 Built-In You can write your tests using ES2015, regardless

    of whether you’re restricting yourself to ES5 in your application code. AVA has Babel built-in so it can’t be easier.
  4. Async / Await AVA lets you use async/await in your

    tests, even if you’re not using it in the rest of your codebase. Which is fantastic because it lets you write tests without returning promises all the time. This makes testing asynchronous code a lot easier.
  5. You’ve probably struggled remembering the specific API of your assertion

    library. AVA comes with “power-assert” built-in. This library, by Takuto Wada, lets you write assertions using plain JavaScript expressions. If the assertion fails it’ll show you what the values were, so you can easily spot the error. power-assert: https://github.com/power-assert-js/
  6. t.true(arr.indexOf(zero) === two) | | | | | | |

    | | 2 | -1 0 false [1,2,3] Here’s an example of when an assertion failed. You can see the assertion in question and each intermediate value. You can easily see that the array does not include zero without having to write debug statements. It’s also easier to see what it is your testing without having to look up the documentation of a particular assertion method. Now you don’t need to memorise Chai!
  7. Opinionated Test Interface AVA doesn’t come with any of the

    test interfaces you may be used to, like BDD, TDD or an exports object. Instead you get a test function which you can call with a title and test implementation.
  8. import test from 'ava' test('1 + 1 = 2', t

    => { t.true(1 + 1 === 2) }) AVA lets you write tests, but not test hierarchies. This lets you focus on testing what your program does, without having to mimic how it’s structured. If you have more than one test file AVA will automatically prepend the file name to the test titles, so there’s no need to even repeat it in the file itself.
  9. Process Isolation AVA starts a new Node.js process for every

    test file. This means you can safely modify global state without accidentally affecting unrelated tests. It can also run multiple test files concurrently, each in their own process.
  10. AVA is, at the time of writing*, 79 contributors (and

    that’s just in the main repository). It’s also got well over 5,000 stars so we’ve convinced a lot of people to press that button. * July 24th, 2016
  11. The download count isn’t too shabby either, undoubtedly helped by

    CI installations ;-) According to https://gist.github.com/feross/e0882df2fe673d6ce064 we’re the 80th most depended on package in npm.
  12. AVA was started by Sindre Sorhus and Kevin Mårtensson in

    November 2014. Other members of the core team are Vadim Demedes, James Talmage and Juan Soto. We’re there to help with issues and contributions from others. Whilst we’re quite familiar with AVA’s internals none of us understand all of it. Major contributions have been made by people not on the core team — it’s not about us, it’s about building a great test runner.
  13. We’re not just looking for code contributions either. Updates to

    the documentation and recipes on how to use AVA are incredibly valuable. (Those “closed” PRs were merged manually, before GitHub released squash&merge.)
  14. And then there’s the awesome people who help translate the

    documentation into, so far, eight languages, with 20 contributors outside of the core team!
  15. And then there’s the awesome people who help translate the

    documentation into, so far, eight languages, with 20 contributors outside of the core team!
  16. Moar Features Let’s get back to what AVA does. We’ve

    already talked about async/await, power-assert, the test interface and process isolation.
  17. Per-Process Concurrency AVA runs each test file in its own

    process. But within each test file it by default starts all tests at the same time. So if you have asynchronous tests these will run concurrently, hopefully resulting in a faster test execution.
  18. test('first', async t => { console.log('start first') await new Promise(resolve

    => setTimeout(resolve, 500)) console.log('finish first') }) test('second', async t => { console.log('start second') await new Promise(resolve => setTimeout(resolve, 100)) console.log('finish second') }) This example has two asynchronous tests. The second will finish before the first.
  19. start first start second finish second finish first Which means

    we get this log output. This can be a problem if both tests share state. Ideally you make sure that’s not the case, but if you can’t…
  20. test.serial('first', async t => { console.log('start first') await new Promise(resolve

    => setTimeout(resolve, 500)) console.log('finish first') }) test('second', async t => { console.log('start second') await new Promise(resolve => setTimeout(resolve, 100)) console.log('finish second') }) console.log('start first') await new Promise(resolve => setTimeout(resolve, 500)) console.log('finish first') }) test('second', async t => { console.log('start second') await new Promise(resolve => setTimeout(resolve, 100)) console.log('finish second') }) You can mark individual test as serial. AVA will execute them first, in order, one at a time, before starting the remaining tests. (There’s also a --serial command line flag which causes all tests to run serially.)
  21. test.before(() => {}) test.after(() => {}) These let you register

    hooks that run before the first test, and after the last. Note that due to process isolation you can usually get away with performing shared setup in the test file itself, and you don’t necessarily have to clean up either. (Any code that runs when the test file is loaded delays when tests actually start running, so there is an impact, but I find it nicer not to use test.before)
  22. test.beforeEach(() => {}) test.afterEach(() => {}) These register hooks that

    are run before and after each test. Like tests, hooks run concurrently.
  23. test.beforeEach(t => { t.context.foo = 'bar' }) test(t => {

    t.true(t.context.foo === 'bar') }) beforeEach and afterEach hooks receive the test object. This is mostly useful because of the context property that is shared with the tests. You can set up a new context for each test. (t.context can also be assigned to.)
  24. test.skip('a test to ignore while developing', t => {}) test.only('a

    test to focus on while developing', t => {}) The .skip and .only modifiers should only be used during development. They can be useful when debugging a test, e.g. you can skip a test that distracts you, or run just those tests you’re having trouble getting right.
  25. test.todo('a test you still need to write') test.failing('a test that

    showcases a bug', t => {}) The .todo modifier can be used when sketching out tests. They’re included in the AVA’s output so you won’t forget about them, which is better than using code comments. The .failing modifier is useful if you or a contributor knows how to reproduce a bug, but not yet how to solve it. You can check in the failing test without it breaking CI. This way a contributor gets the credit for finding the bug, and somebody else already has a test case for when they go try and fix it.
  26. test.cb('doSomething() invokes the callback', t => { doSomething((err, result) =>

    { t.true(err === null) t.true(result === 42) t.end() }) }) The .cb modifier can be used when testing (asynchronous) functions that invoke a callback. You must end these tests using t.end().
  27. test.cb('doSomething() invokes the callback', t => { doSomething((err, result) =>

    { t.true(err === null) t.true(result === 42) t.end() }) }) test.cb('doSomething() invokes the callback', t => { doSomething((err, result) => { t.true(err === null) t.true(result === 42) t.end() }) }) The .cb modifier can be used when testing (asychronous) functions that invoke a callback. You must end these tests using t.end().
  28. import pify from 'pify' test('doSomething() invokes the callback', async t

    => { const result = await pify(doSomething)() t.true(result === 42) }) That said, you’re probably better off using async/await and promisifying the callback-taking function. AVA can’t trace uncaught exceptions back to the test. Using a promise you’re less likely to run into this problem. Here I’m using pify to promisify the function. See https://www.npmjs.com/package/pify.
  29. function macro(t, input, expected) { t.true(eval(input) === expected); } test('2

    + 2 === 4', macro, '2 + 2', 4); test('2 * 3 === 6', macro, '2 * 3', 6); Here the same test implementation is used for two different tests. Additional arguments are passed to the macro itself.
  30. function macro(t, input, expected) { t.true(eval(input) === expected) } macro.title

    = (_, input, expected) => { return `${input} === ${expected}` } test(macro, '2 + 2', 4) test(macro, '2 * 3', 6) macro.title = (_, input, expected) => { return `${input} === ${expected}` } test(macro, '2 + 2', 4) test(macro, '2 * 3', 6) Macros can even generate test titles. We encourage you to use macros, rather than generating tests. For instance it’s easier to statically analyse macros.
  31. Assertions AVA has 15 built-in assertions. However with power-assert you

    won’t use all of them. Let’s look at the most useful ones. You can find the full list in our documentation of course: https://github.com/avajs/ava#assertions.
  32. t.true(expression) t.false(expression) You’ve already seen the first one. t.false is

    the opposite. We recommend you use the t.true assertion as much as possible.
  33. t.truthy(expression) t.falsy(expression) These are less strict than t.true and t.false.

    Of course you use the logical not operator to get an actual boolean.
  34. t.truthy(expression) t.falsy(expression) t.truthy(expression) ! t.true(!!expression) t.falsy(expression) ! t.true(!expression) These are

    less strict than t.true and t.false. Of course you use the logical not operator to get an actual boolean.
  35. t.deepEqual(value, expected) t.notDeepEqual(value, expected) These compare whether the actual value

    structurally matches the expectation. Value types (e.g. strings and numbers) are tested using strict equality.
  36. t.throws(fn, TypeError) t.notThrows(fn, TypeError) These check whether the function throws

    the expected error. The assertion is currently a bit overloaded, I recommend you test by passing an error constructor.
  37. const err = t.throws(fn, TypeError) t.true(err.message === 'Bad type') t.throws()

    returns the error that was thrown. So you can use power-assert to do further comparisons.
  38. t.throws(fn(), TypeError) Also, if you’ve ever done this (calling the

    function rather than passing it to t.throws(), causing the exception to be thrown and your test to fail), AVA does a pretty good job at catching it and warning you about it.
  39. t.pass() t.fail() You can use these assertions to ensure certain

    code paths are run, or indeed are not run.
  40. t.fail('This should never happen') All assertions take a message as

    their last argument. For those rare cases where that’s actually useful.
  41. t.plan(5) If you want you can specify how many assertions

    need to pass for your test to succeed. This can be useful in certain asynchronous or loop situations. If too few or too many assertions pass your test will fail. Note that AVA doesn’t automatically end your test once the expected count is reached.
  42. t.skip.true(expression) You can also skip individual assertions. These are still

    counted so you don’t need to change the plan count.
  43. ⌚⌚⌚ Watch Mode AVA’s watch mode is fantastic. If you

    change a test file it’ll rerun just that file. If you change a source file it’ll rerun all tests that depend on it. And if can’t figure out which test to rerun it just reruns all. Demo time!
  44. "scripts": { "test": "ava" }, "devDependencies": { "ava": "^0.15.2" }

    You should add AVA to your devDependencies. Then configure the test script to call AVA.
  45. npm test -- --serial Like the .serial test modifier, however

    this causes all tests in all test files to run serially. Note that AVA still runs test files concurrently. (You could also use these modifiers in your test script definition, of course, but you’re probably better off configuring them).
  46. npm test -- --match '*end' npm test -- --match 'start*'

    Use match to run just those tests that match the expression. The expression syntax is purposefully limited, you can find more details in the documentation: https:// github.com/avajs/ava#running-tests-with-matching-titles.
  47. npm test -- --timeout=5s AVA doesn’t support per-test timeouts. Instead

    you can tell it to exit if no test completes in the given timeout window.
  48. npm test -- --tap npm test -- --verbose You can

    output test results in TAP format, or using the verbose reporter. Otherwise you get minimal output.
  49. npm test -- my-test.js 'my-dir/**/*.js' You can also pass test

    files or glob patterns as the last arguments. Make sure to use quotes around the glob patterns to prevent your shell from trying to expand them. The same goes for the match argument!
  50. Configuration CLI flags can be a bit cumbersome. Luckily AVA

    can also be configured in the package.json file.
  51. "ava": { "failFast": true, "files": [ "my-test.js", "my-dir/**/*.js" ], "match":

    [ "*end", "start*" ], "serial": true, "timeout": "5s" "verbose": true } Use the AVA stanza to add your configuration. Some options can only be configured here.
  52. "ava": { "source": [ "**/*.{js,jsx}", "!dist/**/*" ] } For instance

    this is where you’d customise the patterns used by the watcher to detect changes to your source files.
  53. Purrfection So that’s it. AVA’s perfect and there are definitely

    no issues we need to talk about. OK so that’s not true.
  54. So that’s it. AVA’s perfect and there are definitely no

    issues we need to talk about. OK so that’s not true.
  55. Concurrency AVA starts a process for each of your test

    files, all at once. If you have lots of test files that can be a bit extreme, to the point where your machine freezes up and your CI build fails. We’re working on several improvements in this area. For now you should use the concurrency flag and option.
  56. "ava": { "concurrency": 5 } npm test -- --concurrency=5 This

    limits AVA to five concurrent child processes. It’s currently an experimental feature while we figure out a good default and resolve issues with the .only modifier.
  57. ES2015 Sources AVA transpiles your tests, but not your source

    files. We’re working on integrating the precompiler in AVA’s main process. In the meantime you have to require babel- register when you run AVA.
  58. "ava": { "require": ["babel-register"] } npm test -- --require=babel-register Note

    that you need to make sure babel-register is installed. babel-register can be quite slow. It’ll be more efficient if you precompile your source files and test your precompiled code. This is my preferred strategy. Also, you can configure the Babel config AVA uses to compile the test files. I won’t go into those details though, please see the documentation: https://github.com/avajs/ ava/blob/master/docs/recipes/babelrc.md.
  59. Browser Testing Currently AVA doesn’t support browser testing. We really

    want to make this work though, it’s just very difficult. That said, you can usually get pretty far by emulating the DOM. We’ve got a recipe for that: https://github.com/avajs/ava/blob/master/docs/recipes/browser-testing.md.
  60. chdir Currently AVA sets the current working directory to that

    of the test file. This is more confusing than beneficial so we’ll be changing it. In the future the current working directory will be that of the package.json file.
  61. Ecosystem There’s a fair amount that AVA does not do.

    Luckily there are other tools that can help.
  62. Code Coverage Currently AVA does not come with any code

    coverage support. We recommend you use nyc,
  63. We recommend you use nyc. It’s now the official command

    line interface for Istanbul so you can’t go wrong. Both me and James Talmage are also contributors to nyc. https://www.npmjs.com/package/nyc
  64. Editor Plugins We have plugins for Sublime, Atom and VSCode.

    As well as integrations with Grunt, Gulp, and so on. Find the links in our documentation: https://github.com/avajs/ ava#related.
  65. And if you really like BDD, there’s a third-party package

    that lets you use describe and it to your harts content.
  66. ava.li @ava__js gitter.im/avajs/ava You can find AVA on https://ava.li. We’re

    also on Gitter: https://gitter.im/avajs/ava, and of course Twitter: https://twitter.com/ava__js.