Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Visual Regression Testing - from a tool to a process

Visual Regression Testing - from a tool to a process

We work with many JavaScript based tools in our day to day lives.

But how do we take those tools from being used by a single person on a machine and integrate it into a team's workflow?

How can we link those tool into chains and go from a tool to a process? :)

In this talk I will discuss how I converted PhantomCSS, a tool for visual regression testing, and integrated it into our workflow. I will also cover the challenges we faced which ranged from integrating the tests into our CI, test execution taking too long, ability to browse the test results and notifying developers about failed regressions.

Nikhil Verma

May 10, 2017

More Decks by Nikhil Verma

Other Decks in Technology


  1. Local • Used by developers • Choose the tests to

    run • Manage your own “baseline” • Develop/Refactor/Debug
  2. • Feature/Staging branches • Runs the entire test suite •

    Baseline from production • Catch regressions and bugs Automated
  3. 1. Download baseline from server 2. Generate and compare screenshots

    3. Upload results back to the server 4. Send notifications to developer/team How does it work?
  4. Tests were slow • Don’t know when the page will

    be ready • Wait for images to download
  5. // On the client, when a page is rendered $page.attr('data-vrt-ready',

    'true'); Client adds an attribute when ready
  6. casperjs -- --config=config/phantomjs.json —concise target=https://m.badoo.com test instructions/mobileweb/tests/ login.js instructions/mobileweb/tests/chat.js instructions/

    mobileweb/tests/gifts.js instructions/mobileweb/tests/liked- you.js --pre=instructions/mobileweb/pre.js -- post=instructions/mobileweb/post.js
  7. Convert an array of questions module.exports = [{ type: 'list',

    name: 'suite', message: 'What do you want to test?', choices: […], default: 'styleguide' }, { type: 'list', name: 'environment', message: 'Which environment do you want to use?', choices: function (answers) {…} }];
  8. +

  9. Trigger • Run a bash script • Start a CI

    agent (TeamCity/Jenkins/CircleCI) • Boot a docker VM/Remote machine • Cron job (no hook needed!)
  10. + !

  11. See the results • Email • Hosted website with results

    • Commit to repository • Hard Disk/Network Attached Storage
  12. A hook in our deploy scripts which triggers VRT on

    a TeamCity agent which sends a notification via HipChat and lets us see the results on a website
  13. • Stability of tests • Test for different conditions •

    User agents • Resolutions • Expand test coverage