Visual Regression Testing with Huxley

Visual Regression Testing with Huxley

Huxley is a tool crated by Pete Hunt at Instagram (@floydophone).

D547672826d436bfb0dfe0a316ced5c0?s=128

Nathan Birrell

February 11, 2014
Tweet

Transcript

  1. Visual Regression Testing with Huxley @nathanbirrell ! 15m

  2. Visual Regression Testing with Huxley Outline Why? Workflows and Demo

    Limitations 1 2 3
  3. Visual Regression Testing with Huxley Problem Testing the user interface

    can be slow and fragile.
 Unit testing does not test the browser- rendered UI.
 ◦ A style change for one component can have unforeseen repercussions.
  4. Visual Regression Testing with Huxley Huxley ! ! ! !

    Record your actions Take screenshots Action playback Update screenshots with changes
  5. Visual Regression Testing with Huxley How it Works Selenium Server

    Selenium WebDriver (browser-specific plugin) A server (Middleman) Huxleyfile
  6. Visual Regression Testing with Huxley Huxleyfile [Test01] url=http://0.0.0.0:4567/PageToTest1.html ! [Test02]

    url=http://0.0.0.0:4567/PageToTest2.html screensize=1024x768 sleepfactor=0.5 browser=firefox ! ... Name URL Screen size (for responsive testing) Browser selection (currently unreliable) Speed
  7. Visual Regression Testing with Huxley Files Test01/diff.png Test01/record.json Test01/

  8. Visual Regression Testing with Huxley Workflows Default: 2. Make normal

    code changes 3. Run Huxley [Screenshots overwritten to show visual changes] 4. Commit changes to git 5. Compare screenshots to previous commits (using an image comparison app) 1. Record all tests Screenshots and click events registered to the millisecond Alternative: 2. Make normal code changes in separate branches 3. Keep a master branch for a base to test from 4. Run Huxley on the base branch, then on the working branch for a pixel-perfect comparison.
  9. Visual Regression Testing with Huxley Workflows Default: 2. Make normal

    code changes 3. Run Huxley [Screenshots overwritten to show visual changes] 4. Commit changes to git 5. Compare screenshots to previous commits (using an image comparison app) 1. Record all tests Screenshots and click events registered to the millisecond Alternative: 2. Make normal code changes in separate branches 3. Keep a master branch for a base to test from 4. Run Huxley on the base branch, then on the working branch for a pixel-perfect comparison.
  10. Visual Regression Testing with Huxley Limitations Time Recordings take a

    while to get through on large projects. ☼ Browsers Browser selection is supported, but seems a little complicated. Animation Sometimes .gif spinners and JS movements throw errors.
  11. Visual Regression Testing with Huxley To come… Integration with VirtualBox

    VMs Better browser integration (IE and Chrome)
  12. Demo

  13. Visual Regression Testing with Huxley More Slides: speakerdeck.com/nathanbirrell Github: github.com/facebook/huxley

    Jira guide: bit.ly/ddhuxley Podcast on Huxley: bit.ly/LS7iTL