Visual Regression Testing with Huxley

Huxley is a tool crated by Pete Hunt at Instagram (@floydophone).

Transcript

1. Visual Regression Testing with Huxley @nathanbirrell ! 15m
2. Visual Regression Testing with Huxley Outline Why? Workflows and Demo Limitations 1 2 3
3. Visual Regression Testing with Huxley Problem Testing the user interface can be slow and fragile.
 Unit testing does not test the browser- rendered UI.
 ◦ A style change for one component can have unforeseen repercussions.
4. Visual Regression Testing with Huxley Huxley ! ! ! ! Record your actions Take screenshots Action playback Update screenshots with changes
5. Visual Regression Testing with Huxley How it Works Selenium Server Selenium WebDriver (browser-specific plugin) A server (Middleman) Huxleyfile
6. Visual Regression Testing with Huxley Huxleyfile [Test01] url=http://0.0.0.0:4567/PageToTest1.html ! [Test02] url=http://0.0.0.0:4567/PageToTest2.html screensize=1024x768 sleepfactor=0.5 browser=firefox ! ... Name URL Screen size (for responsive testing) Browser selection (currently unreliable) Speed
7. Visual Regression Testing with Huxley Files Test01/diff.png Test01/record.json Test01/
8. Visual Regression Testing with Huxley Workflows Default: 2. Make normal code changes 3. Run Huxley [Screenshots overwritten to show visual changes] 4. Commit changes to git 5. Compare screenshots to previous commits (using an image comparison app) 1. Record all tests Screenshots and click events registered to the millisecond Alternative: 2. Make normal code changes in separate branches 3. Keep a master branch for a base to test from 4. Run Huxley on the base branch, then on the working branch for a pixel-perfect comparison.
9. Visual Regression Testing with Huxley Workflows Default: 2. Make normal code changes 3. Run Huxley [Screenshots overwritten to show visual changes] 4. Commit changes to git 5. Compare screenshots to previous commits (using an image comparison app) 1. Record all tests Screenshots and click events registered to the millisecond Alternative: 2. Make normal code changes in separate branches 3. Keep a master branch for a base to test from 4. Run Huxley on the base branch, then on the working branch for a pixel-perfect comparison.
10. Visual Regression Testing with Huxley Limitations Time Recordings take a while to get through on large projects. ☼ Browsers Browser selection is supported, but seems a little complicated. Animation Sometimes .gif spinners and JS movements throw errors.
11. Visual Regression Testing with Huxley To come… Integration with VirtualBox VMs Better browser integration (IE and Chrome)
12. Demo
13. Visual Regression Testing with Huxley More Slides: speakerdeck.com/nathanbirrell Github: github.com/facebook/huxley Jira guide: bit.ly/ddhuxley Podcast on Huxley: bit.ly/LS7iTL