Visual Regression Testing with PhantomCSS Pavels Jelisejevs, C.T.Co 

By "visual regression testing" (VRT) I mean: Tools that allow to automatically check if the UI is displayed correctly, including the dimensions of elements, colors, images and etc. 

No content

Why would we want to automate this? Because people are bad at detecting visual changes. This phenomenon is referred to as "change blindness". 

No content

No content

No content

No content

No content

No content

No content

No content

Major use cases for VRT are: •Testing the appearance of UI elements on the page: headers, footers, button etc for visual defects •Testing responsive designs •Testing complex visualizations (think charts, diagrams, graphs) which cannot be easily covered with unit or functional tests 

How can we test visual aspects? We can try to retrieve certain CSS properties from elements and compare them to the benchmark: assert(element.getCss("color") === "red") Hardly an option due to the amount of code need to be written and complex maintenance. 

How can we test visualizations? Ain't no testing framework gonna help your here 

Introducing PhantomCSS PhantomCSS is a tool that allows you to: 1. Open a page and take a screenshot of an element 2. Store the screenshot as a baseline 3. After a change is done, take another screenshot and compare it to the baseline 

What's under the hood? PhantomCSS uses: •PhantomJS 2 or SlimerJS for browsing •CasperJS for interacting with the browser •Resemble.js for screenshot comparison •Python required for CasperJS 

How does the test look like? casper.
 start( url )
 .then(function() {
 // Take a screenshot of the UI component
 phantomcss.screenshot('#my-elemenmt', 'a screenshot of my element');
 
 })
 .then(function() {
 // compare screenshots
 phantomcss.compareAll();
 })
 .run(function() {
 casper.test.done();
 }); 

Where can I try it out? You can try out a demo available at GitHub Run: npm install casperjs test demo/testsuite.js 

Now what? New screenshot at C:/PhantomCSS/screenshots\coffee machine dialog_1.png 

What if I change something? FAIL Visual change found for screenshot C:/PhantomCSS/screenshots\coffee machine dialog_1.png (21.81% mismatch) # type: fail # file: demo/testsuite.js # subject: false 

How do I accept the change? casperjs test demo/testsuite.js --rebase 

Additional configuration are supported You can set pass additional parameters to CasperJS PhantomCSS. Most popular ones are: - Resolution - Tolerance - Delays 

A word of advice •PhantomCSS is not a replacement for functional tests •Take screenshots of certain elements rather than whole pages •Use your VCS system to store baseline screenshots 

A word of advice •Set up a testing environment: •If you’re testing a whole website, mock all dynamic data •If you’re testing components - showcase them on a separate page (style guide) 

Problems Running PhantomCSS on CI environment occasionally leads to unexpected errors. 

Problems Changes in the dynamic content provided by the backend will cause the tests to break. How do we easily decouple them? 

Popular questions Q: At what project stages should we use VRT? A: When you have something to take pictures of. Q: How to organise VRT in a team? A: In the same manner as all other tests. 

Thank you! • PhantomCSS on GitHub - https://github.com/ Huddle/PhantomCSS • The phenomenon of Change Blindness - https:// en.wikipedia.org/wiki/Change_blindness 

You’re welcome to join our team. Contact [email protected]