Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PhantomCSS - From a tool to a process

PhantomCSS - From a tool to a process

I talked about how we at Badoo used PhantomCSS and VRT from a tool used by a single developer on a machine into a process which is used by the team and is integrated into our continuous delivery system.

620ecdb7215ec072617bfd7f75c98760?s=128

Nikhil Verma

March 21, 2017
Tweet

Transcript

  1. PhantomCSS from a tool to a process Nikhil Verma

  2. Nikhil Verma Front-End / JS Developer for 4 years MOBILE

    WEB TEAM @
  3. Why do we need VRT? We already have unit testing,

    automation testing and manual testing process
  4. Swiss cheese model

  5. Unit testing Automation testing Visual Regression testing Human testing Change

  6. The process today

  7. Local Visual Regression Testing • Used by developers • Choose

    what to test • Manage your own baseline • Develop/Refactor/Debug
  8. Automated Visual Regression Testing • Feature/Staging • Runs the entire

    test suite • Baseline from production • Catch regressions and bugs
  9. Running tests locally

  10. None
  11. git checkout master git checkout -b MW-1234_fix_bugs git ls-files |

    xargs git rm -r git add -A git commit -m “fixed all bugs”
  12. • Run visual regression locally if needed Feature Website Staging

    Website On Production Develop
  13. • Download baseline images from server • PhantomCSS: Generate and

    compare images • Upload screenshots to server • Send notifications to developer Feature Website Staging Website On Production Develop
  14. None
  15. None
  16. Feature Website Staging Website On Production Develop • Download baseline

    images from server • PhantomCSS: Generate and compare images • Upload screenshots to server • Send notifications to release room
  17. None
  18. Regenerate baseline from production Feature Website Staging Website On Production

    Develop
  19. Examples of issues caught by VRT

  20. None
  21. None
  22. None
  23. None
  24. How it started

  25. Cristiano Rastelli Front-End / CSS Developer didoo.net / @areaweb MOBILE

    WEB TEAM
  26. Tests were slow Hard to run individual tests Hardcoded to

    a specific user No integration with our process
  27. Let’s start improving it! one problem at a time

  28. Tests were slow .thenEvaluate(function() { require('Core/History').navigate('popularity'); }) .wait(5000) // wait

    for page to render .thenEvaluate(function() { $('.brick__image').css('background-image','url(http:// placehold.it/90x90)'); }) .wait(5000) // ensure placeholder images are downloaded .then(function() { phantomcss.screenshot(".page.popularity", 5, '', "Pages/Popularity"); })
  29. Wait for page to render

  30. Add data attributes on the client readyForVrt() { this.parent.attr('data-vrt-ready', 'true');

    }
  31. Add data attributes on the client // Inside a page

    implementation someNetworkRequest() .then(data => { this.$el.html(someTemplate.render(data)); }) .then(() => this.readyForVrt());
  32. Ensure placeholder images are downloaded

  33. Replace placehold.it images with base64 module.exports = { PLACEHOLDER_90x90: 'data:image/png;base64,iVBORw0KGgoA

    PLACEHOLDER_180x180: 'data:image/png;base64,iVBORw0KGg PLACEHOLDER_400x400: 'data:image/png;base64,iVBORw0KGg RAINBOW: 'data:image/png;base64,iVBORw0KGgoAAAAN...' };
  34. Replace placehold.it images with base64 var PLACEHOLDER_90x90 = require('config/ constants').PLACEHOLDER_90x90;

    .thenEvaluate(function (placeholder90) { $(‘.brick__image') .css( 'background-image', 'url("' + placeholder90 + '")' ); }, PLACEHOLDER_90x90)
  35. Replace placehold.it images with base64

  36. navigate('popularity') .thenEvaluate(function (placeholder90) { $('.brick__image') .css( 'background-image', 'url("' + placeholder90

    + '")' ); }, PLACEHOLDER_90x90) .then(function () { phantomcss.screenshot(...); });
  37. Before: 6.8 mins After: 1.2 mins ~6x faster!

  38. Tests were slow Hard to run individual tests Hardcoded to

    a specific user No integration with our process
  39. Hard to run individual tests

  40. Inquirer - Convert an array of objects https://www.npmjs.com/package/inquirer

  41. Inquirer - To a list of questions

  42. inquirer.prompt(questions).then(function (answers) { var testList = answers.split ? answers.tests :

    getFullTestList(answers.suite); launchSuite( answers.host, answers.suite, testList, answers.login, answers.password ); });
  43. casperjs -- --config=config/phantomjs.json --concise --login=nikhil10 —password=***** --target=https:// mw-1245.mshot.badoo.com test instructions/mobileweb/

    tests/login.js instructions/mobileweb/tests/chat.js instructions/mobileweb/tests/gifts.js instructions/ mobileweb/tests/liked-you.js --pre=instructions/ mobileweb/pre.js --post=instructions/mobileweb/post.js This is what one would have to type
  44. Tests were slow Hard to run individual tests Hardcoded to

    a specific user No integration with our process
  45. Hardcoded to a specific user

  46. Enter: QAAPI QAAPI is an internal testing API in Badoo

    exposed to developers and QAs. It allows them to modify test accounts and trigger various functionality in the application.
  47. http://qaapi/userRegister http://qaapi/forcePromoNotification http://qaapi/generateWantYou http://qaapi/userAddFriend http://qaapi/userAddFavorite http://qaapi/chatSendMessage and many more…

  48. Convert QAAPI calls to JS functions

  49. Generate a brand new test user

  50. Login with the test user casper.user = user; // in

    tests .then(function () { this.fillSelectors('.form-login', { 'input[name=name]': this.user.email, 'input[name=password]': this.user.password }, false); })
  51. Tests were slow Hard to run individual tests Unreliable due

    to depending on a specific user No integration with our process
  52. • Plug into existing CI system • Allow developers to

    access the results
  53. Plug into existing CI system • After deploying feature/staging we

    want to trigger a build agent to run VRT • Build agent checks out our visual regression repo • Run a script
  54. Triggering a build: TeamCity http://teamcity/httpAuth/action.html? add2Queue=Mobileweb_VisualRegressionTests&name= build.shot_name&value={hostName} https://www.jetbrains.com/teamcity/

  55. Build agent: Checkout git and execute script git clone git@git.mlan:mobileweb/vrt.git

    cd vrt/ ./run.sh --host={hostName}
  56. npm install # make directories mkdir screenshots mkdir screenshots/$VRT_URL #

    download baseline ./utils/downloadMasterResultScreens.sh $VRT_URL # run tests ./visualregression.js --suite=styleguide --host=$VRT_URL ./visualregression.js --suite=mobileweb --host=$VRT_URL # upload results ./utils/uploadResultScreens.sh $VRT_URL # check for errors and notify developers ERRORS_FILE="screenshots/$VRT_URL/error.log" if [ -f $ERRORS_FILE ]; then php ./utils/notifyIssueUsers.php "$(cat $ERRORS_FILE)" $VRT_URL fi
  57. Allow developers to access the results • Results are stored

    in one folder • Read the folder and match file names to generate a JS object *.fail.png - Test failed *.diff.png - Test passed *.png - New screenshot • Use the JS object to render a template and serve it over http
  58. A simple webapp using Koa which is a frontend to

    the “screenshots” folder http://koajs.com/
  59. The final architecture

  60. Smart Prompt > Build Agent Node application PhantomCSS CasperJS ResembleJS

    PhantomJS Scripts Screenshots Diffs Results Web Application Baseline - Master Results - MW1234 Results - MW9876 Results - MW5432 Node.js App GIT Results Baseline Diffs TeamCity App Trigger Event HipChat
  61. Tests were slow Hard to run individual tests Unreliable due

    to depending on a specific user No integration with our process
  62. None
  63. Branch naming conventions VRT-1234_my_feature MW-1234_my_feature Use a matching VRT branch

    to run tests in case of refactoring
  64. Javascript Error logging Log all JS errors during the text

    execution in a file then upload it along with the regression results to make debugging easier.
  65. Utilise npm config to store your user npm config set

    vrtuser xxxxxxxx npm config set vrtpass xxxxxxxx Now we don’t have to enter our test user details all the time
  66. Q: Is everything perfect? Answer : Nope

  67. • Stability of tests (It’s not unit testing) • Reliability

    of QAAPI/data received • Test for different conditions (user agents, viewports) • Expand test coverage
  68. Thank you Answer : Nope