Measuring Web Performance Using Selenium

Measuring Web Performance Using Selenium

You are the Selenium master and automate tests in your sleep. You make sure the quality of your companies web page is 100%. But then the team wants you to measure the performance of your site. How do you do that? Is that even possible with Selenium?

In this talk we will look at how we use Selenium in Browsertime & sitespeed.io to measure the speed of your web application and how you can implement it yourself in your own Selenium tests. We will also briefly look at strategies to make sure you can catch regressions in your sites performance.

You can checkout the talk at Youtube: https://www.youtube.com/watch?v=cLlZhnZvi1M

http://2016.seleniumconf.co.uk/sessions

Dfb236c2d968fbbf6fa4dd7d0541b6b6?s=128

Peter Hedenskog

November 16, 2016
Tweet

Transcript

  1. 4.
  2. 6.
  3. 7.
  4. 8.
  5. 9.
  6. 11.
  7. 14.
  8. 18.

    - Web performance metrics - Selenium - Performance vs functional

    testing - Strategies for catching performance regression Today
  9. 25.
  10. 26.
  11. 28.
  12. 29.
  13. 30.
  14. 31.
  15. 32.

    const script = 'var wp = window.performance.timing;' + 'return wp.domComplete

    - wp.navigationStart;'; driver.get('http://2016.seleniumconf.co.uk/') .then(() => { return driver.executeScript(script) }) .then((result) => { console.log('DOMComplete: %d ms', result); }) .finally(driver.quit); Javascript
  16. 37.

    // Chrome if (window.chrome && window.chrome.loadTimes) { // Convert to

    ms var firstPaint = window.chrome.loadTimes().firstPaintTime*1000; var startTime = window.chrome.loadTimes().startLoadTime*1000; console.log(firstPaint - startTime); } First paint https://github.com/addyosmani/timing.js/blob/ master/timing.js#L47-L56
  17. 38.

    // IE if (typeof window.performance.timing.msFirstPaint === 'number') { var firstPaint

    = window.performance.timing.msFirstPaint; var startTime = window.performance.timing.navigationStart; console.log(firstPaint - startTime); } https://github.com/addyosmani/timing.js/blob/ master/timing.js#L47-L56 First paint
  18. 42.
  19. 43.
  20. 44.
  21. 45.
  22. 48.

    Chrome meaningfulPaint chromeOptions.setLoggingPrefs(logPrefs); chromeOptions.setPerfLoggingPrefs({enableNetwork: true, enablePage: true, traceCategories: "blink.user_timing"}); ...

    case 'Tracing.dataCollected': // Here we can catch things as // firstImagePaint // firstContentfulPaint // firstTextPaint // firstMeaningfulPaint
  23. 50.
  24. 53.
  25. 54.

    Firefox HAR 'use strict'; const webdriver = require('selenium-webdriver'); const firefox

    = require('selenium-webdriver/firefox'); const profile = new firefox.Profile(); // HAR export - see http://www.softwareishard.com/blog/har-export-trigger/ profile.setPreference('extensions.netmonitor.har.enableAutomation', true); profile.setPreference('extensions.netmonitor.har.contentAPIToken', 'supersecrettoken'); profile.setPreference('extensions.netmonitor.har.autoConnect', true); profile.setPreference('devtools.netmonitor.har.includeResponseBodies', false); // Download from the version page, the default URL shows wrong latest version // https://addons.mozilla.org/sv-se/firefox/addon/har-export-trigger/versions/? page=1#version-0.5.0-beta.10 profile.addExtension('har_export_trigger-0.5.0-beta.10-fx.xpi'); 1/3
  26. 55.

    const script = ` var callback = arguments[arguments.length - 1];

    function triggerExport() { HAR.triggerExport({'token':'supersecrettoken', 'getData':true}) .then((result) => { var har = JSON.parse(result.data); har.log.pages[0].title = document.title; return callback({'har': JSON.stringify(har)}); }) .catch((e) => callback({'error': e})); }; if (typeof HAR === 'undefined') { addEventListener('har-api-ready', triggerExport, false); } else { triggerExport(); }`; 2/3 Firefox HAR
  27. 56.

    driver.get('http://2016.seleniumconf.co.uk/') .then(() => driver.executeAsyncScript(script) ) .then((result) => { if (result.error)

    { console.error('Could not get the HAR %s', result.error); } else { console.log(result.har); } }) .finally(driver.quit); 3/3 Firefox HAR
  28. 58.

    Chrome HAR const logPrefs = new webdriver.logging.Preferences(); logPrefs.setLevel(webdriver.logging.Type.PERFORMANCE, webdriver.logging.Level.INFO); const

    chromeOptions = new chrome.Options(); chromeOptions.setLoggingPrefs(logPrefs); chromeOptions.setPerfLoggingPrefs({enableNetwork: true, enablePage: true});
  29. 64.

    Firefox setup const defaultFirefoxPreferences = { 'browser.safebrowsing.enabled': false, 'browser.safebrowsing.malware.enabled': false,

    'browser.safebrowsing.remotelookups': false, 'browser.shell.checkDefaultBrowser': false, 'browser.startup.homepage': 'about:blank' // IRL you need more! } const profile = new firefox.Profile(); Object.keys(defaultFirefoxPreferences).forEach(function(pref) { profile.setPreference(pref, defaultFirefoxPreferences[pref]); });
  30. 67.
  31. 71.

    tc - simulating 3g sudo tc qdisc add dev eth0

    root netem delay 300ms loss 0% rate 1600kbps
  32. 74.
  33. 77.
  34. 78.
  35. 79.
  36. 80.
  37. 82.

    IRL

  38. 88.
  39. 93.
  40. 95.

    Connectivity: https://www.flickr.com/photos/cogdog/15638928284/ Catch regression: https://www.flickr.com/photos/testlab/21496317363/ Fika: https://www.flickr.com/photos/andreasivarsson/5745405973/ Boris 1: https://www.flickr.com/photos/53797600@N04/6849855824

    Broken heart: https://www.flickr.com/photos/miguelpdl/4356975474 Queen: https://www.flickr.com/photos/foreignoffice/8283323803 Ikea: https://www.flickr.com/photos/dahlstroms/4406947248 Apple/Orange: https://www.flickr.com/photos/nubobo/8226113305 Snowflakes: https://www.flickr.com/photos/dan1984/15678152054 Crowd: https://www.flickr.com/photos/vin60/15386141803 Synthetic: https://www.flickr.com/photos/-adam/4674856117 Queen: https://www.flickr.com/photos/aftab/5578122981/ Credits
  41. 96.

    Traffic: https://www.flickr.com/photos/griff69 Safetybelt: https://www.flickr.com/photos/torsteinsaltvedt/2460156876 Free hugs: https://www.flickr.com/photos/clement127/13661779374 Alone: https://www.flickr.com/photos/thomashawk/98867886 Golden:

    https://www.flickr.com/photos/haeresis07/16191841366 Ten commandments: https://www.flickr.com/photos/vancityscapes/512782411 Angry: https://www.flickr.com/photos/stevendepolo/4605621230 Puzzle: https://www.flickr.com/photos/acefrenzy/8163097 Super heroes: https://www.flickr.com/photos/chuckles396/7145919879 Credits