Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Measuring Web Performance Using Selenium

Measuring Web Performance Using Selenium

You are the Selenium master and automate tests in your sleep. You make sure the quality of your companies web page is 100%. But then the team wants you to measure the performance of your site. How do you do that? Is that even possible with Selenium?

In this talk we will look at how we use Selenium in Browsertime & sitespeed.io to measure the speed of your web application and how you can implement it yourself in your own Selenium tests. We will also briefly look at strategies to make sure you can catch regressions in your sites performance.

You can checkout the talk at Youtube: https://www.youtube.com/watch?v=cLlZhnZvi1M

http://2016.seleniumconf.co.uk/sessions

Peter Hedenskog

November 16, 2016
Tweet

More Decks by Peter Hedenskog

Other Decks in Technology

Transcript

  1. - Web performance metrics - Selenium - Performance vs functional

    testing - Strategies for catching performance regression Today
  2. const script = 'var wp = window.performance.timing;' + 'return wp.domComplete

    - wp.navigationStart;'; driver.get('http://2016.seleniumconf.co.uk/') .then(() => { return driver.executeScript(script) }) .then((result) => { console.log('DOMComplete: %d ms', result); }) .finally(driver.quit); Javascript
  3. // Chrome if (window.chrome && window.chrome.loadTimes) { // Convert to

    ms var firstPaint = window.chrome.loadTimes().firstPaintTime*1000; var startTime = window.chrome.loadTimes().startLoadTime*1000; console.log(firstPaint - startTime); } First paint https://github.com/addyosmani/timing.js/blob/ master/timing.js#L47-L56
  4. // IE if (typeof window.performance.timing.msFirstPaint === 'number') { var firstPaint

    = window.performance.timing.msFirstPaint; var startTime = window.performance.timing.navigationStart; console.log(firstPaint - startTime); } https://github.com/addyosmani/timing.js/blob/ master/timing.js#L47-L56 First paint
  5. Chrome meaningfulPaint chromeOptions.setLoggingPrefs(logPrefs); chromeOptions.setPerfLoggingPrefs({enableNetwork: true, enablePage: true, traceCategories: "blink.user_timing"}); ...

    case 'Tracing.dataCollected': // Here we can catch things as // firstImagePaint // firstContentfulPaint // firstTextPaint // firstMeaningfulPaint
  6. Firefox HAR 'use strict'; const webdriver = require('selenium-webdriver'); const firefox

    = require('selenium-webdriver/firefox'); const profile = new firefox.Profile(); // HAR export - see http://www.softwareishard.com/blog/har-export-trigger/ profile.setPreference('extensions.netmonitor.har.enableAutomation', true); profile.setPreference('extensions.netmonitor.har.contentAPIToken', 'supersecrettoken'); profile.setPreference('extensions.netmonitor.har.autoConnect', true); profile.setPreference('devtools.netmonitor.har.includeResponseBodies', false); // Download from the version page, the default URL shows wrong latest version // https://addons.mozilla.org/sv-se/firefox/addon/har-export-trigger/versions/? page=1#version-0.5.0-beta.10 profile.addExtension('har_export_trigger-0.5.0-beta.10-fx.xpi'); 1/3
  7. const script = ` var callback = arguments[arguments.length - 1];

    function triggerExport() { HAR.triggerExport({'token':'supersecrettoken', 'getData':true}) .then((result) => { var har = JSON.parse(result.data); har.log.pages[0].title = document.title; return callback({'har': JSON.stringify(har)}); }) .catch((e) => callback({'error': e})); }; if (typeof HAR === 'undefined') { addEventListener('har-api-ready', triggerExport, false); } else { triggerExport(); }`; 2/3 Firefox HAR
  8. driver.get('http://2016.seleniumconf.co.uk/') .then(() => driver.executeAsyncScript(script) ) .then((result) => { if (result.error)

    { console.error('Could not get the HAR %s', result.error); } else { console.log(result.har); } }) .finally(driver.quit); 3/3 Firefox HAR
  9. Chrome HAR const logPrefs = new webdriver.logging.Preferences(); logPrefs.setLevel(webdriver.logging.Type.PERFORMANCE, webdriver.logging.Level.INFO); const

    chromeOptions = new chrome.Options(); chromeOptions.setLoggingPrefs(logPrefs); chromeOptions.setPerfLoggingPrefs({enableNetwork: true, enablePage: true});
  10. Firefox setup const defaultFirefoxPreferences = { 'browser.safebrowsing.enabled': false, 'browser.safebrowsing.malware.enabled': false,

    'browser.safebrowsing.remotelookups': false, 'browser.shell.checkDefaultBrowser': false, 'browser.startup.homepage': 'about:blank' // IRL you need more! } const profile = new firefox.Profile(); Object.keys(defaultFirefoxPreferences).forEach(function(pref) { profile.setPreference(pref, defaultFirefoxPreferences[pref]); });
  11. tc - simulating 3g sudo tc qdisc add dev eth0

    root netem delay 300ms loss 0% rate 1600kbps
  12. IRL

  13. Connectivity: https://www.flickr.com/photos/cogdog/15638928284/ Catch regression: https://www.flickr.com/photos/testlab/21496317363/ Fika: https://www.flickr.com/photos/andreasivarsson/5745405973/ Boris 1: https://www.flickr.com/photos/53797600@N04/6849855824

    Broken heart: https://www.flickr.com/photos/miguelpdl/4356975474 Queen: https://www.flickr.com/photos/foreignoffice/8283323803 Ikea: https://www.flickr.com/photos/dahlstroms/4406947248 Apple/Orange: https://www.flickr.com/photos/nubobo/8226113305 Snowflakes: https://www.flickr.com/photos/dan1984/15678152054 Crowd: https://www.flickr.com/photos/vin60/15386141803 Synthetic: https://www.flickr.com/photos/-adam/4674856117 Queen: https://www.flickr.com/photos/aftab/5578122981/ Credits
  14. Traffic: https://www.flickr.com/photos/griff69 Safetybelt: https://www.flickr.com/photos/torsteinsaltvedt/2460156876 Free hugs: https://www.flickr.com/photos/clement127/13661779374 Alone: https://www.flickr.com/photos/thomashawk/98867886 Golden:

    https://www.flickr.com/photos/haeresis07/16191841366 Ten commandments: https://www.flickr.com/photos/vancityscapes/512782411 Angry: https://www.flickr.com/photos/stevendepolo/4605621230 Puzzle: https://www.flickr.com/photos/acefrenzy/8163097 Super heroes: https://www.flickr.com/photos/chuckles396/7145919879 Credits