PhantomJS for Web Page Automation

PhantomJS for Web Page Automation

Presented at Selenium San Jose meetup:
http://www.meetup.com/SeleniumSanJose/events/154836972/.

PhantomJS, the scriptable headless WebKit-based automation tool, has gained a lot of traction in its first 3 years. It is a popular choice for continuous integration, whether to run basic tests or to catch rendering regressions. Many JavaScript test frameworks work well with PhantomJS and even Selenium-based tests can be integrated thanks to the support for WebDriver via Ghost Driver. In addition, because PhantomJS permits the inspection of network traffic, it is suitable to run various analysis on the network behavior and performance. This talk will highlight the basic usages of PhantomJS and explore various PhantomJS-tools for the purpose of headless testing of web applications, screen capture, and network monitoring.

0284b8950e0f4a57bcc092d4dbb98d97?s=128

Ariya Hidayat

January 08, 2014
Tweet

Transcript

  1. PhantomJS for Web Page Automation @ariyahidayat Mountain View Jan 8,

    2014 South Bay Selenium Meetup
  2. /usr/bin/whoami shapesecurity.com

  3. “Software Provocateur” PhantomJS Esprima

  4. 11-Second Elevator Pitch Scriptable Simulate user interactions Headless Command-line, no

    GUI WebKit Layout and rendering
  5. Important Traits Multi platforms Zero dependencies (*) Works with Other

    Tools
  6. Who is Using PhantomJS http://phantomjs.org/users.html

  7. Versions vs Downloads http://phantomjs.org/releases.html Announced Jan 23, 2011 WebDriver Winter

    2012 Pure headless Linux Spring 2012
  8. Web Page Processing Screen Capture Headless Testing Network Monitoring

  9. 5-Minute Primer

  10. Hello, World! console.log('Hello, world!'); phantom.exit();

  11. Capture to Image var page = require('webpage').create(); page.open('http://google.com', function (status)

    { page.render('google.png'); phantom.exit(); });
  12. Get Page Title var page = require('webpage').create(); page.open('http://linkedin.com', function (status)

    { if (status !== 'success') { console.log('FAIL to load the address'); } else { var title = page.evaluate(function () { return document.title; }); console.log('Page title is', title); } phantom.exit(); });
  13. Use jQuery (or Other Libs) var page = require('webpage').create(); page.open('http://www.sample.com',

    function() { page.includeJs("http://code.jquery.com/jquery-1.10.2.js", function() { page.evaluate(function() { $("button").click(); }); phantom.exit() }); });
  14. Behind the Scene C++ Framework Operating Systems WebKit PhantomJS Ghost

    Driver
  15. Web Page Processing

  16. Ghost Driver https://speakerdeck.com/detronizator/boston Ghost Driver is a pure JavaScript implementation

    of the WebDriver Wire Protocol for PhantomJS. It's a Remote WebDriver that uses PhantomJS as back-end. https://github.com/detro/ghostdriver
  17. Python Example from selenium import webdriver driver = webdriver.PhantomJS('phantomjs') driver.get('http://www.phantomjs.org')

    print driver.title driver.quit() pip install selenium
  18. PhantomJS (Remote) WebDriver phantomjs --webdriver=9134 Port

  19. Ruby Example require "selenium-webdriver" driver = Selenium::WebDriver.for(:remote, :url => "http://localhost:9134")

    driver.navigate.to "http://google.com" element = driver.find_element(:name, 'q') element.send_keys "PhantomJS" element.submit puts driver.title driver.quit gem install selenium-webdriver
  20. Node.js JavaScript Example var wd = require('wd'); var driver =

    wd.remote('localhost', 9134); driver.init(function() { driver.get("http://casperjs.org", function() { driver.title(function(err, title) { console.log(title); }); }); }); npm install wd
  21. .NET and C# See http://michael-whelan.net/using-phantomjs-with-webdriver http://www.andykelk.net/tech/headless-browser-testing-with-phantomjs- selenium-webdriver-c-nunit-and-mono

  22. None
  23. Using CasperJS var casper = require('casper').create(); casper.start('http://duckduckgo.com', function() { this.echo(this.getTitle());

    }); casper.then(function() { this.fill('form', { q: 'Selenium San Jose' }, true); }); casper.then(function() { var snippet = this.evaluate(function() { return document.querySelector('div#links > div').textContent; }); this.echo(snippet); }); casper.run();
  24. Headless Testing

  25. None
  26. On Being Defensive http://ariya.ofilabs.com/2012/12/quality-code-via-multiple-layers-of-defense.html

  27. Pre-commit Hook http://ariya.ofilabs.com/2012/03/git-pre-commit-hook-and-smoke-testing.html if [ `date +%w` -eq 6 ];

    then echo "Enjoy your life. Do not work on Sunday!" exit 1 fi exit 0 make test
  28. Jasmine Mocha QUnit YUITest Capybara tapedeck Karma Testem Buster.JS Travis

    CI Grunt Laika
  29. “YOU SHALL NOT PASS!” — Darth Vader

  30. Screen Capture

  31. HTML-to-Anything var page = require('webpage').create(); page.open(title, function (status) { if

    (status !== 'success') { console.log('FAIL to load the address'); } else { page.render(filename); } phantom.exit(); }); Outputs: external file, base64 string Formats: PNG, JPEG, GIF, PDF
  32. Rendering Aspects Zoom Factor Scale the page up and down

    Viewport Size Simulates different window dimensions Clipping Rectangle Crop portions of the page
  33. Media Queries, Responsive Design http://mediaqueri.es/

  34. Dichromacy http://ariya.ofilabs.com/2012/10/web-page-screenshot-with-phantomjs.html

  35. De-CSS-ify (No Stylesheets) http://ariya.ofilabs.com/2013/06/capturing-web-page-without-stylesheets.html

  36. Resources Blocking page.onResourceRequested = function(requestData, request) { if ((/http:\/\/.+?\.css$/gi).test(requestData['url'])) {

    console.log('Skipping', requestData['url']); request.abort(); } }; Skipping http://static.bbci.co.uk/frameworks/barlesque/2.45.9/mobile/3.5/style/main.css Skipping http://static.bbci.co.uk/bbcdotcom/0.3.184/style/mobile/bbccom.css Skipping http://static.bbci.co.uk/news/1.7.1-259/stylesheets/core.css Skipping http://static.bbci.co.uk/news/1.7.1-259/stylesheets/compact.css
  37. Visual Regression http://tldr.huddle.com/blog/css-testing/ https://github.com/Huddle/PhantomCSS https://github.com/facebook/huxley https://github.com/bslatkin/dpxdt

  38. Network Monitoring

  39. Requests + Responses Capture var page = require('webpage').create(); page.onResourceRequested =

    function(request) { console.log('Request ' + JSON.stringify(request, undefined, 4)); }; page.onResourceReceived = function(response) { console.log('Receive ' + JSON.stringify(response, undefined, 4)); }; page.open(url);
  40. Network Waterfall

  41. YSlow + PhantomJS http://yslow.org/phantomjs/

  42. Future Outlook

  43. Engine vs Browser Pratt & Whitney JT9D http://ariya.ofilabs.com/2011/06/your-webkit-port-is-special-just-like-every-other-port.html

  44. Diversity = Better

  45. Future (with Caveat Emptor) PhantomJS 2.x Fresher, multi-process QtWebKit PhantomJS

    3.x Chromium-based
  46. Thank You @ariyahidayat ariya.ofilabs.com/highlights speakerdeck.com/ariya Some artworks are from http://openclipart.org.