PhantomJS for Web Page Automation

Presented at Selenium San Jose meetup:
http://www.meetup.com/SeleniumSanJose/events/154836972/.

PhantomJS, the scriptable headless WebKit-based automation tool, has gained a lot of traction in its first 3 years. It is a popular choice for continuous integration, whether to run basic tests or to catch rendering regressions. Many JavaScript test frameworks work well with PhantomJS and even Selenium-based tests can be integrated thanks to the support for WebDriver via Ghost Driver. In addition, because PhantomJS permits the inspection of network traffic, it is suitable to run various analysis on the network behavior and performance. This talk will highlight the basic usages of PhantomJS and explore various PhantomJS-tools for the purpose of headless testing of web applications, screen capture, and network monitoring.

Transcript

1. PhantomJS for Web Page Automation @ariyahidayat Mountain View Jan 8, 2014 South Bay Selenium Meetup
2. /usr/bin/whoami shapesecurity.com
3. “Software Provocateur” PhantomJS Esprima
4. 11-Second Elevator Pitch Scriptable Simulate user interactions Headless Command-line, no GUI WebKit Layout and rendering
5. Important Traits Multi platforms Zero dependencies (*) Works with Other Tools
6. Who is Using PhantomJS http://phantomjs.org/users.html
7. Versions vs Downloads http://phantomjs.org/releases.html Announced Jan 23, 2011 WebDriver Winter 2012 Pure headless Linux Spring 2012
8. Web Page Processing Screen Capture Headless Testing Network Monitoring
9. 5-Minute Primer
10. Hello, World! console.log('Hello, world!'); phantom.exit();
11. Capture to Image var page = require('webpage').create(); page.open('http://google.com', function (status) { page.render('google.png'); phantom.exit(); });
12. Get Page Title var page = require('webpage').create(); page.open('http://linkedin.com', function (status) { if (status !== 'success') { console.log('FAIL to load the address'); } else { var title = page.evaluate(function () { return document.title; }); console.log('Page title is', title); } phantom.exit(); });
13. Use jQuery (or Other Libs) var page = require('webpage').create(); page.open('http://www.sample.com', function() { page.includeJs("http://code.jquery.com/jquery-1.10.2.js", function() { page.evaluate(function() { $("button").click(); }); phantom.exit() }); });
14. Behind the Scene C++ Framework Operating Systems WebKit PhantomJS Ghost Driver
15. Web Page Processing
16. Ghost Driver https://speakerdeck.com/detronizator/boston Ghost Driver is a pure JavaScript implementation of the WebDriver Wire Protocol for PhantomJS. It's a Remote WebDriver that uses PhantomJS as back-end. https://github.com/detro/ghostdriver
17. Python Example from selenium import webdriver driver = webdriver.PhantomJS('phantomjs') driver.get('http://www.phantomjs.org') print driver.title driver.quit() pip install selenium
18. PhantomJS (Remote) WebDriver phantomjs --webdriver=9134 Port
19. Ruby Example require "selenium-webdriver" driver = Selenium::WebDriver.for(:remote, :url => "http://localhost:9134") driver.navigate.to "http://google.com" element = driver.find_element(:name, 'q') element.send_keys "PhantomJS" element.submit puts driver.title driver.quit gem install selenium-webdriver
20. Node.js JavaScript Example var wd = require('wd'); var driver = wd.remote('localhost', 9134); driver.init(function() { driver.get("http://casperjs.org", function() { driver.title(function(err, title) { console.log(title); }); }); }); npm install wd
21. .NET and C# See http://michael-whelan.net/using-phantomjs-with-webdriver http://www.andykelk.net/tech/headless-browser-testing-with-phantomjs- selenium-webdriver-c-nunit-and-mono
23. Using CasperJS var casper = require('casper').create(); casper.start('http://duckduckgo.com', function() { this.echo(this.getTitle()); }); casper.then(function() { this.fill('form', { q: 'Selenium San Jose' }, true); }); casper.then(function() { var snippet = this.evaluate(function() { return document.querySelector('div#links > div').textContent; }); this.echo(snippet); }); casper.run();
24. Headless Testing
26. On Being Defensive http://ariya.ofilabs.com/2012/12/quality-code-via-multiple-layers-of-defense.html
27. Pre-commit Hook http://ariya.ofilabs.com/2012/03/git-pre-commit-hook-and-smoke-testing.html if [ `date +%w` -eq 6 ]; then echo "Enjoy your life. Do not work on Sunday!" exit 1 fi exit 0 make test
28. Jasmine Mocha QUnit YUITest Capybara tapedeck Karma Testem Buster.JS Travis CI Grunt Laika
29. “YOU SHALL NOT PASS!” — Darth Vader
30. Screen Capture
31. HTML-to-Anything var page = require('webpage').create(); page.open(title, function (status) { if (status !== 'success') { console.log('FAIL to load the address'); } else { page.render(filename); } phantom.exit(); }); Outputs: external file, base64 string Formats: PNG, JPEG, GIF, PDF
32. Rendering Aspects Zoom Factor Scale the page up and down Viewport Size Simulates different window dimensions Clipping Rectangle Crop portions of the page
33. Media Queries, Responsive Design http://mediaqueri.es/
34. Dichromacy http://ariya.ofilabs.com/2012/10/web-page-screenshot-with-phantomjs.html
35. De-CSS-ify (No Stylesheets) http://ariya.ofilabs.com/2013/06/capturing-web-page-without-stylesheets.html
36. Resources Blocking page.onResourceRequested = function(requestData, request) { if ((/http:\/\/.+?\.css$/gi).test(requestData['url'])) { console.log('Skipping', requestData['url']); request.abort(); } }; Skipping http://static.bbci.co.uk/frameworks/barlesque/2.45.9/mobile/3.5/style/main.css Skipping http://static.bbci.co.uk/bbcdotcom/0.3.184/style/mobile/bbccom.css Skipping http://static.bbci.co.uk/news/1.7.1-259/stylesheets/core.css Skipping http://static.bbci.co.uk/news/1.7.1-259/stylesheets/compact.css
37. Visual Regression http://tldr.huddle.com/blog/css-testing/ https://github.com/Huddle/PhantomCSS https://github.com/facebook/huxley https://github.com/bslatkin/dpxdt
38. Network Monitoring
39. Requests + Responses Capture var page = require('webpage').create(); page.onResourceRequested = function(request) { console.log('Request ' + JSON.stringify(request, undefined, 4)); }; page.onResourceReceived = function(response) { console.log('Receive ' + JSON.stringify(response, undefined, 4)); }; page.open(url);
40. Network Waterfall
41. YSlow + PhantomJS http://yslow.org/phantomjs/
42. Future Outlook
43. Engine vs Browser Pratt & Whitney JT9D http://ariya.ofilabs.com/2011/06/your-webkit-port-is-special-just-like-every-other-port.html
44. Diversity = Better
45. Future (with Caveat Emptor) PhantomJS 2.x Fresher, multi-process QtWebKit PhantomJS 3.x Chromium-based
46. Thank You @ariyahidayat ariya.ofilabs.com/highlights speakerdeck.com/ariya Some artworks are from http://openclipart.org.