What is the Holy Grail? • Browser independent • All tests written in JavaScript • Unit and integration support • Test against a single browser in development • Test against all target browsers in CI • Runs from the CLI--not by opening some random HTML page.
Where are We Now? • Individual aspects, but nothing working in concert • Classic choose any 3: cheap, fast, scalable • Choose one implementation and lose one of the three
How did We Get Here? • Javascript not really considered a first class language until recently • Javascript traditionally confined to the browser • Focus on Single Page Applications/Client side JS MVC/Insert your term here
var page = require('webpage').create(); var url = 'http://www.phantomjs.org/'; page.open(url, function (status) { page.includeJs("jquery.min.js", function() { // jQuery is loaded, now manipulate the DOM }); page.evalute(function() { // evaluate code in the remote context $("title").html("ZOMG"); }); phantom.exit(); });
• Built on top of PhantomJS • Used to write integration tests • DOM interaction API. Example: fillIn(“foo”, “bar”) • Should not be used to write unit tests • Good for a small # of test cases • Tests bound PhantomJS
• Runs from the command line • Supports multiple browsers • Unclear how to do integration tests • Browser independent, but test runner dependent • Must maintain browser installations • Tests bound to test runner
// Load the page from localhost browser = new Browser() browser.visit("http://localhost:3000/", function () { // Fill email, password and submit form browser. fill("email", "[email protected]"). fill("password", "eat-the-living"). pressButton("Sign Me Up!", function() { // Form submitted, new page loaded. assert.ok(browser.success); assert.equal(browser.text("title"), "BRAINZ"); }); });
# rspec describe "the signup process", :type => :request do it "signs me in" do # capybara within("#session") do fill_in 'Login', :with => '[email protected]' fill_in 'Password', :with => 'password' end click_link 'Sign in' end end
• Multiple adapters for headless or real browsers • Easy to maintain large test suites • Write tests in Ruby • Test code and app code exist in separate threads. This can be confusing.
• No unified framework for unit tests and integration tests • Dependency hell (c++, python, node, java, or ruby) • Tests bound to test runner • Not all methods CLI friendly
1. Remove test runner dependence 2. Unify integration and unit tests 3. Run in a headless browser for local tests 4. Run in real browsers in CI 5. Try to do it with Javascript
// this is a unit test test("hello test", function() { ok(1 == "1", "Passed!"); }); // this is an integration test test("user can fill in the form", function() { fillIn("#todo", "Fix JS Testing"); press("#todo-form submit"); assertContains("#todos", "Fix JS Testing"); });
# Use phantomJS to open the HTML page # and capture the results $ phantomjs run_qunit.js tests.html ..................... 10 Tests, 10 Passes, 0 Failures
# Upload that HTML file to your CI server # so it can be accessed with test swarm $ swarm execute tests.js Running in all browsers.... IE9: 10 tests, 10 Passed, 0 Failures IE6, 10 tests, 0 Passed, 12349871234987 Failures Chrome: 10 tests, 10 Passed, 0 Failures FireFox: 10 tests, 10 Passed, 0 Failures
# ENV variables determine build # Ideally set in your CI process $ swarm execute tests.js BROWSER=IE9 $ swarm execute tests.js BROWSER=FF,SAFARI,CHROME
Why? • Completely browser independent • All tests javascript • Test runner independent • CI Friendly • Tests and app code running in the same thread. Read: direct access to app + ui.
Why? • Requirements change test system • Testing backend + frontend requires a different approach • The holy grail for that is use testing tools for what language your backend is in • Things like HTML5 Audio/Video can induce complexity
Moral of the Story • Testing Javascript applications is difficult but possible • Decide if you want to test your frontend and backend together. • Oh ya....test that shit!
ahawkins
shinmiy
syumai
k0kubun
nanya3737
punchdrunker
sagawa23
pangmoo
akitotsukahara
katsyoshi
mame
yui_knk
masatoshiitoh
jcasabona
kevinliebholz
lara
kastner
sonatard
searls
brad_frost
destraynor
paulrobertlloyd
malarkey
lynnandtonic
jonyablonski