Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Javascript Testing: The Holy Grail

Javascript Testing: The Holy Grail

Approaching the Holy Grail of Javascript application testing. Presented at RejectJS in Berlin

Adam Hawkins

October 04, 2012
Tweet

More Decks by Adam Hawkins

Other Decks in Programming

Transcript

  1. View Slide

  2. Presented at RejectJS Berlin 2012

    View Slide

  3. Ich bin
    • Adam Hawkins
    • tw://adman65
    • adn://twinturbo
    • gh://twinturbo
    • http://broadcastingadam.com

    View Slide

  4. Should I listen to this guy?

    View Slide

  5. You’re god damn right

    View Slide

  6. Who, What, Why?

    View Slide

  7. What is the Holy Grail?
    • Browser independent
    • All tests written in JavaScript
    • Unit and integration support
    • Test against a single browser in
    development
    • Test against all target browsers in CI
    • Runs from the CLI--not by opening some
    random HTML page.

    View Slide

  8. Why Care?
    • Testing + Multi browser CI is the only way
    to know that your application is functioning
    correctly.
    • Manually test if you want....

    View Slide

  9. Where are We Now?
    • Individual aspects, but nothing working in
    concert
    • Classic choose any 3: cheap, fast, scalable
    • Choose one implementation and lose one
    of the three

    View Slide

  10. How did We Get Here?
    • Javascript not really considered a first class
    language until recently
    • Javascript traditionally confined to the
    browser
    • Focus on Single Page Applications/Client
    side JS MVC/Insert your term here

    View Slide

  11. Learning to test my
    javascript app
    What have you been
    doing all these years?

    View Slide

  12. This is harder than Haskell

    View Slide

  13. Let’s See What’s Out
    There

    View Slide

  14. *this is for testing
    Javascript applications

    View Slide

  15. PhantomJS

    View Slide

  16. $ phantomjs
    phantomjs> console.log("Yo Dawg");
    Yo Dawg
    undefined
    phantomjs>

    View Slide

  17. var page = require('webpage').create();
    var url = 'http://www.phantomjs.org/';
    page.open(url, function (status) {
    page.includeJs("jquery.min.js", function() {
    // jQuery is loaded, now manipulate the DOM
    });
    page.evalute(function() {
    // evaluate code in the remote context
    $("title").html("ZOMG");
    });
    phantom.exit();
    });

    View Slide

  18. • Headless browser
    • Execute JavaScript programmatically in a
    real content
    • Real DOM

    View Slide

  19. CasperJS

    View Slide

  20. casper.start "http://www.google.fr/", ->
    @test.assertTitle "Google", "google homepage title incorrect"
    @test.assertExists 'form[action="/search"]', "main form is found"
    @fill 'form[action="/search"]', q: "foo", true
    casper.then ->
    @test.assertTitle "foo - Recherche Google", "google title is ok"
    @test.assertUrlMatch /q=foo/, "search term has been submitted"
    @test.assertEval (->
    __utils__.findAll("h3.r").length >= 10
    ), "google search for \"foo\" retrieves 10 or more results"
    casper.run ->
    @test.renderResults true

    View Slide

  21. • Built on top of PhantomJS
    • Used to write integration tests
    • DOM interaction API. Example:
    fillIn(“foo”, “bar”)
    • Should not be used to write unit tests
    • Good for a small # of test cases
    • Tests bound PhantomJS

    View Slide

  22. JSTestDriver
    Made by:

    View Slide

  23. GreeterTest = TestCase("GreeterTest");
    GreeterTest.prototype.testGreet = function() {
    var greeter = new myapp.Greeter();
    assertEquals("Hello World!", greeter.greet("World"));
    };

    View Slide

  24. JSTestDriver Architecture

    View Slide

  25. • Runs from the command line
    • Supports multiple browsers
    • Unclear how to do integration tests
    • Browser independent, but test runner dependent
    • Must maintain browser installations
    • Tests bound to test runner

    View Slide

  26. ZombieJS

    View Slide

  27. // Load the page from localhost
    browser = new Browser()
    browser.visit("http://localhost:3000/", function () {
    // Fill email, password and submit form
    browser.
    fill("email", "[email protected]").
    fill("password", "eat-the-living").
    pressButton("Sign Me Up!", function() {
    // Form submitted, new page loaded.
    assert.ok(browser.success);
    assert.equal(browser.text("title"), "BRAINZ");
    });
    });

    View Slide

  28. • Clear support for integration tests
    • Decent DOM interaction API
    • Built on Node
    • Tests bound to ZombieJS
    • Similar to CasperJS + PhantomJS

    View Slide

  29. Capybara

    View Slide

  30. # rspec
    describe "the signup process", :type => :request do
    it "signs me in" do
    # capybara
    within("#session") do
    fill_in 'Login', :with => '[email protected]'
    fill_in 'Password', :with => 'password'
    end
    click_link 'Sign in'
    end
    end

    View Slide

  31. • Multiple adapters for headless or real
    browsers
    • Easy to maintain large test suites
    • Write tests in Ruby
    • Test code and app code exist in separate
    threads. This can be confusing.

    View Slide

  32. What do these have in
    Common?

    View Slide

  33. • No unified framework for unit tests and
    integration tests
    • Dependency hell (c++, python, node, java,
    or ruby)
    • Tests bound to test runner
    • Not all methods CLI friendly

    View Slide

  34. Approaching the Holy Grail

    View Slide

  35. How can we combine
    these things?

    View Slide

  36. 1. Remove test runner dependence
    2. Unify integration and unit tests
    3. Run in a headless browser for local tests
    4. Run in real browsers in CI
    5. Try to do it with Javascript

    View Slide

  37. Step 1: Writing Tests

    View Slide

  38. View Slide

  39. // this is a unit test
    test("hello test", function() {
    ok(1 == "1", "Passed!");
    });
    // this is an integration test
    test("user can fill in the form", function() {
    fillIn("#todo", "Fix JS Testing");
    press("#todo-form submit");
    assertContains("#todos", "Fix JS Testing");
    });

    View Slide

  40. * Use jQuery to implement a higher level
    DOM interaction UI similar to capybara for tests

    View Slide

  41. Step 2: Run The Tests

    View Slide





  42. QUnit Example








    View Slide

  43. # Use phantomJS to open the HTML page
    # and capture the results
    $ phantomjs run_qunit.js tests.html
    .....................
    10 Tests, 10 Passes, 0 Failures

    View Slide

  44. Step 3: Multiple
    Browsers

    View Slide

  45. View Slide





  46. Integration Tests










    View Slide

  47. # Upload that HTML file to your CI server
    # so it can be accessed with test swarm
    $ swarm execute tests.js
    Running in all browsers....
    IE9: 10 tests, 10 Passed, 0 Failures
    IE6, 10 tests, 0 Passed, 12349871234987 Failures
    Chrome: 10 tests, 10 Passed, 0 Failures
    FireFox: 10 tests, 10 Passed, 0 Failures

    View Slide

  48. # ENV variables determine build
    # Ideally set in your CI process
    $ swarm execute tests.js BROWSER=IE9
    $ swarm execute tests.js BROWSER=FF,SAFARI,CHROME

    View Slide

  49. Is this Better?

    View Slide

  50. You’re god damn right

    View Slide

  51. Why?
    • Completely browser independent
    • All tests javascript
    • Test runner independent
    • CI Friendly
    • Tests and app code running in the same thread.
    Read: direct access to app + ui.

    View Slide

  52. The community can
    solve this problem

    View Slide

  53. I’ve Already Started
    • https://github.com/radiumsoftware/iridium

    View Slide

  54. Bitte warten Sie

    View Slide

  55. TL;DR: There is no
    Holy Grail

    View Slide

  56. View Slide

  57. Why?
    • Requirements change test system
    • Testing backend + frontend requires a
    different approach
    • The holy grail for that is use testing tools
    for what language your backend is in
    • Things like HTML5 Audio/Video can induce
    complexity

    View Slide

  58. Moral of the Story
    • Testing Javascript applications is difficult but
    possible
    • Decide if you want to test your frontend
    and backend together.
    • Oh ya....test that shit!

    View Slide

  59. Die Slides
    https://speakerdeck.com/u/twinturbo

    View Slide