Client-side Testing with Jasmine, CasperJS, and JsTestDriver

Slides from a 15-minute talk given to the NYC Selenium Meetup about how we do Client-side testing at Intent Media currently and what next-generation technologies we're looking at.

Transcript

1. Client-side Testing with Jasmine, CasperJS, and JsTestDriver
2. Who am I? Hi, I’m Andrew Childs Lead UI / Designer / Front-end Engineer at Intent Media andrew@intentmedia.com github.com/andrewchilds
3. 1. Problems 2. Current Stack 3. Technical Overview 4.What’s Next
4. Problems: We Have Them 1. Javascript Unit Testing We have a large, complicated Javascript codebase that needs testing We want to ensure our codebase works in all major browsers, back to IE 7 We want the experience of reading, writing, and running tests to be as fast and easy as possible 2. Remote Dependency Testing We place sophisticated, tightly integrated ads on sites that don’t belong to us We have dependencies on DOM elements, meta tags, cookies, etc We want to ensure we don’t break our partner sites and vice versa
5. Our Current Stack 1. Javascript Unit Testing Framework: Jasmine (+ CoffeeScript) Test runner: JsTestDriver (+ Ant) CI server: TeamCity We use a captured Firefox process for local & TeamCity testing 2. Remote Dependency Testing CasperJS + our own custom library (to be open-sourced soon) Runs every hour on TeamCity Hourly screenshots of our ads are visible at a private URL and on our kitchen dashboard
6. 1. Problems 2. Current Stack 3. Technical Overview • Jasmine • CasperJS 4.What’s Next
7. Our Jasmine Suite 1. By the Numbers 60 CoffeeScript files, 1000 tests total Takes 60 seconds to run locally using JsTestDriver and Firefox (assuming CoffeeScript files are already compiled) Every branch pushed to Github runs through our Jasmine tests along with all unit/integration tests (RSpec, JUnit, Cucumber) 2. Jasmine Code Style CoffeeScript, while not perfect, means less boilerplate We use custom aliases of the “describe” keyword to make logical structure of units more explicit and human-readable and to expose code smells beforeEach steps should map to “context” (or “say”) steps Text
8.        describe  "#convert_YYYYMMDD_to_MonDDYYYY",  -­‐>                subject  =  IntentMedia.Date.convert_YYYYMMDD_to_MonDDYYYY                say  "the  date  is  formatted  correctly",  -­‐>                        it  "should  handle  double-­‐digit  months  and  days",  -­‐>                                expect(subject('20121015')).toBe  'Oct  15,  2012'                        it  "should  handle  single-­‐digit  months  and  days",  -­‐>                                expect(subject('20120105')).toBe  'Jan  5,  2012'                        it  "should  handle  double-­‐digit  years",  -­‐>                                expect(subject('990201')).toBe  'Feb  1,  1999'                say  "the  date  is  falsey",  -­‐>                        it  "should  return  an  empty  string",  -­‐>                                expect(subject(null)).toBe  ''                                expect(subject('')).toBe  ''
9. About Writing JS Tests... In our experience: • It’s possible to over-test and write tests that don’t add much value • JavaScript functions don’t always cleanly map to “units”
10. function  wakeUpInTheMorning()  {    if  (isMorning())  {        wakeUp();    } } describe  "#wakeUpInTheMorning",  -­‐>    subject  =  wakeUpInTheMorning    beforeEach  -­‐>        spyOn('wakeUp')    say  "it  is  morning",  -­‐>        beforeEach  -­‐>            spyOn('isMorning').andReturn  true        it  "should  call  wakeUp",  -­‐>            subject()            expect(wakeUp).toHaveBeenCalled()    say  "it  is  not  morning",  -­‐>        beforeEach  -­‐>            spyOn('isMorning').andReturn  false        it  "should  not  call  wakeUp",  -­‐>            subject()            expect(wakeUp).not.toHaveBeenCalled() Not every function is worth testing
11. CasperJS Features: Built on PhantomJS, a headless webkit browser Capture screenshots of a page or a particular DOM element Can execute arbitrary Javascript or load external JS into the remote window context Assert selector visibility, HTTP status, redirection Easy to write an even more terse library on top, which we did
12. describe  'Orbitz  Hotel  Details  page',  -­‐>    url  =  'http://www.orbitz.com/yadda/yadda'    open  url,  once_orbitz_page_is_valid,  -­‐>        assert_selectors            'ad  target':  '#rail  div.hotelLinksMod'            'ad  unit':  '#rail  .IM_ad_unit'          assert_meta_tags            'DCSext.LNG':  'en_US'  #  locale            'WT.pn_sku':  'HOT'  #  Product  category          custom_assertions            'page  is  seen  as  a  details  page':  -­‐>  !!IntentMedia.is_hotel_details_page()        capture_selectors            'SSN_orbitz_hotel_details_page.png':  '#rail  .IM_ad_unit'
13. describe  'Requests  for  http[s]://www.intentmedia.com/  should  be  served',  -­‐>    assert_remote_resources  [        'http://www.intentmedia.com/'        'https://www.intentmedia.com/'    ] describe  'Requests  for  https://  should  redirect  to  http://www.intentmedia.com',  -­‐>    assert_redirects        'http://intentmedia.com':  'http://www.intentmedia.com/'        'http://intentmedia.com/privacy':  'http://www.intentmedia.com/privacy'        'https://intentmedia.com':  'http://www.intentmedia.com/'
14. 1. Problems 2. Current Stack 3. Technical Overview 4.What’s Next
15. JsTestDriver is good, not great 1. Pros We finally had the ability to run/test real JS in real browsers Before that we were testing our JS in JUnit using an emulated browser environment, and it was not good Every modern browser can be captured, even IE6 in a VM can run tests (though easy to get into a bad state) 2. Cons Stability issues, could be faster Dependency on Java Browser start/capture/stop process is difficult to automate No native support for watching files Limited configuration options
16. Next-generation JS Test Runners
17. Testem Uses Node.js and socket.io Similar to JsTestDriver but more powerful configuration options Handles the tedium of browser launching & capturing Handles the tedium of pre/post-processing files (coffeescript compile, jslint, etc - we use Ant to do that right now) github.com/airportyh/testem
18. Testacular Built by an Angular.js team dev (who was unhappy with JsTestDriver) to test Angular.js apps Uses Node.js and socket.io, faster and more stable than JsTestDriver’s heartbeat implementation (though it sounds like IE8 has speed issues) Can watch files and re-run on file change (like Guard) Demo video shows a 1,535 Jasmine test run taking 3 seconds Is called “Testacular” github.com/vojtajina/testacular
19. BrowserStack Not a test runner, but could be used to automate testing on dozens of browser/OS combinations, including IE 6-10 on Windows XP, 7 and 8 This is basically the Holy Grail of client-side testing, many projects are working on BrowserStack integration or something similar (e.g. Bunyip, Buster.JS, Testling, Testacular, TestSwarm) Supports a command line interface and localhost tunneling which could point to a locally-running test runner
20. What do these have in common? The test runner owns browser opening/capturing/closing The test runner owns file watching + auto-run “Browsers in the cloud” quickly becoming a reality (BrowserStack, Testling, AWS) Faster and more stable = a better dev/test experience Better debugging and IDE integration (see Testacular demo) Written for large-scale JS apps (Angular.js, Ember.js, Backbone, etc)