Sencha Oslo Meetup

Sencha Oslo Meetup

Transcript

1. © 2014 Bryntum AB @bryntum TDD + CI WITH SIESTA
2. ABOUT ME ! • Web & Ext JS developer since 2007 • Started Bryntum 2009 • Components & tools for the Sencha ecosystem ! • www.bryntum.com • @bryntum
3. AGENDA • Challenges at Bryntum • Our CI process • Intro to Siesta • Code sanity & quality tests • Monkey tests • Logging errors • Pre-commit hooks • In-IDE testing with PhantomJS
4. Challenges @ Bryntum
5. CHALLENGES • Limited resources, want focus on product dev • Distributed team • Multi-browser (Ext JS helps), multi-OS, multi-timezone • Multi-Ext JS versions • Knowing health of product X • Too long release cycle (build, test, docs, upload etc) • Bug reports submitted in forum, hard to track
6. Our CI process
7. BRYNTUM CI IN 2010
9. Epic Fail Decent Fail Awesome Good thousand CFS AWESOME-O-METER Epic Fail Decent Fail Awesome Good AWESOME-O-METER
12. BRYNTUM CI IN 2014
13. VM FARM 10 9 8
14. ~12K ASSERTIONS 2 RUNS / DAY
18. DST TESTS
19. DST TESTS
20. PRE COMMIT HOOKS
21. AUTOMATED RELEASES
23. Epic Fail Decent Fail Awesome Good Epic Fail Decent Fail Awesome Good AWESOME-O-METER
25. INTRODUCING SIESTA
27. Component Inspector Show fails only Show/hide DOM View test source Settings Recorder
28. SIESTA FEATURES Each test is sandboxed in its own iframe Unit testing + Functional testing Written completely in JS, extensible. Test any JS/NodeJS code Adapters for Ext JS + Sencha Touch
29. SIESTA FEATURES Automatically detects global variable leaks Support for async testing BDD layer (it/describe/expect etc) Code coverage (via Istanbul) Event Recorder
30. SENCHA AWARENESS waitForStoresToLoad(userStore, callback); waitForRowsVisible(gridPanel, callback); waitForEvent(someList, ‘selectionchange’, cb);
31. BDD / SUBTESTS You can group your assertions infinitely using t.it. t.iit to isolate and run only a certain test group. t.xit to skip describe(“User model tests”, function(t) { t.it(“Nested test here”, function(t) { var user = new User({name : ‘Bob’}); t.expect(user.name).toBe(‘Bob’); }); });
32. A SIESTA UNIT TEST Is pure JS, can contain any code Should be wrapped by either “StartTest” or “describe” ! Callback is called when all preloads are done and page is ‘ready’
33. BASIC ASSERTIONS describe(‘Basic stuff’, function(t) { t.is(1, 1, '1 is 1') t.isnt(1, 2, "1 isn't 2") t.like('Yo', /yo/i, '"Yo" is like "yo"') t.throwsOk(function () { throw "yo" }, /yo/, 'Exception thrown, and its "yo"') t.livesOk(function () { var a = 1; }, 'Exception not thrown') t.isDeeply({ a : 1, b : 2 }, { a : 1, b : 2 }, 'Correct') })
34. UNIT TESTS Focus on unit tests first, test your model, utility layer You can also test views in isolation, a “UI unit test”. The less code executed in a test, the faster you will be able to find an error if a test fails.
35. APPLICATION TESTS Siesta will visit a URL All application code will be executed, errors harder to find Still has high value, does the application work or not
36. A SIMPLE UNIT TEST Harness.start( { group : 'Model tests', preload : [ ‘class1.js’ ‘user.js’ ], items : [ '001_user.t.js' ] } );
37. 001_USER_T.JS // test files usually named foo.t.js describe(“User model tests”, function(t) { var user = new User({name : ‘Bob’}); ! t.expect(user.name).toBe(‘Bob’); });
38. UI TESTS
39. UI TESTS More complex by nature, as they involve DOM User interactions, waiting and assertions Test UI components in isolation, reveals tight coupling.
40. THE CHAIN COMMAND Since most UI testing scenarios are async, we use the t.chain to push commands onto an asynchronous queue. ! Each “step” in the queue can be an object or a function. ! Functions receive a continue callback as the first argument
41. THE CHAIN COMMAND t.chain( { action : ‘click’, target : ‘#foo’ }, function(next) { // Any code can go here, remember to call next next(); }, function(next) { // Advancing the chain by passing the callback to another method t.click(‘#foo’, next); }, ... );
42. SIMULATING USER INPUT Simply use the various chain actions to interact with the UI. t.chain( { waitFor : ‘selector’, args : ‘#someinput’ }, { click : ‘#someinput’ }, { type : ‘foo bar[ENTER]’ }, ! function(next) { // Assert that login was successful next(); } );
43. TARGETING THE DOM Siesta supports many ways of targeting the DOM content. ! * DOM/Ext element * Ext JS component * CSS selector * CQ selector * CSQ selector * Coordinate (avoid) * A function returning any of the above * You can also provide an ‘offset’ to click exactly at a certain point within your target
44. DOM/EXT ELEMENT // HTMLElement var el2 = document.getElementById(“foo”) ! // Ext Element var el = Ext.get(“foo”)
45. SELECTORS // CSS “div.foo” “#someId” ! // Component Query, prefixed by >> “>> textfield” ! // Composite Query “gridpanel => .x-grid-cell:nth-child(2)”
46. TARGETING THE DOM var myGrid = new Ext.GridPanel({ id : ‘myGrid’, ... }); ! t.chain( { action : 'click', target : [10, 10] }, { action : 'click', target : '#myGrid' }, { action : 'click', target : myGrid }, { action : 'click', target : doc.getElementById('myGrid') }, { action : 'click', target : myGrid() }, { action : 'click', target : '>> gridpanel', offset : [“50%”, 10] }, { action : 'click', target : 'gridpanel => .x-grid-row' }, function() { return myGrid.el; } );
47. WAITING With UI tests, you often need to wait: ! Grid : waitForRowsVisible ! Data : waitForStoresToLoad ! CSS : waitForSelector ! 40+ waitForXXX commands in the API
48. WAITING t.chain( { waitForSelector : ‘#someinput’ }, { waitFor : 1000 }, { waitForEvent : [document.body, ‘click’] }, ! function(next) { // Programmatically wait t.waitFor(1000, next); } );
49. CODE QUALITY If you have a large or a distributed team, achieving and maintaining a high quality JS code base is hard. ! Write tests to help you catch simple errors early. ! We use a lot of sanity check tests.
50. COMPONENT QUALITY TESTS 1. Global variable leak checks. 2. Your component can be loaded on demand with Ext.Loader 3. Your component doesn't create any global Ext JS overrides 4. It passes basic JsHint rules (no syntax errors, trailing commas, debugger) 5. It does not use global style rules ('.x-panel' etc) 6. It can be sub-classed 7. It does not leak any additional components or DOM elements 8. It doesn't override any private Ext JS methods in your component 9. It can be created & destroyed 10. It passes a basic monkey test (random interactions with it), no exceptions https://github.com/matsbryntse/Component-maker-test-pack
51. MONKEY TESTS
52. Ext Scheduler ships with ~40 examples ! We monkey test each of those 40, if an exception is thrown, test fails ! Free testing help MONKEY TESTS
53. Ext Scheduler has ~40 examples online ! If exceptions is thrown, email sent to the team ! Again, free testing help TESTING ONLINE SAMPLES
55. Ideal place to stop broken code entering repo ! As part of your build copy the hook into all developers workspace. ! Ensures the most obvious bugs don’t pollute your repo. PRE-COMMIT HOOKS
56. JSHINT #!/bin/bash! ! # 1. Run JsHint! # ----------------------! jshint $(git diff --name-only --diff-filter=ACMRTUXB | while IFS= read -r line; do! ! if [[ $line =~ ^js\/.*\.js$ ]]; then! echo $line! fi! ! done)! ! if [ "$?" != "0" ]; then! exit 1! fi! ! echo "JSHint sanity test passed correctly"
57. FIND IIT STATEMENTS ! # 2. Look for forgotten t.iit statements in tests! # ----------------------! git diff --name-only --diff-filter=ACMRTUXB | while IFS= read -r line; do! if [[ $line =~ ^tests\/.*\.t\.js$ ]]; then! ! cat $line | grep -q '.iit('! ! if [ $? == "0" ]; then! echo "t.iit() statement found in file: $line"! # will exit a pipe-subshell only, need additional check below! exit 1! fi! fi! done
58. SMOKE TESTS # 3. Run smoke tests! # ----------------------! ! phantomjs http://lh/ExtScheduler2.x/tests/index-no-ui.html?smoke=1 --pause 0 --no-color --filter run-smoketests!
59. AUTOMATION
60. AUTOMATION Siesta tests can be automated using either PhantomJS or Selenium ! For in-IDE use, we recommend PhantomJS (fast) ! For nightly builds and CI, we recommend WebDriver. !
61. PHANTOMJS
62. IDE INTEGRATION Absolutely essential to not waste time ! Otherwise, to much time spent Alt+Tab to browser ! Very easy to setup in any decent IDE ! Bind to a one-hand key command for max speed
63. IDE OUTPUT
64. INTEGRATES WITH WebStorm ! Eclipse ! Visual Studio ! and many more...
65. CODE COVERAGE
68. EVENT RECORDER Generate UI tests quickly ! Creates stable targets using Ext.ComponentQuery ! Available in our latest Siesta release
69. EVENT RECORDER
70. EVENT RECORDER
71. DEMOS
72. THANKS FOR LISTENING! © 2014 Bryntum AB