The Future of JavaScript Language Tooling

The Future of JavaScript Language Tooling

Silicon Valley Code Camp 2013:
http://www.siliconvalley-codecamp.com/Session/2013/the-future-of-javascript-language-tooling

It is inevitable that future JavaScript applications will grow to be more complex. Keeping the quality of such a complex system is far from trivial. Unfortunately, our JavaScript tools recently still move at a glacial speed. Many code analyzers scream at you when a semicolon is omitted, yet they are totally oblivious to a copy-paste mistake. This talk will discuss the development of emerging JavaScript language tooling designed to solve:
* code inspection: autocomplete, code outline
* static analysis: code metrics, API harness, cyclomatic complexity
* dynamic analysis: code coverage, execution tracing, run-time scalability
* source transformation: coding style, language extension (module, class, macro, …)

0284b8950e0f4a57bcc092d4dbb98d97?s=128

Ariya Hidayat

October 06, 2013
Tweet

Transcript

  1. 5.

    Every tool is open-source Tweak/customize/run with it! There are links

    (everywhere) to detailed articles/blog posts
  2. 14.

    This is the Unix Philosophy Write programs that do one

    thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface. Doug McIlroy, the inventor of Unix pipes and one of the founders of the Unix tradition
  3. 15.

    JavaScript in the Browser User Interface Browser Engine Graphics Stack

    Data Persistence Render Engine JavaScript Engine Networking I/O
  4. 16.

    JavaScript Engine Virtual Machine/ Interpreter Parser Runtime Source Syntax Tree

    Built-in objects, host objects, ... Fast and conservative
  5. 17.

    Parser var answer = 42 keyword equal sign identifier number

    Variable Declaration Identifier Literal Constant Tokenization → Tokens Parsing → Syntax Tree
  6. 18.

    { type: "Program", body: [ { type: "VariableDeclaration", declarations: [

    { type: "VariableDeclarator", id: { type: "Identifier", name: "answer" }, init: { type: "Literal", value: 42, raw: "42" } } ], kind: "var" } ] } Syntax Tree Mozilla Parser API var answer = 42; Terms → ECMAScript 5.1 Specification https://developer.mozilla.org/en-US/docs/SpiderMonkey/Parser_API
  7. 21.

    Building Blocks → Tools Parser Code Generator Source Syntax Tree

    Source Obvious examples: minifier, obfuscator, ...
  8. 22.

    Another Case: CoffeeScriptRedux JavaScript Parser Code Generator JavaScript Syntax Tree

    Generated JavaScript JavaScript Source CoffeeScript Source CoffeeScript Parser + Transformer
  9. 26.

    Strict Mode Validator 'use strict'; block = { color: 'blue',

    height: 20, width: 10, color: 'red' }; Duplicate data property in object literal not allowed in strict mode http://ariya.ofilabs.com/2012/10/validating-strict-mode.html
  10. 27.

    Linter vs Validator Validator Linter Looks for specification violations Does

    not care about coding style Works well on generated/minified code Searches for suspicious pattern Warns on style inconsistencies Works well on hand-written code
  11. 28.

    Protection Layers, Again JUnit XML + Jenkins files=$(git diff-index --name-

    only HEAD | grep -P '\.js$') for file in $files; do esvalidate $file if [ $? -eq 1 ]; then echo "Syntax error: $file" exit 1 fi done Git Precommit Hook http://ariya.ofilabs.com/2012/03/git-pre-commit-hook-and-smoke-testing.html
  12. 29.
  13. 30.

    Application Structure MyApp.create('MyApp.Person', { name: 'Joe Sixpack', age: 42, constructor:

    function(name) {}, walk: function(steps) {} run: function(steps) {} }); { objectName: 'MyApp.Person', functions: ['walk', 'run'], properties: ['name', 'age'] } Metadata
  14. 36.

    Polluting Variables var height; // some fancy processing heigth =

    200; Leaks to global test.js:3 heigth = 200; ^ LeakError: global leak detected: heigth https://github.com/kesla/node-leaky http://ariya.ofilabs.com/2012/11/polluting-and-unused-javascript-variables.html
  15. 37.

    Unused Variables var height; // some fancy processing heigth =

    200; http://ariya.ofilabs.com/2012/11/polluting-and-unused-javascript-variables.html Declared but not used test.js height - on line 1 https://github.com/Kami/node-unused
  16. 38.

    Stray Logging function detect_console(code) { function check(node) { if (node.type

    === 'CallExpression') { if (node.callee.type === 'MemberExpression') { if (node.callee.object.name === 'console') { alert('console call at line', node.loc.start.line); } } } } var tree = esprima.parse(code, { loc: true }); estraverse.traverse(tree, { enter:check }); } var answer = 42; console.log(answer); http://ariya.ofilabs.com/2013/04/automagic-removal-of-javascript-logging.html
  17. 39.

    Nested Ternary Conditionals var str = (age < 1) ?

    "baby" : (age < 5) ? "toddler" : (age < 18) ? "child": "adult"; http://ariya.ofilabs.com/2012/10/detecting-nested-ternary-conditionals.html
  18. 40.

    “Boolean Trap” Finder Can you make up your mind? treeItem.setState(true,

    false); event.initKeyEvent("keypress", true, true, null, null, false, false, false, false, 9, 0); The more the merrier? Obfuscated choice var volumeSlider = new Slider(false); Double-negative component.setHidden(false); filter.setCaseInsensitive(false); http://ariya.ofilabs.com/2012/06/detecting-boolean-traps-with-esprima.html
  19. 41.

    Code Complexity http://jscomplexity.org/ if (true) "foo"; else "bar"; Control Flow

    Graph 6 edges 6 nodes 1 exit Cyclomatic Complexity = 2
  20. 42.

    Most Complex Function http://ariya.ofilabs.com/2013/05/continuous-monitoring-of-javascript-code-complexity.html var cr = require('complexity-report'), content =

    require('fs').readFileSync('index.js', 'utf-8'), list = []; cr.run(content).functions.forEach(function (entry) { list.push({ name: entry.name, value: entry.complexity.cyclomatic }); }); list.sort(function (x, y) { return y.value - x.value; }); console.log('Most cyclomatic-complex functions:'); list.slice(0, 6).forEach(function (entry) { console.log(' ', entry.name, entry.value); });
  21. 46.

    Dynamic Code Coverage via Instrumentation var answer = 42; alert(answer);

    var __cov_l$4m7m$L464yvav5F$qhNA = __coverage__['hello.js']; __cov_l$4m7m$L464yvav5F$qhNA.s['1']++; var answer = 42; __cov_l$4m7m$L464yvav5F$qhNA.s['2']++; alert(answer); http://gotwarlost.github.io/istanbul/
  22. 47.

    Latent Trap of Statement Coverage function inc(p, q) { if

    (q == undefined) q = 1; return p + q/q; } assert("inc(4) must give 5", inc(4) == 5); Does not catch the missing code sequence http://ariya.ofilabs.com/2012/09/the-hidden-trap-of-code-coverage.html
  23. 48.

    Workaround: Explicit Path function inc(p, q) { if (q ==

    undefined) return p + 1; return p + q/q; } assert("inc(4) must give 5", inc(4) == 5);
  24. 49.

    Branch Coverage http://ariya.ofilabs.com/2012/12/javascript-code-coverage-with-istanbul.html function inc(p, q) { if (q ==

    undefined) q = 1; return p + q/q; } assert("inc(4) must give 5", inc(4) == 5); E = Else is not taken
  25. 52.

    Code Coverage for Jasmine Tests Specs Karma Istanbul http://ariya.ofilabs.com/2013/10/code-coverage-of-jasmine-tests-using-istanbul-and-karma.html INFO

    [karma]: Karma v0.10.2 server started at http://localhost:9876/ INFO [launcher]: Starting browser PhantomJS INFO [PhantomJS 1.9.2 (Linux)]: Connected on socket N9nDnhJ0Np92NTSPGx-X PhantomJS 1.9.2 (Linux): Executed 1 of 1 SUCCESS (0.029 secs / 0.003 secs)
  26. 56.

    http://ariya.ofilabs.com/2012/12/javascript-performance-analysis-sampling-tracing-and-timing.html Timing Prepare the stopwatch Mark the start and the

    end Sampling Periodically check which function is being executed Tracing Track all function calls and exits
  27. 58.

    What you write Optimized by the engine function test() {

    var a = 0, b = 0; for (var i = 0; i < 100; i++) { a += i; b += i * i; } return a; } function test() { var a = 0; for (var i = 0; i < 100; i++) { a += i; } return a; } Dead Code Elimination
  28. 59.

    Fast = Enough? Alice Bob Chuck Dan ... Bob Alice

    Dan Chuck ... Address Book Application Sort How’s the speed? 2 ms to sort 10 contacts
  29. 60.

    Array.prototype.swap = function (i, j) { var k = this[i];

    this[i] = this[j]; this[j] = k; } function sort(list) { var items = list.slice(0), swapped = false, p, q; for (p = 1; p < items.length; ++p) { for (q = 0; q < items.length - p; ++q) { if (items[q + 1] < items[q]) { items.swap(q, q + 1); swapped =true; } } if (!swapped) break; } return items; } Bubble Sort ???
  30. 61.

    Profile-Guide Optimization function isDigit(ch) { return '0123456789'.indexOf(ch) >= 0; }

    function isDigit(ch) { return ch !==' ' && '0123456789'.indexOf(ch) >= 0; } Letters Digits Spaces http://ariya.ofilabs.com/2013/07/profile-guided-javascript-optimization.html
  31. 62.
  32. 63.

    Object in a Set http://ariya.ofilabs.com/2012/08/determining-objects-in-a-set-examples-in-javascript.html var valid_words = { 'foobar':

    true, 'bar': true, 'baz': true, 'quux': true }; function is_valid(word) { return valid_words.hasOwnProperty(word); } is_valid('fox'); // false Dictionary Spell checker Alternatives: array lookup, switch case, prefix/suffix tree, perfect hash
  33. 64.

    Tiered Conditionals function is_valid(word) { switch (word.length) { case 3:

    return word === 'bar' || word === 'baz'; case 4: return word === 'quux'; case 6: return word === 'foobar'; } return false; } Filter #1: Length http://ariya.ofilabs.com/2012/08/determining-objects-in-a-set-examples-in-javascript.html
  34. 65.

    Empirical Run-time Complexity Array.prototype.swap = function (i, j) { var

    k = this[i]; this[i] = this[j]; this[j] = k; } Array.prototype.swap = function (i, j) { Log({ name: 'Array.prototype.swap', lineNumber: 1, range: [23, 94] }); var k = this[i]; this[i] = this[j]; this[j] = k; } http://esprima.org/demo/functiontrace.html Try online!
  35. 66.

    Run-time Analysis 0 250000 500000 0 500 1000 Calls to

    swap() Input Size http://ariya.ofilabs.com/2012/01/scalable-web-apps-the-complexity-issue.html
  36. 67.

    Execution Tracing http://ariya.ofilabs.com/2012/02/tracking-javascript-execution-during-startup.html https://gist.github.com/1823129 jQuery Mobile startup log 4640 function

    calls jquery.js 26 jQuery jquery.js 103 init undefined, undefined, [object Object] jquery.js 274 each (Function) jquery.js 631 each [object Object], (Function), undefined jquery.js 495 isFunction [object Object] jquery.js 512 type [object Object] jquery.mobile.js 1857 [Anonymous] jquery.mobile.js 642 [Anonymous] jquery.mobile.js 624 enableMouseBindings jquery.mobile.js 620 disableTouchBindings
  37. 69.

    Source Transformation Parser Code Generator Source Syntax Tree Source In-place

    Modification Modified Source Regenerative Non-Destructive http://ariya.ofilabs.com/2013/06/javascript-source-transformation-non-destructive-vs-regenerative.html
  38. 70.

    String Literal Quotes [ { type: "Identifier", value: "console", range:

    [0, 7] }, { type: "Punctuator", value: ".", range: [7, 8] }, { type: "Identifier", value: "log", range: [8, 11] }, { type: "Punctuator", value: "(", range: [11, 12] }, { type: "String", value: "\"Hello\"", range: [12, 19] }, { type: "Punctuator", value: ")", range: [19, 19] } ] console.log('Hello') console.log("Hello") List of tokens http://ariya.ofilabs.com/2012/02/from-double-quotes-to-single-quotes.html
  39. 71.

    Style Formatter CodePainter Source Sample code Formatted Infer coding styles

    Indentation Quote for string literal Whitespace esformatter Source Formatted Style options Indentation Line breaks Whitespaces https://github.com/millermedeiros/esformatter
  40. 73.

    function f() { var j = data.length; console.log(j, 'items'); for

    (var i = 0; i < j; ++i) { var j$0 = data[i] * data[i]; console.log(j$0); // squares } } Lexical Block Scope function f() { let j = data.length; console.log(j, 'items'); for (let i = 0; i < j; ++i) { let j = data[i] * data[i]; console.log(j); // squares } } https://github.com/olov/defs New in ECMAScript 6 http://ariya.ofilabs.com/2013/05/es6-and-block-scope.html
  41. 76.

    Adaptive Tools Explicit Implicit Customize analysis options Define new sets

    of rules Infer from high-quality sample Observe the engineer’s behavior