Improving JavaScript Code Quality: Strategies and Tools

Presented at O'Reilly Fluent 2013, San Francisco.

The quality of JavaScript-based web applications can be improved by choosing the development strategies which minimize common mistakes, avoid API ambiguities, and reduce any syntax confusion. Applying these approaches systematically as part of the development strategies also requires the use of advanced code quality tools. This talk highlights the recent developments on language tools which aim at solving near-future JavaScript quality analysis, everything from run-time complexity profiling to framework-aware static code inspection.

Transcript

1. Improving JavaScript
   Code Quality:
   Strategies and Tools
   Ariya Hidayat
   @ariyahidayat
   1
   

   View Slide

2. whoami
   2
   

   View Slide

3. Multi-Layer Defense
   3
   

   View Slide

4. Defensive = Paranoid
   http://ariya.ofilabs.com/2012/12/quality-code-via-multiple-layers-of-defense.html
   4
   

   View Slide

5. Placement of Layers
   Code Editor
   CI Server
   VCS Hooks
   Smoke Tests
   Pull Request
   Build Tasks
   5
   

   View Slide

6. Feedback is Important
   Engineer
   Tools
   Feedback Cycle
   Boring
   Repetitive
   Time-consuming
   6
   

   View Slide

7. Track Quality Metrics
   Application revision
   Execution time
   Baseline
   7
   

   View Slide

8. Metrics Quadrant
   http://ariya.ofilabs.com/2012/11/optimization-journey-vs-destination.html
   8
   

   View Slide

9. Be Reasonable
   “First Things First”, Steven Covey
   “The Big Rock”
   9
   

   View Slide

10. Foundation
    10
    

    View Slide

11. JavaScript in the Browser
    User Interface
    Browser Engine
    Graphics
    Stack
    Data Persistence
    Render Engine
    JavaScript
    Engine
    Networking I/O
    11
    

    View Slide

12. JavaScript Engines
    SpiderMonkey Firefox
    JavaScriptCore/Nitro Safari
    V8 Node.js, Chrome
    JScript/Chakra Internet Explorer
    Carakan Opera
    12
    

    View Slide

13. Engine Building Blocks
    Virtual
    Machine/
    Interpreter
    Parser
    Runtime
    Source
    Syntax Tree
    Built-in objects,
    host objects, ...
    Fast and conservative
    13
    

    View Slide

14. Parser
    var answer = 42
    keyword equal sign
    identifier number
    Variable Declaration
    Identifier
    Literal Constant
    Tokenization →
    Tokens
    Parsing → Syntax Tree
    14
    

    View Slide

15. Syntax Visualization http://esprima.org/demo/parse.html
    Try online!
    http://ariya.ofilabs.com/2012/04/javascript-syntax-tree-visualization-with-esprima.html
    15
    

    View Slide

16. Execution Visualization
    http://int3.github.io/metajs/
    16
    

    View Slide

17. Live Coding/Editing
    http://nornagon.github.com/scrubby/
    17
    

    View Slide

18. Composition: Chains of Responsibilities
    https://speakerdeck.com/constellation/escodegen-and-esmangle-using-mozilla-javascript-ast-as-an-ir
    Parser
    Code
    Generator
    Source
    Syntax Tree
    Source
    Obvious examples: minifier, obfuscator, ...
    18
    

    View Slide

19. Quasi-Endless Possibilities
    Static analysis
    Inspection
    Dynamic analysis
    Transformation
    19
    

    View Slide

20. Standard, Off-the-Shelf Tools
    20
    

    View Slide

21. Syntax Validation
    http://ariya.ofilabs.com/2012/10/javascript-validator-with-esprima.html
    http://esprima.org/demo/validate.html
    Try online!
    Also available:
    Grunt plugin, Ant task, ...
    21
    

    View Slide

22. Strict Mode Validator
    'use strict';
    block = {
    color: 'blue',
    height: 20,
    width: 10,
    color: 'red'
    };
    Duplicate data property in object
    literal not allowed in strict mode
    http://ariya.ofilabs.com/2012/10/validating-strict-mode.html
    22
    

    View Slide

23. Linter vs Validator
    Validator
    Linter
    Looks for specification violations
    Does not care about coding style
    Works well on generated/minified code
    Searches for suspicious pattern
    Warns on style inconsistencies
    Works well on hand-written code
    23
    

    View Slide

24. Polluting Variables
    var height;
    // some fancy processing
    heigth = 200;
    Leaks to
    global
    test.js:3
    heigth = 200;
    ^
    LeakError: global leak
    detected: heigth
    https://github.com/kesla/node-leaky
    http://ariya.ofilabs.com/2012/11/polluting-and-unused-javascript-variables.html
    24
    

    View Slide

25. Unused Variables
    var height;
    // some fancy processing
    heigth = 200;
    http://ariya.ofilabs.com/2012/11/polluting-and-unused-javascript-variables.html
    Declared
    but not used
    test.js
    height - on line 1
    https://github.com/Kami/node-unused
    25
    

    View Slide

26. Protection Layers, Again
    JUnit XML + Jenkins
    files=$(git diff-index --name-only HEAD
    | grep -P '\.js$')
    for file in $files; do
    esvalidate $file
    if [ $? -eq 1 ]; then
    echo "Syntax error: $file"
    exit 1
    fi
    done
    Git Precommit Hook
    http://ariya.ofilabs.com/2012/03/git-pre-commit-hook-and-smoke-testing.html
    26
    

    View Slide

27. View Slide

28. Code Complexity
    if (true) "foo"; else "bar";
    ◦ Maintainability index: 139.99732896539635
    ◦ Physical LOC: 1
    ◦ Logical LOC: 4
    ◦ Aggregate cyclomatic complexity: 2
    http://jscomplexity.org/
    28
    

    View Slide

29. Continuous Monitoring of Complexity
    http://ariya.ofilabs.com/2013/05/continuous-monitoring-of-javascript-code-complexity.html
    var cr = require('complexity-report'),
    content = require('fs').readFileSync('index.js', 'utf-8'),
    list = [];
    cr.run(content).functions.forEach(function (entry) {
    list.push({ name: entry.name, value: entry.complexity.cyclomatic });
    });
    list.sort(function (x, y) {
    return y.value - x.value;
    });
    console.log('Most cyclomatic-complex functions:');
    list.slice(0, 6).forEach(function (entry) {
    console.log(' ', entry.name, entry.value);
    });
    29
    

    View Slide

30. Complexity Visualization with Plato
    https://ariya.ofilabs.com/2013/01/javascript-code-complexity-visualization.html
    https://github.com/jsoverson/plato
    30
    

    View Slide

31. Editing Autocomplete
    http://ariya.ofilabs.com/2013/03/javascript-editing-with-autocomplete.html
    http://esprima.org/demo/autocomplete.html
    Try online!
    31
    

    View Slide

32. Scope and Highlight
    http://ariya.ofilabs.com/2013/04/javascript-variable-scope-and-highlight.html
    http://esprima.org/demo/highlight.html
    Try online!
    32
    

    View Slide

33. Scope Visualization http://mazurov.github.io/eslevels-demo/
    33
    

    View Slide

34. Rename Refactoring Assistant
    http://ariya.ofilabs.com/2013/04/rename-refactoring-for-javascript-code.html
    http://esprima.org/demo/rename.html
    Try online!
    34
    

    View Slide

35. Source Transformation
    Parser
    Code
    Generator
    Source
    Syntax Tree
    Source
    In-place
    Modification
    Modified Source
    Regenerative
    Non-Destructive
    35
    

    View Slide

36. Dynamic Code Coverage via Instrumentation
    var answer = 42;
    alert(answer);
    var __cov_l$4m7m$L464yvav5F$qhNA = __coverage__['hello.js'];
    __cov_l$4m7m$L464yvav5F$qhNA.s['1']++;
    var answer = 42;
    __cov_l$4m7m$L464yvav5F$qhNA.s['2']++;
    alert(answer);
    http://gotwarlost.github.io/istanbul/
    36
    

    View Slide

37. Statement Coverage and Latent Trap
    http://ariya.ofilabs.com/2012/09/the-hidden-trap-of-code-coverage.html
    function inc(p, q) {
    if (q == undefined) q = 1;
    return p + q/q;
    }
    assert("inc(4) must give 5", inc(4) == 5);
    function inc(p, q) {
    if (q == undefined)
    return p + 1;
    return p + q/q;
    }
    assert("inc(4) must give 5", inc(4) == 5);
    Does not catch the missing
    code sequence
    37
    

    View Slide

38. Branch Coverage
    http://ariya.ofilabs.com/2012/12/javascript-code-coverage-with-istanbul.html
    function inc(p, q) {
    if (q == undefined) q = 1;
    return p + q/q;
    }
    assert("inc(4) must give 5", inc(4) == 5);
    E = Else is not taken
    38
    

    View Slide

39. Hard Thresholds on Code Coverage
    http://ariya.ofilabs.com/2013/05/hard-thresholds-on-javascript-code-coverage.html
    istanbul check-coverage --statement -5 --branch -3 --function 100
    39
    

    View Slide

40. If you think JSLint hurts your feelings,
    wait until you use Istanbul.
    @davglass
    40
    

    View Slide

41. BYOT (Bring Your Own Tools)
    41
    

    View Slide

42. Consistency
    Convention
    Scalability
    42
    

    View Slide

43. String Literal Quotes
    [
    { type: "Identifier", value: "console", range: [0, 7] },
    { type: "Punctuator", value: ".", range: [7, 8] },
    { type: "Identifier", value: "log", range: [8, 11] },
    { type: "Punctuator", value: "(", range: [11, 12] },
    { type: "String", value: "\"Hello\"", range: [12, 19] },
    { type: "Punctuator", value: ")", range: [19, 19] }
    ]
    console.log('Hello')
    console.log("Hello")
    List of tokens
    http://ariya.ofilabs.com/2012/02/from-double-quotes-to-single-quotes.html
    43
    

    View Slide

44. Style Formatter
    CodePainter
    Source
    Sample code
    Formatted
    Infer
    coding
    styles
    Indentation
    Quote for string literal
    Whitespace
    esformatter
    Source Formatted
    Style options
    Indentation
    Line breaks
    Whitespaces
    https://github.com/millermedeiros/esformatter
    44
    

    View Slide

45. Application Structure
    MyApp.create('MyApp.Person', {
    name: 'Joe Sixpack',
    age: 42,
    constructor: function(name) {},
    walk: function(steps) {}
    run: function(steps) {}
    });
    {
    objectName: 'MyApp.Person',
    functions: ['walk', 'run'],
    properties: ['name', 'age']
    }
    Metadata
    45
    

    View Slide

46. Custom Linting
    var fs = require('fs'),
    esprima = require('./esprima'),
    files = process.argv.splice(2);
    files.forEach(function (filename) {
    var content = fs.readFileSync(filename, 'utf-8'),
    syntax = esprima.parse(content, { loc: true });
    JSON.stringify(syntax, function (key, value) {
    if (key === 'test' && value.operator === '==')
    console.log('Line', value.loc.start.line);
    return value;
    });
    });
    if (x == 9) {
    // do Something
    }
    Not a strict equal
    46
    

    View Slide

47. “Boolean Trap” Finder
    Can you make up your mind? treeItem.setState(true, false);
    event.initKeyEvent("keypress", true, true, null, null,
    false, false, false, false, 9, 0);
    The more the merrier?
    Obfuscated choice
    var volumeSlider = new Slider(false);
    Double-negative
    component.setHidden(false);
    filter.setCaseInsensitive(false);
    http://ariya.ofilabs.com/2012/06/detecting-boolean-traps-with-esprima.html
    47
    

    View Slide

48. Nested Ternary Conditionals
    var str = (age < 1) ? "baby" :
    (age < 5) ? "toddler" :
    (age < 18) ? "child": "adult";
    http://ariya.ofilabs.com/2012/10/detecting-nested-ternary-conditionals.html
    48
    

    View Slide

49. Performance: Measurement Confidence
    Accuracy
    Precision
    49
    

    View Slide

50. Fast = Enough?
    Alice
    Bob
    Chuck
    Dan
    ...
    Bob
    Alice
    Dan
    Chuck
    ...
    Address Book Application
    Sort
    How’s the speed?
    2 ms to sort 10 contacts
    50
    

    View Slide

51. Array.prototype.swap = function (i, j) {
    var k = this[i]; this[i] = this[j]; this[j] = k;
    }
    function sort(list) {
    var items = list.slice(0), swapped = false, p, q;
    for (p = 1; p < items.length; ++p) {
    for (q = 0; q < items.length - p; ++q) {
    if (items[q + 1] < items[q]) {
    items.swap(q, q + 1);
    swapped =true;
    }
    }
    if (!swapped) break;
    }
    return items;
    }
    Bubble Sort ???
    51
    

    View Slide

52. Empirical Run-time Complexity
    Array.prototype.swap = function (i, j) {
    var k = this[i]; this[i] = this[j]; this[j] = k;
    }
    Array.prototype.swap = function (i, j) {
    Log({ name: 'Array.prototype.swap', lineNumber: 1, range: [23, 94] });
    var k = this[i]; this[i] = this[j]; this[j] = k;
    }
    http://esprima.org/demo/functiontrace.html
    Try online!
    52
    

    View Slide

53. Run-time Analysis
    0
    250000
    500000
    0 500 1000
    Calls to swap()
    Input Size
    http://ariya.ofilabs.com/2012/01/scalable-web-apps-the-complexity-issue.html
    53
    

    View Slide

54. Execution Tracing
    http://ariya.ofilabs.com/2012/02/tracking-javascript-execution-during-startup.html
    https://gist.github.com/1823129
    jQuery Mobile startup log
    4640 function calls
    jquery.js 26 jQuery
    jquery.js 103 init undefined, undefined,
    [object Object]
    jquery.js 274 each (Function)
    jquery.js 631 each [object Object],
    (Function), undefined
    jquery.js 495 isFunction [object Object]
    jquery.js 512 type [object Object]
    jquery.mobile.js 1857 [Anonymous]
    jquery.mobile.js 642 [Anonymous]
    jquery.mobile.js 624 enableMouseBindings
    jquery.mobile.js 620 disableTouchBindings
    54
    

    View Slide

55. Embrace the Future
    55
    

    View Slide

56. Copy Paste (Mistake) Detector
    function inside(point, rect) {
    return (point.x >= rect.x1) && (point.y >= rect.y1) &&
    (point.x <= rect.x2) && (point.y <= rect.y1);
    }
    Wrong check
    56
    

    View Slide

57. Syntax Augmentation
    ES.Future:
    Polyfills, Transpiler, ...
    Exoskeleton:
    LLJS, Sweet.js, ...
    57
    

    View Slide

58. Adaptive Tools
    Explicit Implicit
    Customize analysis options
    Define new sets of rules
    Infer from high-quality sample
    Observe the engineer’s behavior
    58
    

    View Slide

59. Syntax Query
    if (x = 0) { /* do Something */ }
    IfStatement.test AssigmentExpression[operator='=']
    Which syntax family should be the model?
    CSS selector? XPath? SQL?
    59
    

    View Slide

60. And Many More...
    Semantic Diff
    Symbolic execution
    Informative syntax error
    Declarative transformation
    Pattern Matching
    60
    

    View Slide

61. Her five-year mission: to explore strange new worlds, to
    seek out new lifeforms and new civilizations;
    to boldly go where no one has gone before.
    61
    

    View Slide

62. Thank You
    ariya.hidayat@gmail.com
    @AriyaHidayat
    ariya.ofilabs.com/highlights
    speakerdeck.com/ariya
    Credits: Some artworks are from http://openclipart.org
    62
    

    View Slide