Dynamic Code Analysis for JavaScript

Dynamic Code Analysis for JavaScript

Presented at jQuery Conference 2014 in San Diego: http://events.jquery.org/2014/san-diego/.

These days, publishing a project without a comprehensive test suite is frown upon. However, the tests themselves do not always tell the full story. We need to level up and provide a more confident level of dynamic code testing. In this talk, two types of such analysis will be covered: code coverage and run-time complexity profiling. Code coverage is instrumental in deciding the quality of the existing sets of unit tests. An important aspect of coverage testing, branch coverage, plays an important role in discovering possible latent bugs due to untested code flow. In addition to code coverage, complexity profiling permits selective instrumentation of a particular code block for the purpose of performance analysis. More than just measuring elapsed time, run-time profiling also needs to reveal the algorithmic complexity which can be tracked via this empirical approach.

Additional details: http://ariya.ofilabs.com/2014/02/the-majestic-jquery-conference-san-diego-2014.html.

0284b8950e0f4a57bcc092d4dbb98d97?s=128

Ariya Hidayat

February 12, 2014
Tweet

Transcript

  1. Dynamic Code Analysis for JavaScript @ariyahidayat Feb 12, 2014

  2. /usr/bin/whoami shapesecurity.com

  3. “Software Provocateur” PhantomJS Esprima

  4. None
  5. “YOU SHALL NOT PASS!” — Darth Vader

  6. Utility Belt

  7. Parser Code Coverage Code Generator Function Instrumentation +

  8. Syntax Parser var answer = 42 keyword equal sign identifier

    number Variable Declaration Identifier Literal Constant Tokenization → Tokens Parsing → Syntax Tree
  9. { type: "Program", body: [ { type: "VariableDeclaration", declarations: [

    { type: "VariableDeclarator", id: { type: "Identifier", name: "answer" }, init: { type: "Literal", value: 42, raw: "42" } } ], kind: "var" } ] } Syntax Tree var answer = 42; Terms → ECMAScript 5.1 Specification https://developer.mozilla.org/en-US/docs/SpiderMonkey/Parser_API
  10. Syntax Visualization http://esprima.org/demo/parse.html

  11. None
  12. Source Transformation Parser Code Generator In-place Modification Non-Destructive Regenerative http://ariya.ofilabs.com/2013/06/javascript-source-transformation-non-destructive-vs-regenerative.html

  13. Code Coverage

  14. Regenerative Transformation Parser Code Generator http://ariya.ofilabs.com/2013/06/javascript-source-transformation-non-destructive-vs-regenerative.html

  15. Istanbul

  16. Code Instrumentation var answer = 42; alert(answer); var __cov_l$4m7m$L464yvav5F$qhNA =

    __coverage__['hello.js']; __cov_l$4m7m$L464yvav5F$qhNA.s['1']++; var answer = 42; __cov_l$4m7m$L464yvav5F$qhNA.s['2']++; alert(answer);
  17. Statement/Expression Coverage http://ariya.ofilabs.com/2012/12/javascript-code-coverage-with-istanbul.html function fibo(n) { return (n < 2)

    ? n : fibo(n-1) + fibo(n-2); } assert("fibo(1) must give 1", fibo(1) == 1);
  18. Branch Coverage function inc(p, q) { if (q == undefined)

    q = 1; return p + q/q; } assert("inc(4) must give 5", inc(4) == 5); http://ariya.ofilabs.com/2012/12/javascript-code-coverage-with-istanbul.html
  19. http://ariya.ofilabs.com/2012/09/the-hidden-trap-of-code-coverage.html Does not catch the missing code sequence

  20. None
  21. Profile-Guided Optimization http://ariya.ofilabs.com/2013/07/profile-guided-javascript-optimization.html function isDigit(ch) { return '0123456789'.indexOf(ch) >= 0;

    } function isDigit(ch) { return (ch !== ' ') && '0123456789'.indexOf(ch) >= 0; }
  22. Istanbul, QUnit, Karma http://ariya.ofilabs.com/2013/10/code-coverage-of-qunit-tests-using-istanbul-and-karma.html test("sqrt", function() { deepEqual(My.sqrt(4), 2, "square

    root of 4 is 2"); });
  23. “If you think JSLint hurts your feelings, wait until you

    use Istanbul.” @davglass
  24. ...I gave you my heart But the very next day

    you gave it away...
  25. Function Instrumentation

  26. Non-Destructive Transformation Parser In-place Modification http://ariya.ofilabs.com/2013/06/javascript-source-transformation-non-destructive-vs-regenerative.html

  27. String Literal Quotes http://ariya.ofilabs.com/2012/02/from-double-quotes-to-single-quotes.html console.log('Hello') console.log("Hello")

  28. Timing Prepare the stopwatch Mark the start and the end

    Sampling Periodically check which function is being executed Tracing Track all function calls and exits http://ariya.ofilabs.com/2012/12/javascript-performance-analysis-sampling-tracing-and-timing.html
  29. Fast = Enough? Alice Bob Chuck Dan ... Bob Alice

    Dan Chuck ... Sort How’s the speed? 2 ms to sort 10 contacts
  30. Array.prototype.swap = function (i, j) { var k = this[i];

    this[i] = this[j]; this[j] = k; } function sort(list) { var items = list.slice(0), swapped = false, p, q; for (p = 1; p < items.length; ++p) { for (q = 0; q < items.length - p; ++q) { if (items[q + 1] < items[q]) { items.swap(q, q + 1); swapped =true; } } if (!swapped) break; } return items; } Bubble Sort ???
  31. Run-time Complexity http://ariya.ofilabs.com/2012/01/scalable-web-apps-the-complexity-issue.html Array.prototype.swap = function (i, j) { var

    k = this[i]; this[i] = this[j]; this[j] = k; } Array.prototype.swap = function (i, j) { Log({ name: 'Array.prototype.swap', lineNumber: 1, range: [23, 94] }); var k = this[i]; this[i] = this[j]; this[j] = k; } http://esprima.org/demo/functiontrace.html
  32. None
  33. “First Things First”, Steven Covey “The Big Rock” http://calendar.perfplanet.com/2013/javascript-performance-big-picture/

  34. Layers of Defense

  35. Being Defensive http://ariya.ofilabs.com/2012/12/quality-code-via-multiple-layers-of-defense.html

  36. Being Objective Mr. Reviewer Your code sucks! Your code sucks!

  37. Tests Madness T D D B D D

  38. Coverage Thresholds http://ariya.ofilabs.com/2013/05/hard-thresholds-on-javascript-code-coverage.html istanbul check-coverage --statement -5 --branch -3 --function

    100
  39. Tools separate us

  40. Final Words Dynamic code analysis for quality: • Code coverage

    (statement, branch,...) • Function instrumentation Build, write, tweak code analysis tools
  41. Thank You @ariyahidayat ariya.ofilabs.com speakerdeck.com/ariya Some artworks are from http://openclipart.org.