Building JavaScript Tools

A beginner's look at building JavaScript static code analysis tools

Transcript

1. Building JavaScript Tools Greg Franko 2/12/2014 http://gregfranko.com/building-javascript-tools-talk/
2. About The Speaker Greg Franko JavaScript Engineer at AddThis Open Source Developer Author Speaker
3. What Types of Tools Are We Talking About?
4. Static Code Analysis
5. What Do These Tools Typically Do?
6. Analyze Code Performance, Maintainability, Validation
7. Generate Code For Transpilers, Minifiers, Visualizers
8. Could You Provide Some Examples?
9. UglifyJS Compressor/Minifier That Reduces The Size Of Your JS Code
10. JSHint Helps Detect Problems In Your JS Code Before It Is Too Late
11. CSS Lint Helps Detect Problems In Your CSS Code Before It Is Too Late
12. CoffeeScript Language That Compiles To JavaScript
13. Plato Code Complexity Visualizations
14. RequireJS Optimizer Concatenates AMD Projects Into A Single File
15. AMDClean Converts AMD code To Standard JavaScript
16. Browserify Concatenates CommonJS Projects Into A Single File For The Browser
17. Have You Used A Static Code Analysis Tool? You Probably Have
18. Have You Written A Static Code Analysis Tool? You Probably Haven't
19. Why Not? I have a theory...
20. You Don't Know Where To Start
21. Let's Change That
22. Static Code Analysis Basics Let's Learn
23. It Starts With A String Of Code var code = 'function jqcon() {}';
24. That Is Converted To An Object { "type": "FunctionDeclaration", "id": { "type": "Identifier", "name": "jqcon" }, "params": [], "body": { "type": "BlockStatement", "body": [] }, "expression": false } This Object Is Called an Abstract Syntax Tree (AST)
25. Abstract Syntax Tree An Object (Tree) That Represents The Structure Of Your Code Does Not Store Everything - It is abstract e.g. Does not store that a JS function starts/ends with open/close curly brackets Can Be Traversed and Updated
26. Common AST Questions When Would I Ever Use An AST? Whenever you want to analyze your code Couldn't I Just Use Regular Expressions? Pattern matching will only get you so far. It is also notoriously ugly to read. How Do I Generate An AST? By Using A Parser Library (or writing your own parser)
27. Let's Generate an AST!
28. Esprima JavaScript Parser That Generates ASTs Can Be Used In A Node.js Or Web Environment Adheres To The Mozilla SpiderMonkey Parser API Created by Ariya Hidayat
29. Generating An AST With Esprima // Node.js Environment var code = 'function jqcon() {}', esprima = require('esprima'), ast = esprima.parse(code); That's All There Is To Creating An AST
30. Awesome, It Works! ASTs Are So Cool But I Still Have A Few Questions About Esprima...
31. Common Esprima Questions Do I Have To Use Esprima? No, there are alternatives out there. A popular alternative is Acorn Why Does Esprima Adhere To The Mozilla SpiderMonkey Parser API? Because the SpiderMonkey Parse API is recognized by the community as a standard for structured JS representation (and many other tools adhere to it as well)
32. What Now? Now That We Have Generated Our AST... Let's Traverse and Update It!
33. Estraverse JavaScript Library That Provides AST Traversal and Update Methods Can Be Used In A Node.js Or Web Environment Adheres To The Mozilla SpiderMonkey Parser API Created by Yusuke Suzuki
34. Traversing An AST With Estraverse // Node.js Environment var code = 'function jqcon() {}', esprima = require('esprima'), ast = esprima.parse(code), estraverse = require('estraverse'); estraverse.traverse(ast, { enter: function (node, parent) {}, leave: function(node, parent) {} });
35. Traversing And Updating An AST With Estraverse estraverse.traverse(ast, { enter: function (node, parent) { if(node.type === 'Identifier' && node.name === 'jqcon') { // Changes the 'jqcon' function name to 'jqcon_is_awesome' node.name = node.name + '_is_awesome'; } }, });
36. You Could Also Use the Estraverse.replace() Method...
37. Traversing And Updating An AST With Estraverse estraverse.replace(ast, { enter: function (node, parent) { if(node.type === 'Identifier' && node.name === 'jqcon') { // Changes the 'jqcon' function name to 'jqcon_is_awesome' return { 'type': 'Identifier', 'name': 'jqcon_is_awesome' }; } } });
38. Awesome, It Works! Traversing and Updating ASTs Is Fun But I Still Have A Few Questions About Estraverse...
39. Common Estraverse Questions Do I Have To Use Estraverse? No, there are alternatives out there. A popular alternative is esquery But Couldn't I Traverse And Update The AST Myself? You could, but traversing ASTs is tricky since not all AST child nodes have a unified interface. e.g. VariableDeclaration has nested children nodes within a declarations property, while an IfStatement has nested children within a consequent property.
40. Let's Review Use Esprima To Generate An AST From Code Use Estraverse To Traverse And Update That AST
41. What Now? Now That We Have Generated, Traversed, and Updated Our AST... Let's Generate Code!
42. Escodegen JavaScript Library That Generates Code From An AST Can Be Used In A Node.js Or Web Environment Adheres To The Mozilla SpiderMonkey Parser API Created by Yusuke Suzuki
43. Generating Code From An AST With Escodegen // Node.js Environment var code = 'function jqcon() {}', esprima = require('esprima'), ast = esprima.parse(code), escodegen = require('escodegen'), // Returns a string of code represented by the AST regenerated_code = escodegen.parse(ast); View On JSBin
44. Amazing! Generating Code From An AST Is Hot But I Still Have A Few Questions About Escodegen...
45. Common Escodegen Questions Do I Have To Use Escodegen? Good question. I'm not sure of a good alternative. Anyone in the audience know? But Couldn't I Write an AST Code Generator Myself? You could. I'll see you next year, when you're finished.
46. Let's Review Use Esprima To Generate An AST From Code Use Estraverse To Traverse And Update That AST Use Escodegen To Generate Code From That Ast
47. Wrap Up Questions Your Examples Are Kind Of Simple. Got Anything More Complex? Yep, check out the AMDClean website and source You Didn't Cover CSS Static Analysis. What Gives? Not enough time my friend. But I created an example that you can look at later. Are There Any More Resources You Would Recommend? Yes, the Esprima Parser Demo and the Mozilla SpiderMonkey Parser API will be your best friends. You current best friends will understand.
48. Thanks! Twitter: GregFranko Github: gfranko