SinnerSchrader Tech Session '18 - What the AST?

Slide 1

Slide 1 text

AST? Dominik Kundel | @dkundel | #s2techsession | #whatTheAST What the Photo by Paweł Czerwiński on Unsplash

Slide 2

Slide 2 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST

Slide 3

Slide 3 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Have you used these tools?

Slide 4

Slide 4 text

AST? Dominik Kundel | @dkundel | #s2techsession | #whatTheAST What the Photo by Paweł Czerwiński on Unsplash

Slide 5

Slide 5 text

Dominik Kundel | @dkundel | #s2techsession | console.log(` Hi! I’m Dominik Kundel `); dkundel.com @dkundel dkundel@twilio.com github/dkundel Developer Evangelist && JavaScript Hacker #whatTheAST

Slide 6

Slide 6 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST OnesieJS

Slide 7

Slide 7 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST How do we ﬁnd / modify code?

Slide 8

Slide 8 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Ctrl/Cmd + F

Slide 9

Slide 9 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST grep / sed

Slide 10

Slide 10 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST /RegularExpressions/gi

Slide 11

Slide 11 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Unreliable & Hard to maintain

Slide 12

Slide 12 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Let’s talk about those tools again Linters / Code Formatters Frameworks Bundlers Compilers / Transpilers

Slide 13

Slide 13 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Code > ✨ > Output Linters / Code Formatters Frameworks Bundlers Compilers / Transpilers

Slide 14

Slide 14 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST What’s that Magic? ✨

Slide 15

Slide 15 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Picture by @_lrlna

Slide 16

Slide 16 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Lexical Analysis Tokenization

Slide 17

Slide 17 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST console.log('Hello');

Slide 18

Slide 18 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST isPanda('');

Slide 19

Slide 19 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST const { tokenize } = require('esprima'); const tokens = tokenize("isPanda('')");

Slide 20

Slide 20 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST [ { type: 'Identifier', value: 'isPanda' }, { type: 'Punctuator', value: '(' }, { type: 'String', value: "''" }, { type: 'Punctuator', value: ')' }, ];

Slide 21

Slide 21 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Practical Examples Lexical Analysis Practical Examples

Slide 22

Slide 22 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Syntax Highlighting

Slide 23

Slide 23 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST https://code.visualstudio.com/blogs/2017/02/08/syntax-highlighting-optimizations

Slide 24

Slide 24 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Abstract Syntax Trees ASTs

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST { type: 'Program', body: [ { type: 'ExpressionStatement', expression: { type: 'CallExpression', callee: { type: 'Identifier', name: 'isPanda' }, arguments: [{ type: 'Literal', value: '', raw: "''" }], }, }, ], sourceType: 'script', }

Slide 27

Slide 27 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST https://github.com/estree/estree ESTree Spec

Slide 28

Slide 28 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Oﬃcial ESTree speciﬁcation ESTree Spec Alterations Esprima Syntax Tree Format Babel AST Format Facebook JSX Extension

Slide 29

Slide 29 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Practical Examples ASTs Practical Examples

Slide 30

Slide 30 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Linting

Slide 31

Slide 31 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Code Analysis angular.io/guide/language-service

Slide 32

Slide 32 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Bundling

Slide 33

Slide 33 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Modifying ASTs

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST { type: 'Program', body: [ { type: 'ExpressionStatement', expression: { type: 'Literal', value: true, raw: 'true' }, }, ], sourceType: 'script', }

Slide 36

Slide 36 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Practical Examples Modifying ASTs Practical Examples

Slide 37

Slide 37 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Code Modiﬁcations

Slide 38

Slide 38 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Language Extensions

Slide 39

Slide 39 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Language Transpiling

Slide 40

Slide 40 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Code Coverage https://istanbul.js.org/

Slide 41

Slide 41 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST nyc instrument sum.js ./output

Slide 42

Slide 42 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST sum.js function sum(a, b) { return a + b; } module.exports = sum;

Slide 43

Slide 43 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST ./output/sum.js var cov_2mh38x6lsj = (function () { /* ... */ })(); function sum(a, b) { cov_2mh38x6lsj.f[0] ++; cov_2mh38x6lsj.s[0] ++; return a + b; } cov_2mh38x6lsj.s[1] ++; module.exports = sum;

Slide 44

Slide 44 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST var cov_2mh38x6lsj = (function() { var path = 'sum.js', hash = 'e40ca58a296221aa5a52fed6ee0f25cc89e6447b', Function = function() {}.constructor, global = new Function('return this')(), gcv = ' __coverage __', coverageData = { /* ... */ }, coverage = global[gcv] || (global[gcv] = {}); if (coverage[path] && coverage[path].hash === hash) { return coverage[path]; } coverageData.hash = hash; return (coverage[path] = coverageData); })();

Slide 45

Slide 45 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST coverageData = { path: 'sum.js', statementMap: { '0': { start: { line: 2, column: 2 }, end: { … } }, '1': { start: { line: 4, column: 0 }, end: { … } }, }, fnMap: { '0': { name: 'sum', decl: { start: { … }, end: { … } }, loc: { start: { … }, end: { … } }, line: 1, }, }, branchMap: {}, s: { '0': 0, '1': 0 }, f: { '0': 0 }, b: {}, _coverageSchema: 'd34fc3e6b8297bcde183f5492bcb8fcb36775295', },

Slide 46

Slide 46 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Understanding Code Coverage engineering.semantics3.com/understanding-code-coverage-1074e8fccce0 More Info

Slide 47

Slide 47 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Running other languages in the Browser

Slide 48

Slide 48 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST codecombat.com

Slide 49

Slide 49 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST esper.js.org

Slide 50

Slide 50 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Rendering ASTs Rendering ASTs

Slide 51

Slide 51 text

Slide 52

Slide 52 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST const escodegen = require('escodegen'); escodegen.generate({ type: 'Program', body: [ { type: 'ExpressionStatement', expression: { type: 'Literal', value: true, raw: 'true' }, }, ], sourceType: 'script', });

Slide 53

Slide 53 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST true;

Slide 54

Slide 54 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Practical Examples Rendering ASTs Rendering ASTs Practical Examples

Slide 55

Slide 55 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Code Formatting

Slide 56

Slide 56 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Miniﬁcation

Slide 57

Slide 57 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Why would I care?

Slide 58

Slide 58 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Babel Transform Plugins

Slide 59

Slide 59 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST grep / sed

Slide 60

Slide 60 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST /RegularExpressions/gi

Slide 61

Slide 61 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST /RegularExpressions/gi ( )

Slide 62

Slide 62 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Building a Babel Plugin Live Demo

Slide 63

Slide 63 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Summary Photo by Jan Kahánek on Unsplash

Slide 64

Slide 64 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST 1. You use Tokens & ASTs daily

Slide 65

Slide 65 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST 2. Tokens represent format

Slide 66

Slide 66 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST 3. ASTs represent structure

Slide 67

Slide 67 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST 4. ASTs are “easy” to manipulate

Slide 68

Slide 68 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST 5. AST for safer code alterations

Slide 69

Slide 69 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST 6. ESTree for interoperability

Slide 70

Slide 70 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST 7. Don’t reinvent the wheel

Slide 71

Slide 71 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Libraries to create ASTs • @babel/parser (aka. babylon) • espree (ESLint’s parser) • typescript-estree (Used by ESLint & prettier) • flow-parser • esprima • uglify-js • acorn (the foundation of a lot of parsers)

Slide 72

Slide 72 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Libraries to walk ASTs • @babel/traverse • burrito, falafel • rocambole • babylon-walk (lightweight alternative to @babel/ traverse) • acorn-walk • jscodeshift

Slide 73

Slide 73 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Libraries to render ASTs • @babel/generator • escodegen • astring

Slide 74

Slide 74 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST 8. Replace grep & sed

Slide 75

Slide 75 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST astexplorer.net

Slide 76

Slide 76 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST git.io/babel-handbook

Slide 77

Slide 77 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Everything Open Source

Slide 78

Slide 78 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST hacktoberfest.digitalocean.com

Slide 79

Slide 79 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST www.twilio.com/quest

Slide 80

Slide 80 text

Dominik Kundel | @dkundel | #s2techsession | #whatTheAST Stickers!!

Slide 81

Slide 81 text

console.log(` Thank You! `); dkundel.com @dkundel dkundel@twilio.com github/dkundel d-k.im/s2-ast Dominik Kundel | @dkundel | #s2techsession | #whatTheAST