Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Master Tooling by Hacking the Angular Compiler

7764379521726735c164889159c8f387?s=47 Dominic Elm
December 06, 2019

Master Tooling by Hacking the Angular Compiler

Nowadays, we can hardly imagine the web without tooling. There are tools for code generation, transformations, and, most importantly, build tools. Build tools play an important role in modern web development. They make frameworks like Angular, React, or Vue more approachable and simplify the entire process from start to end. For example, a build tool manipulates our code in powerful ways; it downlevels our code, makes it smaller, and creates a production-grade bundle that is ready for deployment. But how does that work? To understand this, we will specifically look at Angular's build system to implement our own custom template syntax. Along the way, we explore tools such as Webpack, the Angular Compiler, and, most importantly, an Abstract Syntax Tree (AST). ASTs are the basic building blocks of most web tooling that empowers tools like Angular Schematics, Prettier, Babel, or TSLint to do their job. This talk should give you an insight into how Angular builds our applications, and act as an intro to Abstract Syntax Trees, how they work, and what we can do with them.

Special thanks to Kwinten Pisman (https://twitter.com/KwintenP) for helping me putting this talk together.

Links:
https://github.com/typebytes/ngx-template-streams
https://astexplorer.net/
https://github.com/phenomnomnominal/tsquery
https://github.com/urish/tsquery-playground

7764379521726735c164889159c8f387?s=128

Dominic Elm

December 06, 2019
Tweet

Transcript

  1. Master Tooling by Hacking the Angular Compiler by Dominic Elm

  2. None
  3. AST @elmd_

  4. Abstract Syntax Tree @elmd_

  5. @elmd_ NGULAR CLI

  6. @elmd_ NGULAR CLI Schematics

  7. @elmd_ ESLint NGULAR CLI Schematics

  8. @elmd_ ESLint NGULAR CLI Schematics

  9. @elmd_ ESLint NGULAR CLI Schematics

  10. @elmd_ AST ESLint NGULAR CLI Schematics

  11. None
  12. None
  13. @elmd_ Dominic Elm thoughtram https://thoughtram.io StackBlitz https://stackblitz.com Google Developer Expert

    https://bit.ly/2nAqRrZ
  14. Angular Checklist @elmd_

  15. @elmd_ @KwintenP Kwinten Pisman Thanks!

  16. ngx-template-streams? @elmd_

  17. Angular library that tries to embrace reactivity and supercharges templates

    with Observables. @elmd_ “
  18. <button (click)=“clicks$.next($event)”>Click Me</button> @Component({...}) export class AppComponent implements OnInit {

    clicks$ = new Subject(); ngOnInit() { // we can either manually subscribe // or use the async pipe this.clicks$.subscribe(console.log); } }
  19. <button (click)=“clicks$.next($event)”>Click Me</button> @Component({...}) export class AppComponent implements OnInit {

    clicks$ = new Subject(); ngOnInit() { // we can either manually subscribe // or use the async pipe this.clicks$.subscribe(console.log); } }
  20. <button (click)=“clicks$.next($event)”>Click Me</button> @Component({...}) export class AppComponent implements OnInit {

    clicks$ = new Subject(); ngOnInit() { // we can either manually subscribe // or use the async pipe this.clicks$.subscribe(console.log); } }
  21. None
  22. <button (click)=“clicks$.next($event)”>Click Me</button> @Component({...}) export class AppComponent implements OnInit {

    clicks$ = new Subject(); ngOnInit() { // we can either manually subscribe // or use the async pipe this.clicks$.subscribe(console.log); } }
  23. <button (*click)=“clicks$”>Click Me</button> @Component({...}) export class AppComponent implements OnInit {

    @ObservableEvent() clicks$: Observable<any>; ngOnInit() { // we can either manually subscribe // or use the async pipe this.clicks$.subscribe(console.log); } }
  24. <button (*click)=“clicks$”>Click Me</button> @Component({...}) export class AppComponent implements OnInit {

    @ObservableEvent() clicks$: Observable<any>; ngOnInit() { // we can either manually subscribe // or use the async pipe this.clicks$.subscribe(console.log); } }
  25. The Angular Build System @elmd_

  26. @elmd_ Angular CLI

  27. Webpack @elmd_

  28. Webpack @elmd_ Ahead of Time Compiler for the Browser

  29. Webpack @elmd_ Ahead of Time Compiler for the Browser Common

    Misconception: It’s not only for JavaScript ⚠
  30. Webpack *.bundle.js *.bundle.js main.bundle.js App @elmd_ Ahead of Time Compiler

    for the Browser Common Misconception: It’s not only for JavaScript ⚠
  31. Webpack @elmd_ webpack.config.js

  32. Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin AngularCompilerPlugin

    AngularCompilerPlugin @elmd_ webpack.config.js
  33. Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin AngularCompilerPlugin

    <emits> Angular Code AngularCompilerPlugin @elmd_ webpack.config.js
  34. Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin AngularCompilerPlugin

    <emits> Angular Code <waits for> AngularCompilerPlugin @elmd_ webpack.config.js
  35. Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin AngularCompilerPlugin

    <emits> Angular Code <waits for> Resolve Dependencies, Bundle AngularCompilerPlugin @elmd_ webpack.config.js
  36. Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin AngularCompilerPlugin

    <emits> Angular Code <waits for> Resolve Dependencies, Bundle Bundle AngularCompilerPlugin @elmd_ webpack.config.js
  37. webpack.config.js Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin

    AngularCompilerPlugin <emits> Generated Angular Code <waits for> Resolve Dependencies, Bundle AngularCompilerPlugin @elmd_ WebpackCompilerHost ts.CompilerHost
  38. webpack.config.js Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin

    AngularCompilerPlugin <emits> Generated Angular Code <waits for> Resolve Dependencies, Bundle AngularCompilerPlugin ngProgram AOT ts.Program JIT @elmd_ WebpackCompilerHost ts.CompilerHost
  39. webpack.config.js Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin

    AngularCompilerPlugin <emits> Generated Angular Code <waits for> Resolve Dependencies, Bundle AngularCompilerPlugin Analysis AOT ngProgram AOT ts.Program JIT @elmd_ WebpackCompilerHost ts.CompilerHost
  40. webpack.config.js Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin

    AngularCompilerPlugin <emits> Generated Angular Code <waits for> Resolve Dependencies, Bundle AngularCompilerPlugin Resolve AOT Analysis AOT ngProgram AOT ts.Program JIT @elmd_ WebpackCompilerHost ts.CompilerHost
  41. webpack.config.js Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin

    AngularCompilerPlugin <emits> Generated Angular Code <waits for> Resolve Dependencies, Bundle AngularCompilerPlugin Resolve AOT Analysis AOT ngProgram AOT ts.Program JIT @elmd_ WebpackCompilerHost ts.CompilerHost Type Checking AOT
  42. webpack.config.js Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin

    AngularCompilerPlugin <emits> Generated Angular Code <waits for> Resolve Dependencies, Bundle AngularCompilerPlugin Resolve AOT Analysis AOT ngProgram AOT ts.Program JIT Transformer @elmd_ WebpackCompilerHost ts.CompilerHost Type Checking AOT Emit
  43. webpack.config.js Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin

    AngularCompilerPlugin <emits> Generated Angular Code <waits for> Resolve Dependencies, Bundle AngularCompilerPlugin Angular Code Resolve AOT Analysis AOT ngProgram AOT ts.Program JIT Transformer @elmd_ WebpackCompilerHost ts.CompilerHost Type Checking AOT Emit
  44. Cool. But what’s next? @elmd_

  45. Loader .html @elmd_

  46. <button (click)=“"__clicks$.next()”> Angular rocks! "</button> Loader .html <button (*click)=“clicks$”> Angular

    rocks! "</button> @elmd_
  47. webpack.config.js Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin

    AngularCompilerPlugin <emits> Angular Code <waits for> Resolve Dependencies, Bundle Bundle AngularCompilerPlugin @elmd_
  48. webpack.config.js Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin

    AngularCompilerPlugin <emits> Angular Code <waits for> Resolve Dependencies, Bundle Bundle AngularCompilerPlugin HTML Loader Custom Loader @elmd_
  49. Builders @elmd_

  50. ngx-build-plus @elmd_ Thanks Manfred Steyer

  51. $ @elmd_

  52. $ ng add ngx-build-plus $ @elmd_

  53. $ ng add ngx-build-plus touch extra-webpack-config.js $ $ @elmd_

  54. module.exports = { module: { rules: [ { test: /\.html$/,

    use: [ { loader: path.resolve('path/to/custom/html/loader') }, { loader: 'raw-loader' "// import file as a string } ] } ] } }; @elmd_
  55. None
  56. $ ng add ngx-build-plus touch extra-webpack-config.js $ $ @elmd_

  57. $ ng add ngx-build-plus touch extra-webpack-config.js $ touch our-plugin.js $

    $ @elmd_
  58. export default { pre() { "// before build }, post()

    { "// after build }, config(config: webpack.Configuration) { "// after webpack config was built by the CLI } }; @elmd_
  59. export default { pre() { "// before build }, post()

    { "// after build }, config(config: webpack.Configuration) { const acp = findAngularCompilerPlugin(config) as AngularCompilerPlugin; if (!acp) { throw new Error('AngularCompilerPlugin not found'); } const options: AngularCompilerPluginOptions = { ""...acp.options, directTemplateLoading: false "// ""<-- IMPORTANT }; config.plugins = removeCompilerPlugin(config.plugins, acp); const newCompilerPlugin = new AngularCompilerPlugin(options); @elmd_
  60. export default { pre() { "// before build }, post()

    { "// after build }, config(config: webpack.Configuration) { const acp = findAngularCompilerPlugin(config) as AngularCompilerPlugin; if (!acp) { throw new Error('AngularCompilerPlugin not found'); } const options: AngularCompilerPluginOptions = { ""...acp.options, directTemplateLoading: false "// ""<-- IMPORTANT }; config.plugins = removeCompilerPlugin(config.plugins, acp); const newCompilerPlugin = new AngularCompilerPlugin(options); @elmd_
  61. if (!acp) { throw new Error('AngularCompilerPlugin not found'); } const

    options: AngularCompilerPluginOptions = { ""...acp.options, directTemplateLoading: false "// ""<-- IMPORTANT }; config.plugins = removeCompilerPlugin(config.plugins, acp); const newCompilerPlugin = new AngularCompilerPlugin(options); config.plugins.push(newCompilerPlugin); return config; } }; @elmd_
  62. if (!acp) { throw new Error('AngularCompilerPlugin not found'); } const

    options: AngularCompilerPluginOptions = { ""...acp.options, directTemplateLoading: false "// ""<-- IMPORTANT }; config.plugins = removeCompilerPlugin(config.plugins, acp); const newCompilerPlugin = new AngularCompilerPlugin(options); config.plugins.push(newCompilerPlugin); return config; } }; @elmd_
  63. if (!acp) { throw new Error('AngularCompilerPlugin not found'); } const

    options: AngularCompilerPluginOptions = { ""...acp.options, directTemplateLoading: false "// ""<-- IMPORTANT }; config.plugins = removeCompilerPlugin(config.plugins, acp); const newCompilerPlugin = new AngularCompilerPlugin(options); config.plugins.push(newCompilerPlugin); return config; } }; @elmd_
  64. if (!acp) { throw new Error('AngularCompilerPlugin not found'); } const

    options: AngularCompilerPluginOptions = { ""...acp.options, directTemplateLoading: false "// ""<-- IMPORTANT }; config.plugins = removeCompilerPlugin(config.plugins, acp); const newCompilerPlugin = new AngularCompilerPlugin(options); config.plugins.push(newCompilerPlugin); return config; } }; @elmd_
  65. if (!acp) { throw new Error('AngularCompilerPlugin not found'); } const

    options: AngularCompilerPluginOptions = { ""...acp.options, directTemplateLoading: false "// ""<-- IMPORTANT }; config.plugins = removeCompilerPlugin(config.plugins, acp); const newCompilerPlugin = new AngularCompilerPlugin(options); config.plugins.push(newCompilerPlugin); return config; } }; @elmd_
  66. Inline templates? @elmd_

  67. webpack.config.js Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin

    AngularCompilerPlugin <emits> Generated Angular Code <waits for> Resolve Dependencies, Bundle AngularCompilerPlugin Angular Code Resolve AOT Analysis AOT ngProgram AOT ts.Program JIT Transformer @elmd_ WebpackCompilerHost ts.CompilerHost Type Checking AOT Emit
  68. webpack.config.js Webpack Loader Loader Loader ngc <register> <register> Plugin Plugin

    AngularCompilerPlugin <emits> Generated Angular Code <waits for> Resolve Dependencies, Bundle AngularCompilerPlugin Angular Code Resolve AOT Analysis AOT ngProgram AOT ts.Program JIT Transformer @elmd_ WebpackCompilerHost ts.CompilerHost Type Checking AOT Emit Transformer Custom TS Transformer
  69. TypeScript Transformers @elmd_

  70. export function transformer(context: ts.TransformationContext) { return (sourceFile: ts.SourceFile) "=> {

    "// change sourceFile (AST) }; } @elmd_
  71. let a = 1; @elmd_ VariableStatement VariableDeclaration VariableDeclarationList NumericalLiteral text

    = “1” text = “a” Identifier SourceFile
  72. let a = 1; SourceFile VariableStatement VariableDeclaration VariableDeclarationList text =

    “a” Identifier @elmd_ NumericalLiteral text = “1”
  73. let a = 1; SourceFile VariableStatement VariableDeclaration VariableDeclarationList NumericalLiteral text

    = “1” @elmd_ text = “a” Identifier
  74. let a = 1; SourceFile VariableStatement VariableDeclarationList NumericalLiteral text =

    “1” @elmd_ text = “a” Identifier VariableDeclaration
  75. let a = 1; SourceFile VariableStatement VariableDeclarationList NumericalLiteral text =

    “1” @elmd_ text = “a” Identifier VariableDeclaration
  76. let a = 1; SourceFile VariableStatement VariableDeclarationList NumericalLiteral text =

    “1” @elmd_ text = “a” Identifier VariableDeclaration
  77. SourceFile VariableStatement VariableDeclarationList NumericalLiteral text = “1” @elmd_ text =

    “a” Identifier VariableDeclaration let a = 1;
  78. @elmd_

  79. export function transformer(context: ts.TransformationContext) { return (sourceFile: ts.SourceFile) "=> {

    /** * Todos: * 1. Find `template` property within @Component decorator * 2. Apply HTML transformation * 3. Update AST node * 4. Return updated SourceFile "*/ }; } @elmd_
  80. export function transformer(context: ts.TransformationContext) { return (sourceFile: ts.SourceFile) "=> {

    const template = findInlineTemplate(sourceFile); return updateInlineTemplate(template, sourceFile, context); }; } @elmd_
  81. export function transformer(context: ts.TransformationContext) { return (sourceFile: ts.SourceFile) "=> {

    const template = findInlineTemplate(sourceFile); return updateInlineTemplate(template, sourceFile, context); }; } function findInlineTemplate(sourceFile: ts.SourceFile) { return sourceFile.statements[1].decorators[0].expression .arguments[0].properties[0].initializer.text; } @elmd_
  82. export function transformer(context: ts.TransformationContext) { return (sourceFile: ts.SourceFile) "=> {

    const template = findInlineTemplate(sourceFile); return updateInlineTemplate(template, sourceFile, context); }; } function findInlineTemplate(sourceFile: ts.SourceFile) { return sourceFile.statements[1].decorators[0].expression .arguments[0].properties[0].initializer.text; } @elmd_
  83. TSQuery @elmd_ Thanks Craig Spence CSS-like Selectors for the TS

    AST
  84. @elmd_ export function transformer(context: ts.TransformationContext) { return (sourceFile: ts.SourceFile) "=>

    { const template = findInlineTemplate(sourceFile); return updateInlineTemplate(template, sourceFile, context); }; } function findInlineTemplate(sourceFile: ts.SourceFile) { return tsquery.query(sourceFile, INLINE_TEMPLATE_QUERY); }
  85. @elmd_ export function transformer(context: ts.TransformationContext) { return (sourceFile: ts.SourceFile) "=>

    { const template = findInlineTemplate(sourceFile); return updateInlineTemplate(template, sourceFile, context); }; }
  86. @elmd_ export function transformer(context: ts.TransformationContext) { return (sourceFile: ts.SourceFile) "=>

    { const template = findInlineTemplate(sourceFile); return updateInlineTemplate(template, sourceFile, context); }; } function updateInlineTemplate( nodes: Array<ts.Node>, sourceFile: ts.SourceFile, context: ts.TransformationContext) { const visitor: ts.Visitor = (node: ts.Node) "=> { "// ast walker function return ts.visitEachChild(node, visitor, context); }; return ts.visitNode(sourceFile, visitor); }
  87. @elmd_ export function transformer(context: ts.TransformationContext) { return (sourceFile: ts.SourceFile) "=>

    { const template = findInlineTemplate(sourceFile); return updateInlineTemplate(template, sourceFile, context); }; } function updateInlineTemplate( nodes: Array<ts.Node>, sourceFile: ts.SourceFile, context: ts.TransformationContext) { const visitor: ts.Visitor = (node: ts.Node) "=> { "// ast walker function return ts.visitEachChild(node, visitor, context); }; return ts.visitNode(sourceFile, visitor); }
  88. @elmd_ export function transformer(context: ts.TransformationContext) { return (sourceFile: ts.SourceFile) "=>

    { const template = findInlineTemplate(sourceFile); return updateInlineTemplate(template, sourceFile, context); }; } function updateInlineTemplate( nodes: Array<ts.Node>, sourceFile: ts.SourceFile, context: ts.TransformationContext) { const visitor: ts.Visitor = (node: ts.Node) "=> { "// ast walker function return ts.visitEachChild(node, visitor, context); }; return ts.visitNode(sourceFile, visitor); }
  89. @elmd_ export function transformer(context: ts.TransformationContext) { return (sourceFile: ts.SourceFile) "=>

    { const template = findInlineTemplate(sourceFile); return updateInlineTemplate(template, sourceFile, context); }; } function updateInlineTemplate( nodes: Array<ts.Node>, sourceFile: ts.SourceFile, context: ts.TransformationContext) { const visitor: ts.Visitor = (node: ts.Node) "=> { "// ast walker function return ts.visitEachChild(node, visitor, context); }; return ts.visitNode(sourceFile, visitor); }
  90. @elmd_ function updateInlineTemplate( nodes: Array<ts.Node>, sourceFile: ts.SourceFile, context: ts.TransformationContext) {

    const visitor: ts.Visitor = (node: ts.Node) "=> { "// ast walker function return ts.visitEachChild(node, visitor, context); }; return ts.visitNode(sourceFile, visitor); }
  91. function updateInlineTemplate( nodes: Array<ts.Node>, sourceFile: ts.SourceFile, context: ts.TransformationContext) { const

    visitor: ts.Visitor = (node: ts.Node) "=> { if (nodes.includes(node)) { const prop = node as ts.PropertyAssignment; const currentTpl = prop.initializer.getFullText(); const inlineTpl = ts.createNoSubstitutionTemplateLiteral( updateDirectives(currentTpl, true) ); return ts.updatePropertyAssignment(prop, prop.name, inlineTpl); } return ts.visitEachChild(node, visitor, context); }; return ts.visitNode(sourceFile, visitor); @elmd_
  92. if (!acp) { throw new Error('AngularCompilerPlugin not found'); } const

    options: AngularCompilerPluginOptions = { ""...acp.options, directTemplateLoading: false "// ""<-- IMPORTANT }; config.plugins = removeCompilerPlugin(config.plugins, acp); const newCompilerPlugin = new AngularCompilerPlugin(options); config.plugins.push(newCompilerPlugin); return config; } }; @elmd_
  93. if (!acp) { throw new Error('AngularCompilerPlugin not found'); } const

    options: AngularCompilerPluginOptions = { ""...acp.options, directTemplateLoading: false "// ""<-- IMPORTANT }; config.plugins = removeCompilerPlugin(config.plugins, acp); const newCompilerPlugin = new AngularCompilerPlugin(options); acp._transformers = [inlineTplTransformer, ""...acp._transformers]; config.plugins.push(newCompilerPlugin); return config; } }; @elmd_
  94. None
  95. https://github.com/typebytes/ngx-template-streams @elmd_

  96. Key Takeaways @elmd_

  97. Key Takeaways @elmd_ The Angular Compiler doesn’t need to be

    scary
  98. Key Takeaways @elmd_ TypeScript Transformers can be used to transform

    our own source code The Angular Compiler doesn’t need to be scary
  99. Key Takeaways @elmd_ TypeScript Transformers can be used to transform

    our own source code ASTs are powerful and the foundation of many tools The Angular Compiler doesn’t need to be scary
  100. Key Takeaways @elmd_ TypeScript Transformers can be used to transform

    our own source code ASTs are powerful and the foundation of many tools Tooling and code automation can be so much fun and save lots of time The Angular Compiler doesn’t need to be scary
  101. Thank You NG-BE! @elmd_