Build you a static site generator

UIT meetup vol.11

Transcript

1. Build you a static site generator UIT meetup vol.11 2020-12-18
2. • Software Engineer at Google. • This talk has nothing to do with my employer though. Speaker: Jun
3. What would you do with a static site generator?
4. Blogging
5. Existing SSGs for personal blogging • Difficult to understand how they works internally ◦ Most of their features are unneeded though • Difficult to customize and personalize ◦ Custom styling? ◦ Custom directory structure? ◦ Custom Markdown parsing? ◦ Software Trade-off: Customizability v.s. Integrity • Good for common purposes (documentation, book, etc), but may not fit well for personal usages.
6. • 100% in control. • Can learn how SSG works. • Easy and fun. Why not build your own?
7. Design a static site generator • Static site generator: pipeline of files • Pipeline: a set of data processing elements connected in series, where the output of one element is the input of the next one. Wikipedia
8. A pipe FileData • Path • Content • Metadata (e.g. date and tags for posts) class Pipe { nextPipes = []; // A virtual function to be overridden. operate(input) { ... } pipe(nextPipe) { if (!this.nextPipes.includes(nextPipe)) this.nextPipes.push(nextPipe); return nextPipe; } async execute(input) { const output = await this.operate(input); if (output) this.nextPipes.map(next => next.execute(output)); } } Pipe FileData FileData
9. FileReader • No input (source pipe) • Read from file system. class FileReader extends Pipe { file = new FileData(path); async operate() { this.file.content = await fs.readFile(this.file.path(srcDir)); return this.file; } } File Reader FileData
10. class FileWriter extends Pipe { async operate(file) { await fs.mkdir(file.dir(outDir), {recursive: true}); await fs.writeFile(file.path(options.out), file.content); } } FileWriter • No output (sink pipe) • Write to file system. File Writer FileData
11. // The singleton writer. const writer = new FileWriter(); // Reader for a static file. const reader = new FileReader('sample.jpg'); reader.pipe(writer) // Readers for multiple files const readers = await fileReadersInDir('static'); staticReaders.map(reader => reader.pipe(writer)); File Reader Static files • JS, CSS, fonts, images, etc. File Writer
12. • Templates, Markdown, TypeScript, etc. • Example: Post parser Compilation class PostParser extends Pipe { operate(file) { const {metadata, remainingContent} = this.parseMetadata( file.content.toString('utf8')); file.metadata = metadata; // Convert the markdown content into HTML. file.content = markdownConverter .makeHtml(remainingContent); file.extname = '.html'; return file; } } Target FileData Source FileData Compiler
13. • A compiler can use multiple inputs to produce a single output. • Example: Pug template compiler Compilation (cont.) class PugCompiler extends Pipe { ... // Uses 2 inputs; one is a template file and // the other is a date file. Each data file // will produce an output. async operate(input) { if (input instanceof FileData && input.extname == '.pug') { this.resolveTemplate( pug.compile(input.content)); } else { const template = await this.promisedTemplate; input.content = template(input); return input; } } } Target FileData Source FileData Compiler
14. Aggregation class Aggregator extends Pipe { file = new FileData(path); inputs = []; // The count of inputs to be aggregated per // output. aggregationCount; // Determines the order of the aggregated // inputs. compareFunction; operate(input) { this.inputs.push(input); if (this.inputs.length == this.aggregationCount) { this.inputs.sort(this.compareFunction); this.file.metadata.inputs = this.inputs; this.inputs = []; return this.file; } } } Serial FileData Aggregator A single FileData • Aggregate multiple inputs into a single input. • Example: Post list page
15. Pug Compiler The entire pipeline FileReaders (static files) FileReaders (Posts) FileReader (Post template) FileReader (Post list template) FileWriter PostParser Pug Compiler Aggregator
16. • https://github.com/hatashiro/uit-meetup-11 • The main script contains only ~170 lines of code (and other lines for detailed comments). Demo
17. Further (interesting) ideas • Watch files ◦ FileWatcher pipe (similar to FileReader) • Pagination ◦ Pager pipe (similar to Aggregator) • Testability ◦ StringWriter pipe (instead of FileWriter) ◦ ... but who cares of tests for a personal blog? • Cache ◦ Per-pipe cache using hash • Have fun!
18. Thanks!