Slide 1
Slide 1 text
Build you
a static site generator
UIT meetup vol.11
2020-12-18
Slide 2
Slide 2 text
● Software Engineer at Google.
● This talk has nothing to do with my employer though.
Speaker: Jun
Slide 3
Slide 3 text
What would you do
with a static site
generator?
Slide 4
Slide 4 text
Blogging
Slide 5
Slide 5 text
Existing SSGs for personal blogging
● Difficult to understand how they works internally
○ Most of their features are unneeded though
● Difficult to customize and personalize
○ Custom styling?
○ Custom directory structure?
○ Custom Markdown parsing?
○ Software Trade-off: Customizability v.s. Integrity
● Good for common purposes (documentation, book, etc), but may not fit
well for personal usages.
Slide 6
Slide 6 text
● 100% in control.
● Can learn how SSG works.
● Easy and fun.
Why not build your own?
Slide 7
Slide 7 text
Design a static site generator
● Static site generator: pipeline of files
● Pipeline: a set of data processing elements connected in series, where the
output of one element is the input of the next one. Wikipedia
Slide 8
Slide 8 text
A pipe
FileData
● Path
● Content
● Metadata (e.g. date and tags for posts)
class Pipe {
nextPipes = [];
// A virtual function to be overridden.
operate(input) { ... }
pipe(nextPipe) {
if (!this.nextPipes.includes(nextPipe))
this.nextPipes.push(nextPipe);
return nextPipe;
}
async execute(input) {
const output = await this.operate(input);
if (output)
this.nextPipes.map(next =>
next.execute(output));
}
}
Pipe FileData
FileData
Slide 9
Slide 9 text
FileReader
● No input (source pipe)
● Read from file system.
class FileReader extends Pipe {
file = new FileData(path);
async operate() {
this.file.content = await
fs.readFile(this.file.path(srcDir));
return this.file;
}
}
File
Reader FileData
Slide 10
Slide 10 text
class FileWriter extends Pipe {
async operate(file) {
await fs.mkdir(file.dir(outDir),
{recursive: true});
await fs.writeFile(file.path(options.out),
file.content);
}
}
FileWriter
● No output (sink pipe)
● Write to file system.
File
Writer
FileData
Slide 11
Slide 11 text
// The singleton writer.
const writer = new FileWriter();
// Reader for a static file.
const reader = new FileReader('sample.jpg');
reader.pipe(writer)
// Readers for multiple files
const readers = await
fileReadersInDir('static');
staticReaders.map(reader =>
reader.pipe(writer));
File
Reader
Static files
● JS, CSS, fonts, images, etc.
File
Writer
Slide 12
Slide 12 text
● Templates, Markdown,
TypeScript, etc.
● Example: Post parser
Compilation
class PostParser extends Pipe {
operate(file) {
const {metadata, remainingContent} =
this.parseMetadata(
file.content.toString('utf8'));
file.metadata = metadata;
// Convert the markdown content into HTML.
file.content =
markdownConverter
.makeHtml(remainingContent);
file.extname = '.html';
return file;
}
}
Target
FileData
Source
FileData
Compiler
Slide 13
Slide 13 text
● A compiler can use multiple
inputs to produce a single
output.
● Example: Pug template
compiler
Compilation (cont.)
class PugCompiler extends Pipe {
...
// Uses 2 inputs; one is a template file and
// the other is a date file. Each data file
// will produce an output.
async operate(input) {
if (input instanceof FileData &&
input.extname == '.pug') {
this.resolveTemplate(
pug.compile(input.content));
} else {
const template = await
this.promisedTemplate;
input.content = template(input);
return input;
}
}
}
Target
FileData
Source
FileData
Compiler
Slide 14
Slide 14 text
Aggregation class Aggregator extends Pipe {
file = new FileData(path);
inputs = [];
// The count of inputs to be aggregated per
// output.
aggregationCount;
// Determines the order of the aggregated
// inputs.
compareFunction;
operate(input) {
this.inputs.push(input);
if (this.inputs.length ==
this.aggregationCount) {
this.inputs.sort(this.compareFunction);
this.file.metadata.inputs = this.inputs;
this.inputs = [];
return this.file;
}
}
}
Serial
FileData
Aggregator A single
FileData
● Aggregate multiple inputs into
a single input.
● Example: Post list page
Slide 15
Slide 15 text
Pug
Compiler
The entire pipeline
FileReaders
(static files)
FileReaders
(Posts)
FileReader
(Post
template)
FileReader
(Post list
template)
FileWriter
PostParser Pug
Compiler
Aggregator
Slide 16
Slide 16 text
● https://github.com/hatashiro/uit-meetup-11
● The main script contains only ~170 lines of code (and other lines for
detailed comments).
Demo
Slide 17
Slide 17 text
Further (interesting) ideas
● Watch files
○ FileWatcher pipe (similar to FileReader)
● Pagination
○ Pager pipe (similar to Aggregator)
● Testability
○ StringWriter pipe (instead of FileWriter)
○ ... but who cares of tests for a personal blog?
● Cache
○ Per-pipe cache using hash
● Have fun!
Slide 18
Slide 18 text
Thanks!