The future of (javascript) modules (in node)

Slide 1

Slide 1 text

The future of modules

Slide 2

Slide 2 text

The future of javascript modules & node

Slide 3

Slide 3 text

C J Silverio CTO, @ceejbot

Slide 4

Slide 4 text

you use a lot of modular javascript billions & billions of packages

Slide 5

Slide 5 text

if you use code from npm the future of modules matters to you

Slide 6

Slide 6 text

what's a module anyway?

Slide 7

Slide 7 text

a module is a scope plus an API that lets you reuse it

Slide 8

Slide 8 text

ﬁles are modules groups of ﬁles can be

Slide 9

Slide 9 text

give that group of ﬁles a package.json and it's a package

Slide 10

Slide 10 text

module syntax: this is what I'm sharing this is what I'm re-using

Slide 11

Slide 11 text

CommonJS is one module syntax ECMAScript Modules is another

Slide 12

Slide 12 text

you've seen the syntax a million times but here's a fast refresher

Slide 13

Slide 13 text

const chalk = require('chalk'); // cjs import chalk from 'chalk'; // esm console.log(chalk.bold('hello world!'));

Slide 14

Slide 14 text

const { yellow, bold } = require('chalk'); // cjs import { yellow, bold } from 'chalk'; // esm console.log(`${bold('hello')} ${yellow('world!')}`);

Slide 15

Slide 15 text

function red(word) = { ... }; // etc module.exports = { red, green, blue, bold, ...}; // cjs export { red, green, blue, bold, ...}; // esm

Slide 16

Slide 16 text

why is this worth a talk? it isn't because of syntax

Slide 17

Slide 17 text

it's because the transition has not been smooth

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

node needs both and it still doesn't do ESM natively

Slide 20

Slide 20 text

ESM is the future of modular JavaScript

Slide 21

Slide 21 text

in order to understand the future we must ﬁrst understand the past

Slide 22

Slide 22 text

in 2009 JS had no module spec but CommonJS had one

Slide 23

Slide 23 text

there was a lot of server-side js going on and people needed a way to share code

Slide 24

Slide 24 text

node adopted the CommonJS syntax and wrote a loader

Slide 25

Slide 25 text

node's resolution algorithm turns a human-friendly module name into a path to code that can be loaded

Slide 26

Slide 26 text

const chalk = require('chalk'); // loads './node_modules/chalk/index.js' const other = require('./chalk'); // loads './chalk.js' if it !nds it

Slide 27

Slide 27 text

node's take on modules is useful you're using the heck out of it today

Slide 28

Slide 28 text

browser-side js was still concatenating ﬁles

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

TC39 started working on the problem in 2009

Slide 31

Slide 31 text

import & export arrived in ES2015

Slide 32

Slide 32 text

esm design choices differ from cjs! its intended uses are different!

Slide 33

Slide 33 text

inherently asynchronous not synchronous

Slide 34

Slide 34 text

(mostly) not dynamic: can't import from a variable source

Slide 35

Slide 35 text

designed to support static analysis the import() variation is, however, dynamic

Slide 36

Slide 36 text

see Lin Clark's cartoon deep-dive for an exploration of the details

Slide 37

Slide 37 text

static analysis is important when your target is the browser

Slide 38

Slide 38 text

node's nested deps are a great solution server-side, where disk space is cheap

Slide 39

Slide 39 text

static analysis & tree shaking help minimize download burden in FE

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

so front-end focused developers want ESM

Slide 42

Slide 42 text

in between 2009 and now something very interesting happened

Slide 43

Slide 43 text

node's user base changed

Slide 44

Slide 44 text

throwbacks like me still write server-side js most of you use node for frontend tooling

Slide 45

Slide 45 text

you didn't want to wait for new js features you wanted them now

Slide 46

Slide 46 text

transpilation took off babel was the new hotness

Slide 47

Slide 47 text

babel transpiled the ESM syntax into CJS and the future looked like it was here

Slide 48

Slide 48 text

but babel was CJS behind the scenes which meant node's loader was doing the work

Slide 49

Slide 49 text

problem: there is no esm loader

Slide 50

Slide 50 text

no loader? no loader. well, html's loader.

Slide 51

Slide 51 text

import {chalk} from 'https://example.com/js/chalk/index.js'; import {foo} from '../js/foo.js';

Slide 52

Slide 52 text

ﬁle paths are super-predictable and super-annoying to write by hand

Slide 53

Slide 53 text

node's loader is part of its magic there's no magic in spec-compliant ESM loading

Slide 54

Slide 54 text

babel set some expectations for that magical resolution algorithm

Slide 55

Slide 55 text

meanwhile, back in Gotham City...

Slide 56

Slide 56 text

node's been using CJS for 9 years npm has 725K public packages

Slide 57

Slide 57 text

you will recall, front-end dev is now what people are using node to do

Slide 58

Slide 58 text

node should use JS standards when those standards exist

Slide 59

Slide 59 text

therefore node should use ESM. but what does this mean?

Slide 60

Slide 60 text

the devil is in the details

Slide 61

Slide 61 text

ESM in node looked like it was not going good places

Slide 62

Slide 62 text

very depressing talk by Myles Borins about how hard the integration was

Slide 63

Slide 63 text

the systems have to co-exist but they have little in common

Slide 64

Slide 64 text

how do CJS & ESM interoperate?

Slide 65

Slide 65 text

can you require() an esm ﬁle? can you import a cjs ﬁle?

Slide 66

Slide 66 text

how do you know which is which without parsing everything?

Slide 67

Slide 67 text

node has no type="module" marker for its entry points

Slide 68

Slide 68 text

so it proposed .mjs as a ﬁle sufﬁx for javascript using the esm api

Slide 69

Slide 69 text

.mjs means no parsing: every module must advertise api in name

Slide 70

Slide 70 text

No content

Slide 71

Slide 71 text

I'm the Stig table-ﬂipping.

Slide 72

Slide 72 text

"Well," I said. "What happens if we choose different constraints?"

Slide 73

Slide 73 text

No content

Slide 74

Slide 74 text

well, not very hard. Chris Dickinson & I implemented a usable take on ESM + CJS in node.

Slide 75

Slide 75 text

no mjs, parse entry points; require module consumer know its api in advance

Slide 76

Slide 76 text

our aim was to prove with a bit of parsing you could get a lot of usability

Slide 77

Slide 77 text

reception was mixed

Slide 78

Slide 78 text

No content

Slide 79

Slide 79 text

No content

Slide 80

Slide 80 text

technical problems are easy

Slide 81

Slide 81 text

people problems are hard

Slide 82

Slide 82 text

what is node for? how do we use node? where is node going?

Slide 83

Slide 83 text

fundamental differences in answers lead to fundamental differences in designs

Slide 84

Slide 84 text

talking thinking typing

Slide 85

Slide 85 text

the node modules problem needed more talking

Slide 86

Slide 86 text

I didn't want our solution adopted. I wanted the talking phase restarted.

Slide 87

Slide 87 text

No content

Slide 88

Slide 88 text

node has an unchartered modules team doing the hard work of talking

Slide 89

Slide 89 text

it started with use cases and it's continuing with some deep questions

Slide 90

Slide 90 text

see github.com/nodejs/modules to understand how complicated the tradeoffs are

Slide 91

Slide 91 text

this brings us to the present state ESM is not quite there yet, but soon!

Slide 92

Slide 92 text

if you are using babel in your build you have no reason not to use ESM now

Slide 93

Slide 93 text

those of us not using Babel had few options until recently

Slide 94

Slide 94 text

github.com/standard-things/esm

Slide 95

Slide 95 text

npm init esm -y

Slide 96

Slide 96 text

record scratch noise npm install -g npm@latest

Slide 97

Slide 97 text

npm@6 has some goodies

Slide 98

Slide 98 text

like templated init even better than npm init -y

Slide 99

Slide 99 text

npm audit & npm audit !x report and automatically ﬁx known vulnerabilities in your dep tree

Slide 100

Slide 100 text

npm, Inc, is a company that sells goods and services

Slide 101

Slide 101 text

you can pay us to run a very nice single-tenant registry for you

Slide 102

Slide 102 text

now back to modules...

Slide 103

Slide 103 text

ESM in node without a build step: npm init esm -y

Slide 104

Slide 104 text

No content

Slide 105

Slide 105 text

transpilation on the ﬂy by monkey-patching the module object

Slide 106

Slide 106 text

require = require('esm')(module); const Widget = require('./main.js').default; // over in main.js import fs from 'fs'; import cjs from './cjs.js'; const esm = require('./esm.js'); export default class Widget { ... }