Hands-on Native ESM @ JSConf JP 2022

© 2022 Wantedly, Inc.
Hands-on Native ESM
Understanding Node.js module discrepancy
Nov. 26 2022 - Masaki Hara
https://github.com/qnighy
https://www.wantedly.com/id/qnighy

View Slide

The aim of this workshop
● Understand how CJS and ESM (does not
smoothly) interoperate in Node.js
● Understand how it becomes more difficult when
module budler is involved
It does not cover the entire solution, but it will nonetheless help
you migrating to Native ESM.

View Slide

Related materials
● 実践 Node.js Native ESM — Wantedlyでのアプリケー
ション移行事例[1]
● Node.jsのネイティブES Modulesサポートが抱える問題を
解決するBabelプラグインを書いた[2]
● Native ESM + TypeScript 拡張子問題: 歯にものが挟まっ
たようなスッキリしない書き流し[3]
[1] https://www.wantedly.com/companies/wantedly/post_articles/410531
[2] https://zenn.dev/qnighy/articles/6267716578c76d
[3] https://zenn.dev/qnighy/articles/19603f11d5f264

View Slide

Advertisement
We are looking for JS ecosystem enthusiasts!
WebpackやBabelの設定を通じて開発を加速させたい人募集
! - Wantedly[1]
[1] https://www.wantedly.com/projects/1145492

View Slide

Advertisement
Our frontend team also welcomes you!
今見ているWantedlyのWebフロントエンドをもっと良くしませんか？ - Wantedly[1]
[1] https://www.wantedly.com/projects/1159802

View Slide

Hands-on guide
● Materials can be found at:
https://github.com/wantedly/hands-on-native-
esm-2022
● Requires Node.js ≧ 16
● I will demonstrate chapters 1 to 4.
● Read and exercise the remaining chapters (5 to
11) if you are interested.

View Slide

1: Let’s try ESM!

View Slide

Try Native ESM
● Node.js has two module types:
○ ESM = ES Modules (*.mjs, *.js)
○ CJS = CommonJS Modules (*.cjs, *.js)
● They interoperate badly 😵
● You usually write JS in ESM
but execute it as CJS.
(as opposed to Native ESM)

View Slide

Try Native ESM
Try ESM import/export
export default function(x) {
return x * x;
}
import square from "./lib.mjs";
console.log(square(2));
lib.mjs app.mjs

View Slide

Try Native ESM
Try ESM import/export
return x * x;
}
lib.mjs app.mjs
$ node app.mjs
4

View Slide

Try Native ESM
Open https://babeljs.io/repl

View Slide

Try Native ESM
Try CJS transpilation
"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
exports.default = _default;
function _default(x) {
return x * x;
}
"use strict";
var _a =
_interopRequireDefault(require("./a.cjs"));
function _interopRequireDefault(obj) {
return obj && obj.__esModule
? obj
: { default: obj };
}
console.log((0, _a.default)(2));
lib.cjs (before Babel) app.cjs (before Babel)
return x * x;
}
import square from "./lib.cjs";

View Slide

Try Native ESM
"use strict";
value: true
});
return x * x;
}
"use strict";
var _a =
_interopRequireDefault(require("./lib.cjs"));
? obj
: { default: obj };
}
lib.cjs app.cjs

View Slide

Try Native ESM
"use strict";
value: true
});
return x * x;
}
"use strict";
var _a =
_interopRequireDefault(require("./lib.cjs"));
? obj
: { default: obj };
}
lib.cjs app.cjs
$ node app.cjs
4

View Slide

Try Native ESM
CJS → ESM
"use strict";
var _a =
_interopRequireDefault(require("./lib.mjs"));
? obj
: { default: obj };
}
app.cjs
return x * x;
}
lib.mjs

View Slide

Try Native ESM
CJS → ESM
"use strict";
var _a =
_interopRequireDefault(require("./lib.mjs"));
? obj
: { default: obj };
}
app.cjs
return x * x;
}
lib.mjs
$ node app.cjs
node:internal/modules/cjs/loader:1031
throw new ERR_REQUIRE_ESM(filename, true);
^
Error [ERR_REQUIRE_ESM]: require() of ES
Module lib.mjs not supported.
Instead change the require of lib.mjs to a
dynamic import() which is available in all
CommonJS modules.

View Slide

Try Native ESM
ESM → CJS
"use strict";
value: true
});
return x * x;
}
lib.cjs
app.mjs

View Slide

Try Native ESM
ESM → CJS
"use strict";
value: true
});
return x * x;
}
lib.cjs
app.mjs
$ node app.mjs
app.mjs:3
^
TypeError: square is not a function
at app.mjs:3:13

View Slide

There is no easy path
● Library-first upgrade: essential difficulty
○ Synchronous vs. Asynchronous
● Application-first upgrade: avoidable pitfalls
○ Default import problem
● Dual package: difficulty in some packages
○ State sharing problem

View Slide

But we must do it
● Some libraries are already in Native ESM format.
○ node-fetch
○ d3
○ etc.
● Depending applications are (almost) forced to do
the migration.

View Slide

2: Module detection

View Slide

How Node.js determines module types
● Node.js determines[1] module types using:
○ File extensions (*.cjs / *.mjs)
○ Package metadata (“type”: “module”)
if it is *.js
*.cjs *.js *.mjs
“type”: “commonjs” CJS CJS ESM
no “type” field CJS CJS ESM
“type”: “module” CJS ESM ESM
Conversely, they are not taken into account:
● File contents
● How the import had been routed
(e.g. exports map)
[1] https://nodejs.org/api/packages.html#packagejson-and-file-extensions

View Slide

Check module types
if (typeof await /0/["test"] === "function") {
console.log("I'm ESM");
} else if (typeof require === "function") {
console.log("I'm CJS");
} else {
console.log("I'm other");
}
module-type.js

View Slide

Check module types
} else {
}
module-type.js $ node module-type.js
I’m CJS
$ node module-type.cjs
I’m CJS
$ node module-type.mjs
I’m ESM

View Slide

Module type depends on package.json
{
"type": "module"
}
package.json

View Slide

Module type depends on package.json
{
"type": "module"
}
package.json $ node module-type.js
I’m ESM
$ node module-type.cjs
I’m CJS
$ node module-type.mjs
I’m ESM

View Slide

3: Semantics of execution
Module asynchrony

View Slide

Semantics of execution
● Modules are executed
○ synchronously in CJS
○ asynchronously in ESM
● Consequence👉 CJS cannot import ESM[1]
[1]: Precisely speaking, it can wait for the imported module to finish.

View Slide

Do a time-consuming initialization
import { setTimeout } from "node:timers/promises";
await setTimeout(3000);
// For Node.js < 16.0.0
// await new Proise((r) => setTimeout(r, 3000));
console.log("slept 1s");
sleep.mjs
Explicit asynchrony: top-level await

View Slide

Dependent module may also be asynchronous
import { setTimeout } from "node:timers/promises";
await setTimeout(3000);
export const value = 42;
sleep-lib.mjs
Implied asynchrony

View Slide

import { value } from "./sleep-lib.mjs";
console.log("value =", value);
sleep-app.mjs
Implied asynchrony

View Slide

const { value } = await import("./sleep-lib.mjs");
sleep-app.mjs (alternative)
Expanding implied asynchrony

View Slide

Require — this is a mere function
const { value } = require("./sleep-lib.mjs");
sleep-app.cjs
Discrepancy in asynchrony
Error!

View Slide

Require — this is a mere function
const { value } = require("./sleep-lib.mjs");
sleep-app.cjs
Error!
$ node sleep-app.cjs
throw new ERR_REQUIRE_ESM(filename,
true);
^
Error [ERR_REQUIRE_ESM]: require() of
ES Module sleep-lib.mjs not supported.

View Slide

TLA — this is not possible
sleep-app.cjs
Error!

View Slide

TLA — this is not possible
sleep-app.cjs
Error!
$ node sleep-app.cjs
sleep-app.cjs:1
const { value } = await
import("./sleep-lib.mjs");
^^^^^
SyntaxError: await is only valid in
async functions and the top level
bodies of modules

View Slide

Import-then
import("./sleep-lib.mjs").then(({ value }) => {
});
sleep-app.cjs

View Slide

Import-then: it’s too late for exporting
import("./sleep-lib.mjs").then(({ value }) => {
exports.value2 = value * 2;
});
sleep-app.cjs
Too late!

View Slide

4: Default export semantics

View Slide

Default export semantics
● Default export is
○ fancy namespace export in CJS
○ special named export in ESM
● Consequence👉 incompatible translations
● My library node-cjs-interop[1] helps here.
[1] https://github.com/qnighy/node-cjs-interop

View Slide

Default export semantics
Default
Named
export default f;
export { f };
module.exports = f;
exports.f = f;

View Slide

Default export semantics from importer’s viewpoint
import * as ns from ""; ✅ Namespace
❌ Default
✅ Named
import f from "";
import { f } from "";
const ns = require("");
const f = require("");
const { f } = require("");
=
∈

View Slide

Default export semantics from importer’s viewpoint
import * as ns from ""; ✅ Namespace
❌ Default
✅ Named
import { default as f }
from "";
=
∈

View Slide

Default export interop 1: Node.js rule (CJS-preserving)
import * as ns from "";
import f from "";
=
∈
(except “default”)
(except “default”)

View Slide

Default export interop 2: simple mapping (ESM-preserving)
import * as ns from "";
import f from "";
=
∈
(Loss of
information)

View Slide

Default export interop 3: Babel mapping
var _react = _interopRequireDefault(require("react"));
? obj
: { default: obj };
}
Simple mapping (ESM-preserving)

View Slide

Default export interop 3: Babel mapping
var _react = _interopRequireDefault(require("react"));
? obj
: { default: obj };
}
Node.js rule (CJS-preserving)

View Slide

Default export interop – tool support
Node.js rule Babel rule
(hybrid)
Simple mapping
Node.js ✅ ❌ but see:
node-cjs-interop[1]
❌
Webpack javascript/esm javascript/auto ❌
Babel interop = node interop = babel interop = none
TypeScript ❌ esModuleInterop =
true
esModuleInterop =
false
[1] https://github.com/qnighy/node-cjs-interop

View Slide

4.1: Default export semantics (1)
Default export in ESM

View Slide

Default export and default import in ESM
return x * x;
}
lib.mjs app.mjs

View Slide

Default export and default import in ESM
return x * x;
}
lib.mjs app.mjs
$ node app.mjs
4

View Slide

Default import is a syntax sugar
return x * x;
}
import {
default as square
} from "./lib.mjs";
lib.mjs app.mjs

View Slide

Default import is a syntax sugar
return x * x;
}
import {
default as square
} from "./lib.mjs";
lib.mjs app.mjs
$ node app.mjs
4

View Slide

Default export is a syntax sugar
function square(x) {
return x * x;
}
export { square as default };
lib.mjs app.mjs

View Slide

Default export is a syntax sugar
function square(x) {
return x * x;
}
export { square as default };
lib.mjs app.mjs
$ node app.mjs
4

View Slide

Default export in CJS

View Slide

Default export and default import in CJS
module.exports = 42; const x = require("./lib.cjs");
console.log(x);
lib.cjs app.cjs

View Slide

Default export and default import in CJS
module.exports = 42; const x = require("./lib.cjs");
console.log(x);
lib.cjs app.cjs
$ node app.mjs
42

View Slide

Node.js rule for default export
Default export interop
module.exports = 42; import x from "./app.cjs";
console.log(x);
lib.cjs app.mjs

View Slide

Node.js rule for default export
Default export interop
module.exports = 42; import x from "./app.cjs";
console.log(x);
lib.cjs app.mjs
$ node app.mjs
42

View Slide

Named exports in CJS

View Slide

exports.foo = 1;
exports.bar = 2;
const m = require("./lib.cjs");
console.log(m);
lib.cjs app.cjs

View Slide

exports.foo = 1;
exports.bar = 2;
const m = require("./lib.cjs");
console.log(m);
lib.cjs app.cjs
$ node app.cjs
{ foo: 1, bar: 2 }

View Slide

Node.js rule for named exports
Named exports interop
exports.foo = 1;
exports.bar = 2;
import m1 from "./lib.cjs";
import * as m2
from "./lib.cjs";
console.log(m1);
console.log(m2);
lib.cjs app.mjs

View Slide

Node.js rule for named exports
Named exports interop
exports.foo = 1;
exports.bar = 2;
import * as m2
from "./lib.cjs";
console.log(m1);
console.log(m2);
lib.cjs app.mjs
$ node app.mjs
{ foo: 1, bar: 2 }
[Module: null prototype] {
bar: 2,
default: { foo: 1, bar: 2 },
foo: 1
}

View Slide

Node.js rule for named “default” exports
Named “default” exports interop
exports.foo = 1;
exports.bar = 2;
exports.default = 3;
import * as m2
from "./lib.cjs";
console.log(m1);
console.log(m2);
lib.cjs app.mjs

View Slide

Node.js rule for named “default” exports
Named “default” exports interop
exports.foo = 1;
exports.bar = 2;
import * as m2
from "./lib.cjs";
console.log(m1);
console.log(m2);
lib.cjs app.mjs
$ node app.mjs
{ foo: 1, bar: 2, default: 3 }
bar: 2,
default: { foo: 1, bar: 2,
default: 3 },
foo: 1
}

View Slide

Babel-style interop

View Slide

Mixing CJS named and default exports
Named + default exports in CJS
exports.foo = 1;
exports.bar = 2;
module.exports = 3;
lib.cjs
import * as m2
from "./lib.cjs";
console.log(m1);
console.log(m2);
app.mjs

View Slide

Mixing CJS named and default exports
Named + default exports in CJS
exports.foo = 1;
exports.bar = 2;
module.exports = 3;
lib.cjs
import * as m2
from "./lib.cjs";
console.log(m1);
console.log(m2);
app.mjs
$ node app.cjs
3
bar: undefined,
default: 3,
foo: undefined
}

View Slide

Babel-style interop
Special flag for simple mapping
exports.foo = 1;
exports.bar = 2;
exports.__esModule = true;
lib.cjs app.mjs
import * as m2
from "./lib.cjs";
console.log(m1);
console.log(m2);

View Slide

Babel-style interop
exports.foo = 1;
exports.bar = 2;
var m2 =
_interopRequireWildcard(
require("./lib.cjs"));
console.log(m2.default);
console.log(m2);
lib.cjs app.cjs (transpiled from app.mjs)

View Slide

Babel-style interop
exports.foo = 1;
exports.bar = 2;
"use strict";
var m2 = _interopRequireWildcard(require("./lib.cjs"));
function _getRequireWildcardCache(nodeInterop) { if (typeof WeakMap !== "function") return null; var cacheBabelInterop = new
WeakMap(); var cacheNodeInterop = new WeakMap(); return (_getRequireWildcardCache = function (nodeInterop) { return
nodeInterop ? cacheNodeInterop : cacheBabelInterop; })(nodeInterop); }
function _interopRequireWildcard(obj, nodeInterop) { if (!nodeInterop && obj && obj.__esModule) { return obj; } if (obj ===
null || typeof obj !== "object" && typeof obj !== "function") { return { default: obj }; } var cache =
_getRequireWildcardCache(nodeInterop); if (cache && cache.has(obj)) { return cache.get(obj); } var newObj = {}; var
hasPropertyDescriptor = Object.defineProperty && Object.getOwnPropertyDescriptor; for (var key in obj) { if (key !== "default"
&& Object.prototype.hasOwnProperty.call(obj, key)) { var desc = hasPropertyDescriptor ? Object.getOwnPropertyDescriptor(obj,
key) : null; if (desc && (desc.get || desc.set)) { Object.defineProperty(newObj, key, desc); } else { newObj[key] = obj[key];
} } } newObj.default = obj; if (cache) { cache.set(obj, newObj); } return newObj; }
console.log(m2);

View Slide

Babel-style interop
exports.foo = 1;
exports.bar = 2;
"use strict";
var m2 = _interopRequireWildcard(require("./lib.cjs"));
function _getRequireWildcardCache(nodeInterop) { if (typeof WeakMap !== "function") return null; var cacheBabelInterop = new
WeakMap(); var cacheNodeInterop = new WeakMap(); return (_getRequireWildcardCache = function (nodeInterop) { return
nodeInterop ? cacheNodeInterop : cacheBabelInterop; })(nodeInterop); }
function _interopRequireWildcard(obj, nodeInterop) { if (!nodeInterop && obj && obj.__esModule) { return obj; } if (obj ===
null || typeof obj !== "object" && typeof obj !== "function") { return { default: obj }; } var cache =
_getRequireWildcardCache(nodeInterop); if (cache && cache.has(obj)) { return cache.get(obj); } var newObj = {}; var
hasPropertyDescriptor = Object.defineProperty && Object.getOwnPropertyDescriptor; for (var key in obj) { if (key !== "default"
&& Object.prototype.hasOwnProperty.call(obj, key)) { var desc = hasPropertyDescriptor ? Object.getOwnPropertyDescriptor(obj,
key) : null; if (desc && (desc.get || desc.set)) { Object.defineProperty(newObj, key, desc); } else { newObj[key] = obj[key];
} } } newObj.default = obj; if (cache) { cache.set(obj, newObj); } return newObj; }
console.log(m2);
$ node app.cjs
3
{
foo: 1,
bar: 2,
default: 3,
__esModule: true
}

View Slide

Babel-style interop
Node.js does not support __esModule
exports.foo = 1;
exports.bar = 2;
lib.cjs
import * as m2
from "./lib.cjs";
console.log(m1);
console.log(m2);
app.mjs

View Slide

Babel-style interop
Node.js does not support __esModule
exports.foo = 1;
exports.bar = 2;
lib.cjs
import * as m2
from "./lib.cjs";
console.log(m1);
console.log(m2);
app.mjs
$ node app.mjs
{ foo: 1, bar: 2, default: 3 }
__esModule: true,
bar: 2,
default: { foo: 1, bar: 2,
default: 3 },
foo: 1
}

View Slide

End of demonstration
Enjoy the remaining exercises if you are interested in.

View Slide

5: Export analysis

View Slide

Export analysis
● Static analysis is mandatory in Native ESM,
and it also applies to CJS imported from ESM.
● It sometimes fails to detect exports in CJS.

View Slide

Export analysis
Node.js analyzes ESM exports before execution
console.log("Initializing...");
export const foo = 1;
lib.mjs
import { bar }
from "./lib.mjs";
console.log(bar);
app.mjs

View Slide

Export analysis
lib.mjs
import { bar }
from "./lib.mjs";
console.log(bar);
app.mjs
$ node app.mjs
app.mjs:1
import { bar } from "./lib.mjs";
^^^
SyntaxError: The requested module
'./lib.mjs' does not provide an
export named 'bar'

View Slide

Export analysis
lib.mjs
import { bar }
from "./lib.mjs";
console.log(bar);
app.mjs
$ node app.mjs
app.mjs:1
import { bar } from "./lib.mjs";
^^^
SyntaxError: The requested module
'./lib.mjs' does not provide an
export named 'bar'
No “Initializing…” printed

View Slide

Export analysis
It analyzes CJS too (when imported from ESM)
exports.foo = 1;
lib.cjs
import { bar }
from "./lib.cjs";
console.log(bar);
app.mjs

View Slide

Export analysis
exports.foo = 1;
lib.cjs
import { bar }
from "./lib.cjs";
console.log(bar);
app.mjs
$ node app.mjs
app.mjs:1
import { bar } from "./lib.cjs";
^^^
SyntaxError: Named export 'bar'
not found. The requested module
'./lib.cjs' is a CommonJS module,
which may not support all
module.exports as named exports.

View Slide

Export analysis
exports.bar = 2;
lib.cjs
import { bar }
from "./app.cjs";
console.log(bar);
app.mjs

View Slide

Export analysis
exports.bar = 2;
lib.cjs
import { bar }
from "./app.cjs";
console.log(bar);
app.mjs
$ node app.mjs
Initializing...
2

View Slide

Export analysis
exports["bar"] = 2;
lib.cjs
import { bar }
from "./lib.cjs";
console.log(bar);
app.mjs

View Slide

Export analysis
exports["bar"] = 2;
lib.cjs
import { bar }
from "./lib.cjs";
console.log(bar);
app.mjs
$ node app.mjs
Initializing...
2

View Slide

Export analysis
Not all exports can be analyzed
exports["ba" + "r"] = 2;
lib.cjs
import { bar }
from "./lib.cjs";
console.log(bar);
app.mjs

View Slide

Export analysis
Not all exports can be analyzed
exports["ba" + "r"] = 2;
lib.cjs
import { bar }
from "./lib.cjs";
console.log(bar);
app.mjs
$ node app.mjs
app.mjs:1
import { bar } from "./lib.cjs";
^^^
SyntaxError: Named export 'bar'
not found. The requested module
'./lib.cjs' is a CommonJS module,
which may not support all
module.exports as named exports.

View Slide

Export analysis rules
● Rules can be found at cjs-module-lexer[1]
[1] https://github.com/nodejs/cjs-module-lexer
exports.foo = 1;
module.exports.foo = 1;
exports["foo"] = 1;
module.exports = { foo };
module.exports = { foo: bar };
Object.defineProperty(exports, "foo",
);
{ value: 1 }
{ enumerable: true, value: 1 }
{ get() { return bar; } }
{ get: function() { return bar; } }
{ get() { return m.bar; } }
{ get() { return m["bar"]; } }

View Slide

Export analysis rules
● Rules can be found at cjs-module-lexer[1]
[1] https://github.com/nodejs/cjs-module-lexer
module.exports =
require("mod");
module.exports =
{ ...require("mod") };
__export(require("mod"));
__exportStar(require("mod"),
exports);
var m = require("mod");
const m = require("mod");
let m = require("mod");
var m = _interopRequireWildcard
(require("mod"));
Object.keys(m).forEach(...);
+

View Slide

6: Live exports

View Slide

Live binding of exports
● In ESM, imported variables reference the latest
value of the original variable.
● This is not the case with CJS interop in Node.js.

View Slide

counter evaluates to the latest value
export let counter = 0;
export function countUp() {
counter++;
}
lib.mjs
import { counter, countUp }
from "./lib.cjs";
console.log(counter);
countUp();
app.mjs

View Slide

counter evaluates to the latest value
export let counter = 0;
export function countUp() {
counter++;
}
lib.mjs
from "./lib.cjs";
countUp();
app.mjs
$ node app.mjs
0
1

View Slide

counter evaluates to the value when CJS is imported
exports.counter = 0;
exports.countUp = function() {
exports.counter++;
}
lib.cjs
from "./lib.cjs";
countUp();
app.mjs

View Slide

counter evaluates to the value when CJS is imported
exports.counter = 0;
exports.countUp = function() {
exports.counter++;
}
lib.cjs
from "./lib.cjs";
countUp();
app.mjs
$ node app.mjs
0
0

View Slide

7: Module path resolution

View Slide

Module path resolution
● import() no longer automatically adds
extensions.
● It no longer automatically resolves directories as
“index.js”.
● It no longer resolves “folder main”.
● *.cjs is not added for either CJS or ESM.

View Slide

“.js” is added in CJS
console.log("imported");
lib.js
require("./lib");
app.cjs

View Slide

“.js” is added in CJS
lib.js
require("./lib");
app.cjs
$ node app.cjs
imported

View Slide

“.js” is no longer added in ESM
lib.js
import "./lib";
app.mjs

View Slide

“.js” is no longer added in ESM
lib.js
import "./lib";
app.mjs
$ node app.mjs
node:internal/process/esm_loader:97
internalBinding('errors').triggerUncaughtE
xception(
^
Error [ERR_MODULE_NOT_FOUND]: Cannot find
module 'lib' imported from app.mjs
Did you mean to import ../lib.js?

View Slide

./index.js is added in CJS
lib/index.js
require("./lib");
app.cjs

View Slide

./index.js is added in CJS
lib/index.js
require("./lib");
app.cjs
$ node app.cjs
imported

View Slide

./index.js is no longer added in ESM
lib/index.js
import "./lib";
app.mjs

View Slide

./index.js is no longer added in ESM
lib/index.js
import "./lib";
app.mjs
$ node app.mjs
internalBinding('errors').triggerUncaughtExcep
tion(
^
Error [ERR_UNSUPPORTED_DIR_IMPORT]: Directory
import 'lib' is not supported resolving ES
modules imported from app.mjs
Did you mean to import ../lib/index.js?

View Slide

“Folder main” in CJS
lib/main.js
require("./lib");
app.cjs
{ "main": "./main.js" }
lib/package.json

View Slide

“Folder main” in CJS
lib/main.js
require("./lib");
app.cjs
lib/package.json
$ node app.cjs
imported

View Slide

“Folder main” is no longer available
lib/main.js
import "./lib";
app.mjs
lib/package.json

View Slide

“Folder main” is no longer available
lib/main.js
import "./lib";
app.mjs
lib/package.json
$ node app.mjs
internalBinding('errors').triggerUncaughtExcep
tion(
^
Error [ERR_UNSUPPORTED_DIR_IMPORT]: Directory
import 'lib' is not supported resolving ES
modules imported from app.mjs
Did you mean to import ../lib/main.js?

View Slide

“Package main” is still valid
node_modules/lib/main.js
import "lib";
app.mjs
node_modules/lib/package.json

View Slide

“Package main” is still valid
node_modules/lib/main.js
import "lib";
app.mjs
node_modules/lib/package.json
$ node app.mjs
imported

View Slide

“.cjs” is not added even from CJS
lib.cjs
require("./lib");
app.cjs

View Slide

“.cjs” is not added even from CJS
lib.cjs
require("./lib");
app.cjs
$ node app.mjs
throw err;
^
Error: Cannot find module './lib

View Slide

8: Module metadata

View Slide

Module metadata
● These CJS variables are usually available in
to-be-transpiled ESM code:
○ require
○ exports
○ module
○ __filename
○ __dirname
● This is no longer the case in Native ESM.

View Slide

Module metadata
CJS metavariables are not available
console.log(`I'm ${__filename}`);
export {};
meta.mjs

View Slide

Module metadata
CJS metavariables are not available
export {};
meta.mjs
$ node meta.mjs
file:////meta.mjs:1
^
ReferenceError: __filename is not
defined in ES module scope

View Slide

Module metadata
They are usually transpiled as-is
"use strict";
value: true
});
meta.cjs (transpiled from meta.mjs)

View Slide

Module metadata
They are usually transpiled as-is
"use strict";
value: true
});
meta.cjs (transpiled from meta.mjs)
$ node meta.cjs
I’m /meta.cjs

View Slide

Module metadata
Use import.meta instead in Native ESM
console.log(`I'm ${import.meta.url}`);
export {};
meta.mjs

View Slide

Module metadata
Use import.meta instead in Native ESM
console.log(`I'm ${import.meta.url}`);
export {};
meta.mjs
$ node meta.mjs
I’m file:///meta.mjs

View Slide

9: Dual package

View Slide

Dual package
● Export maps allow packages to provide different
module types at once.
● There is a pitfall when the package is stateful (or
impure in some sense).

View Slide

Dual package
Native ESM dual package
{
"name": "esm-test",
"exports": {
"import": "./lib.mjs",
"require": "./lib.cjs",
}
}
package.json
console.log("ESM entrypoint");
lib.mjs
console.log("CJS entrypoint");
lib.cjs

View Slide

Dual package
Using import and require
import "esm-test";
app.mjs
require("esm-test");
app.cjs
package.json
lib.mjs
lib.cjs

View Slide

Dual package
Using import and require
import "esm-test";
app.mjs
app.cjs
package.json
lib.mjs
lib.cjs
$ node app.mjs
ESM entrypoint
$ node app.cjs
CJS entrypoint

View Slide

Dual package
Double side effects
import "esm-test";
import "./app.cjs";
app.mjs
app.cjs
package.json
lib.mjs
lib.cjs

View Slide

Dual package
Double side effects
import "esm-test";
import "./app.cjs";
app.mjs
app.cjs
package.json
lib.mjs
lib.cjs
$ node app.mjs
ESM entrypoint
CJS entrypoint
Imagine this is state initialization.
There are two different states
now!

View Slide

10: Modules in Webpack

View Slide

Modules in bundlers
● ESM story in Node.js gets more complicated when
module bundlers are involved.
● In this workshop, we use Webpack (version 5) as
an example because it is still the most influential
bundler.

View Slide

Modules in Webpack
● Webpack generally adheres to Node.js behavior.
● One big difference: Webpack has two different
ESM modes.
● It complicates the matter in default export
interop.

View Slide

Autodetected ESM (Fake ESM)
● Webpack has “auto detection” module mode.
● It semantically behaves like CJS, even if the
module is detected to be ESM.
*.cjs *.js *.mjs
no “type” field CJS Auto ESM
In Webpack terminology:
● CJS = javascript/dynamic
● Auto = javascript/auto
● ESM = javascript/esm

View Slide

Autodetected ESM (Fake ESM)
● Webpack has “auto detection” module mode.
● It semantically behaves like CJS, even if the
module is detected to be ESM.
*.cjs *.js *.mjs
no “type” field CJS Auto ESM
*.cjs *.js *.mjs
no “type” field CJS CJS ESM
Recap: Node.js behavior

View Slide

Resolution for autodetected ESM
● Special lookup rule to detect Webpack
{
"module": "esm/main.js",
"main": "main.js",
"exports": {
"module": "./esm/main.js",
"default": "./main.js"
}
}
package.json
This is usually CJS
This is usually an ESM module
to be auto-detected by Webpack

View Slide

Resolution for autodetected ESM
● Two types of dual package
{
"exports": {
"module": "./esm/main.js",
"default": "./main.js"
}
}
package.json
{
"exports": {
"import": "./esm/main.mjs",
"require": "./main.cjs"
}
}
package.json
Webpack uses ESM.
Node.js always uses CJS.
Webpack uses ESM.
Node.js (import) uses ESM.
Node.js (require) uses CJS.

View Slide

Default export semantics in Node.js
CJS from Babel
(*.js, *.cjs)
Native ESM
(*.js, *.mjs)
Breaks default export

View Slide

Default export semantics in Webpack
Autodetected ESM
(*.js)
CJS from Babel
(*.js, *.cjs)
Native ESM
(*.js, *.mjs)
Preserves default export
Preserves default export
Breaks default export

View Slide

Set up Webpack
Install Webpack
{}
package.json
$ yarn add --dev webpack webpack-cli
Or: npm install --dev webpack webpack-cli

View Slide

Set up Webpack
Install Webpack
{
"devDependencies": {
"webpack": "^5.75.0",
"webpack-cli": "^5.0.0"
}
}
package.json (result)

View Slide

Set up Webpack
Check import result
lib1.js
import a from "./lib1.js";
console.log("a =", a);
export default 1;
lib2.js
import b from "./lib1.js";
import c from "./lib2.js";
console.log("b =", b);
console.log("c =", c);
app.mjs

View Slide

Set up Webpack
Check import result
lib1.js
export default 1;
lib2.js
app.mjs
$ yarn webpack ./app.mjs --mode
production --target node -o .
(..snip..)
webpack 5.75.0 compiled
successfully in 122 ms
✨ Done in 0.50s.

View Slide

Set up Webpack
Check import result
lib1.js
export default 1;
lib2.js
app.mjs
$ node main.js
a = 1
b = {
default: 1,
__esModule: true
}
c = 1

View Slide

11: Meta APIs for modules

View Slide

Meta APIs for modules
● Node.js implements `import` and `require`
separately.
○ Not just the parser, but the whole module
lifecycle is implemented differently.
● Consequently, they expose different APIs for ESM
and CJS.

View Slide

Loader Hook: CJS
● require.extensions[2] exists for this.
● Isn’t it deprecated? Yes, but even @babel/register
continues using it.
● Things could easily go wrong with this API. Use the
pirate[2] package for reliability.
[1] https://nodejs.org/api/modules.html#requireextensions
[2] https://www.npmjs.com/package/pirates

View Slide

Loader Hook: CJS
const oldLoader = require.extensions[".js"];
require.extensions[".js"] = function(mod, filename) {
const oldCompile = mod._compile;
mod._compile = function(code, filename) {
const newCode = `console.log(__filename);` + code;
return oldCompile.call(this, newCode, filename);
};
oldLoader(mod, filename);
};
register.cjs

View Slide

Loader Hook: CJS
module.exports = 42;
lib.cjs
const x = require("./lib.cjs");
console.log(x);
app.cjs
register.cjs

View Slide

Loader Hook: CJS
lib.cjs
const x = require("./lib.cjs");
console.log(x);
app.cjs
register.cjs
$ node -r ./register.cjs app.cjs
app.cjs
lib.cjs
42

View Slide

Loader Hook: ESM
● Conversely, Node.js has a (truly) public loader
API[1] for ESM.
● It is still experimental and subject to change.
[1] https://nodejs.org/api/esm.html#loaders

View Slide

Loader Hook: ESM
export async function load(url, context, nextLoad) {
const m = await nextLoad(url, context);
if (m.format !== "module") return m;
if (typeof m.source !== "string") {
m.source = new TextDecoder().decode(m.source);
}
m.source = `console.log(import.meta.url); ${m.source}`;
return m;
}
loader.mjs

View Slide

Loader Hook: ESM
export default 42;
lib.mjs
import x from "./lib.mjs";
console.log(x);
app.mjs
loader.mjs

View Slide

Loader Hook: ESM
export default 42;
lib.mjs
import x from "./lib.cjs";
console.log(x);
app.mjs
loader.mjs
$ node --experimental-loader
./loader.mjs ./app.mjs
file:///app.cjs
file:///lib.cjs
42

View Slide

Parsing modules
● Want to re-implement module loader?
○ Jest does this
● For CJS, Function suffices.
● For ESM, we need vm.Module (experimental).

View Slide

Parsing CJS modules
const fs = require("node:fs");
const m = { exports: {} };
const f = new Function(
"exports", "require", "module", "__filename", "__dirname",
fs.readFileSync("lib.cjs", "utf-8"));
f(m.exports, null, m, "", "");
console.log(m.exports);
app.cjs

View Slide

Parsing CJS modules
app.cjs
lib.cjs

View Slide

Parsing CJS modules
app.cjs
lib.cjs
$ node app.cjs
42

View Slide

Parsing ESM modules
import fs from "node:fs";
import vm from "node:vm";
const m = new vm.SourceTextModule(
fs.readFileSync("lib.mjs", "utf-8"));
await m.link(() => {});
await m.evaluate();
console.log(m.namespace.default);
app.mjs

View Slide

Parsing ESM modules
app.mjs
export default 42;
lib.mjs

View Slide

Parsing ESM modules
app.mjs
export default 42;
lib.mjs
$ node --experimental-vm-modules
app.cjs
42
(node:10167) ExperimentalWarning:
VM Modules is an experimental
feature. This feature could change
at any time

View Slide

Key takeaways

View Slide

Key takeaways
● There are dialects of ES Modules:
○ transpiled to CJS and then run
○ fed to Node.js as-is
○ fed to Webpack as autodected ESM
● Lots of things complicate interoperation between
CJS and ESM
● But now your hands remember what is happening
behind the scenes. Nothing to fear.

View Slide

Appendix A
package.json

View Slide

package.json roles in Node.js
Fields Which package.json to use
Module type detection type Nearest ancestor of the resolved module
Package exports exports
main
`segment` or `@scope/segment` part of
the request
Folder main (only in CJS/require) main The requested directory
Package imports imports Nearest ancestor of the requesting
module
Self name
exports
Nearest ancestor of the requesting
module

View Slide

Appendix B
Module polyglot

View Slide

Recap
Snippet to check module types
} else {
}
module-type.js

View Slide

How ESM check works
typeof await /0/["test"] === "function"
typeof (await (/0/["test"])) === "function"
((typeof await) / 0) / ["test"] === "function"
= RegExp.prototype.test
no-op
variable
(nonexistent)
= “undefined” cast to NaN
ESM: [~Yield, +Await, ~Return]
CJS: [~Yield, ~Await, +Return]

View Slide

Advanced application (not recommended for production use!)
Module polyglot
"use strict";
typeof await /[//]/; export default /*
]; module.exports = /**/ 42;
module-polyglot.js

View Slide

Hands-on Native ESM @ JSConf JP 2022

Hands-on Native ESM @ JSConf JP 2022

Masaki Hara

More Decks by Masaki Hara

Other Decks in Programming

Featured

Transcript