JavaScriptのモジュール解決の相互運用性 / JSConf JP 2024 - Interoperability of Module Resolutions in JavaScript

https://jsconf.jp/2024/talk/berlysia/

Transcript

1. JavaScriptの モジュール解決の相互運用性 @berlysia / JSConf JP 2024 Interoperability of Module Resolutions in JavaScript
2. Disclaimer スピーカーは紹介するどのコードベースにおいてもメンテナなどではありません The speaker is not a maintainer of any of the tools mentioned in this talk
3. Agenda - Motivation - Not in this talk - Chapter 1: ESM, CJS and … - Interlude: Should we use Native ESM? Yes, but … - Chapter 2: TypeScript, ESM and "in-place" runtimes - Interlude: Who needs to resolve modules, except runtime and bundler? - Chapter 3: Import path alias / path mapping today
4. Motivation I want to write the import path in a somewhat consistent way that works well. - If it's an application, it should work well at the various runtimes you want to use. - If it is a library, it must be resolvable by the application that wants to use it, and it must be resolvable and work well at multiple target runtimes. - If it is a peripheral toolkit that needs to interpret module resolution, it should work well and be good in a good configuration.
5. Not in this talk - Non-relative paths - “npm:*”, “jsr:*”, “node:*”, “https://*” - Runtime issues - module.exports can not treat with named import - Lack of __esModule in Native ESM, default import/export issue - Dual package hazard - TypeScript issues - See arethetypeswrong/arethetypeswrong.github.io
6. Chapter 1 ESM, CJS and …
7. Module Resolution
8. Who resolves modules in your code? - What runtime runs your artifact? - Browser? Server? Edge? In-app runtime? - How does runtime load it? - Just takes “./src” - Runtime resolves it - Transpiled into “./dist” and keep the file tree - Runtime resolves it (and maybe transpilers also do) - Single bundle - Bundler resolves it - Multiple chunks - Bundler resolves it (and maybe runtimes also do for chunks)
9. Module Systems in JavaScript - CommonJS (CJS) - ECMAScript Modules (ESM)
10. CommonJS (CJS) - `const m = require('module')` style - CommonJSにはちゃんと仕様もありますが、ここではNode.jsの挙動を指して言葉 を使います - The name "CommonJS" can be used to refer to the original module resolution behavior of Node.js - Algorithm: https://nodejs.org/api/modules.html#all-together
11. How CJS works on Node.js
12. ECMAScript Modules (ESM) - `import m from 'module'` style - ESMの仕様は構文と意味論の話だけをしていて、具体的なモジュール解決につい て述べていない - ESM spec says only syntax and semantics - https://tc39.es/ecma262/multipage/ecmascript-language-scripts-and-modules.html#sec-HostL oadImportedModule
13. How ESM generally works - “Extensions everywhere” style - why? - Browsers need it. Deno and Node.js respect that. - "Knocking file" across network is horrible.
14. How the ESM and CJS are determined (simplified) https://nodejs.org/api/packages.html#determining-module-system - Nearest parent package.json “type” field - type: module → *.js file is ESM - type: commonjs(default) → *.js file is CJS - File extensions - *.cjs file is CJS - *.mjs file is ESM - Syntax detection - A file consists only of ESM syntax, the file is ESM
15. "exports” field, Conditional Exports, Subpath Exports “provide a way to map to different paths depending on some conditions” - import: loaded via import - require: loaded via require - module-sync: no top-level-await ESM file https://nodejs.org/api/packages.html#conditional-exports
16. By the way: do you write a code like this? - `import d, { n1, n2 } from "./some/module";` - using Import Statement - without file extension - 🤷 2 patterns here: - Runs as CJS (maybe). Syntax only ESM. The code should be transpiled internally or explicitly. - “Fake ESM” / “Faux ESM” - Runs as ESM. But they resolved like the CJS on Node.js - “Sloppy import” in Deno - Native ESM can be written like this, but let's talk about that later
17. Patterns of Module Resolution Terminology for this slide: - CJS - CommonJS on Node.js. `require(‘path/to/module’)` - Fake ESM - CommonJS (maybe) with ESM import/export syntax. Additional transpile required. - Sloppy ESM - ESM with extension guessing / directory import. - Native ESM - Respect to browsers, “extensions everywhere” style
18. Patterns of Module system adaption CJS Fake ESM Sloppy ESM Native ESM Actually runs as CJS CJS ESM ESM import or require require import import import Extension guessing ✅ ✅ ✅ ❌ Folders as Modules ✅ ✅ ✅ ⚠ Terminology for this slide
19. What are the frameworks doing? When you initialize your code with create-${something}-app like command: - Next.js: Fake ESM - With `type: module` it works as Sloppy ESM. webpack + fullySpecified: false - Vite ( Nuxt.js, Remix, Astro, … ): Sloppy ESM - Vite is based on Rollup. It works as Sloppy ESM or higher - Angular: Fake ESM - With `type: module` it works as Sloppy ESM or higher because of esbuild - Expo: Fake ESM - Metro bundler supports CommonJS only today.
20. CJS Fake ESM Sloppy ESM Native ESM Actually runs as CJS CJS ESM ESM import or require require import import import Extension guessing ✅ ✅ ✅ ❌ Folders as Modules ✅ ✅ ✅ ⚠ Node.js CJS Metro bundler (react-native) Browsers, Deno, Node.js ESM Deno(sloppy) Rollup(Vite) webpack　　　　　　　　　　　　　　　　　 webpack fullySpecified: false Bun, esbuild, TypeScript ～ Volume Zone (maybe) ～
21. Summary of Chapter 1 - There are 2 module system in JavaScript - CommonJS(CJS), ECMAScript Modules(ESM) - How to specify / detect module system - type: module or commonjs - *.cjs, *.mjs - Conditional Exports - Gradient of module system adaption - CJS, Fake ESM, Sloppy ESM, Native ESM
22. Should we use Native ESM (for Application)? - Recommendation: Yes, use Native ESM. Or Sloppy ESM is fine - Node.js can accept Sloppy ESM with custom loader - Sloppy ESM is fine because It’s enough that your runtime can handle them CJS Fake ESM Sloppy ESM Native ESM Actually runs as CJS CJS ESM ESM import or require require import import import Extension guessing ✅ ✅ ✅ ❌ Folders as Modules ✅ ✅ ✅ ⚠
23. Should we use Native ESM (for Application)? - Recommendation: Yes, use Native ESM. Or Sloppy ESM is fine - Node.js can accept Sloppy ESM with custom loader - Sloppy ESM is fine because It’s enough that your runtime can handle them - Tree shaking / static analysis friendly - Better development experience - No bundle in development (Native ESM or transpile time resolution, prebundle) - Reduce dependencies - If libraries support only Native ESM, apps are forced to be Native ESM? - This is relaxed by require(esm) landed - My app is based on Expo, react-native! - Keep going with Fake ESM
24. Should we use Native ESM (for Library)? - Recommendation: Yes, use Native ESM - To achieve broadest interoperability, yes. - Maybe Sloppy ESM would be OK in development, but the results are harder to predict. - Should we also provide CJS version?: Yes, but relaxed by require(esm) - For “Fake ESM” codebases, truly yes - Watch an adaption rate of Node.js, require(esm) supported version CJS Fake ESM Sloppy ESM Native ESM Actually runs as CJS CJS ESM ESM import or require require import import import Extension guessing ✅ ✅ ✅ ❌ Folders as Modules ✅ ✅ ✅ ⚠
25. Should we use Native ESM (for Library)? - Recommendation: Yes, use Native ESM - To achieve broadest interoperability, yes. - Maybe Sloppy ESM would be OK in development, but the results are harder to predict. - Should we also provide CJS version?: Yes, but relaxed by require(esm) - For “Fake ESM” codebases, truly yes - Watch an adaption rate of Node.js, require(esm) supported version - It’s hard to maintain “exports” field properly - Use package bundler like tshy. 📝require(esm) support is not yet - My library is for react-native! - Keep going with Fake ESM
26. Chapter 2 TypeScript, ESM and "in-place" runtimes
27. TypeScript in Native ESM World - Is `import m from "./foo.ts";` style is valid? - This question is super hard especially nowadays
28. Historical Background nowadays - TypeScript 4.7 Native ESM support - Use `import m from “./foo.js”` for foo.ts file. So the answer is invalid at this time. 🔗 - TypeScript’s principle: “preserve JavaScript as written” - To keep Single-File Transpilability, they don't want to rely on context outside the file 🔗 - TypeScript 5.0 `—moduleResolution bundler` and correspondences - TypeScript “in-place” runtimes support, like Bun - We can write `import m from “./foo.ts”` for foo.ts file, under `--allowImportingTsExtensions`. - Instead, it’s forced to set `--noEmit` or `--emitDeclarationOnly` - Node.js v23.0.0 Built-in TypeScript support - Importing TypeScript file requires “extensions everywhere” style evenly CJS for compatibility. - TypeScript 5.7 `—rewriteRelativeImportExtensions` - For users who develop with TypeScript and publish / execute as JavaScript - Should be able to write *.ts extension at development time - But it should be written as "*.js" extension for final artifacts
29. Again: `import m from “./foo.ts”` is valid? - If you can safely set `--allowImportingTsExtensions`: Yes, it’s valid. - This requires `--noEmit` or `--emitDeclarationOnly` - This means 2 patterns - If you are pre-processing all files using a bundler - Most web frontend projects would be this - Often pre-bundled with a server as well - When running at "in-place" runtime
30. When do we use `--rewriteRelativeImportExtensions` - When developing with an "in-place" runtime and the end product is a set of unbundled JS files - And when the conversion is done by tsc - A tool used only by those who fully understand it - Only works with relative paths and extensions of the "ts" series - directory "foo.ts" also becomes "foo.js - Does not work under import path aliases https://x.com/SeaRyanC/status/1840922680725553237
31. Summary of Chapter 2 - When writing in native ESM with TypeScript, the *.ts extension is valid. - If you can safely enable `--allowImportingTsExtensions - Some environments have emerged that can run TypeScript directly. - In the case of Node.js, even if it is CommonJS, you have to write the extension - ⚠ is not stable yet, so be careful in the future. - If you want to end up using *.js, there are more options. - However, import path aliases don't currently work in this case. - There are many edge cases. -
32. Who needs to resolve modules? - Runtimes - Bundlers - TypeScript - ESLint + eslint-plugin-import and forks - Jest, Vitest(same as Vite) - Storybook(webpack or Vite) - Other document generation tools, etc. - Most tools since the CJS days can also resolve ESM import paths. - It’s just like a subset without Extension guessing / Folders as modules
33. Chapter 3 Import path alias / path mapping today
34. Representative examples: - Setup bundlers - Vite, webpack: “resolve.alias” field or plugin - Setup static analyzers - TypeScript: tsconfig.json “paths” field - ESLint + eslint-plugin-import(-x): install suitable `eslint-import-resolver-${resolverName}` - Setup a testing tool - Jest: “moduleNameMapper” or “resolver” option - Vitest: same as Vite We need to maintain them all coming out the same. 🤯 Configuring the "import path alias" is a boring task
35. We can treat it easier with TypeScript Use a plugin to treat tsconfig.json (or jsconfig.json) as Single Source of Truth - Setup bundlers - Vite: vite-tsconfig-paths, @rollup/plugin-typescript - esbuild: built-in tsconfig/jsconfig support - webpack: tsconfig-paths-webpack-plugin - Setup static analyzers - TypeScript: tsconfig.json “paths” field - ESLint + eslint-plugin-import(-x): eslint-import-resolver-typescript - Setup a testing tool - Jest: ts-jest-resolver - Vitest: same as Vite - Setup for runtime - tsc-alias, if you don’t use bundler
36. Overview of tsconfig.json “paths” field Fallback
37. package.json “imports” field, Subpath imports Added in v14.6.0, 2020-07 No fallback Accept array, but
38. Subpath imports support is widespread - webpack 2021-01 - esbuild v0.13.9 2021-10 - TypeScript v4.7, 2022-03 - Vite v4.2.0, 2023-05 - Is it a good idea to make this feature an SSoT?
39. The ability to express the complexity of a path is enough Mapping expressiveness classificaion - Exact match replacement: "#foo" -> "./src/foo.js" - An alias of an external module is covered in a different way - Prefix match replacement: "#components/*" -> "./src/components/*" - All of “module resolver” support them - Single Capturing: "#components/*.js" -> "./src/components/*.alias" - TypeScript “paths” field and Node.js subpath imports can handle this - This class can rewrite extensions in a path - Multiple Capturing, Regular Expressions or JavaScript functions - You can do everything with this class
40. Subpath imports behaviors for “paths” field users - It applies to a runtime! Node.js can handle them - All alias should starts with “#” - “#/” is exceptionally prohibited. “@” and “~” are prohibited. why? - Cleanly distinguishable within Node.js import path resolution algorithm and absolutely no match to existing packages - `@` is used in a scoped module, `~` is also used in the home directory - 📝 `#` reminds us of URL fragments, private fields, etc. - Just rewrite! - No fallback for multiple entries - No extension guessing / Folders as modules, evenly for CommonJS - This behavior matches to Import Maps spec 🔗 - Folders as modules is replaceable by package.json “exports” field
41. Which is preferable to achieve SSoT for path alias today? - Use subpath imports - Native ESM. No fallback. - Sloppy ESM, Fake ESM, CJS and “extensions everywhere” style also fine - 💡Sloppy ESM + “extensions everywhere” is almost Native ESM - Use tsconfig.json or jsconfig.json - Sloppy ESM, Fake ESM, CJS - If you are using path aliases for limited use, you may be able to safely use subpath imports. - Otherwise, continue to use the paths field for import path consistency in your project. - Fallback behavior required - For react-native users - You cannot rely on subpath imports. Native extensions do not work. - For Deno users - Continue with deno.json "imports" field. It has only exact and prefix matching capabilities.
42. 💡We need official implementation as separate module Either “paths” or “imports” / “exports”: - Literally all of the ecosystem tools that need to resolve modules, they will re-implement an official logic to handle each field. - To handle tsconfig.json: 4 packages, AFAIK - To handle “imports” / “exports”: 2 packages - There are
43. Summary of Chapter 3 - To simplify the configuration of the import path alias - Let's write a central settings file and try to convert from it. - There is 2 options: TypeScript “paths” field, Node.js Subpath imports feature - Subpath imports takes an advantage because it applies to a runtime - However, it will affect the consistency of paths in non-Native ESM projects. - If you could move to "extension everywhere" style, subpath imports can apply them - Or if you can tolerate import paths with and without extensions - If you use Native ESM, you can safely adapt Subpath imports
44. Is this … interoperable? - Yes, I believe - Users are slowly moving toward a Native ESM world. - It's a slow process, because it's not really a problem without much awareness. - Different people are creating different gradients and taking it forward. - The current state, especially Subpath imports behavior is confusing, actually. - I guess Extension guessing and Folders as Modules are used by many people, in many projects. How do you think? - While it would be attractive to be able to resolve paths statically, this currently makes it difficult to accommodate subpath imports. - I'd like to talk to someone who knows what's going on.
45. Wrap-up - There are 4 area, between CJS and ESM - Native ESM is recommended regardless of whether it is apps or libs - It writes in the "extension everywhere" style and does not use “sloppy” behavior - For apps development: Sloppy ESM style is also fine (but) - When writing in Native ESM with TypeScript, the *.ts extension is valid. - Some environments have emerged that can run TypeScript directly. Keep watching. - To simplify the configuration of the import path alias: Single Source of Truth - There is 2 options: TypeScript “paths” field, Node.js Subpath imports feature - If you use Native ESM, you can safely adapt Subpath imports - For non-Native ESM codebases, it's difficult to apply subpath imports. - Keep an eye on what's going on with Subpath imports.
46. Speaker • berlysia ◦ Web engineer (mainly frontend) ◦ 妄想を現実にすることをしている • 株式会社ドワンゴ 教育事業 ◦ Webフロントをやる人 ◦ Webフロントのためにいろいろやる人 • TSKaigiの中の人 ◦ TypeScriptのカンファレンス TSKaigi 2025 ◦ 5月に開催するのでよろしくね ▪ そのうちいろいろ情報が出ます
47. 📝この後の語り アプリケーションの人たちは - Expoの人はFake ESMでがんばろう - インポートパスエイリアスを使いたいなら - Subpath importsに任せるならNative ESM + Extensions everywhereスタイルにしましょう - TypeScriptを書いているなら pathsでもいい - Sloppy ESMかつRuntimeがNode.jsならどこかでパス変換が必要になる - TypeScriptじゃない場合もjsconfig.jsonにpathsを書くといけたりする - そうじゃない人たちはしばらく Sloppy ESMでいられそう - しかし便利だった Folders as ModulesはNode.js的にはlegacyです ライブラリの人たちは Native ESM + CJSのDual packageが良い Subpath imports以外の方法を使う場合はエイリアスを最終成果物に残さない Subpath importsを使うのならconditional exportを駆使して出しわけるとよい TypeScriptのpathsや、Node.jsのimports/exportsの挙動は、別ライブラリに切り出されていて欲しい 世界の損 失なので 時間が余ったらそもそもの是非の話をするつもりだった（がたりない）