Possible Tools: The Present & Future of Tooling in Node.js

The web's evolution and JavaScript in-the-large has pushed Node.js forward as a platform for writing tools--despite its roots as a framework for event-driven I/O. Node.js has enabled innovators to built tools like Babel, ESLint and TypeScript, which are critical to our development workflows. Yet, if you've ever tried to write a CLI app built on Node.js, you'll have the felt rough edges, gaps in functionality, and reality-distorting cross-platform bad weirdness.

Chris leads the Node.js Tooling Group--a team of commiserating tooling authors--who are sculpting Node.js into a superior platform for building tools. In this talk, Chris will highlight the Tooling Group's recent successes, showcase its current initiatives, and visualize the future of possible tools written in Node.js.

Transcript

1. Christopher Hiller / OpenJS World / June 2020 POSSIBLE TOOLS the present & future of tooling in Node.js
2. CHRISTOPHER HILLER Developer Advocate @ IBM / Node.js core collaborator / Mocha maintainer / OpenJSF CPC voting member / JSParty panelist
3. — https://nodejs.org/about “Node.js is designed to build scalable network applications.”
4. — narrator “It was mostly not used to build scalable network applications.”
5. Node.js builds the web.
6. Node.js is a tooling framework.
7. THE NODE.JS SERVER-TOOLING IMPEDANCE MISMATCH
8. THE NODE.JS SERVER-TOOLING IMPEDANCE MISMATCH A Thing I Just Made Up • Object-Relational Impedance Mismatch • Difficult to map the concepts of a relational database to an object-oriented language, and vice-versa • If you’ve ever used an ORM, you’ll encounter this problem • Node.js is designed to serve network applications, but is mostly used for developer tools • Node.js lacks first-class support for building tools • Heroically, the community builds tools anyway
9. The Node.js server-tooling impedance mismatch can be solved.
10. THE NODE.JS TOOLING GROUP https://github.com/nodejs/tooling graphic by monkik from flaticon.com
11. THE NODE.JS TOOLING GROUP 5 6 Things You Need To Know, Age, Dating, Net Worth 1. Formed in 2018 after frustration with the status quo 2. Members maintain npm, mocha, nyc, yargs, create-react-app & others, and include some Node.js core collaborators 3. The group is active, has landed features, and has current and ongoing initiatives 4. We have a Slack: node-tooling.slack.com 5. Open meetings every other week, streaming on YouTube (pls like & subscribe) 6. Join the meeting at the OpenJS Collaborator’s Summit!
12. NODE.JS TOOLING GROUP Completed Initiatives by Members • Recursive filesystem operations (mkdir, rmdir) • Native source map support • Native code coverage support • Flag introspection • npm workspaces landing soon (not core, but close enough)
13. CURRENT INITIATIVES
14. RELOADABLE MODULES Parity with CJS • You can’t easily reload a ES module like a CJS module • Inhibits module-level mocks (proxyquire, rewiremock, rewire, etc.) • Inhibits tools that must watch files • Current reality will become more painful as ESM is adopted in Node.js applications • Collaborate w/ Node.js Modules WG
15. ARGUMENT PARSING process.argv.slice(2)? Really? • Bare minimum of common features • Good for simple tools, code examples, and one-offs • Not expected to replace userland modules • Example: parseArgs( ['--foo', '--bar=baz', 'quux'] ) // returns { foo: true, bar: 'baz', _: ['quux'] }
16. ULTIMATE MEGAHOOKS The Death of Monkeypatching • Ability to add “hooks” to built-in functions • Possible use cases: • Phony filesystem for package managers • APM & instrumentation • Sandboxing • Mocking • Forming an ad-hoc group to gather requirements: see OpenJS Collaborators Summit schedule!
17. MORE FILESYSTEM OPERATIONS Missing APIs • “Recursive” mode for: • chmod • chown • cpFile • others? • glob support
18. POSSIBLE INITIATIVES
19. WINDOWS PARITY Another casualty of the server-tooling impedance mismatch • More developers use Node.js on Windows than any other OS • Bias: most Node.js core developers use Linux or macOS • We have not heard from either of the developers who deploy Node.js services on Windows • Is Windows your daily driver? Do you write tools? Node.js core and the tooling group need you!
20. FIX FS.WATCH() & FS.WATCHFILE() It bad
21. SELF-CONTAINED DISTRIBUTABLES Node apps without the node • Think pkg: “This command line interface enables you to package your Node.js project into an executable that can be run even on devices without Node.js installed.” (https://npm.im/pkg) • Room for performance improvements, tree-shaking • Are you interested in this problem? See https://github.com/nodejs/tooling/issues/32
22. BUILT-IN COMMAND-LINE TOOLS Portable Scripts • Problem: Shell commands in package.json scripts are not very portable • Because Node.js runs in many environments, it is (more) portable • Solution: Node.js provides CLIs based upon its own built-in modules • Recursive remove directory example: • node --requireBuiltin=rmdir -rf ./some/doomed_directory/ • Python ships with executable modules: • python -mSimpleHTTPServer
23. JAVASCRIPT ITSELF Participation in TC39 • TC39 is the team responsible for designing ECMAScript (JavaScript) the language • Tooling authors must participate if they are to have representation • Proposals to watch: • Binary AST: https://github.com/tc39/proposal-binary-ast • Realms: https://github.com/tc39/proposal-realms
24. CHRISTOPHER HILLER Developer Advocate, IBM boneskull@boneskull.com github.com/boneskull twitter.com/b0neskull changelog.com/jsparty
25. Thank you!