Managing multiple npm modules in a single repo

@knolleary Managing multiple npm modules in a single repo Nick
O’Leary Node-RED Project Lead IBM Open Source Developer OpenJS Foundation CPC Member

@knolleary How we tore apart the internals of Node-RED and
glued it back together without users knowing

@knolleary What is Node-RED? Many applications we build today, from
Integration to IoT, often require pulling together different APIs, online services and hardware devices in new and interesting ways. Time spent figuring out how to access a Serial port, or to complete an OAuth flow against Twitter is not time spent on creating the real value of a solution. We need tools that make it easier for developers at all levels to create applications and allow them to focus on what is important.

@knolleary What is Node-RED? The Node-RED editor runs in a
browser and allows you to create applications by drawing flows of events. Each node in a flow represents a well-defined piece of functionality, such as updating a value, making a database query or sending a tweet. The developer configures each node as required, focused on what it does, not how it does it. When the flow is deployed to the Node-RED runtime, each node becomes a running piece of code and messages start passing through the flow. “Flow-based programming for event-driven applications”

@knolleary

@knolleary node-red node.js based runtime browser based editor core nodes

@knolleary

@knolleary node-red runtime editor HTTP Admin API @node-red/editor-api Client-side Editor
@node-red/editor-client Flow Runtime @node-red/runtime Node Registry @node-red/registry Common Utils @node-red/util core nodes Core Nodes @node-red/nodes

@knolleary Goals Be maintainable Not require lots of extra tooling
Keep it all in one repository

@knolleary

@knolleary Git Submodules

@knolleary

@knolleary Roll Our Own

@knolleary

@knolleary All dependencies, both production and development, are declared in
the top- level package.json. Each module has its own package.json that lists only its own production dependencies. At development time, npm install is only run at the top level.

@knolleary A list of five things to talk about 1.
Managing Tests 2. Don’t ignore node_modules 3. Managing package versions 4. Managing dependencies 5. Building a release

@knolleary Managing tests

@knolleary Managing tests Using same node_modules-in-the-tree trick, we introduced nr-test-utils
module that can be used by test files to access files in the main src tree. No point publishing as a standalone module as it is inherently tied to the code layout of the repository.

@knolleary

@knolleary Managing tests

@knolleary Don’t ignore node_modules Don’t put node_modules under version control.
Unless it’s the node_modules you want to put under version control.

@knolleary Don’t ignore node_modules – GitHub edition GitHub will not
show commit diffs for any files under node_modules. Unless you ask it nicely - but even then, only on the Desktop view.

@knolleary Managing package versions Script added to set version of
all modules – keeping them in sync. Includes updating any references in the dependencies section of each package.json

@knolleary Managing dependencies When running locally, dependencies are installed from
top-level package.json. Risk that the individual module dependency versions will get out of sync.

@knolleary Managing dependencies Script added to validate all dependencies. Added
to the build so travis catches it when we don’t Two remaining issues: 1. doesn’t check optionalDeps 2. doesn’t catch if we forget to add a brand new dependency

@knolleary Building a release Existing grunt task for building a
release was updated to build a tgz for each module. They can be tested and manually published to npm Really should write a script to do the publishing – ensuring the right tag is used if it’s a beta release.

@knolleary What’s next? Automate a few more chores around dependency
checking and npm publishing

@knolleary What’s next? Forget about package management and get on
developing Node-RED

@knolleary

@knolleary Credits Nolan Lawson – Why we dropped Lerna from
PouchDB https://gist.github.com/nolanlawson/457cdb309c9ec5b39f0d420266a9faa4 “alle – monorepo publishing” https://github.com/boennemann/alle

@knolleary Managing multiple npm modules in a single repo Nick
O’Leary Node-RED Project Lead IBM Open Source Developer http://nodered.org

Managing multiple npm modules in a single repo

Managing multiple npm modules in a single repo

Transcript

More Decks by Nick O'Leary

Other Decks in Programming

Featured