Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Managing multiple npm modules in a single repo

Nick O'Leary
September 25, 2019

Managing multiple npm modules in a single repo

How we tore apart the internals of Node-RED and glued it back together without users knowing.

Presented at the London Node User Group meetup on September 25th, 2019

https://knolleary.net/2019/03/21/moving-node-red-to-a-monorepo/

Nick O'Leary

September 25, 2019
Tweet

More Decks by Nick O'Leary

Other Decks in Programming

Transcript

  1. @knolleary Managing multiple npm modules in a single repo Nick

    O’Leary Node-RED Project Lead IBM Open Source Developer OpenJS Foundation CPC Member
  2. @knolleary How we tore apart the internals of Node-RED and

    glued it back together without users knowing
  3. @knolleary What is Node-RED? Many applications we build today, from

    Integration to IoT, often require pulling together different APIs, online services and hardware devices in new and interesting ways. Time spent figuring out how to access a Serial port, or to complete an OAuth flow against Twitter is not time spent on creating the real value of a solution. We need tools that make it easier for developers at all levels to create applications and allow them to focus on what is important.
  4. @knolleary What is Node-RED? The Node-RED editor runs in a

    browser and allows you to create applications by drawing flows of events. Each node in a flow represents a well-defined piece of functionality, such as updating a value, making a database query or sending a tweet. The developer configures each node as required, focused on what it does, not how it does it. When the flow is deployed to the Node-RED runtime, each node becomes a running piece of code and messages start passing through the flow. “Flow-based programming for event-driven applications”
  5. @knolleary node-red runtime editor HTTP Admin API @node-red/editor-api Client-side Editor

    @node-red/editor-client Flow Runtime @node-red/runtime Node Registry @node-red/registry Common Utils @node-red/util core nodes Core Nodes @node-red/nodes
  6. @knolleary All dependencies, both production and development, are declared in

    the top- level package.json. Each module has its own package.json that lists only its own production dependencies. At development time, npm install is only run at the top level.
  7. @knolleary A list of five things to talk about 1.

    Managing Tests 2. Don’t ignore node_modules 3. Managing package versions 4. Managing dependencies 5. Building a release
  8. @knolleary Managing tests Using same node_modules-in-the-tree trick, we introduced nr-test-utils

    module that can be used by test files to access files in the main src tree. No point publishing as a standalone module as it is inherently tied to the code layout of the repository.
  9. @knolleary Don’t ignore node_modules Don’t put node_modules under version control.

    Unless it’s the node_modules you want to put under version control.
  10. @knolleary Don’t ignore node_modules – GitHub edition GitHub will not

    show commit diffs for any files under node_modules. Unless you ask it nicely - but even then, only on the Desktop view.
  11. @knolleary Managing package versions Script added to set version of

    all modules – keeping them in sync. Includes updating any references in the dependencies section of each package.json
  12. @knolleary Managing dependencies When running locally, dependencies are installed from

    top-level package.json. Risk that the individual module dependency versions will get out of sync.
  13. @knolleary Managing dependencies Script added to validate all dependencies. Added

    to the build so travis catches it when we don’t Two remaining issues: 1. doesn’t check optionalDeps 2. doesn’t catch if we forget to add a brand new dependency
  14. @knolleary Building a release Existing grunt task for building a

    release was updated to build a tgz for each module. They can be tested and manually published to npm Really should write a script to do the publishing – ensuring the right tag is used if it’s a beta release.
  15. @knolleary Credits Nolan Lawson – Why we dropped Lerna from

    PouchDB https://gist.github.com/nolanlawson/457cdb309c9ec5b39f0d420266a9faa4 “alle – monorepo publishing” https://github.com/boennemann/alle
  16. @knolleary Managing multiple npm modules in a single repo Nick

    O’Leary Node-RED Project Lead IBM Open Source Developer http://nodered.org