over the next few months to get us to a version 1.0 release. Getting to 1.0 will be a significant milestone for the project, and one that reflects the fact many people are already using Node-RED in production. Please send any feedback to the Node-RED mailing list or Slack team https://groups.google.com/forum/#!forum/node-red https://nodered.org/slack
Support modern development practices. Integration with CI/CD pipelines. Collaboration Working within a team of developers. Sharing reusable assets. Reviewing changes. Scaling Handling workloads of increasing size. Supporting both horizontal and vertical scaling. Providing the tools to manage a system at scale. API Stability Giving confidence the API exposed can be developer against for the long-term. Extensibility Ensuring both the runtime and editor provide the necessary extension points for them to be customised and built upon Themes These are high level themes this roadmap focuses on. They reflect the priorities we identified through feedback from the community and our own experiences.
a single instance A Team collaborating but developing locally Hit deploy, test locally, push to remote git when ready Hit deploy, straight to running in production Push to remote, trigger CI that pushes to runtimes Push to remote, trigger CI that pushes to testing env Promote flows from testing to production Deploy to 10,000 devices Cloud hosted editor, multiple runtimes Usage Scenarios to enable These are the sorts of usage scenarios we want to enable through this roadmap. Node-RED won’t solve these scenarios entirely, but it will be much easier to integrate with existing solutions to achieve them.
Version Control x x Library Redesign x x Split Editor/Runtime Packaging x x x Subflow Instance Properties x x Persistable Context x x x Pluggable Routing x x x Node Messaging API x x Epic Features and their Themes We have identified the following features as being the main priorities in order to successfully address the themes.
represent a complete, redistributable Node-RED application. • The Runtime runs a single project at any one time, but the Editor provides an easy way to switch between projects. • A project exists locally on disk, but can be linked to a remote git repository Feature: Projects .node-red/ ├── node_modules ├── package.json ├── projects │ └── my-project │ ├── .git │ ├── flow.json │ ├── flow_cred.json │ ├── package.json │ ├── README.md │ └── settings.json └── settings.js Workflow • Collaboration • Extensibility • API Stability • Scaling
their work and to revert back to previous versions easily. • Allow a team of developers to collaborate on a single set of flows in a controlled manner; each developing locally, but sharing changes and maintaining a history. Feature: Version Control Workflow • Collaboration • Extensibility • API Stability • Scaling
editor. • Allow different library sources to be added; both local-file system based and git repository based. • Allow a team of developers to share reusable assets – provide a common set of flows. Feature: Library Redesign Workflow • Collaboration • Extensibility • API Stability • Scaling
parts: • node-red-core-runtime • node-red-core-nodes • node-red-core-editor • The existing node-red package will become a meta-package that pulls them together to deliver Node-RED as it is used today. Feature: Split Editor/Runtime Packaging runtime editor HTTP Admin API Editor Runtime API core-nodes Workflow • Collaboration • Extensibility • API Stability • Scaling
subflow, whilst retaining the common subflow template. • Building on that, a subflow can be saved to a library and becomes a versioned thing. If the library copy is updated, the editor notifies the user and lets them substitute the updated version into their flow – with their customisations preserved. Feature: Subflow Instance Properties Workflow • Collaboration • Extensibility • API Stability • Scaling
key/value store to be used to persist context variables with a flow. • The current get/set context API is synchronous. Need to consider how that maps to a possibly asychronous external API. • The get/set API may also need to be extended to provide a core set of atomic actions such as ‘increment value’ Feature: Persistable Context count: 23 fruit: [“apple”, “banana”] flow context count: 23 fruit: [“apple”, “banana”] Workflow • Collaboration • Extensibility • API Stability • Scaling
the runtime has no way to know when the node has finished processing the message. • This poses some challenges for systems that require some level of message handling assurance – knowing that a message has been properly handled and not silently dropped. • This feature updates the API nodes implement so they can notify the runtime when they have finished with a message. • The runtime will continue to support the existing API as it will take some time for all 1000+ nodes to be updated. Feature: Node Messaging API Workflow • Collaboration • Extensibility • API Stability • Scaling https://github.com/node-red/node-red/wiki/Design:-Node-Messaging-API
one node to the next will become a pluggable component of the runtime. • This will enable a number of use cases, such as: • Flow Debugger – a router that allows breakpoints to be set • Custom low-level logging of messages/events • Distributing flows across multiple runtime instances Feature: Pluggable Routing Workflow • Collaboration • Extensibility • API Stability • Scaling
- Projects - Version control - Library redesign Themes: - Extensibility - Collaboration - API Stability - Scaling Features: - Split editor/runtime packaging - Subflow Instance properties - Node Messaging API Themes: - Scaling - Extensibility Features: - Persistable context - Pluggable routing 0.20 This timeline proposes how the features will be staged across releases. The precise timings of these releases is not yet known, but the goal is to get to v1.0 by the end of 2017.
It stores saved flows on disk but doesn’t provide any way for a team of developers to automatically share those assets. • Flow JSON can be exported/imported via the clipboard, but that is an entirely manual process • If an asset is updated, there is no convenient way to update instances that use it Library Redesign : As-Is Issues Workflow • Collaboration • Extensibility • API Stability • Scaling
snippets for later reuse • A team of developers want to have a common catalog of reusable assets that can be versioned. • A developer wants to be able to pick-up new versions of an asset as easily as possible. • A developer wants to customise instances of an asset but retain the ability to update the asset – for example, setting developer-specific credentials in a node. Library Redesign : Use Cases Workflow • Collaboration • Extensibility • API Stability • Scaling
editor. • Allow different library sources to be added; both local-file system based and git repository based. • Allow a team of developers to share reusable assets – provide a common set of flows. Library Redesign Workflow • Collaboration • Extensibility • API Stability • Scaling
The drop-down menu version is removed. The sidebar lets the user pick which library to view, what type of library object and a listing of the contents. TBC: whether the listing is a complete tree or a single folder (with ‘..’ to return up a level). Library Sidebar Examples ├── inject │ └── repeated injects ├── projects │ └── my examples │ ├── get-sensor-data │ ├── convert-temperature │ └── something-else └── another folder Local Flows
the libraries that have been added. They can be local-file-system based, or remote git repositories. TBC: also support a read-only HTTP based catalogue Library Sidebar Examples ├── inject │ └── repeated injects ├── projects │ └── my examples │ ├── get-sensor-data │ ├── convert-temperature │ └── something-else └── another folder Local Local Cookbook flows My Team flows Community library
things. Aside from Flows, any node can register a library type in the editor – for example the Function and Template nodes. Selecting the Type then provides a filtered view of the library contents. Library Sidebar Examples ├── inject │ └── repeated injects ├── projects │ └── my examples │ ├── get-sensor-data │ ├── convert-temperature │ └── something-else └── another folder Local Flows Functions Templates UI-Templates
preview below. The preview will include any useful meta-data about the entry. Clicking the Import button will add an instance of this entry to the workspace. Library Sidebar Local Convert Temperature Last updated: 22:03 8/7/2017 Import Flows Examples ├── inject │ └── repeated injects ├── projects │ └── my examples │ ├── get-sensor-data │ ├── convert-temperature │ └── something-else └── another folder │ ├── convert-temperature
Library tab in settings is where a user can configure the available libraries. The drop-down lists the configured libraries. Selecting each one shows the relevant configuration information about the library. For example, the default local file system library. Library Settings Local Type: local file system Location: /home/nol/.node-red/lib Add New…
a new library, the Add New button is clicked. Library Settings My Team Flows Type: git repository Location: /home/nol/.node-red/.lib/my-team-flows Add New… Repository: git://example.com/my-team-lib.git
of library is selected Library Settings Local File System Library Type Add new Library Cancel Add Close Local File System Git Repository HTTP Catalogue
local file system library, it is given a name and location on disk. The location defaults to $HOME/.node-red/.lib/ with the name field appended (after being made ‘safe’) Library Settings Local File System Library Type Add new Library Cancel Add Close My second library Name $HOME/.node-red/.lib/my-new-lib Path
git repository, the location and credentials are provided. TBC: more work around authentication. If the underlying git is configured properly, no credentials are needed. An option is to only present credentials challenge if git clone fails due to auth issue. Library Settings Git repository Library Type Add new Library Cancel Add Close git://example.com/my-team-lib.git Repository Username Password
‘Export dialog’. A new button is added to Save to Library. Save to Library Export flow Export to Clipboard Save to Library Cancel selected nodes current flow all flows Export [{"id":"3dbdbbb5.233fc4","type":"change","z":"423676fd.589fc8","na me":"","rules":[{"t":"set","p":"testGlobal","pt":"global","to":"payload","t ot":"msg"}],"action":"","property":"","from":"","to": "", "reg ":false,"x":340 ,"y":260,"wires":[["c53655c3.147608"]]},{"id":"792edc09.641874","typ e":"change","z":"423676fd.589fc8","name":"","rules":[{"t":"set","p":"pa yload","pt":"msg","to":"$flowContext('flowTest') & 'hi'","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":fals e,"x":400,"y":200,"wires":[["16488989.2bc3c6"]]}]
for the entry. A button is added to the path field that can expand a browser. Save to Library Save to Library Save to Library Cancel Local Examples / path / …
path, or select an existing entry to overwrite. Attempting overwrite will prompt a confirmation. Save to Library Save to Library Save to Library Cancel .. inject projects my-flow-one my-flow-two Local ... Examples / path / …
control. This means developers have to manually manage their flow files if they want to put them under versioning. • Manually merging changes from multiple developers at the raw flow file level is not trivial as it involves comparing JSON changes without any context. The Flow Merge tool added in 0.16 is not available to them. • A local developer’s user data directory can become littered with flow files as they switch between different projects. Switching involves manually restarting Node-RED. Projects + Version Control : As-Is Issues Workflow • Collaboration • Extensibility • API Stability • Scaling
under proper version control. They want to be able to commit versions of the flows and rollback to earlier levels. • A team of developers want to collaborate on a set of flows – each developing locally, but using a shared git repository. • A developer wants to quickly switch between different sets of flows they are working on. Projects + Version Control : Use Cases Workflow • Collaboration • Extensibility • API Stability • Scaling
represent a complete, redistributable Node-RED application. • The Runtime runs a single project at any one time, but the Editor provides an easy way to switch between projects. • A project exists locally on disk, but can be linked to a remote git repository Projects .node-red/ ├── node_modules ├── package.json ├── projects │ └── my-project │ ├── .git │ ├── flow.json │ ├── flow_cred.json │ ├── package.json │ ├── README.md │ └── settings.json └── settings.js Workflow • Collaboration • Extensibility • API Stability • Scaling
flow_cred.json – the flow & credential files • package.json – lists any node module dependencies • README.md – documentation for the flow • settings.json – project specific settings • The editor will provide facilities to edit each of these files in the appropriate context. • TBD: A project could contain other files (such as public/html content). Should NR allows you to edit/manage them as well? We don’t want to become a generic IDE. Projects .node-red/ ├── node_modules ├── package.json ├── projects │ └── my-project │ ├── .git │ ├── flow.json │ ├── flow_cred.json │ ├── package.json │ ├── README.md │ └── settings.json └── settings.js Workflow • Collaboration • Extensibility • API Stability • Scaling
• It lists the npm dependencies and any other project metadata. • The dependencies do not get installed in the project directory – they get installed at the top level of the user directory (as they would today). Projects .node-red/ ├── node_modules ├── package.json ├── projects │ └── my-project │ ├── .git │ ├── flow.json │ ├── flow_cred.json │ ├── package.json │ ├── README.md │ └── settings.json └── settings.js Workflow • Collaboration • Extensibility • API Stability • Scaling
are a subset of those available in the top-level settings.js file. TBD: exactly what settings are supported. • The settings file is strictly JSON. It cannot contain code as it must be reloadable without side-effects and be writeable by the runtime. • Introduces the concept of flow-variables: • Extend the existing $(ENV_VAR) support to include properties defined in the settings file. • This makes it easier to deploy a flow to two different environments with environment-specific flow properties. • TBD: describe the workflow for that. Projects .node-red/ ├── node_modules ├── package.json ├── projects │ └── my-project │ ├── .git │ ├── flow.json │ ├── flow_cred.json │ ├── package.json │ ├── README.md │ └── settings.json └── settings.js Workflow • Collaboration • Extensibility • API Stability • Scaling
their work and to revert back to previous versions easily. • Allow a team of developers to collaborate on a single set of flows in a controlled manner; each developing locally, but sharing changes and maintaining a history. Version Control Workflow • Collaboration • Extensibility • API Stability • Scaling
be hidden behind a feature flag so they can disabled if required. Need to define the likely scenarios for where these features may need to be disabled – for example the existing cloud-hosted offerings. • The first time NR runs with Projects enabled, if there is an existing flow-file, the editor will offer to migrate the files to a project. If there is no existing flow-file, the editor offers to create a new project or open an existing one Projects + Version Control Workflow • Collaboration • Extensibility • API Stability • Scaling
flow-file provided, users are presented with a new Welcome screen. They can either: • Create a new Project • Open an existing Project Welcome Screen Create a new project Open an existing project Welcome to Node-RED
optionally point at a git repository that should be used to clone the initial project from. Create a new project my-project Create a new project Project name Create Clone from repository Cancel
optionally point at a git repository that should be used to clone the initial project from. Create a new project – clone from repository my-project Create a new project Project name Create https://example.com/my-project.git Clone from repository Cancel
failed for authentication reasons, the user is prompted for their credentials. The ‘Remember me’ option will save these details, encrypted on disk, and remembered for subsequent requests. Notes: What about SSH Key auth? Do we require the user to configure that so git Just Works? Clone from repository - credentials Clone from repository Username Create Password Cancel Remember me
and the credentials were encrypted, it will then prompt for the encryption key. This key is stored in runtime settings, not the project’s settings file (as that will be version controlled alongside the credential files). Notes: What if the user doesn’t know it? Clone from repository – credentials key Clone from repository Flow Credentials Key Create Cancel
installed in the runtime, it will prompt the user to install them. Skipping this step will leave the flows stopped with ‘unknown’ nodes. Clone from repository – credentials key Clone from repository This project contains nodes you do not have installed. Install them now? Install Skip
created under the top-level projects directory. A project contains the flow configuration (including credentials) and a project-specific settings file. A default README.md is added The settings file is a .json file – not .js. The is an important distinction as it must be reloadable without any side- effects of executed code. That restricts what project-specific settings can be used. For example, functionGlobalContext cannot be used on a per-project basis. But it can be used to define flow- variables Project create on disk .node-red/ ├── node_modules ├── package.json ├── projects │ └── my-project │ ├── .git │ ├── flow.json │ ├── flow_cred.json │ ├── package.json │ ├── README.md │ └── settings.json └── settings.js
a list of available projects. The user selects one and hits Open Open an existing project Open an existing project Open Select a project my-project another-project top-secret-project my-project Cancel
It’s a place to document what the project is about and any other useful information. It uses markdown formatting. Description edit Dependencies Settings
Project Sidebar # This is the project’s README content. It’s a place to document what the project is about and any other useful information. It uses markdown formatting. Description save Settings cancel Dependencies
that comes of a module not in the project’s dependencies list, it will be flagged up with an option to add it. Equally, any dependencies found to be unused can be removed. Project Sidebar Description Dependencies edit node-red-node-random 0.1.1 node-red-contrib-another-node 0.10.1 node-red-contrib-lower-case 7.1.1 base64 Missing dependencies node-red-node-base64 0.1.1 ! add to project node-red-node-unused 0.1.1 remove Unused dependencies Settings
configured. TBD: a workflow for setting ‘local-only’ values for the flow properties – values that are stored locally, but not saved to the projects settings file. This would allow per-developer customisation of the properties whilst developing locally. Project Sidebar Description Dependencies Settings Flow Properties MQTT_HOSTNAME example.com WS_API_KEY ABC_DEF_123 DB_TABLE production_data add
to the project’s flow file. In git terminology, they are un-staged changes. A new option is added to the Deploy menu – ‘Commit changes’ Selecting that option opens ’Version Control’ sidebar (naming tbc). Version Control my-project Commit changes View history
to: - commit unstaged changes - browse local history - view diff - revert to earlier commit - push local commits to remote - browse remote history - pull changes down The ‘Unsaved’ section contains the uncommitted changes. Hovering over each reveals a ‘diff’ and ‘revert’ button. The user selects the files to commit, then enters a message and clicks commit. Version Control Unsaved Local Remote You have uncommitted changes Enter a commit message… Commit flow.json flow_cred.json package.json README.md settings.json
the local repository and shows where the remote repo is up to. The local changes can be pushed to the remote. Version Control You have no uncommitted changes Local – 2 commits to push Remote The latest commit a-user 10:23 7/7/17 A previous commit a-user 10:23 7/7/17 A previous commit a-user 10:23 7/7/17 A previous commit a-user 10:23 7/7/17 A previous commit a-user 10:23 7/7/17 remote push to remote
of the remote repository, and provides a button to pull those changes down. Version Control You have no uncommitted changes Local Remote – 1 commit to pull Someone else’s work ano-user 10:23 7/7/17 A previous commit a-user 10:23 7/7/17 A previous commit a-user 10:23 7/7/17 A previous commit a-user 10:23 7/7/17 A previous commit a-user 10:23 7/7/17 remote pull changes local
knolleary
kimujun
tsumita
msato
sadnessojisan
kakehashi
blueb
kazuhitotakahashi
nikinusu
oracle4engineer
ryuichi1208
tadaken3
addyosmani
jcasabona
akosma
smashingmag
holman
lara
tanoku
brettharned
keithpitt
maggiecrowley
kneath
rmw
