Dependency Management & Yarn - NodeJS Dublin Meetup

An overview of dependency management and the Yarn package manager, a talk given at the NodeJS Dublin meetup of April 26th 2018, hosted by nearForm.

Original deck by Burak Yiğit Kaya.

https://www.meetup.com/Dublin-Node-js-Meetup/events/249736007/

Transcript

2. #nodejsdublin Production Engineer, DCME Dependency Management & Claudio Procida Burak Yigit Kaya UI Engineer, JS Foundation
3. #nodejsdublin • What are dependencies? • Versioning: semver • Versioning: collisions • Dependency managers • Enter Yarn • Performance improvements • Why do we need lock files? • What does Yarn enable? Agenda
4. #nodejsdublin What are dependencies?
5. Dependencies Hello World Single Page App React Webpack Redux Webpack acorn ajv watchpack ajv fast-deep-equal fast-json-stable- stringify json-schema- traverse uri-js uri-js punycode
6. #nodejsdublin • Copy the code into your code base • Will it get modified from the original? • How to upgrade? Will it work after an upgrade? • What about dependencies of dependencies? • Use a “dependencies file” • How to identify? URLs? Versions? • Needs a tool to resolve recursively • Needs a tool to fetch all properly How to include? Dependencies
7. #nodejsdublin Versioning: semver
8. #nodejsdublin • Meaningful and predictable • <major>.<minor>.<patch> • Patch: only fixes • Minor: new features + above • Major: BREAKING CHANGES + above • Still prone to human error but good enough™ Semantic Versioning Versioning: semver
9. #nodejsdublin • Pinning to specific versions are good but not enough • Prevents progress • Requires everyone to watch all updates in all dependencies • The most used versioning range: ^ • ^1.2.3: all minor and patch versions above this • Allows new features and fixes while providing stability • Other notables: ~, *, 1.x.x, <=, >= Version ranges Versioning: semver
10. #nodejsdublin Versioning: collisions
11. #nodejsdublin • You want: • Package A @ version 1.0.0 • Package B @ version 1.0.0 • Package A wants: • Package B @ version 1.1.0 Conflict management Versioning: collisions [Main] A@1.0.0 B@1.1.0 B@1.0.0
12. #nodejsdublin • Force everyone to use the highest/lowest version • Will break if there are major (breaking) version changes • Give everyone what they want • Waste of disk and memory • Hard to track who uses which version • Can cause issues in code when comparing objects • Will almost always work Potential solutions Versioning: collisions
13. #nodejsdublin Dependency managers
14. #nodejsdublin Dependency managers 01 Version ranges to fixed versions Resolve version ranges to latest available and compatible versions of packages 02 Transitive dependencies Get dependencies of dependencies of dependencies of dependencies… 03 Optimize the dependency tree Try to collapse version ranges and reuse already available packages 04 Ensure consistency Ensure consistency across time and systems. AKA reproducible installs 05 Download & Install Download all the packages, cache them and then put them into the right places.
15. #nodejsdublin • Focused on installing a single package • Less parallel, unoptimized, bound by network • Installation order changed the end result • Had a very deep tree without optimization • Was OK for smaller applications Early npm versions (v1 to v4) Dependency managers
16. #nodejsdublin
17. Yarn Performance: everything is done in parallel
18. #nodejsdublin node_modules – Put resolved dependencies on FS yarn.lock – Record all resolved versions package.json – Declare dependencies yarn.lock
19. #nodejsdublin • Easy to review and auto-merge • Only records range resolutions, not the file tree • Still needs package.json • Promises consistency only across same major Yarn version yarn.lock Yarn
20. #nodejsdublin • Build servers should ideally be cut off from the internet • Running an internal registry is not easy • Keeping up to date • Keeping everything vs. only what is needed • Controversial idea: put package tarballs into version control Offline mirror Yarn
21. #nodejsdublin • Sometimes you need to override a transitive dependency • Impossible w/o forking, worse if deeper in the chain • “resolutions” field in “package.json” • Override resolutions in yarn.lock • Life saver during migrations Selective version resolutions Yarn
22. #nodejsdublin • Monorepos are on the rise • Easier internal dependency management • Encourages code separation • Challenge: working across “internal” packages consistently • Automatic linking across workspace directories • Used inside FB monorepo and open source: Babel, React etc. Workspaces Yarn
23. #nodejsdublin Open Source • A separate org from Facebook • Easy to contribute, large impact: ~40% of npm traffic • Come join: https://github.com/yarnpkg/yarn/
24. Questions
25. Thank you