Unlocking Lockfiles - A Developers Guide to Package Management

by Callum Silcock

Slide 1

Slide 1 text

UNLOCKING LOCKFILES A DEVELOPERS GUID PACKAGE MANAGEM Another CodingWithCallum ses

Slide 2

Slide 2 text

WHAT ARE W COVERING?

Slide 3

Slide 3 text

WHAT ARE W COVERING? • History

Slide 4

Slide 4 text

WHAT ARE W COVERING? • History • Fundamentals

Slide 5

Slide 5 text

HISTORY LESSON

Slide 6

Slide 6 text

HISTORY LESSON NEED (>2010

Slide 7

Slide 7 text

HISTORY LESSON NEED (>2010 • include libraries via

Slide 8

Slide 8 text

HISTORY LESSON NEED (>2010 • include libraries via • copypasta code from stackover

Slide 9

Slide 9 text

HISTORY LESSON NEED (>2010 an extremely embarrassing exam

Slide 10

Slide 10 text

HISTORY LESSON BIRTH (2010

Slide 11

Slide 11 text

HISTORY LESSON BIRTH (2010 • npm is created

Slide 12

Slide 12 text

HISTORY LESSON BIRTH (2010 • npm is created • Originally designed for No

Slide 13

Slide 13 text

HISTORY LESSON BIRTH (2010 • npm is created • Originally designed for No • Created standardised regis

Slide 14

Slide 14 text

HISTORY LESSON BIRTH (2010 • npm is created • Originally designed for No • Created standardised regis • package.json created

Slide 15

Slide 15 text

HISTORY LESSON BIRTH (2010 another... extremely embarrassing e

Slide 16

Slide 16 text

HISTORY LESSON BIRTH (2010

Slide 17

Slide 17 text

HISTORY LESSON BIRTH (2010 • Recursive dependency resol

Slide 18

Slide 18 text

HISTORY LESSON BIRTH (2010 • Recursive dependency resol • 🐢🐢🐢🐢

Slide 19

Slide 19 text

HISTORY LESSON BIRTH (2010 • Recursive dependency resol • 🐢🐢🐢🐢 • Nested node_modules

Slide 20

Slide 20 text

HISTORY LESSON BIRTH (2010 callums-bad-code/ ├── node_modules/ │ └── A/ │ ├── package.json │ └── node_modules/ │ └── B/ │ ├── package.json │ └── node_modules/ │ └── C/ │ └── package.json

Slide 21

Slide 21 text

HISTORY LESSON BIRTH (2010

Slide 22

Slide 22 text

HISTORY LESSON BIRTH (2010 • This was pretty revolutionary

Slide 23

Slide 23 text

HISTORY LESSON BIRTH (2010 • This was pretty revolutionary • node_modules-black-ho

Slide 24

Slide 24 text

HISTORY LESSON BIRTH (2010 • This was pretty revolutionary • node_modules-black-ho • Poor windows

Slide 25

Slide 25 text

HISTORY LESSON RISE (2012-20

Slide 26

Slide 26 text

HISTORY LESSON RISE (2012-20 • Bower / RequireJS / Browserfy / Gru

Slide 27

Slide 27 text

HISTORY LESSON RISE (2012-20 • Bower / RequireJS / Browserfy / Gru • npm gets used for frontend things al

Slide 28

Slide 28 text

HISTORY LESSON RISE (2012-20 • Bower / RequireJS / Browserfy / Gru • npm gets used for frontend things al • node_modules folder becomes tru

Slide 29

Slide 29 text

HISTORY LESSON FLAT (2016)

Slide 30

Slide 30 text

HISTORY LESSON FLAT (2016) • yarn released (from Facebook

Slide 31

Slide 31 text

HISTORY LESSON FLAT (2016) • yarn released (from Facebook ▪ lockfiles are born

Slide 32

Slide 32 text

HISTORY LESSON FLAT (2016) • yarn released (from Facebook ▪ lockfiles are born ▪ oﬀline mode

Slide 33

Slide 33 text

HISTORY LESSON FLAT (2016) • yarn released (from Facebook ▪ lockfiles are born ▪ oﬀline mode ▪ security (spoliers!)

Slide 34

Slide 34 text

HISTORY LESSON FLAT (2016) • yarn released (from Facebook ▪ lockfiles are born ▪ oﬀline mode ▪ security (spoliers!) • npm creates package-lock.

Slide 35

Slide 35 text

HISTORY LESSON FLAT (2016) • yarn released (from Facebook ▪ lockfiles are born ▪ oﬀline mode ▪ security (spoliers!) • npm creates package-lock. • flat dependency to avoid duplic

Slide 36

Slide 36 text

HISTORY LESSON FLAT (2016) node_modules/ ├── package-a/ │ └── node_modules/ │ └── package-c/ (version 1.0.0) └── package-b/ └── node_modules/ └── package-c/ (version 1.0.0)

Slide 37

Slide 37 text

HISTORY LESSON FLAT (2016) node_modules/ ├── package-a/ ├── package-b/ └── package-c/ (version 1.0.0, shared by both pa

Slide 38

Slide 38 text

HISTORY LESSON FLAT (2016) node_modules/ ├── package-a/ ├── package-b/ │ └── node_modules/ │ └── package-c/ (version 2.0.0) └── package-c/ (version 1.0.0, used by package-a

Slide 39

Slide 39 text

HISTORY LESSON FLAT (2016) yarn's new security model

Slide 40

Slide 40 text

HISTORY LESSON FLAT (2016) yarn's new security model • checksums

Slide 41

Slide 41 text

HISTORY LESSON FLAT (2016) yarn's new security model • checksums • oﬀline mode

Slide 42

Slide 42 text

HISTORY LESSON FLAT (2016) yarn's new security model • checksums • oﬀline mode • resolution

Slide 43

Slide 43 text

HISTORY LESSON FLAT (2016) yarn's new security model • checksums • oﬀline mode • resolution • license checks

Slide 44

Slide 44 text

HISTORY LESSON FLAT (2016) yarn's new security model • checksums • oﬀline mode • resolution • license checks • script execution

Slide 45

Slide 45 text

HISTORY LESSON FLAT (2016) yarn's new security model • checksums • oﬀline mode • resolution • license checks • script execution • auditing

Slide 46

Slide 46 text

HISTORY LESSON FLAT (2016) Phantom dependencies (I ain't afraid of (Using things not listed in package Your package.json: { "dependencies": { "package- Package A's package.json: { "dependencies": { "p node_modules/ ├── package-a/ └── package-b/ (hoisted from package-a/node_mod

Slide 47

Slide 47 text

HISTORY LESSON MODERN (201

Slide 48

Slide 48 text

HISTORY LESSON MODERN (201 • pnpm created

Slide 49

Slide 49 text

HISTORY LESSON MODERN (201 • pnpm created ▪ content-addressable stor

Slide 50

Slide 50 text

HISTORY LESSON MODERN (201 • pnpm created ▪ content-addressable stor ▪ symlink all the things

Slide 51

Slide 51 text

HISTORY LESSON MODERN (201 • pnpm created ▪ content-addressable stor ▪ symlink all the things • solved "phantom" dependen

Slide 52

Slide 52 text

HISTORY LESSON MODERN (201 • pnpm created ▪ content-addressable stor ▪ symlink all the things • solved "phantom" dependen • yarn (v2 berry) releases Plug

Slide 53

Slide 53 text

HISTORY LESSON MODERN (201 • pnpm created ▪ content-addressable stor ▪ symlink all the things • solved "phantom" dependen • yarn (v2 berry) releases Plug • Deno switches to URL import

Slide 54

Slide 54 text

HISTORY LESSON MODERN (201 pnpm Content-Addressable Sto

Slide 55

Slide 55 text

HISTORY LESSON MODERN (201 pnpm Content-Addressable Sto • Global store in ~/.pnpm-stor

Slide 56

Slide 56 text

HISTORY LESSON MODERN (201 pnpm Content-Addressable Sto • Global store in ~/.pnpm-stor ▪ All packages x versions are s

Slide 57

Slide 57 text

HISTORY LESSON MODERN (201 pnpm Content-Addressable Sto • Global store in ~/.pnpm-stor ▪ All packages x versions are s • Hash-based addressing

Slide 58

Slide 58 text

HISTORY LESSON MODERN (201 pnpm Content-Addressable Sto • Global store in ~/.pnpm-stor ▪ All packages x versions are s • Hash-based addressing ▪ Unique + Free Integrity chec

Slide 59

Slide 59 text

HISTORY LESSON MODERN (201 pnpm Content-Addressable Sto • Global store in ~/.pnpm-stor ▪ All packages x versions are s • Hash-based addressing ▪ Unique + Free Integrity chec • Immutable

Slide 60

Slide 60 text

HISTORY LESSON MODERN (201 pnpm Symlinks

Slide 61

Slide 61 text

HISTORY LESSON MODERN (201 pnpm Symlinks • Symlink local to the Global

Slide 62

Slide 62 text

HISTORY LESSON MODERN (201 pnpm Symlinks • Symlink local to the Global • Strict node module structur

Slide 63

Slide 63 text

HISTORY LESSON MODERN (201 pnpm Symlinks • Symlink local to the Global • Strict node module structur • Multi-level linking

Slide 64

Slide 64 text

HISTORY LESSON MODERN (201 pnpm Symlinks • Symlink local to the Global • Strict node module structur • Multi-level linking ▪ symlinks on top of sym

Slide 65

Slide 65 text

HISTORY LESSON MODERN (201 node_modules/ ├── express -> ./.pnpm/[email protected]/node_modul └── .pnpm/ ├── [email protected]/ │ └── node_modules/ │ ├── express/ (actual link to global │ ├── body-parser -> ../../body-parser │ └── ... (other express dependencies) ├── [email protected]/ │ └── node_modules/ │ ├── body-parser/ (actual link to gl │ └── ... (body-parser dependencies) └── ... (other packages)

Slide 66

Slide 66 text

HISTORY LESSON MODERN (201 yarn (v2 berry) Plug'n'play

Slide 67

Slide 67 text

HISTORY LESSON MODERN (201 yarn (v2 berry) Plug'n'play • Solves the same problems I just

Slide 68

Slide 68 text

HISTORY LESSON MODERN (201 yarn (v2 berry) Plug'n'play • Solves the same problems I just • Peer dependency issues

Slide 69

Slide 69 text

HISTORY LESSON MODERN (201 yarn (v2 berry) Plug'n'play • Solves the same problems I just • Peer dependency issues • Zero install

Slide 70

Slide 70 text

HISTORY LESSON TODAY (2024

Slide 71

Slide 71 text

HISTORY LESSON TODAY (2024 • Package management is everywhere

Slide 72

Slide 72 text

HISTORY LESSON TODAY (2024 • Package management is everywhere • Security concerns are now just supp attacks

Slide 73

Slide 73 text

HISTORY LESSON TODAY (2024 • Package management is everywhere • Security concerns are now just supp attacks • Monorepo management

Slide 74

Slide 74 text

HISTORY LESSON TODAY (2024 • Package management is everywhere • Security concerns are now just supp attacks • Monorepo management • The rise of bun

Slide 75

Slide 75 text

FUNDAMENTALS

Slide 76

Slide 76 text

FUNDAMENTALS SEMVER

Slide 77

Slide 77 text

FUNDAMENTALS SEMVER • Basic format: MAJOR.MINOR.PATC 2.3.1)

Slide 78

Slide 78 text

FUNDAMENTALS SEMVER • Basic format: MAJOR.MINOR.PATC 2.3.1) • MAJOR: Breaking changes

Slide 79

Slide 79 text

FUNDAMENTALS SEMVER • Basic format: MAJOR.MINOR.PATC 2.3.1) • MAJOR: Breaking changes • MINOR: New features, no breaking c

Slide 80

Slide 80 text

FUNDAMENTALS SEMVER • Basic format: MAJOR.MINOR.PATC 2.3.1) • MAJOR: Breaking changes • MINOR: New features, no breaking c • PATCH: Bug fixes, no new features or changes

Slide 81

Slide 81 text

COMMON VERS RANGES

Slide 82

Slide 82 text

No content

Slide 83

Slide 83 text

COMMON VERS RANGES • Exact version: "react": "17.0.

Slide 84

Slide 84 text

No content

Slide 85

Slide 85 text

COMMON VERS RANGES • Exact version: "react": "17.0. ▪ Only use exactly version 17.0.2

Slide 86

Slide 86 text

No content

Slide 87

Slide 87 text

COMMON VERS RANGES • Exact version: "react": "17.0. ▪ Only use exactly version 17.0.2 • Caret (^): "react": "^17.0.2"

Slide 88

Slide 88 text

No content

Slide 89

Slide 89 text

COMMON VERS RANGES • Exact version: "react": "17.0. ▪ Only use exactly version 17.0.2 • Caret (^): "react": "^17.0.2" ▪ Allow updates to any 17.x.x vers 18.0.0 (MINOR / PATCH)

Slide 90

Slide 90 text

No content

Slide 91

Slide 91 text

COMMON VERS RANGES • Exact version: "react": "17.0. ▪ Only use exactly version 17.0.2 • Caret (^): "react": "^17.0.2" ▪ Allow updates to any 17.x.x vers 18.0.0 (MINOR / PATCH) • Tilde (~): "react": "~17.0.2"

Slide 92

Slide 92 text

No content

Slide 93

Slide 93 text

Slide 94

Slide 94 text

No content

Slide 95

Slide 95 text

Slide 96

Slide 96 text

• Wildcard (*): "react": "17.*.* "react": "17.*"

Slide 97

Slide 97 text

Slide 98

Slide 98 text

• Wildcard (*): "react": "17.*.* "react": "17.*" ▪ Any version starting with 17

Slide 99

Slide 99 text

UN-COMMON VERSION RANG

Slide 100

Slide 100 text

No content

Slide 101

Slide 101 text

UN-COMMON VERSION RANG • Greater than (>): "react": ">17

Slide 102

Slide 102 text

No content

Slide 103

Slide 103 text

UN-COMMON VERSION RANG • Greater than (>): "react": ">17 ▪ Any version higher than 17.0.0

Slide 104

Slide 104 text

No content

Slide 105

Slide 105 text

UN-COMMON VERSION RANG • Greater than (>): "react": ">17 ▪ Any version higher than 17.0.0 • Greater than or equal (>=): "react ">=17.0.0"

Slide 106

Slide 106 text

No content

Slide 107

Slide 107 text

UN-COMMON VERSION RANG • Greater than (>): "react": ">17 ▪ Any version higher than 17.0.0 • Greater than or equal (>=): "react ">=17.0.0" ▪ Version 17.0.0 or higher

Slide 108

Slide 108 text

No content

Slide 109

Slide 109 text

UN-COMMON VERSION RANG • Greater than (>): "react": ">17 ▪ Any version higher than 17.0.0 • Greater than or equal (>=): "react ">=17.0.0" ▪ Version 17.0.0 or higher • Less than (<): "react": "<18.0.

Slide 110

Slide 110 text

No content

Slide 111

Slide 111 text

Slide 112

Slide 112 text

No content

Slide 113

Slide 113 text

Slide 114

Slide 114 text

No content

Slide 115

Slide 115 text

Slide 116

Slide 116 text

▪ Between 16.0.0 and 18.0.0 (exclu

Slide 117

Slide 117 text

Slide 118

Slide 118 text

▪ Between 16.0.0 and 18.0.0 (exclu • OR: "react": "15.0.0 || 16

Slide 119

Slide 119 text

Slide 120

Slide 120 text

▪ Between 16.0.0 and 18.0.0 (exclu • OR: "react": "15.0.0 || 16 ▪ Either exactly 15.0.0 or 16.0.0

Slide 121

Slide 121 text

FUNDAMENTALS LOCKFILE

Slide 122

Slide 122 text

FUNDAMENTALS LOCKFILE • Locking / Freezing your dependenci

Slide 123

Slide 123 text

FUNDAMENTALS LOCKFILE • Locking / Freezing your dependenci • Same immutable state of dependen

Slide 124

Slide 124 text

FUNDAMENTALS LOCKFILE • Locking / Freezing your dependenci • Same immutable state of dependen • Stop drift of dependencies between

Slide 125

Slide 125 text

FUNDAMENTALS UPDATING TH LOCKFILE

Slide 126

Slide 126 text

FUNDAMENTALS UPDATING TH LOCKFILE • Install dependencies within defin

Slide 127

Slide 127 text

FUNDAMENTALS UPDATING TH LOCKFILE • Install dependencies within defin ▪ "react": "^18.3.1"

Slide 128

Slide 128 text

FUNDAMENTALS UPDATING TH LOCKFILE • Install dependencies within defin ▪ "react": "^18.3.1" ◦ registry has react@18

Slide 129

Slide 129 text

FUNDAMENTALS UPDATING TH LOCKFILE • Install dependencies within defin ▪ "react": "^18.3.1" ◦ registry has react@18 • Update lockfile to point to react

Slide 130

Slide 130 text

FUNDAMENTALS UPDATING TH LOCKFILE • Install dependencies within defin ▪ "react": "^18.3.1" ◦ registry has react@18 • Update lockfile to point to react • package.json stays the same

Slide 131

Slide 131 text

FUNDAMENTALS RENOVATE

Slide 132

Slide 132 text

FUNDAMENTALS RENOVATE • Update the package.js

Slide 133

Slide 133 text

FUNDAMENTALS RENOVATE • Update the package.js ▪ Within defined range

Slide 134

Slide 134 text

FUNDAMENTALS RENOVATE • Update the package.js ▪ Within defined range • Update lockfile

Slide 135

Slide 135 text

PHEW we made it

Slide 136

Slide 136 text

QUESTIONS • How does pnpm handle workspace resolution? • You didn't go into peer dependencie them? • How is your beard not grey by now?