Emberconf 2017 - Spin me a Yarn

npm is one of the most popular package managers amongst Javascript developers but it is not without its flaws, especially with regards to speed and security. Yarn is a new player on the field that promises to address some of these flaws.

In this talk we will deep-dive into the inner workings of npm and yarn and demystify how they deal with dependency resolution, authenticity of packages and overall performance. A good understanding of these concepts allows you to make better-informed decisions regarding which package manager you should use for your next Javascript application.

Transcript

1. Spin me a Yarn Serena Fritsch @serifritsch serena@intercom.io Emberconf 2017
4. Once upon a time…
5. ๏ First release in 2010
6. ๏ First release in 2010 ๏ NPM Registry
7. ๏ First release in 2010 ๏ NPM Registry ๏ 347184 published packages https://unpm.nodesource.com/
8. ๏ First release in 2010 ๏ NPM Registry ๏ 347184 published packages ๏ 11,164 packages per week https://unpm.nodesource.com/
11. xkdc.com and reddit
12. ๏ Non-Determinism
13. ๏ Non-Determinism ๏ Performance
14. ๏ Uses npm registry
15. ๏ Uses npm registry ๏ Consistent and reliable dependency resolution
16. ๏ Uses npm registry ๏ Consistent and reliable dependency resolution ๏ Improved performance
17. Dependency Resolution? Non-Determinism? Performance?
19. In the beginning…
20. Disclaimer ✓ You use a package manager ✓ You are familiar with the npm eco system
21. What are packages?
22. Piece of software that can be downloaded
23. May depend on other packages
24. What are dependencies? a^1.0.0 b^1.0.0 App
25. Specified inside the package.json a^1.0.0 b^1.0.0 App
26. a^1.0.0 b^1.0.0 App Follow semantic versioning
27. https://xkcd.com/1172/
28. Multiple levels of dependencies a^1.0.0 s^1.0.0 App b^1.0.0 s^2.0.0
29. a^1.0.0 s^1.0.0 App b^1.0.0 s^2.0.0 No dependency conflicts
30. Under the hood
31. Project Code
32. Project Code Manifest +
33. Project Code Manifest + Dependency Code
34. goo.gl/LJSNmP
36. Disclaimer ✓ First-time installation ✓ Empty node_modules folder ✓ No pre-cached packages
37. Install Phases 1. Dependency Resolution Make requests to the registry and look up dependencies recursively
38. Install Phases 2. Fetching Packages Fetch package tarballs and place in global cache
39. Install Phases 3. Linking Packages Copying files from global cache to local node_modules folder
40. https://github.com/ashleygwilliams/npm-sandbox package.json
41. https://github.com/ashleygwilliams/npm-sandbox package.json Dependency Graph a^1.0.0 b^1.0.0 App s^1.0.0 s^2.0.0
42. . . node_modules folder a^1.0.0 b^1.0.0 App s^1.0.0 s^2.0.0
43. . node_modules folder a1 b1 App s1 s2
44. . . a1 s1 b1 s2 a1 b1 App s1 s2
45. . Dependency Resolution 1. Load the existing node_modules tree from disc a1 b1 App s1 s2
46. . Dependency Resolution 2. Clone the current tree a1 b1 App s1 s2
47. . Dependency Resolution 3.Build the ideal tree a1 b1 App s1 s2
48. . Dependency Resolution a1 b1 App s1 s2
49. . Dependency Resolution a1 a1 b1 App s1 s2
50. . Dependency Resolution a1 a1 b1 App s1 s2
51. a1 s1 . Dependency Resolution a1 b1 App s1 s2
52. a1 s1 . Dependency Resolution a1 b1 App s1 s2
53. a1 s1 b1 . . Dependency Resolution a1 b1 App s1 s2
54. a1 s1 b1 . Dependency Resolution a1 b1 App s1 s2
55. . a1 s1 b1 s2 Dependency Resolution a1 b1 App s1 s2
56. . Dependency Resolution 4.Generate Actions to take add a@1.0.0 add b@1.0.0 add s@2.0.0 add s@1.0.0 a1 s1 b1 s2 a1 b1 App s1 s2
57. . Package Fetching and Linking a1 s1 b1 a1 s1 b1 s2
58. . Dependency Resolution 1.Create a list of Package Requests a1 b1 App s1 s2
59. . Dependency Resolution 2. Find Version on Registry a1 b1 App s1 s2
60. . Dependency Resolution 3. Check existing dependencies a1 b1 App s1 s2
61. . Dependency Resolution 4. Create a new Package Request and repeat… a1 b1 App s1 s2 a1
62. . Dependency Resolution a1 b1 App s1 s2 a1
63. a1 s1 . Dependency Resolution a1 b1 App s1 s2
64. a1 s1 . Dependency Resolution a1 b1 App s1 s2
65. a1 s1 b1 . . Dependency Resolution a1 b1 App s1 s2
66. a1 s1 b1 . . Dependency Resolution a1 b1 App s1 s2
67. a1 s1 b1 . Dependency Resolution a1 b1 App s1 s2 a1 s1 b1 s2
68. . Package Fetching and Linking a1 s1 b1 a1 s1 b1 s2
69. . . Save Lockfile a1 s1 b1 a1 s1 b1 s2
70. “To make it more clear, your package.json states “what i want” for the project whereas your lock file says “what I had” in terms of dependencies” -Dan Abramov . . Save Lockfile
71. .
72. Differences
74. a1 b1 App s1 s2 c1 s1 a1 b1 s2 s1 c1 https://docs.npmjs.com/how-npm-works/npm3-nondet
75. Upgrade of Package A a2 b1 App s2 s2 c1 s1
76. a1 b1 s2 s1 c1 a2 b1 App s2 s2 c1 s1
77. a1 b1 s2 s1 c1 a2 b1 App s2 s2 c1 s1
78. a2 b1 s2 s1 c1 a2 b1 App s2 s2 c1 s1
79. a2 b1 s2 s1 c1 a2 b1 App s2 s2 c1 s1
80. a2 b1 s2 s1 c1 a2 b1 App s2 s2 c1 s1 s2
81. a2 b1 s2 s1 c1 a2 b1 App s2 s2 c1 s1 s2
82. a2 b1 s2 s1 c1 a2 b1 App s2 s2 c1 s1 s2
83. a2 b1 s2 s1 c1 a2 b1 App s2 s2 c1 s1 s2
84. a2 b1 s2 s1 c1 a2 b1 App s2 s2 c1 s1 s2
85. a2 b1 s2 s1 c1 a2 b1 App s2 s2 c1 s1 s2
86. a2 b1 s2 s1 c1 a2 b1 App s2 s2 c1 s1 s2
87. a2 b1 s2 s1 c1 a2 b1 App s2 s2 c1 s1 s2
88. a2 b1 App s2 s2 c1 s1 a2 b1 s2 s1 c1 s2
89. a2 b1 App s2 s2 c1 s1
90. a2 b1 App s2 s2 c1 s1
91. a2 b1 App s2 s2 c1 s1 a2
92. a2 b1 App s2 s2 c1 s1 a2
93. a2 s2 a2 b1 App s2 s2 c1 s1
94. a2 s2 a2 b1 App s2 s2 c1 s1
95. a2 b1 s2 a2 b1 App s2 s2 c1 s1
96. a2 b1 s2 a2 b1 App s2 s2 c1 s1
97. a2 b1 s2 a2 b1 App s2 s2 c1 s1
98. a2 b1 s2 a2 b1 App s2 s2 c1 s1
99. a2 b1 App s2 s2 c1 s1 a2 b1 s2 c1
100. a2 b1 App s2 s2 c1 s1 a2 b1 s2 c1
101. a2 b1 App s2 s2 c1 s1 a2 b1 s2 c1 s1
102. a2 b1 s2 c1 s1 a2 b1 s2 s1 c1 s2
103. Anything I can do? When in doubt, clear node_modules out
104. Save Lockfile npm shrinkwrap
105. Save Lockfile By default turned off
106. Upgrade of Package A a2 b1 App s2 s2 c1 s1
107. a2 b1 App s2 s2 c1 s1
108. a2 b1 App s2 s2 c1 s1 a2
109. a2 b1 App s2 s2 c1 s1 a2
110. a2 s2 a2 b1 App s2 s2 c1 s1
111. a2 s2 a2 b1 App s2 s2 c1 s1
112. a2 b1 s2 a2 b1 App s2 s2 c1 s1
113. a2 b1 s2 a2 b1 App s2 s2 c1 s1
114. a2 b1 s2 a2 b1 App s2 s2 c1 s1
115. a2 b1 s2 a2 b1 App s2 s2 c1 s1
116. a2 b1 App s2 s2 c1 s1 a2 b1 s2 c1
117. a2 b1 App s2 s2 c1 s1 a2 b1 s2 c1
118. a2 b1 App s2 s2 c1 s1 a2 b1 s2 c1 s1
119. a2 b1 s2 c1 s1
120. a2 b1 s2 c1 s1
121. Project Code package.json + + Lockfile
122. Performance
123. Issue opened by Sam Saccone November 2015 https://github.com/npm/npm/issues/10380
124. GET a1 GET b1 GET s1 GET s1 Timeline start 1.0s 1.5s 2.0s 2.5s Multi-Stage Installation
125. GET a1 GET b1 GET s1 GET s2 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
126. GET a1 GET b1 GET s1 GET s1 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
127. GET a1 GET b1 GET s1 GET s2 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
128. GET a1 GET b1 GET s1 GET s2 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
129. GET a1 GET b1 GET s1 GET s2 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
130. GET a1 GET b1 GET s1 GET s2 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
131. GET a1 GET b1 GET s1 GET s2 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
132. GET a1 GET b1 GET s1 GET s2 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
133. GET a1 GET b1 GET s1 GET s2 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
134. GET a1 GET b1 GET s1 GET s2 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
135. GET a1 GET b1 GET s2 Timeline start 1.0s 1.5s 2.0s 2.5s Built-In Parallelism GET s1
136. GET a1 GET b1 GET s2 GET s1 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
137. GET a1 GET b1 GET s2 GET s1 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
138. GET a1 GET b1 GET s2 GET s1 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
139. GET a1 GET b1 GET s2 GET s1 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
140. GET a1 GET b1 GET s2 GET s1 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
141. GET a1 GET b1 GET s2 GET s1 Timeline start 1.0s 1.5s 2.0s 2.5s a1 s1 b1 s2
143. Last Chapter
144. Ember CLI 2.13 onwards is “yarn aware”
145. Migrate your own projects typing yarn https://yarnpkg.com/lang/en/docs/migrating-from-npm/
146. Community project with governance model taken from Ember and Rust Contribute https://github.com/yarnpkg/yarn
147. Watch this Space as well http://blog.npmjs.org
148. And they lived happily ever after…
149. Thank You! @serifritsch serena@intercom.io