The accidental noder

or, how I woke up & found myself running the npm registry, interspersed with diverse lessons learned through a series of epic mistakes with the occasional juicy hint about upcoming npm projects

Transcript

1. the accidental noder
2. or, how I woke up & found myself running the npm registry
3. interspersed with diverse lessons learned through a series of epic mistakes
4. with the occasional juicy hint about upcoming npm projects
5. the accidental noder C J Silverio, @ceejbot
6. what does a CTO do, anyway?
7. npm's engineers and and the systems they build are under my care
8. where they are now & where they're going next
9. npm's cli: 3 people the tool you use to install
10. npm's website: 4 people where you look up the packages
11. npm's registry: 5 people storing & serving 350K packages
12. record-scratch noise
13. Woo. What a job. How many times have I done this before?
14. my background looks nothing like this
15. how the heck did this happen? I didn't set out to do this.
16. let's go back in time
17. I've been in the industry since 1989 this means I'm old enough to be your mom
18. imagine me glaring next time somebody says your mom could do something
19. I worked at a bunch of forgotten places General Magic, WebTV, etc
21. Lesson: adapt or get selected out
22. consolation prize: acquisition funemployment!
23. modern web development
24. woo, ruby on rails + backbone!
25. ruby is too slow
26. node 0.4 allegedly fast & v hipster
27. 0.4 was barely usable, but async i/o was what I wanted
28. javascript was a surprise: fast w/ first-class functions
29. npm existed but was different you found modules by reading a wiki page
31. docs were terrible I had to learn by reading source
32. node was the most fun I'd ever had writing code
33. I found my oldest node project and tried to run it
34. Woah. It runs. info: ---------- ljsnarf run started info: Version: 0.1.3 info: Source account: ceej@www.livejournal.com info: Generating session using challenge/response debug: making !at API call with mode: sessiongenerate debug: requesting fresh challenge (node:87517) DeprecationWarning: util.puts is deprecated. Use console.log instead. (node:87517) DeprecationWarning: util.error is deprecated. Use console.error instead. info: Recording userpic keyword info for ceej info: Read cached data from userpics.json. debug: making !at API call with mode: login
35. Voxer: a walkie-talkie app with a 100% node back end
36. 0.6 was the first node that actually worked
37. attempting to evangelize node among people writing Scala
38. at the end of 2013 npm started falling over
39. Lesson: in a viral success costs outscale goodwill
40. @izs contacted me do devops for npm?
41. heck yeah
43. we got it stable then we rewrote it in node & made it scale
44. 2014 was a good year for node
45. node 0.10 was stable npm became stable
46. April 29, 2014
47. October 25, 2016
48. but wait! that was a month ago!
50. 296,334,144 packages installed last Tuesday
51. you install from npm thoughtlessly & this is awesome
53. Lesson: stable platforms are where ecosystems flourish
54. we did this with a tiny team 12 engineers now; half that to start
55. I had no idea when I started that I knew how to do this
57. Lesson: I probably didn't. But I could learn.
58. you can too
59. record-scratch noise again
60. I've learned more in the last 3 years than in my entire previous career
61. you ready? here comes the drop
62. Lesson: automate everything
63. you're a programmer, right? automating things is what we do!
64. automate it so you don't have to think about it any more
65. Lesson: observability is crucial
66. if you can't observe your software you have no idea what it's doing
67. put a metric on it
69. want to steal our metrics system? github.com/numbat-metrics
70. Lesson: measure & iterate ftw
72. try new things incrementally
73. ship a small thing that improves your situation
74. measure & observe improve it incrementally
75. repeat until victory
76. here's a concrete example: npm's current API is a legacy mess
77. want to replace it all with RESTful APIs for registry data
78. a search API and our first React app in front of it
80. minimal change, measure & iterate one little slice of the problem
81. if this works, we'll do it again repeat until victory
82. Lesson: simple is good.
83. if you want to be fast do as little work as possible
84. make data immutable get out of a db & onto a filesystem
85. Lesson: devops is hard (but you gotta do it)
86. the devops skillset is huge
87. the devops mindset is balanced
88. development is optimistic operations are conservative
89. dev: computers woohoo! ops: computers suck.
90. we only run node LTS releases
91. we only upgrade when we want a new JS language feature
92. Lesson: don't get dogmatic about tech
93. performance work ongoing because y'all show no signs of slowing down
94. we're experimenting with Rust for our perf hot spots
95. because we love javascript but it isn't always the right choice
96. minimal change, measure & iterate test Rust with one µservice
97. Lesson: all databases are terrible
98. you can only love a database you've never put into production
99. corollary: your tools are always terriusefulbad
100. now for some harder problems… the non-technical problems
101. Lesson: my job is only sometimes solving hard problems
102. my job: identify the right problem & hand it to the right person
103. Lesson: people are harder than technology
105. the real challenge is keeping people in a place where they can do their best
106. I'm selfish. I build the engineering organization I want to work in.
107. collaborative, constructive, curious
108. I won't know how I did until I can look back in 5 years
109. node is still the most fun I've had writing code
110. I hate it, of course, because we always hate our tools
111. but node is how I write everything from twitter bots to deployment systems
112. I want you to type npm install & have packages flow like spice
113. npm ❤ you