The accidental noder - Speaker Deck

Slide 1

Slide 1 text

the accidental noder

Slide 2

Slide 2 text

or, how I woke up & found myself running the npm registry

Slide 3

Slide 3 text

interspersed with diverse lessons learned through a series of epic mistakes

Slide 4

Slide 4 text

with the occasional juicy hint about upcoming npm projects

Slide 5

Slide 5 text

the accidental noder C J Silverio, @ceejbot

Slide 6

Slide 6 text

what does a CTO do, anyway?

Slide 7

Slide 7 text

npm's engineers and and the systems they build are under my care

Slide 8

Slide 8 text

where they are now & where they're going next

Slide 9

Slide 9 text

npm's cli: 3 people the tool you use to install

Slide 10

Slide 10 text

npm's website: 4 people where you look up the packages

Slide 11

Slide 11 text

npm's registry: 5 people storing & serving 350K packages

Slide 12

Slide 12 text

record-scratch noise

Slide 13

Slide 13 text

Woo. What a job. How many times have I done this before?

Slide 14

Slide 14 text

my background looks nothing like this

Slide 15

Slide 15 text

how the heck did this happen? I didn't set out to do this.

Slide 16

Slide 16 text

let's go back in time

Slide 17

Slide 17 text

I've been in the industry since 1989 this means I'm old enough to be your mom

Slide 18

Slide 18 text

imagine me glaring next time somebody says your mom could do something

Slide 19

Slide 19 text

I worked at a bunch of forgotten places General Magic, WebTV, etc

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

Lesson: adapt or get selected out

Slide 22

Slide 22 text

consolation prize: acquisition funemployment!

Slide 23

Slide 23 text

modern web development

Slide 24

Slide 24 text

woo, ruby on rails + backbone!

Slide 25

Slide 25 text

ruby is too slow

Slide 26

Slide 26 text

node 0.4 allegedly fast & v hipster

Slide 27

Slide 27 text

0.4 was barely usable, but async i/o was what I wanted

Slide 28

Slide 28 text

javascript was a surprise: fast w/ ﬁrst-class functions

Slide 29

Slide 29 text

npm existed but was different you found modules by reading a wiki page

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

docs were terrible I had to learn by reading source

Slide 32

Slide 32 text

node was the most fun I'd ever had writing code

Slide 33

Slide 33 text

I found my oldest node project and tried to run it

Slide 34

Slide 34 text

Woah. It runs. info: ---------- ljsnarf run started info: Version: 0.1.3 info: Source account: ceej@www.livejournal.com info: Generating session using challenge/response debug: making !at API call with mode: sessiongenerate debug: requesting fresh challenge (node:87517) DeprecationWarning: util.puts is deprecated. Use console.log instead. (node:87517) DeprecationWarning: util.error is deprecated. Use console.error instead. info: Recording userpic keyword info for ceej info: Read cached data from userpics.json. debug: making !at API call with mode: login

Slide 35

Slide 35 text

Voxer: a walkie-talkie app with a 100% node back end

Slide 36

Slide 36 text

0.6 was the ﬁrst node that actually worked

Slide 37

Slide 37 text

attempting to evangelize node among people writing Scala

Slide 38

Slide 38 text

at the end of 2013 npm started falling over

Slide 39

Slide 39 text

Lesson: in a viral success costs outscale goodwill

Slide 40

Slide 40 text

@izs contacted me do devops for npm?

Slide 41

Slide 41 text

heck yeah

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

we got it stable then we rewrote it in node & made it scale

Slide 44

Slide 44 text

2014 was a good year for node

Slide 45

Slide 45 text

node 0.10 was stable npm became stable

Slide 46

Slide 46 text

April 29, 2014

Slide 47

Slide 47 text

October 25, 2016

Slide 48

Slide 48 text

but wait! that was a month ago!

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

296,334,144 packages installed last Tuesday

Slide 51

Slide 51 text

you install from npm thoughtlessly & this is awesome

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

Lesson: stable platforms are where ecosystems ﬂourish

Slide 54

Slide 54 text

we did this with a tiny team 12 engineers now; half that to start

Slide 55

Slide 55 text

I had no idea when I started that I knew how to do this

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

Lesson: I probably didn't. But I could learn.

Slide 58

Slide 58 text

you can too

Slide 59

Slide 59 text

record-scratch noise again

Slide 60

Slide 60 text

I've learned more in the last 3 years than in my entire previous career

Slide 61

Slide 61 text

you ready? here comes the drop

Slide 62

Slide 62 text

Lesson: automate everything

Slide 63

Slide 63 text

you're a programmer, right? automating things is what we do!

Slide 64

Slide 64 text

automate it so you don't have to think about it any more

Slide 65

Slide 65 text

Lesson: observability is crucial

Slide 66

Slide 66 text

if you can't observe your software you have no idea what it's doing

Slide 67

Slide 67 text

put a metric on it

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

want to steal our metrics system? github.com/numbat-metrics

Slide 70

Slide 70 text

Lesson: measure & iterate ftw

Slide 71

Slide 71 text

No content

Slide 72

Slide 72 text

try new things incrementally

Slide 73

Slide 73 text

ship a small thing that improves your situation

Slide 74

Slide 74 text

measure & observe improve it incrementally

Slide 75

Slide 75 text

repeat until victory

Slide 76

Slide 76 text

here's a concrete example: npm's current API is a legacy mess

Slide 77

Slide 77 text

want to replace it all with RESTful APIs for registry data

Slide 78

Slide 78 text

a search API and our ﬁrst React app in front of it

Slide 79

Slide 79 text

No content

Slide 80

Slide 80 text

minimal change, measure & iterate one little slice of the problem

Slide 81

Slide 81 text

if this works, we'll do it again repeat until victory

Slide 82

Slide 82 text

Lesson: simple is good.

Slide 83

Slide 83 text

if you want to be fast do as little work as possible

Slide 84

Slide 84 text

make data immutable get out of a db & onto a ﬁlesystem

Slide 85

Slide 85 text

Lesson: devops is hard (but you gotta do it)

Slide 86

Slide 86 text

the devops skillset is huge

Slide 87

Slide 87 text

the devops mindset is balanced

Slide 88

Slide 88 text

development is optimistic operations are conservative

Slide 89

Slide 89 text

dev: computers woohoo! ops: computers suck.

Slide 90

Slide 90 text

we only run node LTS releases

Slide 91

Slide 91 text

we only upgrade when we want a new JS language feature

Slide 92

Slide 92 text

Lesson: don't get dogmatic about tech

Slide 93

Slide 93 text

performance work ongoing because y'all show no signs of slowing down

Slide 94

Slide 94 text

we're experimenting with Rust for our perf hot spots

Slide 95

Slide 95 text

because we love javascript but it isn't always the right choice

Slide 96

Slide 96 text

minimal change, measure & iterate test Rust with one µservice

Slide 97

Slide 97 text

Lesson: all databases are terrible

Slide 98

Slide 98 text

you can only love a database you've never put into production

Slide 99

Slide 99 text

corollary: your tools are always terriusefulbad

Slide 100

Slide 100 text

now for some harder problems… the non-technical problems

Slide 101

Slide 101 text

Lesson: my job is only sometimes solving hard problems

Slide 102

Slide 102 text

my job: identify the right problem & hand it to the right person

Slide 103

Slide 103 text

Lesson: people are harder than technology

Slide 104

Slide 104 text

No content

Slide 105

Slide 105 text

the real challenge is keeping people in a place where they can do their best

Slide 106

Slide 106 text

I'm selﬁsh. I build the engineering organization I want to work in.

Slide 107

Slide 107 text

collaborative, constructive, curious

Slide 108

Slide 108 text

I won't know how I did until I can look back in 5 years

Slide 109

Slide 109 text

node is still the most fun I've had writing code

Slide 110

Slide 110 text

I hate it, of course, because we always hate our tools

Slide 111

Slide 111 text

but node is how I write everything from twitter bots to deployment systems

Slide 112

Slide 112 text

I want you to type npm install & have packages ﬂow like spice

Slide 113

Slide 113 text

npm ❤ you