Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Stabilizing the npm registry
Search
C J Silverio
February 11, 2015
Programming
2
370
Stabilizing the npm registry
How npm went from a car fire to a very boring service with a very boring uptime.
C J Silverio
February 11, 2015
Tweet
Share
More Decks by C J Silverio
See All by C J Silverio
The economics of package management
ceejbot
4
1.5k
The future of (javascript) modules (in node)
ceejbot
1
290
Keeping JavaScript safe
ceejbot
3
440
ceej's how to solve it
ceejbot
6
760
work-life balance at npm
ceejbot
5
790
hash functions and you!
ceejbot
2
350
The accidental noder
ceejbot
2
150
Design Patterns & Modularity in the npm Registry
ceejbot
3
190
Monitoring on a budget
ceejbot
2
290
Other Decks in Programming
See All in Programming
設計やレビューに悩んでいるPHPerに贈る、クリーンなオブジェクト設計の指針たち
panda_program
6
1.7k
第9回 情シス転職ミートアップ 株式会社IVRy(アイブリー)の紹介
ivry_presentationmaterials
1
250
地方に住むエンジニアの残酷な現実とキャリア論
ichimichi
5
1.4k
Bytecode Manipulation 으로 생산성 높이기
bigstark
2
390
datadog dash 2025 LLM observability for reliability and stability
ivry_presentationmaterials
0
190
データの民主化を支える、透明性のあるデータ利活用への挑戦 2025-06-25 Database Engineering Meetup#7
y_ken
0
330
High-Level Programming Languages in AI Era -Human Thought and Mind-
hayat01sh1da
PRO
0
620
PostgreSQLのRow Level SecurityをPHPのORMで扱う Eloquent vs Doctrine #phpcon #track2
77web
2
410
なんとなくわかった気になるブロックテーマ入門/contents.nagoya 2025 6.28
chiilog
1
240
LT 2025-06-30: プロダクトエンジニアの役割
yamamotok
0
600
5つのアンチパターンから学ぶLT設計
narihara
1
130
型付きアクターモデルがもたらす分散シミュレーションの未来
piyo7
0
810
Featured
See All Featured
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
800
We Have a Design System, Now What?
morganepeng
53
7.7k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
45
7.5k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
181
53k
Mobile First: as difficult as doing things right
swwweet
223
9.7k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
Code Review Best Practice
trishagee
68
18k
The Language of Interfaces
destraynor
158
25k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
10
940
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
710
Making Projects Easy
brettharned
116
6.3k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Transcript
stabilizing the registry
C J Silverio director of engineering, npm @ceejbot
This is the story of a plucky package registry named
npm
scaling problem manifesting itself as a stability problem
"scaling" capacity to meet growing demands
"At scale" huge demand & lots of data
"stability" not falling over under normal demand
What's normal demand?
129K packages 239 GB package tarballs 40 million pkg dls/day
1500 req/sec, peak 3200
"Legacy" Anything you've put into production
this is the story of a legacy system becoming more
flexible
None
January 2013 20K packages .5 million dls/day
Oct 2013 44K packages 108 million dls/month 3.6 million dls/day
None
our plucky little registry had to change
step 1: CDN Put Fastly.com in front of the registry
cache rules everything around me
step 2: tarballs get them out of couchdb
tarballs are huge! couch runs better without them base64 decoding
is work.
None
January 2014 60K packages 6+ million dls/day
step 3: visibility are things going wrong? what's going wrong?
reactive monitoring monitor deeply fix things quickly
proactive monitoring self-healing (also things don't break)
monitoring is unit testing Add monitoring after every outage
visibility is a prerequisite but not a solution
act on what monitoring and metrics reveal
step 4: redundancy several CouchDBs! reads, writes, & replication
fewer responsibilities for each piece isolates errors
step 5: automation ansible no server is special
June 2014 Superficially similar.
June 2014 80K packages 10 million dls/day
step 6: simplification now that it's not on fire we
can modify at leisure
None
Nov 2014 105K packages 28 million dls/day peak
50/50 AWS region split no AWS-specific magic Ubuntu 14.04 Trusty
Fastly: geoloc + varnish haproxy + CouchDB nginx + a
filesystem
where's the node?
registry 2 electric boogaloo with 500% more node
None
haproxy + node services couchdb ➜ postgres redis for caching
nginx + filesystem
more complicated more flexible & redundant more scaling dials to
turn
excited about postgres ad-hoc queries are fun
scaling node is exactly like scaling everything else
Understand system get visibility cool down hot spots add redundancy
npm client <3 npm install -g npm@latest
npm loves you