Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Necessary Performance Measures: npm at Scale
Raquel Vélez
April 30, 2014
Technology
1
110
Necessary Performance Measures: npm at Scale
Talk on caching and distribution at MelbJS meetup in Melbourne, Australia
Raquel Vélez
April 30, 2014
Tweet
Share
More Decks by Raquel Vélez
See All by Raquel Vélez
rockbot
0
95
rockbot
0
110
rockbot
1
91
rockbot
4
410
rockbot
1
490
rockbot
0
74
rockbot
0
420
rockbot
0
250
rockbot
1
270
Other Decks in Technology
See All in Technology
daisukehirama41
1
360
hagifoo
0
100
supership
0
200
thehajime
0
120
lmi
5
3.8k
bolonio
4
580
eayedi
1
110
sei88888
0
450
line_developers
PRO
2
520
ainehanta
0
110
lain21
14
5.4k
harshbothra
0
240
Featured
See All Featured
jrom
116
7.2k
keavy
108
14k
hannesfritz
29
980
tanoku
259
24k
jakevdp
776
200k
garrettdimon
289
110k
caitiem20
311
17k
mza
80
4.2k
holman
448
140k
jnunemaker
PRO
40
4.7k
hatefulcrawdad
257
17k
eileencodes
114
25k
Transcript
Necessary Performance Measures: at Scale Brought to you by @rockbot
Package Management for Node.js • The Open Source Dream !
• ~70k packages • up to 10 million downloads/ day
221,845,731) !10,000!! !20,000!! !30,000!! !40,000!! !50,000!! !60,000!! !70,000!! !80,000!! !90,000!!
!-!!!! !10,000,000!! !20,000,000!! !30,000,000!! !40,000,000!! !50,000,000!! !60,000,000!! !70,000,000!! !80,000,000!! !90,000,000!! !100,000,000!! !110,000,000!! !120,000,000!! !130,000,000!! !140,000,000!! !150,000,000!! !160,000,000!! !170,000,000!! !180,000,000!! !190,000,000!! !200,000,000!! !210,000,000!! !220,000,000!! !230,000,000!! !240,000,000!! 8/29/12! 12/7/12! 3/17/13! 6/25/13! 10/3/13! 1/11/14! 4/21/14! Downloads)of)npm)packages)in)previous)month)) &)Total)number)of)npm)Packages)of)node.js)code) !!!Downloads!of!npm!packages!in!previous!month! !!!Total!Number!of!npm!Packages!
Performance = Perception • Best request vs worst request •
Focus on the few, super-slow instances: improve performance like whoa ! • Caching vs Distribution
Caching
Data Layer Query Caching Database-level performance enhancements ! innoDB buffer
pool, CouchDB’s ACID architecture
Query Layer Save CPU Preprocessed/aggregated data; scheduled handling of data
! CouchDB views, Download Counts
App Layer Per-User Data that doesn’t change per user !
Storing user session data in Redis
Edge Server Layer Centralized/Global CDN, static assets ! Fastly
Client Layer Distributed HTTP headers, local storage, etc. ! Browser
caching
Distribution
Split Load More boxes, more specialized boxes read-only, write-only !
One process per box Redis, CouchDB, ElasticSearch
Add Resiliency Multi-cloud, multi-zone East/West, multiple datacenters ! Fallback to
least effort/ most common usage i.e. read-only
Boost with Third-Party Services Global content distribution network CDN
The End?
Thank You! ! Raquel Vélez @rockbot raquel@ js.com