Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Node.js Uncanny Lessons

Node.js Uncanny Lessons

Talk given at DenverJS 3/15/2012
Lessons Learned

Mike Brevoort

March 16, 2012

More Decks by Mike Brevoort

Other Decks in Programming


  1. Node.js uncanny lessons Mike Brevoort @mbrevoort work at... (though opinions

    expressed in this presentation are my own and not meant to represent Pearson)
  2. Node.js uncanny lessons know your powers •don’t try to work

    against the async nature •don’t forget to be an engineer!
  3. Node.js uncanny lessons beware of the diabolical spaghetti code app.get('/bar',

    function(req, res) { foo.fetchSomething(function(error, something) { if(!error) { foo.fetchSomeOne(something, function(error, someone) { if(!error) { foo.fetchBar(function(error, bar) { if(!error) { res.send("we got bar: " + bar); } else { res.send(error.statusCode); } }); } else { res.send(error.statusCode); } }); } else { res.send(error.statusCode); } }); }); it’s easy to write code that looks like this
  4. Node.js uncanny lessons expect errors •typically, on error, node emits

    an ‘error’ event on the corresponding object •if no listeners on object for ‘error’, a top level exception is raised and the process exits
  5. Node.js uncanny lessons expect errors (cont) process.on('uncaughtException', function(error) { console.log("Kaboom....

    handle " + error); }); var server = http.createServer(function (req, res) {}); server.on('error', function(error) { console.log("Caught error! Don't exit!"); }); prudent approach nuclear approach > > however, you’re better off logging and rethrowing!
  6. Node.js uncanny lessons Log Aggressively •The async nature of node

    makes it difficult to diagnose some production problems. •Why? Check out Winston (by ) logging levels + multiple transports
  7. Node.js uncanny lessons don’t fall asleep at the wheel •plan

    for your program to crash! •Automatically restart it with a process monitor •like ‘s Forever or
  8. Node.js uncanny lessons offload computationally intensive tasks •spawn a child

    process require('child_process').spawn() •call out to another system more apt to handle heavy lifting •use a job queue
  9. Node.js uncanny lessons be weary of loops! for (var i=0,

    l=entries.length; i<l; i++) { doSomething(entries[i]); } innocent enough? if # entries = 10,000 doSomething() takes ~1ms you block for 10 seconds!
  10. Node.js uncanny lessons be weary of loops! (cont) // order

    matters function processEntry(entries, index) { index = index || 0; if(index === entries.length) return done(); doSomething(entries[index]); process.nextTick(function() { processEntry(entries, index++) }); } processEntry(entries); non-blocking loops
  11. Node.js uncanny lessons be weary of loops! (cont) non-blocking loops

    // order doesn't matter var leftToProcess = entries.length; for (var i=0, l=entries.length; i<l; i++) { (function(foo) { process.nextTick(function() { doSomething(foo); if(--leftToProcess === 0) { done(); } }); })(entries[i]); }
  12. Node.js uncanny lessons be weary of loops! (cont) or use

    a library like async var async = require('async'); // order doesn't async.forEach(entries, doSomething); // order matters async.forEachSeries(entries, doSomething); // throttled 10 at a time async.forEachLimit(entries, 10, doSomething);
  13. Node.js uncanny lessons Secret Agent Pools • node pools outbound

    http(s) connections by default for each host • default concurrent maxSockets per host is 5 • is this what you want? // disable per request http.get({host:'localhost', port:80, path:'/', agent:false}, function (res) { // Do stuff }) // effectively disable http.globalAgent.maxSockets = 99999;
  14. Node.js uncanny lessons stay lean •watch your heap size •v8

    does stop the world generational GC •profile gc with --trace-gc •Cost of GC is proportional to the number of live objects
  15. Node.js uncanny lessons stay lean (cont) •I try to keep

    heap size < 200mb •Alternatively move data outside of node •instead use Redis, mongodb, CouchDB, etc •encourages statelessness, scalability and durability
  16. Node.js uncanny lessons you can’t do it alone! •don’t build

    monolithic apps •Build loosely coupled, highly cohesive systems
  17. Node.js uncanny lessons unleash your file descriptors! • node can

    handles 1000’s of connections!? • but your OS says... Too many open files • default # file descriptors on most unix systems is 1024 • 1 FD per socket means max open sockets < 1024 • increase the max # of file descriptors • ulimit -n <max # FD> • ulimit -a to see current max
  18. Node.js uncanny lessons git your modules in order •NPM is

    awesome •Maintaining a private NPM registry has been a struggle •Alternatively use Git Module dependencies (one git repo per module) devDependencies: { "eclg-prospero": "git+ssh://[email protected]:PearsonEducation/prospero-node.git#v2.0.0" }
  19. Node.js uncanny lessons count your friends •peg your npm versions

    { "name": "Foo Package", "description": "my Foo package", "version": "1.0.0", "author": "Mike Brevoort <[email protected]>", "dependencies": { "express": "2.3.2", "cluster": ">= 0.6.1", "mongodb": "0.9.x", "connect-gzip": "~0.1", "underscore": "= latest" }, "engines": { "node": "= 0.4.8" } } is this what you really want? really? >
  20. Node.js uncanny lessons check ‘em in • is amazing, but

    check in your module dependencies (in node_modules) •Dependencies of dependencies can change unseeingly! •ignore binaries •npm rebuild on deploy
  21. Node.js uncanny lessons or shrinkwrap ‘em! •npm shrinkwrap (new in

    1.1.2) •Records a full module/ version dependency graph •doesn’t seem to work with git referenced modules
  22. Node.js uncanny lessons Multiplicity •plan to manage multiple node.js versions

    •I prefer n •nvmw on windows $ n 0.4.12 0.4.8 0.6.0 ο 0.6.11 0.6.3 0.6.5 0.6.7 0.7.4 0.7.5 # run with a specific version $ n as 0.6.11 myprogram.js #using bundled npm $ n npm 0.6.11 # install a new version $ n latest $ n 0.6.10
  23. Node.js uncanny lessons if you have a cape, fly •

    use conventions! • check out Felix Geisendorfer’s node style guide http:// nodeguide.com • for example, reserve the first parameter of any callback for an optional error object function foo() { bar.getSomething(callback); } function callback(error, baz) { if (error) return console.log(error.message || error); // something else }
  24. Node.js uncanny lessons buffers are a super friend •great for

    shuffling binary data and streams •not part of the v8 heap so not gc’d (only the reference)
  25. Node.js uncanny lessons these connections aren’t going to clean up

    themselves! app.get('/something', function(req, res, next) { if(req.params.foo) { // go along happy path var foo = getSomething(req.params.foo); res.send(foo); } }); what if this is false?
  26. Node.js uncanny lessons this is supposed to be fun Remember

    If it’s not, maybe it’s time to try Lua??