Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Node.js Uncanny Lessons

Node.js Uncanny Lessons

Talk given at DenverJS 3/15/2012
Lessons Learned


Mike Brevoort

March 16, 2012


  1. Node.js Mike Brevoort 15 March 2012 uncanny lessons

  2. Node.js uncanny lessons Mike Brevoort @mbrevoort work at... (though opinions

    expressed in this presentation are my own and not meant to represent Pearson)
  3. Node.js uncanny lessons Agenda 22 pieces of advice

  4. Node.js uncanny lessons know your strengths •don’t force everything into

    node Fibonacci you say?!?
  5. Node.js uncanny lessons know your powers •don’t try to work

    against the async nature •don’t forget to be an engineer!
  6. Node.js uncanny lessons its a dynamic language! TEST

  7. Node.js uncanny lessons beware of the diabolical spaghetti code app.get('/bar',

    function(req, res) { foo.fetchSomething(function(error, something) { if(!error) { foo.fetchSomeOne(something, function(error, someone) { if(!error) { foo.fetchBar(function(error, bar) { if(!error) { res.send("we got bar: " + bar); } else { res.send(error.statusCode); } }); } else { res.send(error.statusCode); } }); } else { res.send(error.statusCode); } }); }); it’s easy to write code that looks like this
  8. Node.js uncanny lessons expect errors •typically, on error, node emits

    an ‘error’ event on the corresponding object •if no listeners on object for ‘error’, a top level exception is raised and the process exits
  9. Node.js uncanny lessons expect errors (cont) process.on('uncaughtException', function(error) { console.log("Kaboom....

    handle " + error); }); var server = http.createServer(function (req, res) {}); server.on('error', function(error) { console.log("Caught error! Don't exit!"); }); prudent approach nuclear approach > > however, you’re better off logging and rethrowing!
  10. Node.js uncanny lessons Log Aggressively •The async nature of node

    makes it difficult to diagnose some production problems. •Why? Check out Winston (by ) logging levels + multiple transports
  11. Node.js uncanny lessons don’t fall asleep at the wheel •plan

    for your program to crash! •Automatically restart it with a process monitor •like ‘s Forever or
  12. Node.js uncanny lessons get out of the way!

  13. Node.js uncanny lessons get out of the way!

  14. Node.js uncanny lessons offload computationally intensive tasks •spawn a child

    process require('child_process').spawn() •call out to another system more apt to handle heavy lifting •use a job queue
  15. Node.js uncanny lessons be weary of loops! for (var i=0,

    l=entries.length; i<l; i++) { doSomething(entries[i]); } innocent enough? if # entries = 10,000 doSomething() takes ~1ms you block for 10 seconds!
  16. Node.js uncanny lessons be weary of loops! (cont) // order

    matters function processEntry(entries, index) { index = index || 0; if(index === entries.length) return done(); doSomething(entries[index]); process.nextTick(function() { processEntry(entries, index++) }); } processEntry(entries); non-blocking loops
  17. Node.js uncanny lessons be weary of loops! (cont) non-blocking loops

    // order doesn't matter var leftToProcess = entries.length; for (var i=0, l=entries.length; i<l; i++) { (function(foo) { process.nextTick(function() { doSomething(foo); if(--leftToProcess === 0) { done(); } }); })(entries[i]); }
  18. Node.js uncanny lessons be weary of loops! (cont) or use

    a library like async var async = require('async'); // order doesn't async.forEach(entries, doSomething); // order matters async.forEachSeries(entries, doSomething); // throttled 10 at a time async.forEachLimit(entries, 10, doSomething);
  19. Node.js uncanny lessons Secret Agent Pools • node pools outbound

    http(s) connections by default for each host • default concurrent maxSockets per host is 5 • is this what you want? // disable per request http.get({host:'localhost', port:80, path:'/', agent:false}, function (res) { // Do stuff }) // effectively disable http.globalAgent.maxSockets = 99999;
  20. Node.js uncanny lessons stay lean •watch your heap size •v8

    does stop the world generational GC •profile gc with --trace-gc •Cost of GC is proportional to the number of live objects
  21. Node.js uncanny lessons stay lean (cont) •I try to keep

    heap size < 200mb •Alternatively move data outside of node •instead use Redis, mongodb, CouchDB, etc •encourages statelessness, scalability and durability
  22. Node.js uncanny lessons you can’t do it alone! •don’t build

    monolithic apps •Build loosely coupled, highly cohesive systems
  23. Node.js uncanny lessons unleash your file descriptors! • node can

    handles 1000’s of connections!? • but your OS says... Too many open files • default # file descriptors on most unix systems is 1024 • 1 FD per socket means max open sockets < 1024 • increase the max # of file descriptors • ulimit -n <max # FD> • ulimit -a to see current max
  24. Node.js uncanny lessons git your modules in order •NPM is

    awesome •Maintaining a private NPM registry has been a struggle •Alternatively use Git Module dependencies (one git repo per module) devDependencies: { "eclg-prospero": "git+ssh://git@github.com:PearsonEducation/prospero-node.git#v2.0.0" }
  25. Node.js uncanny lessons count your friends •peg your npm versions

    { "name": "Foo Package", "description": "my Foo package", "version": "1.0.0", "author": "Mike Brevoort <mikebre@ecollege.com>", "dependencies": { "express": "2.3.2", "cluster": ">= 0.6.1", "mongodb": "0.9.x", "connect-gzip": "~0.1", "underscore": "= latest" }, "engines": { "node": "= 0.4.8" } } is this what you really want? really? >
  26. Node.js uncanny lessons check ‘em in • is amazing, but

    check in your module dependencies (in node_modules) •Dependencies of dependencies can change unseeingly! •ignore binaries •npm rebuild on deploy
  27. Node.js uncanny lessons or shrinkwrap ‘em! •npm shrinkwrap (new in

    1.1.2) •Records a full module/ version dependency graph •doesn’t seem to work with git referenced modules
  28. Node.js uncanny lessons Multiplicity •plan to manage multiple node.js versions

    •I prefer n •nvmw on windows $ n 0.4.12 0.4.8 0.6.0 ο 0.6.11 0.6.3 0.6.5 0.6.7 0.7.4 0.7.5 # run with a specific version $ n as 0.6.11 myprogram.js #using bundled npm $ n npm 0.6.11 # install a new version $ n latest $ n 0.6.10
  29. Node.js uncanny lessons if you have a cape, fly •

    use conventions! • check out Felix Geisendorfer’s node style guide http:// nodeguide.com • for example, reserve the first parameter of any callback for an optional error object function foo() { bar.getSomething(callback); } function callback(error, baz) { if (error) return console.log(error.message || error); // something else }
  30. Node.js uncanny lessons buffers are a super friend •great for

    shuffling binary data and streams •not part of the v8 heap so not gc’d (only the reference)
  31. Node.js uncanny lessons judge your alliances be scrupulous when choosing

    3rd party modules
  32. Node.js uncanny lessons these connections aren’t going to clean up

    themselves! app.get('/something', function(req, res, next) { if(req.params.foo) { // go along happy path var foo = getSomething(req.params.foo); res.send(foo); } }); what if this is false?
  33. Node.js uncanny lessons this is supposed to be fun Remember

    If it’s not, maybe it’s time to try Lua??
  34. Node.js uncanny lessons Questions?