Node.js Uncanny Lessons

Talk given at DenverJS 3/15/2012
Lessons Learned

Transcript

1. Node.js Mike Brevoort 15 March 2012 uncanny lessons
2. Node.js uncanny lessons Mike Brevoort @mbrevoort work at... (though opinions expressed in this presentation are my own and not meant to represent Pearson)
3. Node.js uncanny lessons Agenda 22 pieces of advice
4. Node.js uncanny lessons know your strengths •don’t force everything into node Fibonacci you say?!?
5. Node.js uncanny lessons know your powers •don’t try to work against the async nature •don’t forget to be an engineer!
6. Node.js uncanny lessons its a dynamic language! TEST
7. Node.js uncanny lessons beware of the diabolical spaghetti code app.get('/bar', function(req, res) { foo.fetchSomething(function(error, something) { if(!error) { foo.fetchSomeOne(something, function(error, someone) { if(!error) { foo.fetchBar(function(error, bar) { if(!error) { res.send("we got bar: " + bar); } else { res.send(error.statusCode); } }); } else { res.send(error.statusCode); } }); } else { res.send(error.statusCode); } }); }); it’s easy to write code that looks like this
8. Node.js uncanny lessons expect errors •typically, on error, node emits an ‘error’ event on the corresponding object •if no listeners on object for ‘error’, a top level exception is raised and the process exits
9. Node.js uncanny lessons expect errors (cont) process.on('uncaughtException', function(error) { console.log("Kaboom.... handle " + error); }); var server = http.createServer(function (req, res) {}); server.on('error', function(error) { console.log("Caught error! Don't exit!"); }); prudent approach nuclear approach > > however, you’re better off logging and rethrowing!
10. Node.js uncanny lessons Log Aggressively •The async nature of node makes it difficult to diagnose some production problems. •Why? Check out Winston (by ) logging levels + multiple transports
11. Node.js uncanny lessons don’t fall asleep at the wheel •plan for your program to crash! •Automatically restart it with a process monitor •like ‘s Forever or
12. Node.js uncanny lessons get out of the way!
13. Node.js uncanny lessons get out of the way!
14. Node.js uncanny lessons offload computationally intensive tasks •spawn a child process require('child_process').spawn() •call out to another system more apt to handle heavy lifting •use a job queue
15. Node.js uncanny lessons be weary of loops! for (var i=0, l=entries.length; i<l; i++) { doSomething(entries[i]); } innocent enough? if # entries = 10,000 doSomething() takes ~1ms you block for 10 seconds!
16. Node.js uncanny lessons be weary of loops! (cont) // order matters function processEntry(entries, index) { index = index || 0; if(index === entries.length) return done(); doSomething(entries[index]); process.nextTick(function() { processEntry(entries, index++) }); } processEntry(entries); non-blocking loops
17. Node.js uncanny lessons be weary of loops! (cont) non-blocking loops // order doesn't matter var leftToProcess = entries.length; for (var i=0, l=entries.length; i<l; i++) { (function(foo) { process.nextTick(function() { doSomething(foo); if(--leftToProcess === 0) { done(); } }); })(entries[i]); }
18. Node.js uncanny lessons be weary of loops! (cont) or use a library like async var async = require('async'); // order doesn't async.forEach(entries, doSomething); // order matters async.forEachSeries(entries, doSomething); // throttled 10 at a time async.forEachLimit(entries, 10, doSomething);
19. Node.js uncanny lessons Secret Agent Pools • node pools outbound http(s) connections by default for each host • default concurrent maxSockets per host is 5 • is this what you want? // disable per request http.get({host:'localhost', port:80, path:'/', agent:false}, function (res) { // Do stuff }) // effectively disable http.globalAgent.maxSockets = 99999;
20. Node.js uncanny lessons stay lean •watch your heap size •v8 does stop the world generational GC •profile gc with --trace-gc •Cost of GC is proportional to the number of live objects
21. Node.js uncanny lessons stay lean (cont) •I try to keep heap size < 200mb •Alternatively move data outside of node •instead use Redis, mongodb, CouchDB, etc •encourages statelessness, scalability and durability
22. Node.js uncanny lessons you can’t do it alone! •don’t build monolithic apps •Build loosely coupled, highly cohesive systems
23. Node.js uncanny lessons unleash your file descriptors! • node can handles 1000’s of connections!? • but your OS says... Too many open files • default # file descriptors on most unix systems is 1024 • 1 FD per socket means max open sockets < 1024 • increase the max # of file descriptors • ulimit -n <max # FD> • ulimit -a to see current max
24. Node.js uncanny lessons git your modules in order •NPM is awesome •Maintaining a private NPM registry has been a struggle •Alternatively use Git Module dependencies (one git repo per module) devDependencies: { "eclg-prospero": "git+ssh://git@github.com:PearsonEducation/prospero-node.git#v2.0.0" }
25. Node.js uncanny lessons count your friends •peg your npm versions { "name": "Foo Package", "description": "my Foo package", "version": "1.0.0", "author": "Mike Brevoort <mikebre@ecollege.com>", "dependencies": { "express": "2.3.2", "cluster": ">= 0.6.1", "mongodb": "0.9.x", "connect-gzip": "~0.1", "underscore": "= latest" }, "engines": { "node": "= 0.4.8" } } is this what you really want? really? >
26. Node.js uncanny lessons check ‘em in • is amazing, but check in your module dependencies (in node_modules) •Dependencies of dependencies can change unseeingly! •ignore binaries •npm rebuild on deploy
27. Node.js uncanny lessons or shrinkwrap ‘em! •npm shrinkwrap (new in 1.1.2) •Records a full module/ version dependency graph •doesn’t seem to work with git referenced modules
28. Node.js uncanny lessons Multiplicity •plan to manage multiple node.js versions •I prefer n •nvmw on windows $ n 0.4.12 0.4.8 0.6.0 ο 0.6.11 0.6.3 0.6.5 0.6.7 0.7.4 0.7.5 # run with a specific version $ n as 0.6.11 myprogram.js #using bundled npm $ n npm 0.6.11 # install a new version $ n latest $ n 0.6.10
29. Node.js uncanny lessons if you have a cape, fly • use conventions! • check out Felix Geisendorfer’s node style guide http:// nodeguide.com • for example, reserve the first parameter of any callback for an optional error object function foo() { bar.getSomething(callback); } function callback(error, baz) { if (error) return console.log(error.message || error); // something else }
30. Node.js uncanny lessons buffers are a super friend •great for shuffling binary data and streams •not part of the v8 heap so not gc’d (only the reference)
31. Node.js uncanny lessons judge your alliances be scrupulous when choosing 3rd party modules
32. Node.js uncanny lessons these connections aren’t going to clean up themselves! app.get('/something', function(req, res, next) { if(req.params.foo) { // go along happy path var foo = getSomething(req.params.foo); res.send(foo); } }); what if this is false?
33. Node.js uncanny lessons this is supposed to be fun Remember If it’s not, maybe it’s time to try Lua??
34. Node.js uncanny lessons Questions?