You Don't Know Node.js

Node.js can be a powerful platform used in the right situations, but it also has more footguns than PHP does for new developers. Asynchronous programming style, concurrency issues, error handling, the event loop, and no default timeouts for HTTP requests or network calls will give new developers more than their fair share of troubles and can cause issues that are difficult to debug and reproduce. This talk gives a brief introduction to node.js and the event loop model (don’t block the event loop!), and covers common pitfalls when using node.js with simple and practical solutions.

Transcript

1. You Don’t Know Node.js Explanations for common foot-guns and gotchas Vance Lucas http://vancelucas.com
2. Who are You? • Vance Lucas • http://vancelucas.com • Twitter: @vlucas • Co-founded JavaScript User Group • Node.js based startup postnix.com
3. What is Node? • JavaScript on the server • V8 engine (Google Chrome team) • Fully async non-blocking I/O • No-frills vanilla JS - not a framework
4. Node Process • Runs forever • Does not cleanup/tear down • Need to watch memory usage • Callbacks for everything • Single threaded?* * or is it?
5. Callbacks in JS $.get('http://nodejs.org', function(data) { console.log(data); });
6. Callbacks in Node var fs = require('fs'); fs.readFile('myfile.txt', function(err, data) { console.log(data); });
7. Everything is async db.users.find({ name: 'John' }, function(err, users) { users.forEach(function(user) { console.log(user); }); });
8. The Event Loop Brief overview
9. https://www.packtpub.com/web-development/build-network-application-node-video
10. How Event Loop Works • http://latentflip.com/loupe/ • Philip Roberts: What the heck is the event loop anyway? • https://www.youtube.com/watch?v=8aGhZQkoFbQ
11. Demo
13. Don’t Block The Event Loop! • Don’t do heavy I/O, large file parsing, or computation in your main process • All major processing in queue • node-blocked NPM package • https://github.com/tj/node-blocked
14. Database • How are you connecting to your db? • Are you using a connection pool? • Remember: Node runs forever, and does not cleanup after each request
15. Before var options = require('../config') // ... , mysql = require('mysql') , connection = mysql.createConnection(options.mysql); connection.query('SELECT id, setting_name, setting_value FROM settings', function(err, results) { // ... });
16. After var options = require('../config') // ... , mysqlPool = require('./mysqlPool') // Use MySQL pool for queries var connection = mysqlPool.create({ logSql: false }, "DBPool", options.mysql); connection.query('SELECT id, setting_name, setting_value FROM settings', function(err, results) { // ... }); https://github.com/felixge/node-mysql/issues/778#issuecomment-41077643
17. Streams • Idiomatic node • Use streams to copy file contents - don’t read it into memory • Gulp.js vs. Grunt.js • NPM ‘multipipe’ package • https://github.com/juliangruber/multipipe
18. File Copy Stream var fs = require('fs'); var zlib = require('zlib'); fs.createReadStream('input.txt.gz') .pipe(zlib.createGunzip()) .pipe(fs.createWriteStream('output.txt'));
19. Error Handling • Node process crashes & exits on error • Need to catch and handle all errors • Try/catch doesn’t work like you think • Promises can help
20. Uncaught Exceptions process.on('uncaughtException', function(err){ log.critical('uncaught', err); setTimeout(function(){ // cleanup and exit... }, 1000); });
21. Naive Try/Catch var fs = require('fs'); try { fs.readFile('doesnotexist.txt', function(err, data) { throw new Error('File does not exist!'); }); } catch(e) { console.log('There was an error, but it\'s cool, because I handled it, right?'); }
22. Try/Catch Blocks • Callbacks will be executed in a different stack than your try/catch block • They are basically useless unless they are everywhere
23. Promises • More elegant code • Can chain calls together • Better error handling
24. Promises doSomething() .then(doThing2()) .then(doThing3()) .then(doThing4()) .then(function(result) { doThing5(someArg, result); }) .catch(function(e) { console.error("unable to read whatever") });
25. Parallel Promise.all([ doThing2(), doThing3(), doThing4() ]) .then(function(results) { doThing5(someArg, results); }) .catch(function(e) { console.error("unable to read whatever") });
26. Callback Style fs.readFile("file.json", function(err, json) { if(err) { console.error("unable to read file"); } else { try { json = JSON.parse(json); console.log(json.success); } catch(e) { console.error("invalid json in file"); } } });
27. Promises var Promise = require('bluebird'); var fs = Promise.promisifyAll(require("fs")); fs.readFileAsync("file.json") .then(JSON.parse) .then(function(json) { console.log(json.success); }) .catch(SyntaxError, function(e) { console.error("invalid json in file"); }) .catch(function(e) { console.error("unable to read file") });
28. Promises ns.mxLookup = function(domain) { return ns.mxLookupLocal(domain) .then(function(mxRecords) { if (!mxRecords) { return ns.mxLookupRemote(domain); } return mxRecords; }); }; Code from postnix.com
29. ns.checkSingleEmail = function(user, email, tag, ip) { return new Promise(function(fulfill, reject) { // First, see if user can check an email return ns.canUserCheckEmail(user, ip) .then(function() { // Local db lookup for cached result return ns.localEmailLookup(user, email, tag, ip); }, function(err) { // Error (no credits or restricted by IP) reject(err); }) .then(function(json) { if (json) { // Return cached JSON response immediately return fulfill(json); } // Perform all lookups return ns.mxLookup(domain) .then(function(mxRecords) { if (!mxRecords) { throw new Error('no MX records exist for domain'); } return ns.smtpCheck(user, email, mxRecords); }) .then(function(json) { fulfill(json); }); }) .catch(function(err) { if(err.code == 'ECONNREFUSED') { err.json = ns.errorJson(email, 'connection to mail server refused'); } else { err.json = ns.errorJson(email, err.message || 'connection timeout'); } db.query("DELETE FROM email_mx WHERE domain = ?", [domain]); reject(err); }); }); };
30. Gotchas Things to watch out for
31. Network I/O • No default timeouts for network requests • Sometimes you still have to .end() the connection itself on timeout event • This is on purpose, by design
32. Adding Timeouts server.on('connection', function(socket) { log.info('SOCKET OPENED'); socket.on('end', function() { log.info('SOCKET END: other end of the socket sends a FIN packet'); }); // Set & listen for timeout socket.setTimeout(2000); socket.on('timeout', function() { log.info('SOCKET TIMEOUT'); socket.end(); // have to sever/end socket manually }); });
33. setTimeout(callback, 2000)
34. ES6 Makes Life Better
35. ES6 Features • https://github.com/lukehoban/es6features • Use io.js! (node fork w/ES6) • Generators, • Workers in io.js (experimental) • https://github.com/iojs/io.js/pull/1159
36. More Resources • Mixu’s Node Book • http://book.mixu.net/node/ • Event Loop Overview from Mozilla • https://developer.mozilla.org/en-US/docs/Web/ JavaScript/EventLoop
37. Questions? @vlucas | vance@vancelucas.com