Upgrade to Pro — share decks privately, control downloads, hide ads and more …

You Don't Know Node.js

You Don't Know Node.js

Node.js can be a powerful platform used in the right situations, but it also has more footguns than PHP does for new developers. Asynchronous programming style, concurrency issues, error handling, the event loop, and no default timeouts for HTTP requests or network calls will give new developers more than their fair share of troubles and can cause issues that are difficult to debug and reproduce. This talk gives a brief introduction to node.js and the event loop model (don’t block the event loop!), and covers common pitfalls when using node.js with simple and practical solutions.

94cb827736e36e0f6343e9640e72fec7?s=128

vlucas

May 08, 2015
Tweet

Transcript

  1. You Don’t Know Node.js Explanations for common foot-guns and gotchas

    Vance Lucas http://vancelucas.com
  2. Who are You? • Vance Lucas • http://vancelucas.com • Twitter:

    @vlucas • Co-founded JavaScript User Group • Node.js based startup postnix.com
  3. What is Node? • JavaScript on the server • V8

    engine (Google Chrome team) • Fully async non-blocking I/O • No-frills vanilla JS - not a framework
  4. Node Process • Runs forever • Does not cleanup/tear down

    • Need to watch memory usage • Callbacks for everything • Single threaded?* * or is it?
  5. Callbacks in JS $.get('http://nodejs.org', function(data) { console.log(data); });

  6. Callbacks in Node var fs = require('fs'); fs.readFile('myfile.txt', function(err, data)

    { console.log(data); });
  7. Everything is async db.users.find({ name: 'John' }, function(err, users) {

    users.forEach(function(user) { console.log(user); }); });
  8. The Event Loop Brief overview

  9. https://www.packtpub.com/web-development/build-network-application-node-video

  10. How Event Loop Works • http://latentflip.com/loupe/ • Philip Roberts: What

    the heck is the event loop anyway? • https://www.youtube.com/watch?v=8aGhZQkoFbQ
  11. Demo

  12. None
  13. Don’t Block The Event Loop! • Don’t do heavy I/O,

    large file parsing, or computation in your main process • All major processing in queue • node-blocked NPM package • https://github.com/tj/node-blocked
  14. Database • How are you connecting to your db? •

    Are you using a connection pool? • Remember: Node runs forever, and does not cleanup after each request
  15. Before var options = require('../config') // ... , mysql =

    require('mysql') , connection = mysql.createConnection(options.mysql); connection.query('SELECT id, setting_name, setting_value FROM settings', function(err, results) { // ... });
  16. After var options = require('../config') // ... , mysqlPool =

    require('./mysqlPool') // Use MySQL pool for queries var connection = mysqlPool.create({ logSql: false }, "DBPool", options.mysql); connection.query('SELECT id, setting_name, setting_value FROM settings', function(err, results) { // ... }); https://github.com/felixge/node-mysql/issues/778#issuecomment-41077643
  17. Streams • Idiomatic node • Use streams to copy file

    contents - don’t read it into memory • Gulp.js vs. Grunt.js • NPM ‘multipipe’ package • https://github.com/juliangruber/multipipe
  18. File Copy Stream var fs = require('fs'); var zlib =

    require('zlib'); fs.createReadStream('input.txt.gz') .pipe(zlib.createGunzip()) .pipe(fs.createWriteStream('output.txt'));
  19. Error Handling • Node process crashes & exits on error

    • Need to catch and handle all errors • Try/catch doesn’t work like you think • Promises can help
  20. Uncaught Exceptions process.on('uncaughtException', function(err){ log.critical('uncaught', err); setTimeout(function(){ // cleanup and

    exit... }, 1000); });
  21. Naive Try/Catch var fs = require('fs'); try { fs.readFile('doesnotexist.txt', function(err,

    data) { throw new Error('File does not exist!'); }); } catch(e) { console.log('There was an error, but it\'s cool, because I handled it, right?'); }
  22. Try/Catch Blocks • Callbacks will be executed in a different

    stack than your try/catch block • They are basically useless unless they are everywhere
  23. Promises • More elegant code • Can chain calls together

    • Better error handling
  24. Promises doSomething() .then(doThing2()) .then(doThing3()) .then(doThing4()) .then(function(result) { doThing5(someArg, result); })

    .catch(function(e) { console.error("unable to read whatever") });
  25. Parallel Promise.all([ doThing2(), doThing3(), doThing4() ]) .then(function(results) { doThing5(someArg, results);

    }) .catch(function(e) { console.error("unable to read whatever") });
  26. Callback Style fs.readFile("file.json", function(err, json) { if(err) { console.error("unable to

    read file"); } else { try { json = JSON.parse(json); console.log(json.success); } catch(e) { console.error("invalid json in file"); } } });
  27. Promises var Promise = require('bluebird'); var fs = Promise.promisifyAll(require("fs")); fs.readFileAsync("file.json")

    .then(JSON.parse) .then(function(json) { console.log(json.success); }) .catch(SyntaxError, function(e) { console.error("invalid json in file"); }) .catch(function(e) { console.error("unable to read file") });
  28. Promises ns.mxLookup = function(domain) { return ns.mxLookupLocal(domain) .then(function(mxRecords) { if

    (!mxRecords) { return ns.mxLookupRemote(domain); } return mxRecords; }); }; Code from postnix.com
  29. ns.checkSingleEmail = function(user, email, tag, ip) { return new Promise(function(fulfill,

    reject) { // First, see if user can check an email return ns.canUserCheckEmail(user, ip) .then(function() { // Local db lookup for cached result return ns.localEmailLookup(user, email, tag, ip); }, function(err) { // Error (no credits or restricted by IP) reject(err); }) .then(function(json) { if (json) { // Return cached JSON response immediately return fulfill(json); } // Perform all lookups return ns.mxLookup(domain) .then(function(mxRecords) { if (!mxRecords) { throw new Error('no MX records exist for domain'); } return ns.smtpCheck(user, email, mxRecords); }) .then(function(json) { fulfill(json); }); }) .catch(function(err) { if(err.code == 'ECONNREFUSED') { err.json = ns.errorJson(email, 'connection to mail server refused'); } else { err.json = ns.errorJson(email, err.message || 'connection timeout'); } db.query("DELETE FROM email_mx WHERE domain = ?", [domain]); reject(err); }); }); };
  30. Gotchas Things to watch out for

  31. Network I/O • No default timeouts for network requests •

    Sometimes you still have to .end() the connection itself on timeout event • This is on purpose, by design
  32. Adding Timeouts server.on('connection', function(socket) { log.info('SOCKET OPENED'); socket.on('end', function() {

    log.info('SOCKET END: other end of the socket sends a FIN packet'); }); // Set & listen for timeout socket.setTimeout(2000); socket.on('timeout', function() { log.info('SOCKET TIMEOUT'); socket.end(); // have to sever/end socket manually }); });
  33. setTimeout(callback, 2000)

  34. ES6 Makes Life Better

  35. ES6 Features • https://github.com/lukehoban/es6features • Use io.js! (node fork w/ES6)

    • Generators, • Workers in io.js (experimental) • https://github.com/iojs/io.js/pull/1159
  36. More Resources • Mixu’s Node Book • http://book.mixu.net/node/ • Event

    Loop Overview from Mozilla • https://developer.mozilla.org/en-US/docs/Web/ JavaScript/EventLoop
  37. Questions? @vlucas | vance@vancelucas.com