Upgrade to Pro — share decks privately, control downloads, hide ads and more …

You Don't Know Node.js

You Don't Know Node.js

Node.js can be a powerful platform used in the right situations, but it also has more footguns than PHP does for new developers. Asynchronous programming style, concurrency issues, error handling, the event loop, and no default timeouts for HTTP requests or network calls will give new developers more than their fair share of troubles and can cause issues that are difficult to debug and reproduce. This talk gives a brief introduction to node.js and the event loop model (don’t block the event loop!), and covers common pitfalls when using node.js with simple and practical solutions.

vlucas

May 08, 2015
Tweet

More Decks by vlucas

Other Decks in Programming

Transcript

  1. You Don’t Know Node.js
    Explanations for common foot-guns and gotchas
    Vance Lucas
    http://vancelucas.com

    View full-size slide

  2. Who are You?
    • Vance Lucas
    • http://vancelucas.com
    • Twitter: @vlucas
    • Co-founded JavaScript User Group
    • Node.js based startup postnix.com

    View full-size slide

  3. What is Node?
    • JavaScript on the server
    • V8 engine (Google Chrome team)
    • Fully async non-blocking I/O
    • No-frills vanilla JS - not a framework

    View full-size slide

  4. Node Process
    • Runs forever
    • Does not cleanup/tear down
    • Need to watch memory usage
    • Callbacks for everything
    • Single threaded?*
    * or is it?

    View full-size slide

  5. Callbacks in JS
    $.get('http://nodejs.org', function(data) {
    console.log(data);
    });

    View full-size slide

  6. Callbacks in Node
    var fs = require('fs');
    fs.readFile('myfile.txt', function(err, data) {
    console.log(data);
    });

    View full-size slide

  7. Everything is async
    db.users.find({ name: 'John' }, function(err, users) {
    users.forEach(function(user) {
    console.log(user);
    });
    });

    View full-size slide

  8. The Event Loop
    Brief overview

    View full-size slide

  9. https://www.packtpub.com/web-development/build-network-application-node-video

    View full-size slide

  10. How Event Loop Works
    • http://latentflip.com/loupe/
    • Philip Roberts: What the heck is the
    event loop anyway?
    • https://www.youtube.com/watch?v=8aGhZQkoFbQ

    View full-size slide

  11. Don’t Block The Event Loop!
    • Don’t do heavy I/O, large file parsing,
    or computation in your main process
    • All major processing in queue
    • node-blocked NPM package
    • https://github.com/tj/node-blocked

    View full-size slide

  12. Database
    • How are you connecting to your db?
    • Are you using a connection pool?
    • Remember: Node runs forever, and
    does not cleanup after each request

    View full-size slide

  13. Before
    var options = require('../config')
    // ...
    , mysql = require('mysql')
    , connection = mysql.createConnection(options.mysql);
    connection.query('SELECT id, setting_name, setting_value FROM
    settings', function(err, results) {
    // ...
    });

    View full-size slide

  14. After
    var options = require('../config')
    // ...
    , mysqlPool = require('./mysqlPool')
    // Use MySQL pool for queries
    var connection = mysqlPool.create({ logSql: false }, "DBPool",
    options.mysql);
    connection.query('SELECT id, setting_name, setting_value FROM
    settings', function(err, results) {
    // ...
    });
    https://github.com/felixge/node-mysql/issues/778#issuecomment-41077643

    View full-size slide

  15. Streams
    • Idiomatic node
    • Use streams to copy file contents -
    don’t read it into memory
    • Gulp.js vs. Grunt.js
    • NPM ‘multipipe’ package
    • https://github.com/juliangruber/multipipe

    View full-size slide

  16. File Copy Stream
    var fs = require('fs');
    var zlib = require('zlib');
    fs.createReadStream('input.txt.gz')
    .pipe(zlib.createGunzip())
    .pipe(fs.createWriteStream('output.txt'));

    View full-size slide

  17. Error Handling
    • Node process crashes & exits on error
    • Need to catch and handle all errors
    • Try/catch doesn’t work like you think
    • Promises can help

    View full-size slide

  18. Uncaught Exceptions
    process.on('uncaughtException', function(err){
    log.critical('uncaught', err);
    setTimeout(function(){
    // cleanup and exit...
    }, 1000);
    });

    View full-size slide

  19. Naive Try/Catch
    var fs = require('fs');
    try {
    fs.readFile('doesnotexist.txt', function(err, data) {
    throw new Error('File does not exist!');
    });
    } catch(e) {
    console.log('There was an error, but it\'s cool,
    because I handled it, right?');
    }

    View full-size slide

  20. Try/Catch Blocks
    • Callbacks will be executed in a different
    stack than your try/catch block
    • They are basically useless unless they
    are everywhere

    View full-size slide

  21. Promises
    • More elegant code
    • Can chain calls together
    • Better error handling

    View full-size slide

  22. Promises
    doSomething()
    .then(doThing2())
    .then(doThing3())
    .then(doThing4())
    .then(function(result) {
    doThing5(someArg, result);
    })
    .catch(function(e) {
    console.error("unable to read whatever")
    });

    View full-size slide

  23. Parallel
    Promise.all([
    doThing2(),
    doThing3(),
    doThing4()
    ])
    .then(function(results) {
    doThing5(someArg, results);
    })
    .catch(function(e) {
    console.error("unable to read whatever")
    });

    View full-size slide

  24. Callback Style
    fs.readFile("file.json", function(err, json) {
    if(err) {
    console.error("unable to read file");
    } else {
    try {
    json = JSON.parse(json);
    console.log(json.success);
    }
    catch(e) {
    console.error("invalid json in file");
    }
    }
    });

    View full-size slide

  25. Promises
    var Promise = require('bluebird');
    var fs = Promise.promisifyAll(require("fs"));
    fs.readFileAsync("file.json")
    .then(JSON.parse)
    .then(function(json) {
    console.log(json.success);
    })
    .catch(SyntaxError, function(e) {
    console.error("invalid json in file");
    })
    .catch(function(e) {
    console.error("unable to read file")
    });

    View full-size slide

  26. Promises
    ns.mxLookup = function(domain) {
    return ns.mxLookupLocal(domain)
    .then(function(mxRecords) {
    if (!mxRecords) {
    return ns.mxLookupRemote(domain);
    }
    return mxRecords;
    });
    };
    Code from postnix.com

    View full-size slide

  27. ns.checkSingleEmail = function(user, email, tag, ip) {
    return new Promise(function(fulfill, reject) {
    // First, see if user can check an email
    return ns.canUserCheckEmail(user, ip)
    .then(function() { // Local db lookup for cached result
    return ns.localEmailLookup(user, email, tag, ip);
    }, function(err) { // Error (no credits or restricted by IP)
    reject(err);
    })
    .then(function(json) {
    if (json) { // Return cached JSON response immediately
    return fulfill(json);
    }
    // Perform all lookups
    return ns.mxLookup(domain)
    .then(function(mxRecords) {
    if (!mxRecords) {
    throw new Error('no MX records exist for domain');
    }
    return ns.smtpCheck(user, email, mxRecords);
    })
    .then(function(json) {
    fulfill(json);
    });
    })
    .catch(function(err) {
    if(err.code == 'ECONNREFUSED') {
    err.json = ns.errorJson(email, 'connection to mail server refused');
    } else {
    err.json = ns.errorJson(email, err.message || 'connection timeout');
    }
    db.query("DELETE FROM email_mx WHERE domain = ?", [domain]);
    reject(err);
    });
    });
    };

    View full-size slide

  28. Gotchas
    Things to watch out for

    View full-size slide

  29. Network I/O
    • No default timeouts for network
    requests
    • Sometimes you still have to .end() the
    connection itself on timeout event
    • This is on purpose, by design

    View full-size slide

  30. Adding Timeouts
    server.on('connection', function(socket) {
    log.info('SOCKET OPENED');
    socket.on('end', function() {
    log.info('SOCKET END: other end of the socket sends
    a FIN packet');
    });
    // Set & listen for timeout
    socket.setTimeout(2000);
    socket.on('timeout', function() {
    log.info('SOCKET TIMEOUT');
    socket.end(); // have to sever/end socket manually
    });
    });

    View full-size slide

  31. setTimeout(callback, 2000)

    View full-size slide

  32. ES6 Makes Life Better

    View full-size slide

  33. ES6 Features
    • https://github.com/lukehoban/es6features
    • Use io.js! (node fork w/ES6)
    • Generators,
    • Workers in io.js (experimental)
    • https://github.com/iojs/io.js/pull/1159

    View full-size slide

  34. More Resources
    • Mixu’s Node Book
    • http://book.mixu.net/node/
    • Event Loop Overview from Mozilla
    • https://developer.mozilla.org/en-US/docs/Web/
    JavaScript/EventLoop

    View full-size slide