Writing robust Node.js applications

Transcript

1. Node.js Tutorial Tom Hughes-Croucher, Principal Jetpacks For Dinosaurs, LLC tom@jetpacksfordinosaurs.com @sh1mmer Wednesday, May 30, 12
2. Introduction • Tom Hughes-Croucher • Principal at Jetpacks for Dinosaurs • Node.js and High-performance web site consulting and training • Walmart, MySpace, Joyent, Yahoo!, NASA, Tesco, etc • Node.js contributor • Lead author of "Node: Up and Running" Wednesday, May 30, 12
3. Scalable Server-Side Code with JavaScript Tom Hughes-Croucher Node Up and Running http://ofps.oreilly.com/titles/9781449398583/ http://shop.oreilly.com/product/0636920015956.do Wednesday, May 30, 12
4. Training Online In person Wednesday, May 30, 12
5. Node refresher Wednesday, May 30, 12
6. Code! Wednesday, May 30, 12
7. Interactive Debugging Wednesday, May 30, 12
8. Enki:~/Code/node-examples $ node --debug helloworld.js debugger listening on port 5858 Server running at http://127.0.0.1:8124/ Wednesday, May 30, 12
9. enki:~ sh1mmer$ ps -A | grep node 19093 ttys001 0:00.39 node index.js 19273 ttys002 0:00.00 grep node enki:~ sh1mmer$ kill -s USR1 19093 Wednesday, May 30, 12
10. enki:~ sh1mmer$ node index.js Hit SIGUSR1 - starting debugger agent. debugger listening on port 5858 Wednesday, May 30, 12
11. Enki:~ $ sudo npm install -g node-inspector node-inspector@0.1.8 ./node_modules/node-inspector ├── websocket-server@1.4.04 └── paperboy@0.0.2 Enki:~ $ node-inspector visit http://0.0.0.0:8080/debug?port=5858 to start debugging Wednesday, May 30, 12
12. Wednesday, May 30, 12
13. Exercises • Modify a basic HTTP server to return the text "I'm learning Node" • Change the HTTP response to HTML and return your text in an HTML page • Return the User-Agent string from the browser as part of your HTML page Wednesday, May 30, 12
14. Express.js Wednesday, May 30, 12
15. Sinatra Style MVC framework Wednesday, May 30, 12
16. var app = require('express').createServer(); app.get('/', function(req, res){ res.send('hello world'); }); app.listen(3000); Wednesday, May 30, 12
17. HTTP Verb Oriented Wednesday, May 30, 12
18. Middleware Wednesday, May 30, 12
19. app.use(express.bodyParser()); app.use(express.cookieParser()); app.post('/', function(req, res){ // Perhaps we posted several items with a form // (use the bodyParser() middleware for this) var items = req.body.items; console.log(items); res.send('logging'); }); Wednesday, May 30, 12
20. Templating Wednesday, May 30, 12
21. var express = require("express"); app.configure(function () { var public = __dirname + "/../public/"; public = require("path").normalize(public); app.set("views", __dirname + "/views"); app.set("view engine", "jade"); }); app.get("/", function (req, res) { res.render("index", { locals : { h1: 'Router Stats', scripts : [ "/public/smoothie.js", "/public/raphael.js", "/public/base.js", "/public/gfx.js", "/public/explore.js", "/public/app.js" ] } } }); Wednesday, May 30, 12
22. •Create an Express server •Make routes for ‘/’ & ‘/products’ •Serve two different pages based on value of the query string parameter "page" •Create a redirect from /old to /new •Set a cookie on the client Exercise Wednesday, May 30, 12
23. Building a chat server Wednesday, May 30, 12
24. Node Architectures Wednesday, May 30, 12
25. The obvious architecture Wednesday, May 30, 12
26. User Node DB Front-end Database Client Wednesday, May 30, 12
27. What about this? Wednesday, May 30, 12
28. User Node DB Front-end Farm Database Client Node Node Node Node Node Node Node Heavy Processor Work Wednesday, May 30, 12
29. Don’t we want to move “heavy work”? Wednesday, May 30, 12
30. Client → Server Server → DB Computation Computation Wednesday, May 30, 12
31. Better architecture Wednesday, May 30, 12
32. User Node DB Front-end Farm Database Client Node Node Node Node Node Logging farm Disk Big Slow Disk Wednesday, May 30, 12
33. Move work which isn’t user-facing out of main event loop Wednesday, May 30, 12
34. Ensure consistency of Node processes Wednesday, May 30, 12
35. Distributing work Wednesday, May 30, 12
36. Cluster Wednesday, May 30, 12
37. var cluster = require('cluster'); var http = require('http'); var numCPUs = require('os').cpus().length; if (cluster.isMaster) { // Fork workers. for (var i = 0; i < numCPUs; i++) { cluster.fork(); } cluster.on('death', function(worker) { console.log('worker ' + worker.pid + ' died'); }); } else { // Worker processes have a http server. http.Server(function(req, res) { res.writeHead(200); res.end("hello world\n"); }).listen(8000); } Wednesday, May 30, 12
38. Use load balancers to distribute across servers Wednesday, May 30, 12
39. node-proxy squid varnish haproxy Wednesday, May 30, 12
40. Unbundle SSL stud, stunnel, etc http://vincent.bernat.im/en/blog/2011- ssl-benchmark.html#conclusion Wednesday, May 30, 12
41. Sharding Wednesday, May 30, 12
42. Disclaimer: Sharding is hard See CAP theorem Wednesday, May 30, 12
43. Socket.io Wednesday, May 30, 12
44. Web Socket library Wednesday, May 30, 12
45. client and server with the shared API Wednesday, May 30, 12
46. var app = require('express').createServer(), io = require('socket.io').listen(app); app.listen(80); app.get('/', function (req, res) { res.sendfile(__dirname + '/index.html'); }); io.sockets.on('connection', function (socket) { socket.emit('news', { hello: 'world' }); socket.on('my other event', function (data) { console.log(data); }); }); Wednesday, May 30, 12
47. <!DOCTYPE html> <html> <body> <script src="/socket.io/socket.io.js"></script> <script> var socket = io.connect('http://localhost'); socket.connect(); socket.on('connect', function(){ console.log('connect') }) socket.on('message', function(){ console.log('message') }) socket.on('disconnect', function() { console.log('disconnect'); }) </script> </body> </html> Wednesday, May 30, 12
48. •WebSocket •Adobe® Flash® Socket •AJAX long polling •AJAX multipart streaming •Forever iframe •JSONP Polling Transports Wednesday, May 30, 12
49. options = { transports: ['websocket'], log: util.log } io.listen(server, options) Wednesday, May 30, 12
50. // note, io.listen() will create a http server for you var io = require('socket.io').listen(80); io.sockets.on('connection', function (socket) { io.sockets.emit('this', { will: 'be received by everyone'}); socket.on('private message', function (from, msg) { console.log('I received a private message by ', from, ' saying ', msg); }); socket.on('disconnect', function () { sockets.emit('user disconnected'); }); }); Wednesday, May 30, 12
51. •Create a socket.io connect to an express server •Create a route which loads a page that includes socket.io •Send "hello world" to the client sockets •Make the client respond with "thanks" and disconnect from the server Exercises Wednesday, May 30, 12
52. Using Redis with Socket.io Wednesday, May 30, 12
53. Why use Redis? • No shared memory • Cluster on server won’t share client data • Multiple servers can use the same Redis Wednesday, May 30, 12
54. var express = require('express') , app = express.createServer(); var sio = require('socket.io') , RedisStore = sio.RedisStore , io = sio.listen(app); app.listen(8080); app.get('/', function(req,res) { res.sendfile(__dirname + '/index.html'); }); io.sockets.on('connection', function (socket) { socket.on('chat', function (data) { console.log(data); socket.broadcast.emit('chat', data); }) }); Wednesday, May 30, 12
55. var cluster = require('cluster'); var http = require('http'); var numCPUs = require('os').cpus().length; if (cluster.isMaster) { // Fork workers. for (var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { var express = require('express') , app = express.createServer(); var sio = require('socket.io') , RedisStore = sio.RedisStore , io = sio.listen(app); app.listen(8080); app.get('/', function(req,res) { res.sendfile(__dirname + '/index.html'); }); // Somehow pass this information to the workers io.set('store', new RedisStore); // Do the work here io.sockets.on('connection', function (socket) { socket.on('chat', function (data) { console.log(data); socket.broadcast.emit('chat', data); }) }); } Wednesday, May 30, 12
56. Node in production Wednesday, May 30, 12
57. Error handling Wednesday, May 30, 12
58. var http = require('http'); var req = http.request({ host: 'www.google.com', path: '/', port: 80, method: 'POST' }, function (response) { // Do stuff with the response here }); Wednesday, May 30, 12
59. node.js:134 throw e; // process.nextTick error, or 'error' event on first tick ^ Error: Uncaught, unspecified 'error' event. at EventEmitter.emit (events.js:47:15) at Object.<anonymous> (/ Users/you/y-u-no-listen-for-errors.js:5:9) at Module._compile (module.js:404:26) at Object..js (module.js:410:10) at Module.load (module.js:336:31) at Function._load (module.js:297:12) at Array.<anonymous> (module.js:423:10) at EventEmitter._tickCallback (node.js:126:26) Wednesday, May 30, 12
60. var http = require('http'); try { var req = http.request({ host: 'www.google.com', path: '/', port: 80, method: 'POST' }, function (response) { // Do stuff with the response here }); } catch(e) { //exception handling } Wednesday, May 30, 12
61. var http = require('http'); var req = http.request({ host: 'www.google.com', path: '/', port: 80, method: 'POST' }, function (response) { // Do stuff with the response here }); req.on('error', function (err) { //safely handle this if possible }); Wednesday, May 30, 12
62. // Suppress errors process.on('uncaughtException', function (err) { // Log it! console.dir(err); }); Wednesday, May 30, 12
63. EventEmitter leaks Wednesday, May 30, 12
64. (node) warning: possible EventEmitter memory leak detected. 11 listeners added. Use emitter.setMaxListeners() to increase limit. Trace: at Pool.<anonymous> (events.js:101:17) at Object.proxyRequest (~/http-proxy/lib/node-http-proxy.js:185:7) at Server.<anonymous> (/Users/some-user/myapp.js:14:9) at Server.emit (events.js:45:17) at HTTPParser.onIncoming (http.js:1078:12) at HTTPParser.onHeadersComplete (http.js:87:31) at Socket.ondata (http.js:977:22) at Socket._onReadable (net.js:654:27) at IOWatcher.onReadable [as callback] (net.js:156:10) Wednesday, May 30, 12
65. var events = require('events'); function doSomethingThenTellMe () { var emitter = new events.EventEmitter(); setTimeout(function () { emitter.emit('done'); }, 2000); return emitter; } var doingIt = doSomethingThenTellMe(); doingIt.on('done', function () { console.log("Ok, it's done"); }); // Why are you using `.on()`? // You only expect this event to fire once. Wednesday, May 30, 12
66. once() is awesome Wednesday, May 30, 12
67. var events = require('events'); function doSomethingThenTellMe () { var emitter = new events.EventEmitter(); setTimeout(function () { emitter.emit('done'); }, 2000); return emitter; } var doingIt = doSomethingThenTellMe(); doingIt.once('done', function () { console.log("Ok, it's done"); }); Wednesday, May 30, 12
68. Process monitoring Wednesday, May 30, 12
69. Upstart Wednesday, May 30, 12
70. description "node.js server" author "croucher`" start on startup stop on shutdown script # We found $HOME is needed. Without it, we ran into problems export HOME="/root" exec /usr/local/bin/node /var/noderoot/index.js 2>&1 >> /var/log/node.log end script Wednesday, May 30, 12
71. forever Wednesday, May 30, 12
72. $ cd /path/to/your/app $ forever start bin/yourapp info: Running action: start info: Forever processing file: examples/server.js $ forever list info: Running action: list info: Forever processes running [0] bin/yourapp [77041, 77040] /Users/You/.forever 20dL.log 0:0:0:1.788 Wednesday, May 30, 12
73. NPM Packaging Wednesday, May 30, 12
74. Enki:~/Code/node-dnsserve(master) $ ls LICENSE README dnsserver.js package.json Enki:~/Code/node-dnsserve(master) $ Wednesday, May 30, 12
75. { "name": "dns-server", "description": "DNS server for Node", "version": "0.0.1a", "author": "Tom Hughes-Croucher <tom.croucher@gmail.com>", "main" : "dnsserver" } Wednesday, May 30, 12
76. Enki:~/Code/node-dnsserve(master) $ sudo npm install . -g dns-server@0.0.1a /usr/local/lib/node_modules/dns- server Enki:~/Code/node-dnsserve(master) $ Wednesday, May 30, 12
77. Enki:~/Code/node-dnsserve(master) $ sudo npm link /usr/local/lib/node_modules/dns-server -> /Users/ sh1mmer/Code/node-dnsserve Enki:~/Code/node-dnsserve(master) $ npm ls -g /usr/local/lib ├── dns-server@0.0.1a -> /Users/sh1mmer/Code/node- dnsserve └─┬ npm@1.0.6 ├── abbrev@1.0.3 ├── node-uuid@1.1.0 ├── nopt@1.0.4 └── semver@1.0.5 Enki:~/Code/node-dnsserve(master) $ Wednesday, May 30, 12
78. Enki:~/Code/node-dnsserve(master) $ npm adduser Username: foobarbaz Password: Email: Wednesday, May 30, 12
79. Enki:~/Code/node-dnsserve(master) $ npm publish npm WARN Sending authorization over insecure channel. Enki:~/Code/node-dnsserve(master) $ Wednesday, May 30, 12
80. { "name": "winston", "description": "A multi-transport async logging library for Node.js", "version": "0.2.7", "author": "Charlie Robbins <charlie.robbins@gmail.com>", "contributors": [ { "name": "Matthew Bergman", "email": "mzbphoto@gmail.com" } ], "repository": { "type": "git", "url": "http://github.com/indexzero/winston.git" }, "keywords": ["logging", "sysadmin", "tools"], "dependencies": { "colors": ">= 0.3.0", "eyes": ">= 0.1.6", "loggly": ">= 0.1.4", "vows": ">= 0.5.2" }, "main": "./lib/winston", "scripts": { "test": "vows test/*-test.js --spec" }, "engines": { "node": ">= 0.3.0" } } Wednesday, May 30, 12
81. { "name": "winston", "description": "A multi-transport async logging library for Node.js", "version": "0.2.7", "author": "Charlie Robbins <charlie.robbins@gmail.com>", "contributors": [ { "name": "Matthew Bergman", "email": "mzbphoto@gmail.com" } ], "repository": { "type": "git", "url": "http://github.com/indexzero/winston.git" }, "keywords": ["logging", "sysadmin", "tools"], "dependencies": { "colors": ">= 0.x.x", "eyes": ">= 0.1.x", "loggly": ">= 0.1.x", "vows": ">= 0.5.x" }, "main": "./lib/winston", "scripts": { "test": "vows test/*-test.js --spec" }, "engines": { "node": ">= 0.3.0" } } Wednesday, May 30, 12
82. tom@jetpacksfordinosaurs.com @sh1mmer Questions Wednesday, May 30, 12