Writing Real Programs with JavaScript!?

Transcript

1. Writing Real Programs
   … with JavaScript!?
   An introduction to
   Node.JS
   Andrew Montalenti
   

   View Slide

2. Act I,
   Functional JavaScript
   Act II,
   Meet Node.JS
   

   View Slide

3. Act I, JavaScript:
   the beast with two backs
   • JS is a cool programming language!
   – Functional
   – Weakly and dynamically typed
   – Function scope and closure
   – Has a >99% desktop install base (no kidding!)
   • JS is not a real programming language!
   – No compilation units, modules and namespaces
   – No classes or object-orientation built-in
   – No standard library (no kidding!)
   – Incoherent implementation standard
   

   View Slide

4. JS boring details in one slide
   • Number: double-precision 64-bit IEEE 754, e.g. 0 or 0.5
   • String: stream of 16-bit Unicode characters, e.g. “hello,
   world!”
   • Boolean: true or false
   • object: {a: b} - like Python dictionary + attribute (dot notation)
   syntax
   • Function: function() {}
   • Array: [a, b] – implemented as hash, includes length
   property
   • Date: supports basic date/time representation and parsing
   from strings
   • RegExp: created using /regex/ syntax; efficient RegEx impl
   • Null: null - deliberate non-value
   • Undefined: undefined - uninitialized value
   no I/O – object literals – common C-style if, for, switch/case, while
   statements – uncommon C-style do/while, tertiary statement, and
   short-circuiting – var keyword – string concatenation and splicing –
   

   View Slide

5. What makes JavaScript cool?
   Our first example is a map/reduce
   implementation written with JS functions.
   1. we build an iterator function called each which
   is found in most JS frameworks
   2. we implement map and reduce using each
   3. we implement a test function that maps pow2
   function (x*x) to each element in a list, and
   then reduces it via addition. Equivalent to the
   following Python:
   sum(x*x for x in items)
   

   View Slide

6. The iterator function
   function each(items, fn) {
   for (var i = 0; i < items.length; i++) {
   var item = items[i];
   fn.apply(item, []);
   }
   };
   

   View Slide

7. The mapper
   function map(fn, items) {
   var results = [];
   each(items, function() {
   var val = fn.apply(this, []);
   results.push(val)
   });
   return results;
   };
   

   View Slide

8. The reducer
   function reduce(items, fn) {
   var result;
   var first = true;
   var that = this;
   each(items, function() {
   if (first) {
   result = this;
   first = false;
   return; // continues
   }
   result = fn.apply(that, [result, this]);
   });
   return result;
   };
   

   View Slide

9. Functional programming in JS
   function test_map_reduce() {
   var adder = function(i1, i2) { return i1 + i2; };
   var pow2 = function() { return this * this; };
   var input = [1, 2, 3, 4, 5, 6];
   console.log("input: " + input);
   var mapped = map(pow2, input);
   console.log("mapped: " + mapped);
   var reduced = reduce(mapped, adder);
   console.log("reduced: " + reduced);
   };
   

   View Slide

10. It works!
    >>> test_map_reduce();
    input: 1,2,3,4,5,6
    mapped: 1,4,9,16,25,36
    reduced: 91
    

    View Slide

11. Grokking Functional JS
    Functions are first-class objects, but they
    have some quirks.
    Three declaration forms:
    • function statements
    • anonymous function expressions
    • named function expressions (exotic)
    The keyword this is used for functional
    calling context, controlled by fn.apply()
    

    View Slide

12. Declaration Forms
    function add1(i1, i2) {
    return i1 + i2;
    }
    var add2 = function(i1, i2) {
    return i1 + i2;
    }
    var add3 = function add3(i1, i2) {
    return i1 + i2;
    }
    

    View Slide

13. Calling context
    function each(items, fn) {
    for (var i = 0; i < items.length; i++) {
    var item = items[i];
    fn.apply(item, []);
    }
    }
    each([0, 1], function() { console.log(this); });
    function context
    this
    fn.apply( item , []);
    

    View Slide

14. Closure: obvious and complex
    Because JavaScript can treat function definitions as
    expressions, they can be used almost anywhere in your
    program.
    Common locations
    • Nested within other functions
    • As values in object literals
    • As arguments to other functions
    Closure allows functions to access the values of variables
    outside the scope of the function definition, even after
    that variable has gone out of scope.
    

    View Slide

15. Common Function Locations
    function nested_argument() {
    var arr = [];
    each(results, function() {
    // do something with arr
    });
    }
    function obj_literal() {
    var obj = {};
    return {
    "do": function() { /* do something with obj */ }
    };
    }
    

    View Slide

16. anonymous
    function
    Closure
    function map(fn, items) {
    var results = [];
    each(items, function() {
    var val = fn.apply(this, []);
    results.push(val)
    });
    return results;
    }
    “closes
    over” outer
    variables
    [1, 2, 3]
    results
    results
    

    View Slide

17. Closure Stack
    function stack() {
    var stack = [];
    var pusher = function(item) {
    stack.push(item);
    };
    var popper = function() {
    return stack.pop();
    };
    var peeker = function() {
    return stack[stack.length-1];
    }
    var obj = { "push": pusher, "pop": popper,
    "peek": peeker };
    return obj; // upon return, stack not in scope
    }
    Each is an
    example of
    closure
    

    View Slide

18. OO JS?
    >>> var s = stack()
    Object
    >>> s.push(0)
    >>> s.peek()
    0
    >>> s.peek()
    0
    >>> s.push(1)
    >>> s.peek()
    1
    >>> s.pop()
    1
    >>> s.pop()
    0
    >>> s.pop()
    >>> s.pop()
    

    View Slide

19. Closure Book
    

    View Slide

20. What hath we wrought?!
    

    View Slide

21. JavaScript OO vs. Java OO
    “Java is to JavaScript what
    Car is to Carpet.”
    - Chris Heilmann
    

    View Slide

22. Classic Stack
    function Stack() {
    var stack = [];
    this.push = function(item) {
    stack.push(item);
    };
    this.pop = function() {
    return stack.pop();
    };
    this.peek = function() {
    return stack[stack.length-1];
    };
    };
    

    View Slide

23. Classic Stack, improved
    var Stack = function() {
    this.stack = [];
    };
    Stack.prototype.push = function(item) {
    this.stack.push(item);
    };
    Stack.prototype.pop = function() {
    return this.stack.pop();
    };
    Stack.prototype.peek = function() {
    return this.stack[stack.length-1];
    };
    

    View Slide

24. Objects and calling context
    call_method
    this
    obj.call_method();
    call_method
    this
    var fn = obj.call_method;
    fn();
    

    View Slide

25. The new keyword
    >>> var s = new Stack()
    Object
    >>> s.push(0)
    >>> s.peek()
    0
    >>> s.peek()
    0
    >>> s.push(1)
    >>> s.peek()
    1
    >>> s.pop()
    1
    >>> s.pop()
    0
    >>> s.pop()
    >>> s.pop()
    What the heck is the
    new keyword doing?
    

    View Slide

26. The new keyword
    >>> var s = {};
    >>> Stack.apply(s, []);
    >>> s.peek()
    0
    >>> s.peek()
    0
    >>> s.push(1)
    >>> s.peek()
    1
    >>> s.pop()
    1
    >>> s.pop()
    0
    >>> s.pop()
    >>> s.pop()
    Yep, that’s all.
    

    View Slide

27. Spot the bug
    >>> var s = Stack();
    >>> s.push(0);
    TypeError: s is undefined
    . . . source=with(_FirebugCommandLine)
    . . . {s.push(0)\n};
    >>> window.push(0);
    >>> window.pop();
    0
    Ugh!
    

    View Slide

28. Now, for the bad parts
    1. In JavaScript, variables are global by default.
    2. The keyword this is bound to “the global object” by
    default, which in browsers is usually window.
    3. Since functions meant to be used with the new keyword can
    be called without the new keyword, functions that rely on
    new/this for OO programming are often accidentally
    misused.
    4. Since this changes based on calling context AND “normal”
    fn() invocation does not alter calling context, this is a
    regular source of bugs.
    

    View Slide

29. What can we do?
    – Define only a SINGLE GLOBAL for your
    application, which acts as a namespace
    – Be highly COGNIZANT OF this, and
    DOCUMENT when you rely on it
    – Name all functions meant to be used with new
    with CamelCase, e.g. Stack
    – Use the MODULE PATTERN and CLOSURE to
    organize your code
    – Follow OO patterns and use a FRAMEWORK
    – Use jslint to check our source code
    

    View Slide

30. Module Pattern
    var NS = function () { // create namespace
    // private module data
    var VERSION = “1.2”;
    // public module classes, functions, and variables
    var m = {};
    m.DateParser = function() {
    this.parse = function() {};
    };
    m.parse_date = function(s) {
    var dp = new m.DateParser();
    return dp.parse(s);
    };
    return m;
    }(); // invoke function to hide private data
    

    View Slide

31. Using a module
    >>> NS.parse_date(“2009-09-01”);
    DateParser(year=2009, month=09, day=01)
    >>> new NS.DateParser().parse(“2009-09-01”);
    DateParser(year=2009, month=09, day=01)
    

    View Slide

32. No Standard Library
    • JavaScript desperately needs a standard
    library
    • Heavy development is underway to
    achieve this:
    – chironjs / narwhal
    – CommonJS
    – ECMAScript standard
    – For server-side, Node.JS
    

    View Slide

33. My pragmatic client-side JS stack
    These are libraries I use every day for real programming with JS:
    – jQuery: CSS-style DOM selector, animations, event binding, HTTP
    requests
    – Sometimes, ExtJS: Observable, ISO8601 DateTime, OO,
    XTemplate, Store, DomHelper, JSON, Forms, TaskRunner, and a
    slew of extensible UI components
    – Underscore.js: functional primitives
    – DateJS: extended date parsing and manipulation
    – BlackbirdJS: cross-browser client-side logging
    – Bootstrap (new!): standard UI components for HTML5
    

    View Slide

34. My Pragmatic Dev Env
    These are the tools I use on a daily basis for doing
    JS prototyping and development:
    – Firebug and Live HTTP Headers
    – Webstorm
    – node, especially node-static
    

    View Slide

35. ACT I, Fin
    This ends our look at the JavaScript language.
    Now, on to Node.JS.
    

    View Slide

36. Useful Resources
    My favorite JavaScript reference documents:
    http://delicious.com/amontalenti/my:jsref
    Other questions?
    @amontalenti
    

    View Slide

37. View Slide

38. View Slide

39. Act II, Node.JS
    

    View Slide

40. View Slide

41. Why Server-Side JS (SSJS)?
    • The Grand Unified Stack: JavaScript front to
    back, baby.
    • The rise of Rich Internet Applications (RIAs)
    has led to a smaller role for servers anyway.
    • Re-use between client/server (e.g. form
    validation logic, utilities, object models)
    may de-duplicate code
    • Single syntax may reduce programmer
    context switching
    

    View Slide

42. Why SSJS has failed thus far?
    • Netscape LiveWire
    – Tied to a commercial vendor
    – JavaScript was not even adopted yet
    • Rhino
    – Open source
    – No community (identity crisis)
    – Poor performance
    – JavaScript was not cool yet
    

    View Slide

43. Node.JS: right place, right time
    • And, the right tool.
    • Node.JS
    – Open source
    – Library to build HTTP servers
    – Riding asynchronous wave (epoll)
    – Riding realtime wave (COMET, Websockets)
    – JavaScript finally cool, thanks to AJAX/RIAs
    – Strong community, and growing
    

    View Slide

44. Revenge of JS
    • “Despite JavaScript's astonishing shortcomings, deep down, in its core, it
    got something very right.
    • JavaScript was the world's most misunderstood programming language.
    • Its obvious defects, its unfashionable programming model, intentional
    mispositioning at its introduction, and its ridiculous name caused it to be
    rejected as unworthy by most knowledgeable programmers. But Ajax gave
    JavaScript a second chance.
    

    View Slide

45. It’s better to be lucky
    • Because JavaScript is the language of the web browser, and
    because the web browser has become the dominant application
    delivery system, and because JavaScript isn't too bad, JavaScript
    has become the World's Most Popular Programming Language. Its
    popularity is growing. It is now being embedded in other
    applications and contexts. JavaScript has become important.
    • It is better to be lucky than smart.”
    – Douglas Crockford
    

    View Slide

46. Does coolness matter?
    

    View Slide

47. What Node.JS isn’t…
    • Django/Rails
    – Pieces of Django/Rails are implemented by
    npm modules, but no unified “web stack” yet
    • Rhino/Spidermonkey
    – Node.JS is one layer up from these (think
    Python/Ruby standard library, not the
    interpreter itself).
    – But… Node.JS uses a different interpreter,
    too.
    

    View Slide

48. What Node.JS is…
    • A standard library for JavaScript
    • An asynchronous server library (epoll)
    – Twisted
    – Tornado
    – EventMachine
    • Because programmable async servers
    were the reason Node.JS was written, its
    standard library has an async bent.
    

    View Slide

49. Node.JS Architecture
    

    View Slide

50. OK, but what is Node.JS, really?
    • It’s clear devs want it to become as
    powerful as Django/Rails for web app
    development
    – express module is starting to provide routing
    and controllers
    – mongodb and redis are popular data stores
    for Node
    – Template languages are ported every day
    

    View Slide

51. The promise:
    Node can simplify the web stack
    • Simplify the backend with JSON-based,
    async write/read data stores
    • Simplify the middle tier with clientside /
    serverside code sharing.
    • Simplify the frontend, with server-backed
    data bindings
    • Simplify deployment, since Node.JS runs
    directly in production.
    

    View Slide

52. Aside: my intro to epoll
    “The C10K Problem”
    

    View Slide

53. YAGNI, right?
    

    View Slide

54. Node.JS is programmable nginx
    

    View Slide

55. I/O is our biggest bottleneck
    

    View Slide

56. HTTP Hello, World
    var http = require('http');
    var server = http.createServer(function(req, rsp) {
    rsp.writeHead(200, {"Content-Type":
    "text/plain"});
    rsp.end("Hello World\n");
    });
    server.listen(8000);
    console.log("Server running at :8000");
    

    View Slide

57. Express Hello, World
    var app = require('express').createServer()
    app.get('/', function(req, rsp) {
    rsp.writeHead(200, {"Content-Type":
    "text/plain"});
    rsp.end("Hello World\n");
    });
    app.listen(8000);
    console.log("Server running at :8000");
    

    View Slide

58. UNIX Hello, World
    var fs = require('fs');
    var lazy = require('lazy');
    lazy(fs.createReadStream("/usr/share/dict/words"))
    .lines
    .forEach(function(line) {
    var line = line.toString();
    if (line.indexOf("hello") !== -1
    || line.indexOf("world") !== -1) {
    console.log(line);
    }
    });
    

    View Slide

59. Client/Server Hello, World (1)
    var
    html = require('fs’).readFileSync(…),
    srv = require('http’).createServer(function(req, res){
    res.end(html);
    });
    server.listen(8080);
    var nowjs = require("now");
    var everyone = nowjs.initialize(server);
    everyone.now.distributeMessage = function(message){
    everyone.now.receiveMessage(this.now.name, message);
    };
    

    View Slide

60. Client/Server Hello, World (2)
    $(document).ready(function(){
    now.receiveMessage = function(name, message){
    $("#messages")
    .append("
    " + name + ": " + message);
    }
    $("#send-button").click(function(){
    now.distributeMessage("Hello, world");
    });
    now.name = prompt("What's your name?", "");
    });
    

    View Slide

61. Notice a trend?
    • Node.JS relies heavily on what you might
    call the Hollywood Principle
    • “Don’t call us, we’ll call you”
    • Node.JS achieves this through a heavy
    use of callbacks and errbacks
    • Luckily, we now know Functional JS!
    

    View Slide

62. Why callbacks?
    • In Node.JS, everything runs in parallel,
    except your code.
    • Let’s say you register two callbacks in
    Express.
    – Node.JS registers those callbacks, and goes to
    sleep.
    – Request comes in, dispatches it to callback –
    meanwhile, any other requests wait in line.
    – As long as your callback doesn’t block,
    performance is guaranteed.
    

    View Slide

63. OK, let’s break it
    • Let’s see what actually happens if one of
    our callbacks takes a long time to
    respond.
    

    View Slide

64. Broken World
    var app = require('express').createServer()
    var i = 1;
    app.get('/', function(req, rsp) {
    if (i === 3) {
    console.log("spin needlessly");
    while (1) {}
    }
    rsp.writeHead(200, {"Content-Type": "text/plain"});
    rsp.end("Hello World " + i + "\n");
    i++;
    });
    app.listen(8000);
    console.log("Server running at :8000");
    

    View Slide

65. Enough Hello, World.
    How about Real World?
    • There is a Node.JS sweet spot right now.
    • Real-time Analytics
    – Low latency, high concurrency HTTP/UDP
    – JSON is a natural fit for data exchange
    – Client-side “pixel” can be synced w/ server-
    side pixel parser
    – Websockets natural fit for viewing live data
    – MongoDB a natural fit for data store
    • We’ll look at three examples.
    

    View Slide

66. View Slide

67. View Slide

68. StatsD
    

    View Slide

69. Parse.ly real-world use case
    • Parse.ly uses Node.JS for what we call
    “dynamic JavaScript configuration”
    • Client-side JavaScript code makes a JSON-P
    call to a “config server”, which returns
    JavaScript configuration settings
    • Allows us to load custom JS for individual
    publishers, or even individuals
    • Run A/B tests, do sampling, etc.
    • It’s perfect for this (esp. with Redis)
    

    View Slide

70. My eye is also on Node.IO
    • Web crawling framework built with Node
    • Borrows from jQuery for selectors
    • Best possible DOM comprehension
    possible with JavaScript
    – No more “real world HTML” and “valid
    XHTML” mismatch problems
    • Evented system fits nicely. I use Scrapy in
    the Python world for this now.
    

    View Slide

71. More use cases born every day
    • Chat applications
    • Collaboration tools
    • JSON API servers
    • Telephony systems
    

    View Slide

72. ACT II, Fin
    • That’s it, a whirlwind tour through old and
    weary functional JavaScript and its
    ambitious younger brother, Node.JS.
    • To follow me:
    – @amontalenti
    – gplus.to/amontalenti
    – linkedin.com/in/andrewmontalenti
    – andrew@parsely.com
    – http://pixelmonkey.org
    

    View Slide