Upgrade to Pro — share decks privately, control downloads, hide ads and more …

jseverywhere2012

Ben Wen
October 26, 2012

 jseverywhere2012

Programmers are building and designing Web APIs, often before the Web or mobile experience. Real-time APIs are now becoming more prevalent with the acceptance of WebSockets and Node.js. In this talk, we will walk through a Socket.io ‘push’ application that uses MongoDB capped collections and a simple Javascript client. The code is running at tractorpush.herokuapp.com and is available on Github at https://github.com/mongolab/tractorpush-server

Ben Wen

October 26, 2012
Tweet

More Decks by Ben Wen

Other Decks in Programming

Transcript

  1. Socket.io & realtime & MongoDB tailable cursors, Oh my Ben

    Wen - VP MongoLab - [email protected] - @benwen - http://linkedin.com/in/benwen JS.Everywhere() 10.24.2012 An API example with Node.js and capped collections
  2. Motivation: The client-server environment has changed •  Web 1.0 – 

    Single client environment: web browser –  Server sends PRESENTATION to client (HTML) –  Server controls “page flow” of application •  Web 2.0 –  Multiple heterogeneous client environments •  Web browser with rich client functionality (HTML5 / AJAX) •  Mobile web •  Mobile apps: iPhone (Objective-C), Android (Java), etc… •  Browser plug-ins •  TV (it’s coming) •  Flash (it’s going) –  Mashups: client receives data from multiple servers –  Server sends DATA to the client (JSON, XML (yuck), RSS, KML) –  Client and server more de-coupled –  ALL IN “SOFT” REALTIME
  3. Client is getting thicker (smarter) server is getting thinner app

    server browser HTML (presentation) database web browser mobile browser database web 1.0 web 2.0 app server a lot of presentation logic / app flow very little presentation logic JSON, XML, RSS, KML (data) apps other Very rich and dynamic (HTML5 / CSS3) Lots of presentation logic
  4. Your new job: make APIs app server browser database web

    browser mobile browser database web 1.0 web 2.0 app server a lot of presentation logic / app flow very little presentation logic apps other Very rich and dynamic (HTML5 / CSS3) Lots of presentation logic BUILD THIS! HTML (presentation) JSON, XML, RSS, KML (data)
  5. We want a stack optimized for moving JSON other web

    services JSON JSON JSON node mongodb AND IN “REALTIME”! (psst, visit nodestack.org)
  6. On multiple Platforms as a Service •  Heroku: http://tractorpush.herokuapp.com/ – 

    Standard bearer PaaS •  AppFog: http://tractorpush.aws.af.cm/ –  Multi-cloud PaaS. Acquired nodester.com •  Nodejitsu: http://tractorpush.jit.su/ –  Node.js dedicated PaaS •  Windows Azure: http://tractorpush.azurewebsites.net/ –  Globally distributed PaaS •  (working on others… )
  7. Node.js: What is it? •  The Javascript part: V8 engine

    •  The Node part: –  single threaded event loop –  non-blocking I/O •  Stunning results –  with 1 process, 1 core, < 1GB RAM –  can push Gbps of traffic, with 100k’s of connections
  8. Threaded vs. Evented servers Threaded •  One thread per request

    •  Creating each thread is expensive •  There is a max # threads Evented •  Single threaded •  Event loop •  Non-blocking I/O •  Can handle lots of open connections
  9. What is Node.js great at? •  Great for I/O bound

    services and APIs •  Perfect for real-time, push / long-polling •  Great for server to server
  10. Socket.io •  Realtime cross-browser ‘push’ communication •  Clients listen on

    event types •  Multiple underlying transports –  WebSockets: ‘Upgrade’ of HTTP for full-duplex communication –  Flash Sockets –  AJAX Long polling –  Etc. (multipart streaming, iframe, JSONP) •  Perfect for Node.js’s asynchronous nature •  MIT License
  11. WebSockets •  Heroku: http://tractorpush.herokuapp.com/ –  No WebSockets •  AppFog: http://tractorpush.aws.af.cm/

    –  No WebSockets •  Nodejitsu: http://tractorpush.jit.su/ –  YES! WebSockets •  Windows Azure: http://tractorpush.azurewebsites.net/ –  No WebSockets
  12. How many of you have done this? ID SSN_LAST F_NAME

    L_NAME PHONE ALT_PHONES 1 “4347” “Bob” “Smith” “4155164347” “4158983787, 4156563987, 4159878787” 2 “9489” “Mary” “Jones” “6502132333” “6503134421, 6509872736” 3 “9898” “David” “Bass” “6504143451” “6503134421, 6509876228, 6502334998” 4 … and felt guilty… but convinced yourself you won’t need to search on this column anyway…
  13. Forget big data (for the moment) , this is a

    data structure thing {
 _id: 1234,
 author: { name: “Bob Davis”, email : [email protected] },
 post: “In these troubled times I like to …“,
 date: { $date: “2010-07-12 13:23UTC” },
 location: [ -121.2322, 42.1223222 ],
 rating: 2.2, 
 comments: [
 { user: “[email protected]”,
 upVotes: 22,
 downVotes: 14, 
 " text: “Great point! I agree” },
 { user: “[email protected]”,
 upVotes: 421,
 downVotes: 22, 
 " text: “You are a moron” } 
 ],
 tags: [ “Politics”, “Virginia” ]
 }"
  14. {
 _id: 1234,
 author: { name: “Bob Davis”, email :

    [email protected] },
 post: “In these troubled times I like to …“,
 date: { $date: “2010-07-12 13:23UTC” },
 location: [ -121.2322, 42.1223222 ],
 rating: 2.2, 
 comments: [
 { user: “[email protected]”,
 upVotes: 22,
 downVotes: 14, 
 " text: “Great point! I agree” },
 { user: “[email protected]”,
 upVotes: 421,
 downVotes: 22, 
 " text: “You are a moron” } 
 ],
 tags: [ “Politics”, “Virginia” ]
 }" This is involves less brain damage…
  15. MongoDB is great for APIs •  JSON-based document (object) database

    •  Flexible query language •  Atomic updates •  Indexes •  Geospatial awareness (geo apis are cool!) •  Ok, and it scales…. –  Replica-set clusters –  Auto-sharding –  ** BUT THIS IS NOT THE TOPIC OF THE TALK **
  16. MongoDB capped collections •  Allows for tailable cursors –  block

    when out of data, instead of issuing an exception •  Fixed in size (‘capped’) •  Very high performance •  Maintain insertion order •  Not shardable
  17. More info •  Nodestack –  http://nodestack.org •  MongoDB / 10gen

    –  http://mongodb.org –  http://10gen.com •  Node.js / Joyent –  http://nodejs.org –  http://joyent.com •  MongoLab –  http://mongolab.com –  [email protected] •  Tractorpush Code –  Server: •  https://github.com/mongolab/ tractorpush-server –  Writer •  https://github.com/mongolab/ tractorpush-writer-ruby –  Doc •  https://devcenter.heroku.com/ articles/realtime-polyglot-app- node-ruby-mongodb-socketio –  Presentation •  https://speakerdeck.com/ benwen/jseverywhere2012
  18. Thank You Ben Wen - VP MongoLab - [email protected] -

    @benwen - http://linkedin.com/in/benwen JS.Everywhere() 10.24.2012