BBC Now ...that's what I call Node

Red Badger gives an overview of the Node project they carried out for the BBC, collating BBC content as it's published and pushing it to the BBC homepage in real time.

Transcript

1. BBC Now ...that's what I call Node
2. Architectural Overview Redis MongoDB Brand Association Feed Data Admin Site CRUD change.[type] firehose.[type] Other iPlayer Now Playing Twitter RSS Content Processors Public Site firehose.* Filtering Backfill Processor Trend Analysis [service] Services firehose.*
3. Real Time Transports • Polling ◦ The traditional approach, AJAX requests every n seconds to check for new content. • WebSockets ◦ Bidirectional, TCP layer, Supports IE10+ but fallbacks with Socket.IO etc. • Server Sent Events ◦ Unidirectional, HTTP, simple protocol.
4. Server Sent Events - The Good • It's just HTTP • Served directly from our express app • Super simple plaintext format • Easy to produce, easy to consume • JS Polyfill available where there's no native support... So it works with IE!
5. Server Sent Events - The Stream Response: id: 101 event: update data: { "title": "Obama Wins : The Reaction" } id: 102 event: popular-topics data: [ "President Obama", "Formula 1", "NSA" ] Request: Last-Event-ID: 100
6. Server Sent Events - The Bad • Its a long running connection, often terminated by corporate proxies etc. • It has an odd MIME type, text/event-stream, this means browsers don't try and read it by default • Does it scale?
7. Loosely Coupled • Frontend ◦ Lightweight express app ◦ Heavily cached by Nginx ◦ In memory cache kept fresh by Redis • Backend ◦ Significantly higher load ◦ Many external dependencies ◦ Higher chance of failure Backend Redis Services Content Processors MongoDB Frontend 1..n Nginx Node Redis Connection * Frontend n
8. Thanks! Questions? @redbadgerteam http://now.pilots.bbcconnectedstudio.co.uk David Wynne @dwynne Joe Stanton @joe_stant