Upgrade to Pro — share decks privately, control downloads, hide ads and more …

So you want to build a WebSocket Abstraction

So you want to build a WebSocket Abstraction

The slide deck my talk at Realtimeconf 2013

Arnout Kazemier

October 18, 2013
Tweet

More Decks by Arnout Kazemier

Other Decks in Technology

Transcript

  1. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland A SO YOU

    WANT TO BUILD WEBSOCKET ABSTRACTION Friday, October 18, 13
  2. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland   

     3rd-Eden 3rdEden arnout.kazemier arnoutkazemier Friday, October 18, 13
  3. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland  Friday, October

    18, 13
  4. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland  Friday, October

    18, 13
  5. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland primus transforming the

    way we use real-time Friday, October 18, 13
  6. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland open source is

    flawed Friday, October 18, 13
  7. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland open source is

    flawed Friday, October 18, 13
  8. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland Friday, October 18,

    13
  9. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland Friday, October 18,

    13
  10. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland Friday, October 18,

    13
  11. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland Friday, October 18,

    13
  12. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland Friday, October 18,

    13
  13. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland 1 line of

    code to switch between excluding adding a framework to your package.json Friday, October 18, 13
  14. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland 1 line of

    code to switch between excluding adding a framework to your package.json different frameworks build-in socket.io, engine.io, sockjs, websockets, browserchannel, but allows addition third party 5 Friday, October 18, 13
  15. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland 1 line of

    code to switch between excluding adding a framework to your package.json different frameworks build-in socket.io, engine.io, sockjs, websockets, browserchannel, but allows addition third party 5 stars, but growing steady star it on http://github.com/primus/primus 279 Friday, October 18, 13
  16. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland 1 amazing plugin

    interface keeps the core light, while maximizing developer flexibility Friday, October 18, 13
  17. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland 1 amazing plugin

    interface keeps the core light, while maximizing developer flexibility different ways to stay connected online/offline detection, client side pings combined with different reconnect strategies 5 Friday, October 18, 13
  18. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland 1 amazing plugin

    interface keeps the core light, while maximizing developer flexibility different ways to stay connected online/offline detection, client side pings combined with different reconnect strategies 5 reasons on why you should use it fixes bugs in real-time frameworks, fixes websocket crashes, it has a cool name, node.js stream interface, an api that never changes ∞ Friday, October 18, 13
  19. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland MADE WITH ♥

    oh, and it’s used in production at nodejitsu and observe.it <3 Friday, October 18, 13
  20. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland ◅ ▻ var

    Primus = require(‘primus’) , http = require(‘http’); // Create a HTTP server, attach Primus, specify transformer & done var server = http.createServer() , primus = new Primus(server, { transformer: ‘websockets’ }); primus.on(‘connection’, function (spark) { spark.write(‘foo’); spark.write({ json: ‘also works’ }); spark.on(‘data’, function (data) { console.log(‘data’, data); }); // Sparks are streams, so pipe data. require(‘fs’).createReadStream(__dirname +’/index.js’).pipe(spark); }); server.listen(80); the server Friday, October 18, 13
  21. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland ◅ ▻ var

    primus = new Primus(‘http://localhost:8080?foo=bar', { // Options to further customize Primus. }); primus.on(‘data’, function (data) { console.log(‘received’, data); }); primus.on(‘end’, function () { console.log(‘connection end’); }); primus.write({ sending: 1337 }); primus.write(‘same syntax as node’); the client Friday, October 18, 13
  22. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland WHAT COULD BE

    YOUR REASON? Friday, October 18, 13
  23. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland Friday, October 18,

    13
  24. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland scalability Friday, October

    18, 13
  25. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland better performance scalability

    Friday, October 18, 13
  26. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland high availability better

    performance scalability Friday, October 18, 13
  27. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland high availability better

    performance scalability NIH Friday, October 18, 13
  28. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland high availability debug

    better performance scalability NIH Friday, October 18, 13
  29. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland high availability debug

    better performance scalability NIH latency Friday, October 18, 13
  30. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland high availability debug

    better performance scalability NIH OMG WHY ARE YOU USING COFFEESCRIPT latency Friday, October 18, 13
  31. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland high availability debug

    better performance scalability NIH MAINTENANCE OMG WHY ARE YOU USING COFFEESCRIPT latency Friday, October 18, 13
  32. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland high availability debug

    better performance biz integration scalability NIH MAINTENANCE OMG WHY ARE YOU USING COFFEESCRIPT latency Friday, October 18, 13
  33. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland high availability missing

    critical features debug better performance biz integration scalability NIH MAINTENANCE OMG WHY ARE YOU USING COFFEESCRIPT latency Friday, October 18, 13
  34. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland high availability missing

    critical features debug better performance biz integration scalability NIH MAINTENANCE OMG WHY ARE YOU USING COFFEESCRIPT latency freedom Friday, October 18, 13
  35. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland THE abstraction requirements

    Friday, October 18, 13
  36. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland Friday, October 18,

    13
  37. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland message encoding Friday,

    October 18, 13
  38. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland message encoding data

    transfer Friday, October 18, 13
  39. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland AUTHENTICATION & HANDSHAKING

    message encoding data transfer Friday, October 18, 13
  40. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland AUTHENTICATION & HANDSHAKINGEXTENDABLE

    BY PLUGINS message encoding data transfer Friday, October 18, 13
  41. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland work in hostile

    user environments AUTHENTICATION & HANDSHAKINGEXTENDABLE BY PLUGINS message encoding data transfer Friday, October 18, 13
  42. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland reliable connections work

    in hostile user environments AUTHENTICATION & HANDSHAKINGEXTENDABLE BY PLUGINS message encoding data transfer Friday, October 18, 13
  43. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland AUTOMATIC RECONNECT reliable

    connections work in hostile user environments AUTHENTICATION & HANDSHAKINGEXTENDABLE BY PLUGINS message encoding data transfer Friday, October 18, 13
  44. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland AUTOMATIC RECONNECT reliable

    connections no packet loss work in hostile user environments AUTHENTICATION & HANDSHAKINGEXTENDABLE BY PLUGINS message encoding data transfer Friday, October 18, 13
  45. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland AUTOMATIC RECONNECT no

    high latency reliable connections no packet loss work in hostile user environments AUTHENTICATION & HANDSHAKINGEXTENDABLE BY PLUGINS message encoding data transfer Friday, October 18, 13
  46. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland AUTOMATIC RECONNECT no

    high latency reliable connections no packet loss work in hostile user environments AUTHENTICATION & HANDSHAKINGEXTENDABLE BY PLUGINS message encoding data transfer no code bloat Friday, October 18, 13
  47. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland AUTOMATIC RECONNECT NETWORK

    CONDITION HEARTBEATS no high latency reliable connections no packet loss work in hostile user environments AUTHENTICATION & HANDSHAKINGEXTENDABLE BY PLUGINS message encoding data transfer no code bloat Friday, October 18, 13
  48. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland AUTOMATIC RECONNECT NETWORK

    CONDITION HEARTBEATS no high latency reliable connections no packet loss CROSS BROWSER PLATFORM DOMAIN / / work in hostile user environments AUTHENTICATION & HANDSHAKINGEXTENDABLE BY PLUGINS message encoding data transfer no code bloat Friday, October 18, 13
  49. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland FYI real-time is

    hard Friday, October 18, 13
  50. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland FYI real-time is

    an adventure Friday, October 18, 13
  51. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland FYI real-time is

    an adventure Friday, October 18, 13
  52. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland metrics should be

    the first component you build knowledge is power, but only for those who use it Friday, October 18, 13
  53. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland data transfer gzip

    ajax and websocket frames keeps the core light, while maximizing developer flexibility Friday, October 18, 13
  54. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland data transfer proxies

    and security software mangle headers again, use secure connections to obfuscate the connection from this software, use regular expressions or direct gzip detection Accept-EncodXng: gzip, deflate X-cept-Encoding: gzip, deflate XXXXXXXXXXXXXXX: XXXXXXXXXXXXX ---------------: ------------- ~~~~~~~~~~~~~~~: ~~~~~~~~~~~~~ source: http://developer.yahoo.com/blogs/ydn/pushing-beyond-gzipping-25601.html Friday, October 18, 13
  55. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland stop inventing protocols

    just use JSON keeps the core light, while maximizing developer flexibility Friday, October 18, 13
  56. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland prevent packet drops

    use packet/message numbering no open source framework implements this, but has been solved for ages Friday, October 18, 13
  57. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland xhr jsonp handshake

    connection authorization generate a custom connection id which should be used for each request / Friday, October 18, 13
  58. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland customize handshake response

    retrieving initial state from server or other data that is needed to give the connection it’s initial state and saves an extra poll request Friday, October 18, 13
  59. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland connect with query

    strings sending initial state to server again, to save inital request, this can save a extra server request Friday, October 18, 13
  60. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland fix multiple tabs

    use multiple subdomains / hosts no open source framework implements this, but has been solved for ages this can also be used for load balancing when done correctly Friday, October 18, 13
  61. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland ping pong heartbeat

    server side ping over come maximum connection holding limitations Friday, October 18, 13
  62. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland ping pong heartbeat

    server side ping over come maximum connection holding limitations Friday, October 18, 13
  63. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland ping pong heartbeat

    client side ping detect connection drops, like wifi changes or 3G to wifi upgrades etc Friday, October 18, 13
  64. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland ping pong heartbeat

    leverage native pings when available websockets have ping/pong protocols which are optimised for this, no api for the browser :( because they didn’t find it useful enough and that it should be implemented by browser vendors (which didn’t implement it) Friday, October 18, 13
  65. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland ping pong heartbeat

    make the messages useful send metrics, messages, something to make these ping/pongs more effecient Friday, October 18, 13
  66. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland support multiple transports

    JSONP-polling, long polling, HTMLFile, EventSource WebSockets and think outside the box allow custom transports maybe SPDY push script files? Datachannels? Friday, October 18, 13
  67. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland support multiple transports

    force HTTPS and upgrade them correctly while handling port upgrades / downgrades for WebSockets Friday, October 18, 13
  68. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland focus on cross

    domain use postMessage / iframes to transform same domain in to cross domain Friday, October 18, 13
  69. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland iframe all the

    things their here to assist you, instead of haunt you sandbox your library with the most under estimated technology currently available on the web actually... Friday, October 18, 13
  70. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland be extremely paranoid

    one single bug can take down your connection watch my previous talks about websuckets and their issues - enough to fill a single talk Friday, October 18, 13
  71. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland be extremely paranoid

    no-cache and random query string EVERYTHING iOS6 caches POST requests, query strings prevent GET caching as well Friday, October 18, 13
  72. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland warning node.js specific

    slides ahead ⚠ Friday, October 18, 13
  73. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland build for failure

    crash only design & service based architecture this in my opinion the easiest to scale and node crashes a lot Friday, October 18, 13
  74. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland make it a

    stream no flow control library can beat stream#pipe yes, I know, streams are slow… but usability & maintenance wins over performance every day ™ Friday, October 18, 13
  75. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland prevent gc at

    all cost stop creating new objects, pool & reuse this is a serious pain point for node.js and real-time.. you might even be better of with erlang Friday, October 18, 13
  76. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland   

     3rd-Eden 3rdEden arnout.kazemier arnoutkazemier Friday, October 18, 13
  77. Arnout Kazemier, founder observe.it Realtimeconf 2013, Portland thanks building your

    own abstraction is rewarding take this journey and start an adventure Friday, October 18, 13