Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Evented Web

The Evented Web

Given at the Denver Open Source Users Group (@dosug) Nov. 1, 2011

7071119714e1a32441aca0c336657d3e?s=128

Mike Brevoort

November 01, 2011
Tweet

Transcript

  1. The Evented Web webhooks and websockets the dynamic duo Mike

    Brevoort @ mbrevoort Denver Open Source User Group November 1, 2011 1
  2. • The evented What? Why? • Webhooks • Websockets •

    Asynchronous I/O and node.js • Putting it all together 2
  3. Semi-related points in time 3

  4. 1990 HTML 1993 Common Gateway Interface (CGI) 1997 Java servlets

    1999 XMLHTTP ActiveX control 2004 Gmail, “web 2.0” 2005 AJAX coined CometD/Bayeux 2006 2010 (Dec 2009) Websockets Draft 4
  5. Conditioned... ...into aone sided conversation wait ...to http://sites.google.com/site/davidnchung/StakeElephant.jpg 5

  6. When we should have been doing this! http://www.tecacentre.net/funpics/funpics/flying_elephant.jpg 6

  7. The traditional web Call Me... I won’t call you. 7

  8. boiling yet? boiling yet? boiling yet? boiling yet? boiling yet?

    boiling yet? boiling yet? boiling yet? boiling yet? boiling yet? boiling yet? http://www.charlesculp.com/isleroyale07/BoilingWater_IMG_3487_800_160.jpg 8
  9. anonymous web the disconnected 9

  10. it’s all about events http://www.flickr.com/photos/cizake/4164756091/ 10

  11. I will call them Webhooks. And they will be good

    Jeff Lindsay @progrium 2006 11
  12. what is a Webhook? callbacks over HTTP ... an HTTP

    POST that occurs when something happens 12
  13. Server Server 13

  14. scalability performance throughput realtime instant data 14

  15. scalability performance throughput realtime instant data 14

  16. 15

  17. I got it! 15

  18. When something happens, I’ll just call you! I got it!

    15
  19. When something happens, I’ll just call you! I got it!

    So... relax 15
  20. it’s simple 16

  21. it’s simple “even a baby can do it” 16

  22. configurable event handling user to infinity and beyond! 17

  23. credit: Jeff Lindsay @progrium 18

  24. receive data in real time 19

  25. receive data in real time Superpoke /remove 19

  26. receive data in real time Facebook User removes app Superpoke

    /remove 19
  27. receive data in real time Facebook User removes app HTTP

    POST signed request w/ user id Superpoke /remove 19
  28. build script it! resolve issue change 20

  29. receive data, do something, pass it on $ cat apple.txt

    | wc | mail -s "The count" joe@me.com http://www.flickr.com/photos/russmorris/2713018257 21
  30. Yo’ App Yo’ Warehouse 22

  31. Customer Pays Yo’ App Yo’ Warehouse 22

  32. Customer Pays Yo’ App Yo’ Warehouse event New Order HTTP

    POST 22
  33. Customer Pays Yo’ App Yo’ Warehouse event New Order HTTP

    POST 22
  34. Customer Pays Yo’ App Yo’ Warehouse event New Order HTTP

    POST New Order HTTP PO ST 22
  35. Customer Pays Yo’ App Yo’ Warehouse event New Order HTTP

    POST New Order HTTP PO ST Come and get it HTTP POST 22
  36. process data, give something in return like a typical API

    call but the contract is dictated by the caller (not the callee) 23
  37. process data, give something in return like a typical API

    call but the contract is dictated by the caller (not the callee) when something happens, I’m going to send you Y 23
  38. process data, give something in return like a typical API

    call but the contract is dictated by the caller (not the callee) when something happens, I’m going to send you Y OK 23
  39. process data, give something in return like a typical API

    call but the contract is dictated by the caller (not the callee) when something happens, I’m going to send you Y just tell me where 23
  40. process data, give something in return like a typical API

    call but the contract is dictated by the caller (not the callee) when something happens, I’m going to send you Y just tell me where OK 23
  41. process data, give something in return like a typical API

    call but the contract is dictated by the caller (not the callee) when something happens, I’m going to send you Y just tell me where Respond back with X OR call me back HERE 23
  42. process data, give something in return like a typical API

    call but the contract is dictated by the caller (not the callee) when something happens, I’m going to send you Y just tell me where Respond back with X OR call me back HERE OK 23
  43. process data, give something in return like a typical API

    call but the contract is dictated by the caller (not the callee) when something happens, I’m going to send you Y just tell me where Respond back with X OR call me back HERE OK 23
  44. 24

  45. Webhook Standards? • Yes, it’s called HTTP • OK, but

    there are a few related conventions/standards including RESTFul webhooks, pubsubhubbub, etc 25
  46. Webhook Standards? • Yes, it’s called HTTP • OK, but

    there are a few related conventions/standards including RESTFul webhooks, pubsubhubbub, etc “even a baby can do it” 25
  47. http://code.google.com/p/pubsubhubbub/ 26

  48. http://code.google.com/p/pubsubhubbub/ 27

  49. http://code.google.com/p/pubsubhubbub/ 28

  50. http://code.google.com/p/pubsubhubbub/ 29

  51. http://code.google.com/p/pubsubhubbub/ 30

  52. http://code.google.com/p/pubsubhubbub/ 31

  53. http://code.google.com/p/pubsubhubbub/ 32

  54. Webhooks • powerful glue • plumbing of the evented web

    • empowering 33
  55. Server Client Websockets 34

  56. Server Websockets web Browser (the hard part) 34

  57. Flash sockets Comet Silverlight Duplex Websockets Server web Browser 35

  58. Websockets enables native bidirectional communication between a server and a

    web browser 36
  59. W3C API and IETF Protocol Shares ports with HTTP/S (80,

    443) Cross-Origin Works across firewalls, proxies and routers (theoretically) Websockets 37
  60. Comet long held HTTP requests umbrella term for multiple techniques

    aka “Ajax Push”, “Reverse Ajax”, “HTTP Streaming” compatible 38
  61. Flash Sockets raw TCP socket requires Flash runtime proxy and

    router woes can implement websocket protocol 39
  62. Silverlight Duplex socket or polling requires Silverlight sockets over ports

    4502-4534 can implement websocket protocol 40
  63. Why not comet? • an inefficient hack? • 1/2 duplex

    • HTTP Overhead • latency http://websocket.org/quantum.html 41
  64. Polling http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=/com.ibm.websphere.ajax.devguide.help/docs/PureAjax_pubsub_clients.html Comet 42

  65. Polling http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=/com.ibm.websphere.ajax.devguide.help/docs/PureAjax_pubsub_clients.html Comet 42

  66. http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=/com.ibm.websphere.ajax.devguide.help/docs/PureAjax_pubsub_clients.html Long Polling Comet 43

  67. Keep-Alive Streaming http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=/com.ibm.websphere.ajax.devguide.help/docs/PureAjax_pubsub_clients.html Comet 44

  68. a protocol http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-03 Websockets 45

  69. an API http://dev.w3.org/html5/websockets/ var socket = new WebSocket("ws://host/socket/"); socket.onopen =

    function() { ! console.log("viva la socket!"); } socket.onmessage = function(msg) { ! console.log(msg);! } Websockets 46
  70. Browser Support http://caniuse.com Websockets 47

  71. Security Flaw • secure Websockets (wss://) not affected • not

    specifically Websockets, rather affects all sockets over HTTP including Flash and Java • IETF study of 47,338 HTTP proxies tested found 0.37% and 0.017% were vulnerable to the two attack types • Browser vendors want to bulletproof Websocket wire protocol to handle even buggy implementations of intercepting proxies • Websocket protocol R6 introduced 2/6/2011, actively being reviewed. More info: http://tools.ietf.org/wg/hybi/ http://security.sys-con.com/node/1642956 Websockets 48
  72. Security Flaw • secure Websockets (wss://) not affected • not

    specifically Websockets, rather affects all sockets over HTTP including Flash and Java • IETF study of 47,338 HTTP proxies tested found 0.37% and 0.017% were vulnerable to the two attack types • Browser vendors want to bulletproof Websocket wire protocol to handle even buggy implementations of intercepting proxies • Websocket protocol R6 introduced 2/6/2011, actively being reviewed. More info: http://tools.ietf.org/wg/hybi/ http://security.sys-con.com/node/1642956 Websockets Resolved 48
  73. What about the server? Websockets 49

  74. Traditional Web Servers just won’t do 50

  75. Traditional Web Servers just won’t do 50

  76. Why? All these webhooks and websockets mean orders of magnitude

    more concurrent, long lived connections Thread per connection is not scalable So much time is wasted waiting... on I/O 51
  77. Typical I/O Latency L1: 3 cycles L2: 14 cycles RAM:

    250 cycles DISK: 41,000,000 cycles NETWORK: 240,000,000 cycles http://nodejs.org/jsconf.pdf L1 L2 RAM Disk Network 0 60,000,000 120,000,000 180,000,000 240,000,000 300,000,000 52
  78. var result = db.query("select * from T"); // wait for

    result // use result console.log( result[0].C ); What’s wrong with this? waiting idle blocking for 240,000,000 cycles 53
  79. Alternatively // the async way db.query("select * from T", function(result)

    { console.log( result[0].C ); }); 54
  80. Asynchronous IO • non-blocking • event loop • callbacks 55

  81. Apache vs Nginx http://blog.webfaction.com/a-little-holiday-present asynchronous vs synchronous I/O event loop

    vs thread per connection 56
  82. Apache vs Nginx http://blog.webfaction.com/a-little-holiday-present asynchronous vs synchronous I/O event loop

    vs thread per connection 57
  83. Some of the contenders • Twisted (python) • EventMachine (Ruby)

    • Node.js (javascript) • etc. (non-blocking, asyncronous I/O, event loop) 58
  84. provide an easy way to build scalable network programs. •

    server-side Javascript • client-side development likeness • Javascript designed specifically to be used with an event loop • non-blocking libraries • FAST, underpinned by V8 • community momentum 59
  85. a different way of thinking var callback = function(response) {

    // do something with response }; obj.getSomething(data, callback); 60
  86. http://www.flickr.com/photos/way2go/5536458691/ But, don’t I have to wait until more browsers

    support Websockets? Ummm... 61
  87. 62

  88. Don’t be ridiculous. We can use fallbacks! 62

  89. Don’t be ridiculous. We can use fallbacks! With Socket.io we

    can eat IE6’s lunch! 62
  90. Don’t be ridiculous. We can use fallbacks! With Socket.io we

    can eat IE6’s lunch! and IE7 62
  91. Don’t be ridiculous. We can use fallbacks! With Socket.io we

    can eat IE6’s lunch! and IE7 and IE8 62
  92. Don’t be ridiculous. We can use fallbacks! With Socket.io we

    can eat IE6’s lunch! and IE7 and IE8 and IE9 62
  93. Don’t be ridiculous. We can use fallbacks! With Socket.io we

    can eat IE6’s lunch! and IE7 and IE8 and IE9 *sigh* I’m getting too old for this crap 62
  94. Socket.io Socket.IO aims to make realtime apps possible in every

    browser and mobile device, blurring the differences between the different transport mechanisms 63
  95. Websocket Flash Socket AJAX Polling & Multipart Forever Frame JSONP

    Polling Socket.io 64
  96. IE 5.5+ Chrome 4+ Firefox 3+ Safari 3+ iOS Safari

    Android Webkit WebOs Webkit Opera 10.6 Socket.io 65
  97. Demo 66

  98. Demo webhooks + websockets /twillio /socket.io 67

  99. Demo webhooks + websockets 1. Call /twillio /socket.io 67

  100. Demo webhooks + websockets 1. Call 2. New Call GET

    (webhook) /twillio /socket.io 67
  101. Demo webhooks + websockets 1. Call 2. New Call GET

    (webhook) 3. Status (websocket) /twillio /socket.io 67
  102. Demo webhooks + websockets 1. Call 2. New Call GET

    (webhook) 3. Status (websocket) 4. Respond Say + Record /twillio /socket.io 67
  103. Demo webhooks + websockets 1. Call 2. New Call GET

    (webhook) 3. Status (websocket) 4. Respond Say + Record 5. Say /twillio /socket.io 67
  104. Demo webhooks + websockets 1. Call 2. New Call GET

    (webhook) 3. Status (websocket) 4. Respond Say + Record 5. Say 6. Record /twillio /socket.io 67
  105. Demo webhooks + websockets 1. Call 2. New Call GET

    (webhook) 3. Status (websocket) 4. Respond Say + Record 5. Say 6. Record 7. New Recording POST (webhook) /twillio /socket.io 67
  106. Demo webhooks + websockets 1. Call 2. New Call GET

    (webhook) 3. Status (websocket) 4. Respond Say + Record 5. Say 6. Record 8. Recording (websocket) 7. New Recording POST (webhook) /twillio /socket.io 67
  107. Demo webhooks + websockets 1. Call 2. New Call GET

    (webhook) 3. Status (websocket) 4. Respond Say + Record 5. Say 6. Record 8. Recording (websocket) 7. New Recording POST (webhook) /twillio /socket.io 9. <audio/> GET Recording 67
  108. https://github.com/mbrevoort/ Evented-Web-Demo 68

  109. Beyond http://www.photostrend.com/wp-content/uploads/2011/09/Abstract-Stream-Light-hd.jpg Realtime = Web 3.0 69

  110. Thank You (you’re free to fly now) 70