APIStrat 2016 | The end of polling: why and how to transform a REST API into a Data Streaming API (Audrey Neveu)

APIStrat 2016 | The end of polling: why and how to transform a REST API into a Data Streaming API (Audrey Neveu)

IoT & Mobile, User eXperience, Interactivity and RESTful APIs...those topics are big trends in development today. The last one brings fast adoption and reduced development costs but is in opposition with the three others needs. IoT & Mobile need low latency whereas UX and interactivity need real-time data to keep user interest alive.

As a developer you need to poll APIs to provide these real-time information, with all the limitations that this involves. On the other side as an API provider, being massively polled can become an issue and you're probably looking for a more resilient solution to offer to your consumer. What if we can apply the "Hollywood principle" to REST APIs? Don't poll us, we will push you! And why not going further by reducing the amount of data to transfer? In this talk, we'll see together which technology you should opt for and how to implement them to turn your API into a Data Streaming API.


  1. 12.
  2. 16.

    PROTOCOLS GET /chat HTTP/1.1 Host: example.com Upgrade: websocket Connection: Upgrade

    Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ== Sec-WebSocket-Protocol: chat Sec-WebSocket-Version: 13 GET /stream HTTP/1.1 1 Host: example.com Accept: text/event-stream HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk= Sec-WebSocket-Protocol: chat Sec-WebSocket-Version: 13 Web-Sockets (RFC-6455) Server-Sent Events @StreamdataIO streamdataio
  3. 18.

    MESSAGES FORMAT var msg = { type: "message", text: "Hello

    APIStrat!", id: 12345, date: Date.now() }; data: Hello APIStrat! data: Hello APIStrat! data: with two lines data: {"time": "16:34:36", "text": "Hello APIStrat!"} id: 12345 event: foo data: Hello APIStrat! retry: 10000 @StreamdataIO streamdataio Web-Sockets Server-Sent Events
  4. 19.

    IMPLEMENTATION Server-Sent Events @StreamdataIO streamdataio Web-Sockets var websocket = new

    WebSocket ('ws://websocketserver/echo'); var eventSource = new EventSource ('http://sseserver/echo'); websocket.onOpen = function(){ ... }; eventSource.onopen = function(){ ... }; websocket.onMessage = function(e){ var data = e.data; var message = data.msg; ... }; eventSource.onMessage = function(e){ var message = JSON.parse(e.data); ... }; websocket.onError = function(e){ ... }; eventSource.onError = function(e){ ... }; ... eventSource .addEventListener('foo',function(e){ // do something }, false); eventSource .addEventListener('bar',function(e){ // do something else }, false);
  5. 21.

    BROWSER SUPPORT Server-Sent Events 53. 49. 10. 39. 11. 14.

    53. 49. 10. 39. 11. UC. @StreamdataIO streamdataio Web-Sockets source : http://caniuse.com/
  6. 22.

    MOBILE BROWSER SUPPORT Server-Sent Events 51. 49. 10. 18. 52.

    source : http://caniuse.com/ 51. 49. 10. 18. 52. @StreamdataIO streamdataio Web-Sockets
  7. 23.

    PERFORMANCES 8s 5s x1.6 8s 6s x1.3 16s 7s x.2.2

    Web-Sockets SSE source: http://matthiasnehlsen.com/blog/2013/05/01/server-sent-events-vs-websockets/ Diff @StreamdataIO streamdataio
  8. 25.

    Proxy-as-a-Service ✓ works with any JSON API ✓ streaming based

    on Server-Sent Events ✓ dynamic cache ✓ incremental updates STREAMDATA.IO @StreamdataIO streamdataio
  9. 26.

    JSON-PATCH (RFC-6902) [{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"}, {"title":"Value 2","price":85,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 3","price":21,"param1":"31","param2":"12","param3":"4"},

    {"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"}, {"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"}, {"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}] [{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"}, {"title":"Value 2","price":5,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 3","price":21,"param1":"31","param2":"32","param3":"4"}, {"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"}, {"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"}, {"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}] [{"op":"replace","path":"/2/price","value":5}, {"op":"replace","path":"/3/param2","value":"32"}] @StreamdataIO streamdataio
  10. 27.

    THE API PROVIDER BEST FRIEND Sample Case: Investment Mobile App

    @StreamdataIO streamdataio Stockmarket API Web & Mobile Apps ➢ 25bn calls/month ✓ 90% reduction of the outgoing traffic & API Server Load ✓ 6 months acceleration in development