APIStrat 2016 | The end of polling: why and how to transform a REST API into a Data Streaming API (Audrey Neveu)

IoT & Mobile, User eXperience, Interactivity and RESTful APIs...those topics are big trends in development today. The last one brings fast adoption and reduced development costs but is in opposition with the three others needs. IoT & Mobile need low latency whereas UX and interactivity need real-time data to keep user interest alive.

As a developer you need to poll APIs to provide these real-time information, with all the limitations that this involves. On the other side as an API provider, being massively polled can become an issue and you're probably looking for a more resilient solution to offer to your consumer. What if we can apply the "Hollywood principle" to REST APIs? Don't poll us, we will push you! And why not going further by reducing the amount of data to transfer? In this talk, we'll see together which technology you should opt for and how to implement them to turn your API into a Data Streaming API.

Transcript

1. @StreamdataIO streamdataio API Streaming Why and how to transform a REST API into a Data Streaming API
2. ABOUT ME Developer Relation Co-Leader France @Audrey_Neveu @StreamdataIO streamdataio
3. MODERN TIMES @StreamdataIO streamdataio
4. OUR GOAL @StreamdataIO streamdataio
5. ANIMATION IS THE KEY... BUT WHY? @StreamdataIO streamdataio
6. BECAUSE OF EVOLUTION @StreamdataIO streamdataio
7. USER INTERFACE @StreamdataIO streamdataio
8. KEEP CALM AND... @StreamdataIO streamdataio LEAVE.
9. REFRESH BUTTON IS EVIL @StreamdataIO streamdataio
10. REAL-TIME USER EXPERIENCE @StreamdataIO streamdataio
11. API STREAMING @StreamdataIO streamdataio
12. SOLUTIONS FOR REAL-TIME APPLICATIONS ✓ Web Sockets ✓ Server-Sent Events ✓ (Long) Polling @StreamdataIO streamdataio
13. IS NOT A SOLUTION POLLING... @StreamdataIO streamdataio (and neither is long polling)
14. *TRUE* SOLUTIONS Push Technologies Web-Sockets Server-Sent Events 2008 2006 @StreamdataIO streamdataio W3C Specification
15. PUSH TECHNOLOGIES Web-Sockets Server-Sent Events Text + Binary Text @StreamdataIO streamdataio
16. PROTOCOLS GET /chat HTTP/1.1 Host: example.com Upgrade: websocket Connection: Upgrade Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ== Sec-WebSocket-Protocol: chat Sec-WebSocket-Version: 13 GET /stream HTTP/1.1 1 Host: example.com Accept: text/event-stream HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk= Sec-WebSocket-Protocol: chat Sec-WebSocket-Version: 13 Web-Sockets (RFC-6455) Server-Sent Events @StreamdataIO streamdataio
17. CONFIGURATION Server-Sent Events @StreamdataIO streamdataio Web-Sockets
18. MESSAGES FORMAT var msg = { type: "message", text: "Hello APIStrat!", id: 12345, date: Date.now() }; data: Hello APIStrat! data: Hello APIStrat! data: with two lines data: {"time": "16:34:36", "text": "Hello APIStrat!"} id: 12345 event: foo data: Hello APIStrat! retry: 10000 @StreamdataIO streamdataio Web-Sockets Server-Sent Events
19. IMPLEMENTATION Server-Sent Events @StreamdataIO streamdataio Web-Sockets var websocket = new WebSocket ('ws://websocketserver/echo'); var eventSource = new EventSource ('http://sseserver/echo'); websocket.onOpen = function(){ ... }; eventSource.onopen = function(){ ... }; websocket.onMessage = function(e){ var data = e.data; var message = data.msg; ... }; eventSource.onMessage = function(e){ var message = JSON.parse(e.data); ... }; websocket.onError = function(e){ ... }; eventSource.onError = function(e){ ... }; ... eventSource .addEventListener('foo',function(e){ // do something }, false); eventSource .addEventListener('bar',function(e){ // do something else }, false);
20. LOST IN CONNECTION Server-Sent Events @StreamdataIO streamdataio Web-Sockets
21. BROWSER SUPPORT Server-Sent Events 53. 49. 10. 39. 11. 14. 53. 49. 10. 39. 11. UC. @StreamdataIO streamdataio Web-Sockets source : http://caniuse.com/
22. MOBILE BROWSER SUPPORT Server-Sent Events 51. 49. 10. 18. 52. source : http://caniuse.com/ 51. 49. 10. 18. 52. @StreamdataIO streamdataio Web-Sockets
23. PERFORMANCES 8s 5s x1.6 8s 6s x1.3 16s 7s x.2.2 Web-Sockets SSE source: http://matthiasnehlsen.com/blog/2013/05/01/server-sent-events-vs-websockets/ Diff @StreamdataIO streamdataio
24. CHOOSE WISELY @StreamdataIO streamdataio
25. Proxy-as-a-Service ✓ works with any JSON API ✓ streaming based on Server-Sent Events ✓ dynamic cache ✓ incremental updates STREAMDATA.IO @StreamdataIO streamdataio
26. JSON-PATCH (RFC-6902) [{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"}, {"title":"Value 2","price":85,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 3","price":21,"param1":"31","param2":"12","param3":"4"}, {"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"}, {"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"}, {"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}] [{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"}, {"title":"Value 2","price":5,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 3","price":21,"param1":"31","param2":"32","param3":"4"}, {"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"}, {"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"}, {"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}] [{"op":"replace","path":"/2/price","value":5}, {"op":"replace","path":"/3/param2","value":"32"}] @StreamdataIO streamdataio
27. THE API PROVIDER BEST FRIEND Sample Case: Investment Mobile App @StreamdataIO streamdataio Stockmarket API Web & Mobile Apps ➢ 25bn calls/month ✓ 90% reduction of the outgoing traffic & API Server Load ✓ 6 months acceleration in development
28. DEMO @StreamdataIO streamdataio
29. VOTE SERVER-SENT EVENTS! @StreamdataIO streamdataio
30. @StreamdataIO streamdataio THANKS! Q&A