APIStrat 2016 | The end of polling: why and how to transform a REST API into a Data Streaming API (Audrey Neveu)

IoT & Mobile, User eXperience, Interactivity and RESTful APIs...those topics are big trends in development today. The last one brings fast adoption and reduced development costs but is in opposition with the three others needs. IoT & Mobile need low latency whereas UX and interactivity need real-time data to keep user interest alive.

As a developer you need to poll APIs to provide these real-time information, with all the limitations that this involves. On the other side as an API provider, being massively polled can become an issue and you're probably looking for a more resilient solution to offer to your consumer. What if we can apply the "Hollywood principle" to REST APIs? Don't poll us, we will push you! And why not going further by reducing the amount of data to transfer? In this talk, we'll see together which technology you should opt for and how to implement them to turn your API into a Data Streaming API.

Transcript

1. @StreamdataIO
   streamdataio
   API Streaming
   Why and how to transform a REST API
   into a Data Streaming API
   

   View Slide

2. ABOUT ME
   Developer Relation Co-Leader France
   @Audrey_Neveu
   @StreamdataIO
   streamdataio
   

   View Slide

3. MODERN TIMES
   @StreamdataIO
   streamdataio
   

   View Slide

4. OUR GOAL
   @StreamdataIO
   streamdataio
   

   View Slide

5. ANIMATION IS THE KEY... BUT WHY?
   @StreamdataIO
   streamdataio
   

   View Slide

6. BECAUSE OF EVOLUTION
   @StreamdataIO
   streamdataio
   

   View Slide

7. USER INTERFACE
   @StreamdataIO
   streamdataio
   

   View Slide

8. KEEP CALM AND...
   @StreamdataIO
   streamdataio
   LEAVE.
   

   View Slide

9. REFRESH BUTTON IS EVIL
   @StreamdataIO
   streamdataio
   

   View Slide

10. REAL-TIME USER EXPERIENCE
    @StreamdataIO
    streamdataio
    

    View Slide

11. API STREAMING
    @StreamdataIO
    streamdataio
    

    View Slide

12. SOLUTIONS FOR REAL-TIME APPLICATIONS
    ✓ Web Sockets
    ✓ Server-Sent Events
    ✓ (Long) Polling
    @StreamdataIO
    streamdataio
    

    View Slide

13. IS NOT A SOLUTION
    POLLING...
    @StreamdataIO
    streamdataio
    (and neither is long polling)
    

    View Slide

14. *TRUE* SOLUTIONS
    Push Technologies
    Web-Sockets Server-Sent Events
    2008 2006
    @StreamdataIO
    streamdataio
    W3C Specification
    

    View Slide

15. PUSH TECHNOLOGIES
    Web-Sockets Server-Sent Events
    Text + Binary Text
    @StreamdataIO
    streamdataio
    

    View Slide

16. PROTOCOLS
    GET /chat HTTP/1.1
    Host: example.com
    Upgrade: websocket
    Connection: Upgrade
    Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
    Sec-WebSocket-Protocol: chat
    Sec-WebSocket-Version: 13
    GET /stream HTTP/1.1 1
    Host: example.com
    Accept: text/event-stream
    HTTP/1.1 101 Switching Protocols
    Upgrade: websocket
    Connection: Upgrade
    Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=
    Sec-WebSocket-Protocol: chat
    Sec-WebSocket-Version: 13
    Web-Sockets
    (RFC-6455)
    Server-Sent Events
    @StreamdataIO
    streamdataio
    

    View Slide

17. CONFIGURATION
    Server-Sent Events
    @StreamdataIO
    streamdataio
    Web-Sockets
    

    View Slide

18. MESSAGES FORMAT
    var msg = {
    type: "message",
    text: "Hello APIStrat!",
    id: 12345,
    date: Date.now()
    };
    data: Hello APIStrat!
    data: Hello APIStrat!
    data: with two lines
    data: {"time": "16:34:36", "text": "Hello APIStrat!"}
    id: 12345
    event: foo
    data: Hello APIStrat!
    retry: 10000
    @StreamdataIO
    streamdataio
    Web-Sockets Server-Sent Events
    

    View Slide

19. IMPLEMENTATION
    Server-Sent Events
    @StreamdataIO
    streamdataio
    Web-Sockets
    var websocket =
    new WebSocket
    ('ws://websocketserver/echo');
    var eventSource =
    new EventSource
    ('http://sseserver/echo');
    websocket.onOpen = function(){
    ...
    };
    eventSource.onopen = function(){
    ...
    };
    websocket.onMessage = function(e){
    var data = e.data;
    var message = data.msg;
    ...
    };
    eventSource.onMessage = function(e){
    var message = JSON.parse(e.data);
    ...
    };
    websocket.onError = function(e){
    ...
    };
    eventSource.onError = function(e){
    ...
    };
    ...
    eventSource
    .addEventListener('foo',function(e){
    // do something
    }, false);
    eventSource
    .addEventListener('bar',function(e){
    // do something else
    }, false);
    

    View Slide

20. LOST IN CONNECTION
    Server-Sent Events
    @StreamdataIO
    streamdataio
    Web-Sockets
    

    View Slide

21. BROWSER SUPPORT
    Server-Sent Events
    53. 49. 10. 39. 11. 14. 53. 49. 10. 39. 11. UC.
    @StreamdataIO
    streamdataio
    Web-Sockets
    source : http://caniuse.com/
    

    View Slide

22. MOBILE BROWSER SUPPORT
    Server-Sent Events
    51. 49. 10. 18. 52.
    source : http://caniuse.com/
    51. 49. 10. 18. 52.
    @StreamdataIO
    streamdataio
    Web-Sockets
    

    View Slide

23. PERFORMANCES
    8s 5s x1.6
    8s 6s x1.3
    16s 7s x.2.2
    Web-Sockets SSE
    source: http://matthiasnehlsen.com/blog/2013/05/01/server-sent-events-vs-websockets/
    Diff
    @StreamdataIO
    streamdataio
    

    View Slide

24. CHOOSE WISELY
    @StreamdataIO
    streamdataio
    

    View Slide

25. Proxy-as-a-Service
    ✓ works with any JSON API
    ✓ streaming based on Server-Sent Events
    ✓ dynamic cache
    ✓ incremental updates
    STREAMDATA.IO
    @StreamdataIO
    streamdataio
    

    View Slide

26. JSON-PATCH (RFC-6902)
    [{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"},
    {"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"},
    {"title":"Value 2","price":85,"param1":"1","param2":"22","param3":"33"},
    {"title":"Value 3","price":21,"param1":"31","param2":"12","param3":"4"},
    {"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"},
    {"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"},
    {"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}]
    [{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"},
    {"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"},
    {"title":"Value 2","price":5,"param1":"1","param2":"22","param3":"33"},
    {"title":"Value 3","price":21,"param1":"31","param2":"32","param3":"4"},
    {"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"},
    {"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"},
    {"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}]
    [{"op":"replace","path":"/2/price","value":5},
    {"op":"replace","path":"/3/param2","value":"32"}]
    @StreamdataIO
    streamdataio
    

    View Slide

27. THE API PROVIDER BEST FRIEND
    Sample Case: Investment Mobile App
    @StreamdataIO
    streamdataio
    Stockmarket
    API
    Web & Mobile
    Apps
    ➢ 25bn calls/month
    ✓ 90% reduction of the outgoing
    traffic & API Server Load
    ✓ 6 months acceleration in
    development
    

    View Slide

28. DEMO
    @StreamdataIO
    streamdataio
    

    View Slide

29. VOTE SERVER-SENT EVENTS!
    @StreamdataIO
    streamdataio
    

    View Slide

30. @StreamdataIO
    streamdataio
    THANKS!
    Q&A
    

    View Slide