From application latency to animated data (and to APIs streaming)

Latence Latence applicative... applicative... @ctranxuan @StreamdataIO

£at€nc€... i$ Møn€¥ £øss £at€nc€... i$ Møn€¥ £øss Amazon found
every 100 milliseconds of Amazon found every 100 milliseconds of apps latency cost them 1% in sales. apps latency cost them 1% in sales. Source: http://perspectives.mvdirona.com/2009/10/31/TheCostOfLatency.aspx Google found an extra 0.5 seconds in Google found an extra 0.5 seconds in search page generation time dropped search page generation time dropped traffic by 20% traffic by 20% Source: http://highscalability.com/latency-everywhere-and-it-costs-you-sales-how-crush-it

-100ms = -1% revenue & -5% -100ms = -1% revenue
& -5% page views page views Source: Source: Source: http://www.webperformancetoday.com/2011/11/08/velocity-faster-mobile-sites-business-kpis-case-studies/ http://www.ebaytechblog.com/2013/03/29/measuring-real-user-experience-with-site-speed-gauge/ https://blog.radware.com/applicationdelivery/applicationaccelerationoptimization/2013/06/web-performance- poverty-line/

0,5 s 0,5 s Latency Graal Latency Graal

Round Trip Delays (ping time) Round Trip Delays (ping time)
In ADSL, ﬁrst useful piece of data in 60ms (HTTPS) In 4G / LTE, ﬁrst useful piece of data in 300 ms (450 ms in 3G)

Round Trip Delays (ping time) Round Trip Delays (ping time)

Constraints Constraints HTTP 1.1 HTTP 1.1 SSL / TLS SSL
/ TLS Increasing global traffic Increasing global traffic Mobile First (3G / 4G) Mobile First (3G / 4G)

Google /Netflix/ Twitter Google /Netflix/ Twitter

Google Google SPDY SPDY ➡ ➡ ➡ ➡ HTTP/2 HTTP/2

HTTP HTTP 1996 1996 1999 1999 HTTP/1.0 HTTP/1.0 HTTP/1.1 HTTP/1.1
static content small size objects small # of objects 201x 201x dynamic content larger size objects more objects Matthew Yohe [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0) or GFDL (http://www.gnu.org/copyleft/fdl.html)], via Wikimedia Commons more objects even more larger objects

HTTP HTTP Web page: ~ 2 Mb ~ 80 elements
~ 38 TCP connections source: http://httparchive.org

HTTP/1.x issues HTTP/1.x issues Web page: Web page: ~ 2
Mb ~ 2 Mb ~ 80 elements ~ 80 elements Head of line blocking Head of line blocking

HTTP/1.x issues HTTP/1.x issues 1996 1996 1999 1999 HTTP/1.0 HTTP/1.0
HTTP/1.1 HTTP/1.1 201x 201x Hacks Hacks

HTTP/1.x: sharding HTTP/1.x: sharding .css .png, .jpg .html, .js

HTTP/1.x: spriting HTTP/1.x: spriting By Kriplozoik CC BY 3.0 (http://creativecommons.org/licenses/by/3.0)],
via Wikimedia Commons #home { left: 0px; width: 46px; background: url('img_navsprites.gif') 0 0; } #prev { left: 63px; width: 43px; background: url('img_navsprites.gif') -47px 0; } #next { left: 129px; width: 43px; background: url('img_navsprites.gif') -91px 0; }

HTTP/1.x: inlining HTTP/1.x: inlining <img src="data:image/png;base64,iVBORw0KGgoAAA ...E2AAA7JN+/OU8bK5CYII=" alt="html_inline_alpesjug.png" title="html_inline_alpesjug.png">

HTTP/1.x: concatenation HTTP/1.x: concatenation <html> <head> <title>AlpesJUG</title> <meta charset="utf-8"> <meta
http-equiv="X-UA-Compatible" content="IE=edge"> <meta content="width=device-width,initial-scale=1" name="viewport"> <link rel="stylesheet" href="assets/styles/main-f11f93b619.css"> <link rel="stylesheet" href="assets/styles/theme-564f6756da.css"> <link rel="stylesheet" href="assets/styles/ie-d41d8cd98f.css"> </head> <body>...</body> <script src="scripts/vendor-a6d64f3f39.js"></script> <script src="scripts/app-0d43bb3b4b.js"></script> <script src="scripts/theme-439fdb6f87.js"> </html> 1 change => invalidate all the cache

HTTP HTTP 1996 1996 1999 1999 2009 2009 2015 2015
HTTP/1.0 HTTP/1.0 HTTP/1.1 HTTP/1.1 HTTP/2 HTTP/2 static content small size objects small # of objects more objects even more larger objects mobile devices Matthew Yohe [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0) or GFDL (http://www.gnu.org/copyleft/fdl.html)], via Wikimedia Commons S SP PD DY Y dynamic content larger size objects more objects RFC 7540

HTTP/2 HTTP/2 Retro-compatibility Retro-compatibility with HTTP/1.x with HTTP/1.x Goal: optimization
of Goal: optimization of the transport the transport

HTTP/2: multiplexing HTTP/2: multiplexing Multiplexing of requests concurrent requests 1
TCP connection streams: bidirectional frames source: http://www.splicemarketing.co.uk/_blog/Splice_Blog/post/the-arrival-of-http2/#.VlrbuiCrSV4

HTTP/2: HTTP/2: headers compression headers compression Method: GET Scheme: HTTPS
Accept: image/webp,image/*,*/*;q=0.8 Accept-Language: en-US,en Path: /resource Host: http://www.alpesjug.org/ User-Agent: Mozilla/5.0 Path: /images + HPACK compression + HPACK compression 2nd, 3rd, etc. req 2nd, 3rd, etc. req

HTTP/2: Server push HTTP/2: Server push

HTTP/2: TLS HTTP/2: TLS optional TLS (but for all browsers,
it's mandatory) TLS 1.2 min no compression no renegociation stronger crypto

<html> <head> <title>AlpesJUG</title> <link rel="stylesheet" href="assets/styles/main-f11f93b619.css"> <link rel="stylesheet" href="assets/styles/theme-564f6756da.css"> <link
rel="stylesheet" href="assets/styles/ie-d41d8cd98f.css"> </head> <body>...</body> <script src="scripts/vendor-a6d64f3f39.js"></script> <script src="scripts/app-0d43bb3b4b.js"></script> <script src="scripts/theme-439fdb6f87.js"> </html> HTTP/2: Benefits HTTP/2: Benefits <img src="data:image/png;base64,iVBORw0KGgoAAA ...E2AAA7JN+/OU8bK5CYII=" alt="html_inline_alpesjug.png" title="html_inline_alpesjug.png">

HTTP/2 HTTP/2

HTTP/2: HTTP/2: browsers support browsers support

HTTP/2: HTTP/2: server implementations server implementations source: https://github.com/http2/http2-spec/wiki/Implementations ... ...

HTTP/2: next HTTP/2: next bottleneck? bottleneck? 1001010101011 01100011010 000010101011010 101010101011
0110100101110010 01010111111110001010 01011101010111 10101010101 00000101010110 101010110101 TCP TCP

Google Google

Google - QUIC Google - QUIC source: https://peering.google.com/about/quicfaq.html

Netflix Netflix source: http://www.programmableweb.com/news/why-rest-keeps-me-night/2012/05/15 2+ billions requests / day 2+
billions requests / day

Netflix Netflix source: http://www.programmableweb.com/news/why-rest-keeps-me-night/2012/05/15 800+ different type 800+ different type
of devices of devices

Netflix Netflix source: http://www.programmableweb.com/news/why-rest-keeps-me-night/2012/05/15 OSFA API (One Size Fits All
API)

Netflix: problematic Netflix: problematic source: http://techblog.netﬂix.com/2013/01/optimizing-netﬂix-api.html

Netflix Netflix OSFA API OSFA API optimized for none of
the devices chatty API (lots of calls)

Netflix Netflix source: http://techblog.netﬂix.com/2013/01/optimizing-netﬂix-api.html

Netflix Netflix

Netflix Netflix single items multiple items synchronous Object Iterable asynchronous
Future Observable

Netflix Netflix Marble diagram

source: Ben Christensen, Functional reactive programming in the Netﬂix API
Netflix Netflix

source: Ben Christensen, Functional reactive programming in the Netﬂix API
Netflix Netflix def Observable<Map> getVideos(userId) { return VideoService.getVideos(userId) // we only want the first 10 of each list .take(10) .flatMap({ Video video -> def m = video.getMetaData(); def b = video.getBookmark(); def r = video.getRating(); // compose these together return Observable.zip(m, b, r { metadata, bookmark, rating -> // no transform to complete dictionary // of data we want for each Video return [id: video.videoId] << metadata << bookmark << rating }) }) } The "zip" operator combines the 3 asynchronous Observables into 1

Netflix: hystrix Netflix: hystrix

Netflix: problematic Netflix: problematic source: http://techblog.netﬂix.com/2013/01/optimizing-netﬂix-api.html

http://netflix.com/genreList/1249?rowOffset=0&rowSize=5&titleprops=name,boxshot http://netflix.com/genreList/1249/setRating?titleId=5&view=movieDetailPage Netflix: problematic Netflix: problematic Rest API Rest API
http://netflix.com/genreList/1249 "Rest-ish" API "Rest-ish" API http://netflix.com/genreList/1249?titleprops=name,ratings "RPC-ish" API "RPC-ish" API

Netflix: problematic Netflix: problematic « Netflix's domain is a «
Netflix's domain is a graph problem not a tree graph problem not a tree problem » problem »

Netflix Netflix

Netflix Netflix 㱺㱺 Middleware Middleware 㱺㱺 Cache Cache
㱺㱺 JSON extension: JSON extension: JSONGraph (Virtual JSON) JSONGraph (Virtual JSON)

Netflix Netflix /model.json

Netflix Netflix Cache HTTP D a t a S o
u r c e https://... /model.json Router DataSource Client View Middleware Recommandation Service Title Service Rating Service Model Express

Netflix Netflix DEMO DEMO

Netflix Netflix « The data is the API: « The
data is the API: if you know the data, you if you know the data, you know the API » know the API »

Netflix: UI refactoring Netflix: UI refactoring + + source: http://techblog.netﬂix.com/2015/08/making-netﬂixcom-faster.html

Twitter Twitter

What's wrong with What's wrong with polling? polling? Bob Bob
Caroll Caroll Alice Alice Erik Erik Pull too fast: waste resources Pull at the wrong moment: get blocked Pull too slow: run behind source: Erik Meijer https://www.youtube.com/watch?v=FvMuPtuvP5w

Real-time UX... i$ Rea£-time Real-time UX... i$ Rea£-time Mon€¥ Mon€¥
Amazon found every 100 milliseconds of Amazon found every 100 milliseconds of apps latency cost them 1% in sales. apps latency cost them 1% in sales. Source: http://perspectives.mvdirona.com/2009/10/31/TheCostOfLatency.aspx Google found an extra 0.5 seconds in Google found an extra 0.5 seconds in search page generation time dropped search page generation time dropped traffic by 20% traffic by 20% Source: http://highscalability.com/latency-everywhere-and-it-costs-you-sales-how-crush-it

Which ingredients for Which ingredients for real-time UX? real-time UX?

Solutions for dynamic Solutions for dynamic data data Long polling
Long polling WebSockets WebSockets Server-Sent Events Server-Sent Events

Long-Polling Long-Polling HTTP Hack HTTP Hack 㱺㱺 Don't use
it anymore Don't use it anymore

WebSockets WebSockets Server-Sent Server-Sent Events Events Push techno W3C 2008
Push techno W3C 2006

WebSockets WebSockets Server-Sent Server-Sent Events Events

WebSockets WebSockets Server-Sent Server-Sent Events Events TCP low-level protocol HTTP
upgrade handshake GET /chat HTTP/1.1 Host: example.com:8000 Upgrade: websocket Connection: Upgrade Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ== Sec-WebSocket-Version: 13 GET /stream HTTP/1.1 1 Host: example.com Accept: text/event-stream

WebSockets WebSockets Server-Sent Server-Sent Events Events proxies and load balancers
reconﬁguration No need to reconﬁgure! (it's HTTP)

WebSockets WebSockets Server-Sent Server-Sent Events Events

WebSockets WebSockets Server-Sent Server-Sent Events Events binary + text binary
+ text native browsers native browsers support support text text ... but polyfills! ... but polyfills!

https://goo.gl/cCMIGf

WebSockets WebSockets Server-Sent Server-Sent Events Events var websocket = new
WebSocket('ws://websocketserver/echo'); websocket.onopen = function () { ... }; websocket.onmessage = function (e) { ... }; websocket.onerror = function (error) { ... }; var eventSource = new EventSource('http://sseserver/echo'); eventSource.onopen = function () { ... }; eventSource.onmessage = function (e) { ... }; eventSource.onerror = function (error) { ... }; eventSource.addEventListener('foo', function(e) { ... }, false);

"Perfs" "Perfs" Use case: Préchargement de 500 Tweets sur une
page web (nginx conﬁguré en tant que proxy) Source http://matthiasnehlsen.com/blog/2013/05/01/server-sent-events-vs-websockets/

Solutions for Dynamic Data Solutions for Dynamic Data

Streaming vs HTTP Polling Streaming vs HTTP Polling 10 messages
of 1 byte : 7ms vs 220 ms 10 messages of 1 byte : 7ms vs 220 ms 100 messages of 10 bytes : 57 vs 986 ms 100 messages of 10 bytes : 57 vs 986 ms 1000 messages of 100 bytes : 200 vs 10210 ms 1000 messages of 100 bytes : 200 vs 10210 ms 5000 messages of 1000 bytes : 1,2 sec vs 54 sec 5000 messages of 1000 bytes : 1,2 sec vs 54 sec Source http://blog.arungupta.me/rest-vs-websocket-comparison-benchmarks/

Support (bis) Support (bis) Some Java Servers Some Java Servers
Clients libraries in... Clients libraries in...

WebSockets WebSockets Server-Sent Server-Sent Events Events chat, chat, chat !
Share living editing GPS GoogleMap-like Games Use cases Use cases Fintech / Trading Betting Games Realtime timetables Animated data apps (charts, monitoring, etc.)

What if...? What if...? ... I have just a ...
I have just a JSON API? JSON API?

Proxy as a Service: push to client Proxy as a
Service: push to client

API Server Load API Server Load polling polling w/ streamdata.io
w/ streamdata.io 10,000 concurrent clients API Server CPU API Server CPU

Cache Cache

JSON-Patch (RFC 6902) JSON-Patch (RFC 6902) [{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"},
{"title":"Value 2","price":85,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 3","price":21,"param1":"31","param2":"12","param3":"4"}, {"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"}, {"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"}, {"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}] [{"op":"replace","path":"/2/price","value":5}, {"op":"replace","path":"/3/param2","value":"32"}] [{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"}, {"title":"Value 2","price":5,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 3","price":21,"param1":"31","param2":"32","param3":"4"}, {"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"}, {"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"}, {"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}]

Efficiently Turn APIs into Efficiently Turn APIs into Real-Time Experiences
Real-Time Experiences Push, don't poll Dynamic Cache Incremental Data

Animated Data Animated Data

Refresh and Waiting Refresh and Waiting Poor UX: low emotional
engagement Poor UX: low emotional engagement Poor SEO: poor dwell time Poor SEO: poor dwell time

Which ingredients for Which ingredients for real-time UX? real-time UX?

Friendly frameworks / libs Friendly frameworks / libs Android Android
Swift Swift ObjectiveC ObjectiveC

Demos Demos

Poll Demo Poll Demo

Xignite Demo Xignite Demo

JCDecaux Demo JCDecaux Demo

0.5s is the latency objective “The more others invest in
amazing UI, the more yours seems lousy” Einstein « Relativity concepts applied to UI », OpenRoadMedia, 1937 Conclusions Conclusions The world is more and more dynamic and we've got the tools Be ready to animate data, become streamers!

Thank you! Thank you! @StreamdataIO @StreamdataIO

https://goo.gl/cCMIGf

References References Streamdata.io fork from AMVTek AMVTek Yaﬄe Remi JS
Polyfills JS Polyfills

References References What is Server-Sent Events? WebSockets vs SSE Getting
started with WebSockets and SSE Server-Sent Events: The simplest realtime browser spec WebSockets / SSE WebSockets / SSE

From application latency to animated data (and ...

From application latency to animated data (and to APIs streaming)

More Decks by ctranxuan

Other Decks in Programming

Featured

Transcript