Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Money is in the Gap @Apidays.io Paris 2016

Money is in the Gap @Apidays.io Paris 2016

1. Real-time API consumptionReal-time API consumption what? why? how? Eric Horesnyi @StreamdataIO Paris, Dec'15 @ApidaysGlobal
2. Our brain is Real-timeOur brain is Real-time
3. Amazon found every 100 milliseconds of apps latency cost them 1% in sales http://perspectives.mvdirona.com/2009/10/31/Th eCostOfLatency.aspx Google found an extra 0.5 seconds in search page generation time dropped traffic by 20%. http://highscalability.com/latency-everywhere-and- it-costs-you-sales-how-crush-it Real-time UX i$...Real-time UX i$... Rea£ Mon€¥Rea£ Mon€¥ Blockbuster apps: 1% only 2,000 apps2,000 apps 1m+ downloads - 10m d.API calls App rating Dwell time Sharing in Real-time
4. {API}{API}(UX)(UX) UX-focus / API-focusUX-focus / API-focus 2012/Single European Railway Area
5. Real-time is relativeReal-time is relative On informationOn information symmetrysymmetry
6. -Nanosec Millisec X00ms Seconds Minutes+ High Frequency Trading Algorithmic Trading Day Trading Retail Traders Investors
7. Real-time sharingReal-time sharing busting insidersbusting insiders GD Saxton, SUNY, 2013: 90-day insider profit 4.4% > 0.9% The End of Information Assymmetry
8. Latency x5Latency x5 Bandwidth /10Bandwidth /10 Data updates x60Data updates x60 Devices x10Devices x10 Data volume x2Data volume x2
9. Polling/Long-PollingPolling/Long-Polling HTTP HackHTTP Hack Don't use it anymoreDon't use it anymore
10. WebSocketsWebSockets Server-SentServer-Sent EventsEvents var websocket = new WebSocket('ws://websocketserver/echo'); websocket.onopen = function () { ... }; websocket.onmessage = function (e) { ... }; websocket.onerror = function (error) { ... }; var eventSource = new EventSource('http://sseserver/echo'); eventSource.onopen = function () { ... }; eventSource.onmessage = function (e) { ... }; eventSource.onerror = function (error) { ... }; eventSource.addEventListener('foo', function(e) { ... }, false);
11. WebSocketsWebSockets Server-SentServer-Sent EventsEvents TCP Error Handling : TBD Protocol upgrade GET /chat HTTP/1.1 Host: example.com:8000 Upgrade: websocket Connection: Upgrade Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ== Sec-WebSocket-Version: 13 GET /stream HTTP/1.1 1 Host: example.com Accept: text/event-stream HTTP=Standard HTTP/2 leverage No Loadbalancer, Proxy, or FW reconfig
12. LatencyLatency Use case: Pre-load of 500 Tweets on a web page web Source http://matthiasnehlsen.com/blog/2013/05/01/server-sent-events-vs-websockets/
13. WebSocketsWebSockets Server-SentServer-Sent EventsEvents
14. Reactive+HTTP/2+SSE=Reactive+HTTP/2+SSE= UX-API end-to-end event-driven data stream .OnData() .OnPatch() .OnClose() #REST API JSON {"EURUSD", "1.1233123"} {"EURGBP", "0.9233123"} {"EURJPY", "98.456788"} {"EURAUD", "1.4828679"} #JSON PATCH { "EURUSD", "1.1233123" } #HashMap {"EURUSD", "1.1233123"} {"EURGBP", "0.9233123"} {"EURJPY", "98.456788"} {"EURAUD", "1.4828679"} #Observables .Mount(tag)
15. JSON-Patch (RFC 6902)JSON-Patch (RFC 6902) [{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"}, {"title":"Value 2","price":85,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 3","price":21,"param1":"31","param2":"12","param3":"4"}, {"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"}, {"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"}, {"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}] [{"op":"replace","path":"/2/price","value":5}, {"op":"replace","path":"/3/param2","value":"32"}] [{"title":"Value 0","price":66,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 1","price":63,"param1":"11","param2":"2","param3":"53"}, {"title":"Value 2","price":5,"param1":"1","param2":"22","param3":"33"}, {"title":"Value 3","price":21,"param1":"31","param2":"32","param3":"4"}, {"title":"Value 4","price":10,"param1":"151","param2":"22","param3":"33"}, {"title":"Value 5","price":6,"param1":"11","param2":"21","param3":"33"}, {"title":"Value 6","price":60,"param1":"11","param2":"222","param3":"33"}]
16. UI speed with Json PatchUI speed with Json Patch http://streamdata.io/blog/ui-latency-json-patch/ Patch x20 faster
17. Coding Session this AfternoonCoding Session this Afternoon
18. DevopsDevops RecipesRecipes
19. Latency Scalability
20. ErlangErlang E=λh λ: intensity h: hold time A.K. Erlang M/M/c/c queue aka Little's Law
21. //US Equity broker // Audience: 1 million Var a = 1000000 // Engagement: 10% Var e = 0.1 // Hold: 3 minutes Var h = 180 // Intensity: 25% during last 30' market close // Total day : 8 hours a day (trading day) Var i = 0.25 * 8 / 0.5 Var E = a * e * h * i >>> print (E) >>> 2500 # E erlang traffic // Cluster: 80% on S&P500 Var c = 0.8 // Frequency: update/sec Var f = 1 // Non-blocking Erlang Pb (E, m) 99.99% non-blocking #recursive search m in range (5 * E) until Pb ( E, m ) > 0.0001 >>>print (m) >>>2650 #streams w/out cache >>>2650*(1-c)=265 #API streams w/cache Sizing APISizing API trunkstrunks
22. API Server & Network LoadAPI Server & Network Load pollingpolling w/ streamdata.iow/ streamdata.io 10,000 concurrent clients API Server CPUAPI Server CPU
23. Battery UsageBattery Usage Regular APIRegular API Diff Streamed APIDiff Streamed API 75 to 99% savings75 to 99% savings
24. APIs in BPA & IoTAPIs in BPA & IoT real-time statesreal-time states
25. Real-time UX i$ Rea£ Mon€¥ Symmetry of information = fairness & real-time sharing HTTP/2+SSE=scalable web standard End-to-end reactive/event-driven architecture can scale Stream your API above 10m calls/day - #100k audience ConclusionConclusion Thank you!Thank you!
26. ReferencesReferences What is Server-Sent Events? WebSockets vs SSE Getting started with WebSockets and SSE Server-Sent Events: The simplest realtime browser spec WebSockets / SSEWebSockets / SSE
27. CreditsCredits GIFs: giphy Photos: mostly Wiki

erichoresnyi

January 02, 2017
Tweet

More Decks by erichoresnyi

Other Decks in Programming

Transcript

  1. $£€ 10Ks <10M 1s 0 OS1:Family Product-Market Fit, Marc Andreesen

    2007 http://web.stanford.edu/class/ee204/ProductMarketFit.html
  2. OS1 2007 > $£€ 10Ks <10M 1s 0 OS1:Family "Do

    things that don't scale", Paul Graham 2013 http://paulgraham.com/ds.html
  3. Experiment Fast & Stay Lean $£€ 10Ks <10M 1s 0

    OS1:Family {API}>Product http://theleanstartup.com/ Eric Ries, 2008
  4. OS4:City Extensibility {API}=Civism $£€ 10Ms 1Bn+ 1Ks 100s x3 Dwell

    Time 90% Resource Savings 1Bn events per day 20x Faster
  5. OS4:City APIs in Law, Jeff Bezos 2002 via {API Evangelist}

    {API}=Civism $£€ All teams will henceforth expose their data and functionality through service interfaces. Teams must communicate with each other through these interfaces. There will be no other form of inter-process communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network. It doesn’t matter what technology they use. All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions. Anyone who doesn’t do this will be fired. Thank you; have a nice day! 10Ms 1Bn+ 1Ks 100s
  6. $£€ 100Ms 5Bn+ 10Ks 1Ks "Salesforce.com generates 50% of its

    revenues through APIs. Expedia.com generates 90%. eBay generates 60%." Harvard Business Review, Jan. 2015 {API}:Platfor m OS5:Nation Open