Event-Driven APIs - Apidays.io Paris Dec'18

Presentation here: https://youtu.be/QVuTTAHXCGM

Title : Event-Driven APIs
Bio: Eric started to code with HTTP the year W3C was created (1994), as part of the founding teams of several startups acquired since then: Internet Way (now Verizon Business), then as high-frequency trading architect and head of exchange sales in New York for Radianz, the financial cloud (acquired by BT plc). Eric now heads Streamdata.io as CEO. Based in France and USA, Streamdata.io is a leader in event-driven infrastructures with its Distribute product and its Journey API Workshops.
Abstract in 140 characters: Event-Driven API is the new best practice for immersive UX, meshing services, data streams, and real-time AI. Use cases from the web giants.
Abstract
Traditional software was based on batches and databases for decades. But the real world is a system of agents changing states based on events sent by other agents, not records updating each other once in a while. To provide better UX, frontends have started to react to events with reactive frameworks. Data hubs too have moved from lakes to streams. AI also is updating its models based on events with streaming algorithms. And finallly dear HTTP/2-3 is now based on streams. Since APIs are -by definition- the interface between domains, successful API strategies by most mature API providers are adding webhooks, and streaming, on top of their existing request and response approach to deliver APIs. Let's take a moment to learn from the API rockstars like Slack, Twitch and Nest, as well as the growing number of API pioneers who have invested heavily on their event-driven infrastructure in 2018.

Transcript

1. Event-Driven APIs Apidays.io Paris, 2018 Eric Horesnyi, CEO 1
2. ©streamdata.io 2018 | confidential 2 December 2018 25 years in the Web
3. ©streamdata.io 2018 | confidential Real Time i$ Rea£ Mone¥ 3 December 2018
4. ©streamdata.io 2018 | confidential The Web 4 December 2018
5. ©streamdata.io 2018 | confidential Streamdata.io 5 December 2018 API Consumers API Product Consumed APIs Distribute™ Provide Events Distribute™ Consume Events Discover™ Be discovered by API consumers Discover™ Discover API Providers Journey™ API Lifecycle
6. Business Drivers Event-Driven APIs 6
7. ©streamdata.io 2018 | confidential Real Time i$ Rea£ Mone¥ : Finance 7 December 2018 Trading 10 msec. = +$8m/y Source: http://www.tabbgroup.com/PublicationDetail.aspx?PublicationID=346 UX 10 msec. = + $1Bn Source: 10% over 1 sec increase applied to ICBC
8. ©streamdata.io 2018 | confidential Real Time i$ Rea£ Mone¥ : Retail UX 10 msec.= +$200m/y Source: http://perspectives.mvdirona.com/2009/10/31/TheCostOfLatency.aspx 8 December 2018 Logistics 1 sec.= +$50m/y Source: How many $m does Amzon make per second
9. ©streamdata.io 2018 | confidential Polling Madness 9 December 2018
10. ©streamdata.io 2018 | confidential Events 10 December 2018
11. ©streamdata.io 2018 | confidential Vocabulary 11 December 2018 REST Stateless Request/Response Resources URL Action Volume: # API Calls Events Stateful Persistent Connections Events Topic Reaction Intensity: # Items / sec
12. ©streamdata.io 2018 | confidential Event-Driven Stack#1 12 December 2018 Real Time Payments (RTP) Restaurant Demo (Slideshow) TARGET Instant Payment Settlement (TIPS) (Slideshow) Financial RoboAdvisor (Slideshow) Amplify & Choreography
13. ©streamdata.io 2018 | confidential Protocols 13 December 2018 AMQP Webhook Websocket Kafka SSE
14. Tech Drivers Event-Driven APIs 1 4
15. ©streamdata.io 2018 | confidential Frontends React to Events Best-of-breed UX is reactive: real-time and self-refreshed, eg. ReactJS 15 December 2018 .onData(...) .onPatch(...) .onError(...) Web Component SSE Event-driven Data Source Source: https://streamdata.io/blog/ui-latency-json-patch/ Events JavaScript EURUSD 1.15331 USGBP 0.632887 EURGPY 138.0965 CHFDKK 6.883
16. ©streamdata.io 2018 | confidential Languages are reactive FRONT-END Manipulate UI and API events, on the Web with RxJS, or on mobile with RxJava or RxSwift 16 December 2018 BACK-END Compose asynchronous and event-based programs CROSS-PLATFORM Available for all major programming languages
17. ©streamdata.io 2018 | confidential No Tables, Just Streams Best-of-breed data backends are based on streams of events, eg Kafka Social networks Market data Alternative data Third-Party Microservices Internal Data Intelligence App & Third Party Apps Table* is reconstructed from stream. Stream and table represent same data. 17 December 2018
18. ©streamdata.io 2018 | confidential Streaming AI Massive online analysis: model updated with each event, AI feeds off data hubs Third-party microservices Internal data intelligence App & third party apps Social networks Market data Sensors data 18 December 2018
19. ©streamdata.io 2018 | confidential K8s and Service Mesh 19 December 2018 APIs for all, even backend IT ! Request/Response & Async
20. ©streamdata.io 2018 | confidential 2019: Data flows with PSD2 + Instant Payment ASPSP #1 ASPSP #2 ASPSP #3 Bank User API #1 API #2 API #3 U. X. Security Governance ASPSP #4 API #4 1 TPP taking updates from 1m customers with 1 transaction/4 day means 7.5m update/month/TPP/ASPSP. This sums up to 25Bn updates/year in USA, 45Bn/year in EU* Security: DDOS all • Shut down polling of APIs • Shut down all web scrapping Governance • No limit to traffic to partners • Third party traffic controlled Cost • Decrease ISSP traffic cost • Increase revenue for VIP users *USA data: https://www.nacha.org/news/ach-volume-grows-more-25-billion-payments-and-43-trillion-value-2016 EU Data: https://www.europeanpaymentscouncil.eu/news-insights/insight/greeces-progress-towards-less-cash-society 20 December 2018
21. ©streamdata.io 2018 | confidential Market Definition By 2020, 50% of organizations managing APIs will incorporate event-driven APIs in their architecture. M.O’Neill, P.Malinverno, Gartner Event-Driven API capabilities is 1 of 13 criteria we use to rank API Management Platforms P.Malinverno, Gartner Symposium 2018 21 December 2018
22. Examples Event-Driven APIs 2 2
23. ©streamdata.io 2018 | confidential Discover on the Web: API Gallery™ 23 December 2018 Curated directory and rating of dynamic APIs 40,000 endpoints organized in 800 topics from 500 organizations Discover web services and where APIs exist
24. ©streamdata.io 2018 | confidential Volume/Revenue of API Product $4Bn, ie 50% of Salesforce Revenue $3Bn, based on Q3.18 entire revenue 1+ Trillion API calls a year Approx. $200m $500m Use Case Consume any Salesforce CRM API data as event Social Network activity Financial Market Data: Stocks, Bonds, Index for apps & ML servers Slack Real Time Messaging (RTM) Business Support Chat Topics SOQL Filtering on phrases (Track), user IDs (Follow), location (BoundingBox), public status Filtering on Financial Instruments 90 + Event Types to subscribe to the most meaningful events on the platform Subscribe to almost 15 chat and support agent topics Messaging CometD: websocket or Ajax/HTTP SOQL to define Pushtopics HTTP / Long Running / Chunked SSE/HTTP Websockets Websockets 10 Event-Driven APIs 24 December 2018
25. ©streamdata.io 2018 | confidential Volume/Revenue of API Product “Most Popular Bitcoin Wallet” $1Bn total NTT Group - $105 Billion 200M people 10Bn API calls a day, 200m+ viewers Use Case Blockchain.com Nest Triathlon and Sporting Events Events gathering people on any topic Data streams about live video streams (!) Topics Based on bitcoin currency Device: home thermostat, smoke and CO2 alarms, indoor and outdoor security cameras Sports ranking, Team, health data from runners’ devices RSVP, Photos, Comments Games, Media, and real time pop culture events and conversations that occur at scale Messaging Websocket SSE Websockets HTTP / Websocket Webhooks / PubSub 10 Event-Driven APIs 25 December 2018
26. ©streamdata.io 2018 | confidential Xignite Cloudstreaming 26 December 2018
27. ©streamdata.io 2018 | confidential Event-Driven Stack#2 : Salesforce 27 December 2018 Distribute™
28. ©streamdata.io 2018 | confidential Discover Air Traffic 28 December 2018
29. ©streamdata.io 2018 | confidential Event-Driven Stack#3 : Talend Datastream 29 December 2018 Distribute™
30. Thank you 30