HTTP/2 Server Push
 and the rise of client-driven REST APIs

@dunglas - vulcain.rocks Kévin Dunglas ❏ Founder of Les-Tilleuls.coop ❏ Creator of Vulcain.rocks, Mercure.rocks, API Platform… ❏ Symfony Core Team Member @dunglas @ .social

@dunglas - vulcain.rocks API, cloud, PHP and JS experts ✊ Self-managed, 100% employee-owned 45 people, 1,000% growth in 6 years [email protected] Les-Tilleuls.coop

@dunglas - vulcain.rocks What’s REST?

@dunglas - vulcain.rocks « REST was developed […] primarily as a means for communicating Web concepts as we wrote the HTTP/1.0 specification and the initial HTTP/1.1 proposal. »
 - Roy T. Fielding

@dunglas - vulcain.rocks REST: the Architectural Style of the Web ❏ client-server ❏ resource oriented ❏ stateless ❏ cacheable by clients and intermediaries ❏ layered (gateways and proxies) ❏ uniform interface (URI, HATEOAS…)

RESTful Web Services

@dunglas - vulcain.rocks REST Performance

@dunglas - vulcain.rocks REST Performance ❏ REST is designed to solve performance and scalability problems at internet-scale ❏ REST relies on a powerful caching model: clients and intermediates can cache ❏ HTTP implements this model © Google

@dunglas - vulcain.rocks The Under-Fetching and the N+1 Problems Not fetching enough data initially forces to issue subsequent requests, so creates waterfalls and increase latency. © Expedia Group

@dunglas - vulcain.rocks The Over-Fetching Problem Generating and fetching too much data (data not used by clients) wastes bandwidth and computation power.

GraphQL: an Alternative to REST « Fixing » its Perf Issues (among other things…)

@dunglas - vulcain.rocks HTTP/2+

@dunglas - vulcain.rocks HTTP/2: Making the Web Faster ❏ Binary protocol, multiplexing, prioritization ❏ Server Push of resources ❏ Headers compression ❏ HTTP/3: good bye TCP (perf, again)

@dunglas - vulcain.rocks HTTP/2: Multiplexing © Narayan Prusty

@dunglas - vulcain.rocks HTTP/2: Server Push

@dunglas - vulcain.rocks HTTP/2 global support:
 95% of all internet users

@dunglas - vulcain.rocks CloudWays Benchmark

@dunglas - vulcain.rocks Fast and idiomatic client-driven REST APIs Introducing Vulcain

No content

@dunglas - vulcain.rocks Pushing Relations

@dunglas - vulcain.rocks Consuming Pushes in JavaScript const outboxResp = await fetch( "/outbox", { headers: { Preload: "/orderedItems/*/object" } } ); const outbox = await outboxResp.json(); outboxJSON.orderedItems.forEach(async activityURL => { // Returns immediately,
 // the resource has been pushed and is already in the push cache const activityResp = await fetch(activityURL); const activity = await activityResp.json(); // Returns immediately too const statusResp = await fetch(activity.object); // Do something with the status response... });

@dunglas - vulcain.rocks Benefits ❏ Fixes the under-fetching and N+1 problems ❏ Resources are downloaded in parallel (multiplexing) ❏ Granular HTTP cache (download only what isn’t in cache) ❏ Respects REST principles ❏ Leverages all features of the HTTP protocol ❏ cache, authorization, content negotiation, extensibility…

@dunglas - vulcain.rocks Filtering Resources

@dunglas - vulcain.rocks Other Capabilities ❏ Selector negotiation: ❏ extended JSON Pointer, XPath, CSS selectors, JMESPath… ❏ Support using Link preload headers and103 Early Hints as a fallback (cross-domain resources, HTTP/1…) ❏ Use query parameters instead of HTTP headers ❏ Support non-hypermedia APIs through OpenAPI

@dunglas - vulcain.rocks The Vulcain Gateway Server

@dunglas - vulcain.rocks The Vulcain Gateway Server ❏ Instantly turns any existing hypermedia or non-hypermedia API into a Vulcain-enabled API ❏ Gateway server (plug’n’play network intermediate) ❏ Free software (open source) ❏ Fast, written in Go ❏ Easy to install (static binary, Docker image) ❏ Best performance: implement the protocol directly at the app server layer

@dunglas - vulcain.rocks Starting the Gateway Server

@dunglas - vulcain.rocks Mapping a « Legacy » API

@dunglas - vulcain.rocks Mapping a « Legacy » API # openapi.yaml openapi: 3.0.0 # ... paths: '/books/{id}': get: # ... responses: default: links: author: operationId: getAuthor parameters: id: '$response.body#/author' '/authors/{id}': get: operationId: getAuthor responses: default: # ... // /books/1984 { "title": "1984", "author": 1 // Not an URI } // /authors/1 { "givenName": "George", "familyName": "Orwell" }

@dunglas - vulcain.rocks So… What About GraphQL?

@dunglas - vulcain.rocks Vulcain vs GraphQL: Performance ❏ Unlike GraphQL, Vulcain sends each pushed resource in a separate HTTP/2 stream (multiplexing): related resources are sent in parallel ❏ Embedding resources is a forced push: the client receive the full JSON documents, even if it already has some parts of it ❏ With Vulcain, clients can cancel the push of resources they already have, saving bandwidth and improving performance

@dunglas - vulcain.rocks But GraphQL is More Than That! ❏ Query Language ❏ Type System ❏ Introspection ❏ Subscriptions ❏ Great tooling

@dunglas - vulcain.rocks Type System and Introspection REST has that too… and a bit more! ❏ JSON-LD: internet-scale type system,
 built on top of RDF, W3C ❏ Hydra: hypermedia introspection system,
 built on top of JSON-LD, W3C ❏ OpenAPI: type system and introspection

@dunglas - vulcain.rocks { "@context": "https://www.w3.org/ns/activitystreams", "type": "Person", "id": "https://social.example/alyssa/", "name": "Alyssa P. Hacker", "preferredUsername": "alyssa", "summary": "Lisp enthusiast hailing from MIT", "inbox": "https://social.example/alyssa/inbox/", "outbox": "https://social.example/alyssa/outbox/", "followers": "https://social.example/alyssa/followers/", "following": "https://social.example/alyssa/following/", "liked": "https://social.example/alyssa/liked/" }

@dunglas - vulcain.rocks Subscriptions https://mercure.rocks

@dunglas - vulcain.rocks Tooling ❏ GraphQL provides great client-side tooling and an unified Developer Experience, REST is still behind ❏ But because GraphQL uses HTTP as a black box: ❏ HTTP Cache servers (Varnish, Nginx…) ❏ standard WAF (ModSecurity…) ❏ HTTP logs analyzer (GoAccess, AWStats…) ❏ API Gateways (Kong, Gravitee…) are almost useless

@dunglas - vulcain.rocks Tooling REST also has modern, easy to use client-side and server-side tooling: ❏ API Platform (PHP & JS): create JSON-LD APIs and smart React & Vue PWA, admin UIs, etc… in minutes! ❏ Graphiti (Ruby On Rails): create REST APIs in minutes ❏ Zeit’s SWR: React Hooks library for HTTP APIs leveraging concurrent rendering (perfect Vulcain’s companion) Vulcain works out of the box with all these tools!

@dunglas - vulcain.rocks Using GraphQL to Query a Vulcain Server ❏ GraphQL is a convenient Query Language for APIs ❏ Write GraphQL queries client-side, and benefit from all the GQL ecosystem ❏ Use apollo-link-rest to convert the GQL query in a single REST+Vulcain request to the server Best of both worlds?

@dunglas - vulcain.rocks Save the Web:
 Stop Centralizing!

No content

@dunglas - vulcain.rocks

No content

@dunglas - vulcain.rocks « The Web is intended to be an Internet-scale distributed hypermedia system […]. The Internet is about interconnecting information networks across multiple organizational boundaries. »
 - Roy T. Fielding

@dunglas - vulcain.rocks The Threat of Data Centralization ❏ Facebook, Microsoft, Google, Apple… participated to NSA’s PRISM global mass surveillance program ❏ Cambridge Analytica manipulated the US election using Facebook data ❏ Facebook blocks pages of Kurdish organizations fighting ISIS ❏ GitHub blocks all Iranian users, including famous OSS contributors ❏ GitHub banned Tsunami Democràtic's app (pro Catalan-independence) Data centralization is harmful!

No content

@dunglas - vulcain.rocks REST /
 Linked Data GraphQL server Origin Scientific community, to serve humanity Facebook, to serve its business Data access « Internet as a global database » Centralized information silos Interoperability Global and built-in Partial Best suited for Distributed systems Public APIs (« Open Data ») The World Wide Web Centralized systems Internal APIs

@dunglas - vulcain.rocks // https://social.example/alyssa/posts/123 { "@context": "https://www.w3.org/ns/activitystreams", "type": "Note", // Not possible with a GraphQL server "to": ["https://chatty.example/ben/"], // reference to an external organization "attributedTo": "https://social.example/alyssa/", "content": "Say, did you finish reading that book I lent you?" } The power of Vulcain: GET https://social.example/alyssa/posts/123 Preload: /to Fields: /to, /content // Some hope for GraphQL? GraphQL-LD, HyperGraphQL: GQL as a query language for RDF

The Rise of the Dweb

@dunglas - vulcain.rocks The Rise of the Dweb REST, JSON-LD, schema.org, RDF, ActivityPub… ❏ Solid (by T. Berners-Lee): web decentralization, privacy by design ❏ : federated social network ❏ Mobilizon: federated Meetup-like ❏ PeerTube: federated video-streaming service ❏ NextCloud: federated file sharing service Vulcain works out of the box with all of them!

@dunglas - vulcain.rocks And What About Symfony?!

@dunglas - vulcain.rocks Vulcain, Symfony and API Platform The project: ❏ an app to send feedback to speakers, as in Lisbon 2018 The stack: ❏ Symfony + API Platform ❏ JSON-LD ❏ Varnish ❏ Vulcain

@dunglas - vulcain.rocks The API Platform Distribution ❏ PHP + Symfony + API Platform ❏ Vulcain (master branch only) ❏ Varnish ❏ Postgres ❏ Docker and Kubernetes config github.com/dunglas/demo-vulcain-api-platform

@dunglas - vulcain.rocks MakerBundle FTW!

@dunglas - vulcain.rocks Our Vulcain API is Ready! A fully-featured API for the Dweb: ❏ JSON-LD + Hydra + schema.org ❏ Vulcain ❏ Varnish ❏ Mercure, auth, pagination, filters, validation, test framework, admin, React/Vue.js/Next.js generators…

@dunglas - vulcain.rocks Benchmark ❏ Inject fixtures (using Alice) ❏ 10 conferences ❏ 100 sessions ❏ 100 feedback ❏ Fetch all the resources client-side

@dunglas - vulcain.rocks Benchmark // Load ALL data from the API // Simulate more data dependencies than they really exist
 // to increase the waterfall effect console.time('download'); const conferences = await fetchJson('/conferences', { headers: { Preload: ‘/hydra:member/*/@id/ sessions/*/feedback/*' }}); for (const conferenceRel of conferences['hydra:member']) { const conference = await fetchJson(conferenceRel['@id']); // Fetch sessions for (const sessionURL of conference.sessions) { const session = await fetchJson(sessionURL); // Fetch feedback for (const feedbackURL of session.feedback) { const feedback = await fetchJson(feedbackURL) } } } console.timeEnd('download');

@dunglas - vulcain.rocks Results: Vulcain kills the Waterfall effect ❏ No Varnish, No Vulcain: 22,152 ms ❏ No Varnish, Vulcain: 9,704 ms ❏ Varnish, No Vulcain: 783 ms ❏ Varnish + Vulcain: 524ms ❏ Vulcain is 2.3x faster than API Platform without Vulcain ❏ Varnish is 28x faster than API Platform without Varnish ❏ The combination of Varnish and Vulcain is 42x faster than API Platform without Varnish, and without Vulcain ❏ The combination of Varnish and Vulcain is 1.5x faster than Varnish alone!

Without Vulcain: 10.4s With Vulcain: 4.72s

@dunglas - vulcain.rocks github.com/dunglas/demo-vulcain-api-platform Contains a benchmark using Symfony HttpClient! « All data retrieved in 0.85s »

@dunglas - vulcain.rocks Summary Vulcain is: ❏ Fast (faster than GraphQL and JSON:API includes) ❏ Designed for the open web and interoperability at Internet-scale ❏ Easy to deploy ❏ Compatible with most existing tools ❏ Swagger/OpenAPI, log analyzers, Varnish, WAF, RDF… ❏ Can be supported in minutes by any web API ❏ including legacy RESTish APIs thanks to its OpenAPI support ❏ You can even use GraphQL to query a Vulcain API! Give it a try!

Thank you! vulcain.rocks - @dunglas - les-tilleuls.coop and come at our booth!