Building Async APIs with HTTP/2+ and Mercure

Building Async APIs with HTTP/2+ and Mercure

Mercure.rocks is a new protocol allowing to push data updates to web browsers and other HTTP clients in a convenient, fast, reliable and battery-efficient way. It is especially useful to publish real-time updates of resources served through web APIs, to reactive web and mobile apps.

From the ground up, Mercure has been designed to work with technologies not able to maintain persistent connections. It's especially relevant in serverless environments, but is also convenient when using PHP or FastCGI scripts.

Mercure is basically a higher-level replacement for WebSockets. But unlike WebSockets, it leverages the new capabilities of HTTP/2 and HTTP/3.
It has been designed with hypermedia APIs in mind, is auto-discoverable through the Web Linking RFC and is also compatible with GraphQL.
It natively supports authorization, reconnection in case of network issue (with refetching of missed events), subscribing to several topics, topics patterns (using templated URIs)...

Because it is built on top of Server-sent Events and plain old HTTP requests, it is already compatible with all modern browsers, and requires 0 client-side dependencies.

The protocol is open (available as an Internet Draft), and a reference open source implementation of the server written in Go is available. Integrations with popular web frameworks are already available!

During this talk, you'll learn everything you need to get started with the Mercure protocol!

E66449b8260b07a1cf51c5ab5eaa8180?s=128

Kévin Dunglas

April 22, 2020
Tweet

Transcript

  1. 2.

    @dunglas - mercure.rocks Kévin Dunglas ❏ Founder of Les-Tilleuls.coop ❏

    Creator of Mercure, Vulcain, API Platform… ❏ Symfony Core Team @dunglas
  2. 3.

    @dunglas - mercure.rocks API and cloud experts ✊ worker-owned, self-managed

    cooperative 50 people, 1,000% growth in 6 years contact@les-tilleuls.coop Les-Tilleuls.coop
  3. 8.

    @dunglas - mercure.rocks WebSocket (RFC 6455) Benefits over SSE: ❏

    Full-duplex communication ❏ Low level: full control Drawbacks over SSE: ❏ Low level: no native ❏ auth ❏ re-connection ❏ state reconciliation ❏ events history ❏ Obsoleted by HTTP/2 & 3 ❏ Hard to secure
  4. 9.

    @dunglas - mercure.rocks The Persistent Connections Problem ❏ WebSocket and

    SSE rely on persistent connections ❏ Serverless platforms (AWS Lambda, Cloud Run, Azure functions…), PHP, FastCGI…
 are designed for short-lived connections ❏ Persistent, long-lived, concurrent connections are better handled by
 dedicated and optimized software and hardware
  5. 10.
  6. 13.

    @dunglas - mercure.rocks Mercure, at a Glance ❏ Full-duplex, but

    plain old HTTP ❏ Publish: HTTP POST ❏ Subscribe: SSE ❏ Built-in: reconnection, retrieving of lost messages, history ❏ Auto-discoverable: designed for REST and GraphQL ❏ JWT-based authorization mechanism (private updates) ❏ Designed for serverless, PHP, FastCGI… ❏ End-2-End encryption support
  7. 16.
  8. 19.
  9. 23.

    @dunglas - mercure.rocks const http = require('http'); const querystring =

    require('querystring'); const postData = querystring.stringify({ 'topic': 'https://example.com/books/1', 'data': JSON.stringify({ '@id': 'https://example.com/books/1', status: 'OufOfStock'}), }); const req = http.request({ hostname: 'demo.mercure.rocks', port: '80', path: '/.well-known/mercure', method: 'POST', headers: { Authorization: 'Bearer snip', 'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': Buffer.byteLength(postData), } }); req.write(postData);req.end(); Publishing: Node.js
  10. 26.

    @dunglas - mercure.rocks Subscribing: Basic Usage const es = new

    EventSource( 'https://demo.mercure.rocks/.well-known/mercure?topic=https://example.com/books/1' ); // The callback will be called every time an update is published es.onmessage = function (e) { console.log(JSON.parse(e.data)); }
  11. 27.

    @dunglas - mercure.rocks Subscribing: Several Topics const url = new

    URL('https://demo.mercure.rocks/.well-known/mercure'); url.searchParams.append('topic', 'https://example.com/books/1') url.searchParams.append('topic', 'https://example.com/reviews/21') const es = new EventSource(url); es.onmessage = function (e) { console.log(JSON.parse(e.data)); }
  12. 28.

    @dunglas - mercure.rocks Subscribing: using patterns const url = new

    URL('https://demo.mercure.rocks/.well-known/mercure'); // URI Template (RFC 6560) url.searchParams.append('topic', 'https://example.com/books/{id}') // Both https://example.com/books/1 and https://example.com/books/33 will match const es = new EventSource(url); es.onmessage = function (e) { console.log(JSON.parse(e.data)); }
  13. 29.
  14. 33.

    @dunglas - mercure.rocks Authorization ❏ Uses JSON Web Token (JWT

    - RFC 7519) ❏ An update can be intended for one or several targets ❏ Publisher: must be authenticated ❏ Subscriber: ❏ Can be anonymous (if allowed by the config) ❏ Must be authenticated to receive private updates ❏ Two transports: cookie and Authorization header
  15. 36.

    @dunglas - mercure.rocks Reference Implementation ❏ Implements 100% of the

    Mercure protocol ❏ Fast, written in Go ❏ Works everywhere: static binaries and Docker ❏ Automatic HTTP/2 and HTTPS (Let’s Encrypt) ❏ CORS support, CSRF protection ❏ Cloud Native (12Factor App) ❏ Open source (AGPL) ❏ Optional: a server can implement directly the protocol
  16. 40.

    @dunglas - mercure.rocks Alternative Implementations ❏ No hubs: the hub

    is optional ❏ Go library (AGPL) ❏ Node.js library (GPL) ❏ HA version (managed and on premise)
  17. 41.

    @dunglas - mercure.rocks iGraal / Glory4Gamers Benchmarks ❏ Open Source

    version (EC2 t3.micro) ❏ 40k concurrent connections ❏ HA version (on premise) ❏ 200k concurrent connections
  18. 42.
  19. 44.

    @dunglas - mercure.rocks Integrations ❏ Official integrations: ❏ Symfony, API

    Platform ❏ Libs: ❏ Go, JS, Python, Java, Dart, PHP, Laravel
  20. 47.

    You get a fully-featured API supporting: ❏ JSON-LD + Hydra

    + schema.org ❏ GraphQL, JSON:API, HAL ❏ OpenAPI ❏ auth, pagination, filters, validation, HTTP/2 push, test framework… ❏ and Mercure support! MIT license
  21. 50.

    @dunglas - mercure.rocks Unleashing the Power of AsyncAPI servers: production:

    url: https://demo.mercure.rocks/.well-known/mercure protocol: mercure channels: 'https://example.com/books/{id}': parameters: id: schema: type: integer subscribe: message: $ref: '#/components/messages/book' publish: message: $ref: '#/components/messages/book'