Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Async APIs with HTTP/2+ and Mercure

Building Async APIs with HTTP/2+ and Mercure

Mercure.rocks is a new protocol allowing to push data updates to web browsers and other HTTP clients in a convenient, fast, reliable and battery-efficient way. It is especially useful to publish real-time updates of resources served through web APIs, to reactive web and mobile apps.

From the ground up, Mercure has been designed to work with technologies not able to maintain persistent connections. It's especially relevant in serverless environments, but is also convenient when using PHP or FastCGI scripts.

Mercure is basically a higher-level replacement for WebSockets. But unlike WebSockets, it leverages the new capabilities of HTTP/2 and HTTP/3.
It has been designed with hypermedia APIs in mind, is auto-discoverable through the Web Linking RFC and is also compatible with GraphQL.
It natively supports authorization, reconnection in case of network issue (with refetching of missed events), subscribing to several topics, topics patterns (using templated URIs)...

Because it is built on top of Server-sent Events and plain old HTTP requests, it is already compatible with all modern browsers, and requires 0 client-side dependencies.

The protocol is open (available as an Internet Draft), and a reference open source implementation of the server written in Go is available. Integrations with popular web frameworks are already available!

During this talk, you'll learn everything you need to get started with the Mercure protocol!


Kévin Dunglas

April 22, 2020


  1. @dunglas - mercure.rocks Building Async APIs with HTTP/2+ and

  2. @dunglas - mercure.rocks Kévin Dunglas ❏ Founder of Les-Tilleuls.coop ❏

    Creator of Mercure, Vulcain, API Platform… ❏ Symfony Core Team @dunglas
  3. @dunglas - mercure.rocks API and cloud experts ✊ worker-owned, self-managed

    cooperative 50 people, 1,000% growth in 6 years contact@les-tilleuls.coop Les-Tilleuls.coop
  4. @dunglas - mercure.rocks Async and Real-Time APIs?!

  5. @dunglas - mercure.rocks

  6. @dunglas - mercure.rocks Why a New Protocol?

  7. © Ilya Grigorik High Performance Browser Networking (O'Reilly)

  8. @dunglas - mercure.rocks WebSocket (RFC 6455) Benefits over SSE: ❏

    Full-duplex communication ❏ Low level: full control Drawbacks over SSE: ❏ Low level: no native ❏ auth ❏ re-connection ❏ state reconciliation ❏ events history ❏ Obsoleted by HTTP/2 & 3 ❏ Hard to secure
  9. @dunglas - mercure.rocks The Persistent Connections Problem ❏ WebSocket and

    SSE rely on persistent connections ❏ Serverless platforms (AWS Lambda, Cloud Run, Azure functions…), PHP, FastCGI…
 are designed for short-lived connections ❏ Persistent, long-lived, concurrent connections are better handled by
 dedicated and optimized software and hardware
  10. None
  11. @dunglas - mercure.rocks The Mercure Protocol

  12. @dunglas - mercure.rocks

  13. @dunglas - mercure.rocks Mercure, at a Glance ❏ Full-duplex, but

    plain old HTTP ❏ Publish: HTTP POST ❏ Subscribe: SSE ❏ Built-in: reconnection, retrieving of lost messages, history ❏ Auto-discoverable: designed for REST and GraphQL ❏ JWT-based authorization mechanism (private updates) ❏ Designed for serverless, PHP, FastCGI… ❏ End-2-End encryption support
  14. @dunglas - mercure.rocks Stream Events to Clients

  15. @dunglas - mercure.rocks Internet Draft: draft-dunglas-mercure

  16. None
  17. @dunglas - mercure.rocks Mercure and HTTP/2+

  18. @dunglas - mercure.rocks © Narayan Prusty What is Multiplexing in

    HTTP/2? HTTP/2 Multiplexing
  19. @dunglas - mercure.rocks HTTP/2 global support:
 95% of all internet

    users ...and SSEs also work with HTTP/1.x
  20. @dunglas - mercure.rocks SSE global support:
 100% of all internet

    users there is a polyfill for IE 5.5+
  21. @dunglas - mercure.rocks Publishing

  22. @dunglas - mercure.rocks Publishing POST /.well-known/mercure Content-Type: application/x-www-form-urlencoded Authorization: Bearer:

    <JWT> topic=<topic-url>&data=<update-content>
  23. @dunglas - mercure.rocks const http = require('http'); const querystring =

    require('querystring'); const postData = querystring.stringify({ 'topic': 'https://example.com/books/1', 'data': JSON.stringify({ '@id': 'https://example.com/books/1', status: 'OufOfStock'}), }); const req = http.request({ hostname: 'demo.mercure.rocks', port: '80', path: '/.well-known/mercure', method: 'POST', headers: { Authorization: 'Bearer snip', 'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': Buffer.byteLength(postData), } }); req.write(postData);req.end(); Publishing: Node.js
  24. @dunglas - mercure.rocks Publishing: Symfony (PHP)

  25. @dunglas - mercure.rocks Subscribing

  26. @dunglas - mercure.rocks Subscribing: Basic Usage const es = new

    EventSource( 'https://demo.mercure.rocks/.well-known/mercure?topic=https://example.com/books/1' ); // The callback will be called every time an update is published es.onmessage = function (e) { console.log(JSON.parse(e.data)); }
  27. @dunglas - mercure.rocks Subscribing: Several Topics const url = new

    URL('https://demo.mercure.rocks/.well-known/mercure'); url.searchParams.append('topic', 'https://example.com/books/1') url.searchParams.append('topic', 'https://example.com/reviews/21') const es = new EventSource(url); es.onmessage = function (e) { console.log(JSON.parse(e.data)); }
  28. @dunglas - mercure.rocks Subscribing: using patterns const url = new

    URL('https://demo.mercure.rocks/.well-known/mercure'); // URI Template (RFC 6560) url.searchParams.append('topic', 'https://example.com/books/{id}') // Both https://example.com/books/1 and https://example.com/books/33 will match const es = new EventSource(url); es.onmessage = function (e) { console.log(JSON.parse(e.data)); }
  29. None
  30. @dunglas - mercure.rocks The Discovery Mechanism

  31. @dunglas - mercure.rocks Discovery Mechanism Web Linking: RFC 5988

  32. @dunglas - mercure.rocks Authorization

  33. @dunglas - mercure.rocks Authorization ❏ Uses JSON Web Token (JWT

    - RFC 7519) ❏ An update can be intended for one or several targets ❏ Publisher: must be authenticated ❏ Subscriber: ❏ Can be anonymous (if allowed by the config) ❏ Must be authenticated to receive private updates ❏ Two transports: cookie and Authorization header
  34. @dunglas - mercure.rocks Authorization

  35. @dunglas - mercure.rocks The Mercure Hub

  36. @dunglas - mercure.rocks Reference Implementation ❏ Implements 100% of the

    Mercure protocol ❏ Fast, written in Go ❏ Works everywhere: static binaries and Docker ❏ Automatic HTTP/2 and HTTPS (Let’s Encrypt) ❏ CORS support, CSRF protection ❏ Cloud Native (12Factor App) ❏ Open source (AGPL) ❏ Optional: a server can implement directly the protocol
  37. @dunglas - mercure.rocks Reference Implementation https://mercure.rocks

  38. @dunglas - mercure.rocks Starting the Hub https://mercure.rocks

  39. @dunglas - mercure.rocks On a Kubernetes Cluster

  40. @dunglas - mercure.rocks Alternative Implementations ❏ No hubs: the hub

    is optional ❏ Go library (AGPL) ❏ Node.js library (GPL) ❏ HA version (managed and on premise)
  41. @dunglas - mercure.rocks iGraal / Glory4Gamers Benchmarks ❏ Open Source

    version (EC2 t3.micro) ❏ 40k concurrent connections ❏ HA version (on premise) ❏ 200k concurrent connections
  42. None
  43. @dunglas - mercure.rocks Integrations

  44. @dunglas - mercure.rocks Integrations ❏ Official integrations: ❏ Symfony, API

    Platform ❏ Libs: ❏ Go, JS, Python, Java, Dart, PHP, Laravel
  45. @dunglas - mercure.rocks Example using
 API Platform & React

  46. @dunglas - mercure.rocks Create a Mercure-enabled Web API api-platform.com

  47. You get a fully-featured API supporting: ❏ JSON-LD + Hydra

    + schema.org ❏ GraphQL, JSON:API, HAL ❏ OpenAPI ❏ auth, pagination, filters, validation, HTTP/2 push, test framework… ❏ and Mercure support! MIT license
  48. @dunglas - mercure.rocks Scaffold Mercure-Enabled Clients api-platform.com

  49. @dunglas - mercure.rocks Try it! bit.ly/2H17hN3

  50. @dunglas - mercure.rocks Unleashing the Power of AsyncAPI servers: production:

    url: https://demo.mercure.rocks/.well-known/mercure protocol: mercure channels: 'https://example.com/books/{id}': parameters: id: schema: type: integer subscribe: message: $ref: '#/components/messages/book' publish: message: $ref: '#/components/messages/book'
  51. @dunglas les-tilleuls.coop .rocks