Building Async APIs with HTTP/2+ and Mercure

Slide 1

Slide 1 text

@dunglas - mercure.rocks Building Async APIs with HTTP/2+ and

Slide 2

Slide 2 text

@dunglas - mercure.rocks Kévin Dunglas ❏ Founder of Les-Tilleuls.coop ❏ Creator of Mercure, Vulcain, API Platform… ❏ Symfony Core Team @dunglas

Slide 3

Slide 3 text

@dunglas - mercure.rocks API and cloud experts ✊ worker-owned, self-managed cooperative 50 people, 1,000% growth in 6 years contact@les-tilleuls.coop Les-Tilleuls.coop

Slide 4

Slide 4 text

@dunglas - mercure.rocks Async and Real-Time APIs?!

Slide 5

Slide 5 text

@dunglas - mercure.rocks

Slide 6

Slide 6 text

@dunglas - mercure.rocks Why a New Protocol?

Slide 7

Slide 7 text

Slide 8

Slide 8 text

@dunglas - mercure.rocks WebSocket (RFC 6455) Benefits over SSE: ❏ Full-duplex communication ❏ Low level: full control Drawbacks over SSE: ❏ Low level: no native ❏ auth ❏ re-connection ❏ state reconciliation ❏ events history ❏ Obsoleted by HTTP/2 & 3 ❏ Hard to secure

Slide 9

Slide 9 text

@dunglas - mercure.rocks The Persistent Connections Problem ❏ WebSocket and SSE rely on persistent connections ❏ Serverless platforms (AWS Lambda, Cloud Run, Azure functions…), PHP, FastCGI…  are designed for short-lived connections ❏ Persistent, long-lived, concurrent connections are better handled by  dedicated and optimized software and hardware

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

@dunglas - mercure.rocks The Mercure Protocol

Slide 12

Slide 12 text

@dunglas - mercure.rocks

Slide 13

Slide 13 text

@dunglas - mercure.rocks Mercure, at a Glance ❏ Full-duplex, but plain old HTTP ❏ Publish: HTTP POST ❏ Subscribe: SSE ❏ Built-in: reconnection, retrieving of lost messages, history ❏ Auto-discoverable: designed for REST and GraphQL ❏ JWT-based authorization mechanism (private updates) ❏ Designed for serverless, PHP, FastCGI… ❏ End-2-End encryption support

Slide 14

Slide 14 text

@dunglas - mercure.rocks Stream Events to Clients

Slide 15

Slide 15 text

@dunglas - mercure.rocks Internet Draft: draft-dunglas-mercure

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

@dunglas - mercure.rocks Mercure and HTTP/2+

Slide 18

Slide 18 text

@dunglas - mercure.rocks © Narayan Prusty What is Multiplexing in HTTP/2? HTTP/2 Multiplexing

Slide 19

Slide 19 text

@dunglas - mercure.rocks HTTP/2 global support:  95% of all internet users ...and SSEs also work with HTTP/1.x

Slide 20

Slide 20 text

@dunglas - mercure.rocks SSE global support:  100% of all internet users there is a polyfill for IE 5.5+

Slide 21

Slide 21 text

@dunglas - mercure.rocks Publishing

Slide 22

Slide 22 text

@dunglas - mercure.rocks Publishing POST /.well-known/mercure Content-Type: application/x-www-form-urlencoded Authorization: Bearer: topic=&data=

Slide 23

Slide 23 text

@dunglas - mercure.rocks const http = require('http'); const querystring = require('querystring'); const postData = querystring.stringify({ 'topic': 'https://example.com/books/1', 'data': JSON.stringify({ '@id': 'https://example.com/books/1', status: 'OufOfStock'}), }); const req = http.request({ hostname: 'demo.mercure.rocks', port: '80', path: '/.well-known/mercure', method: 'POST', headers: { Authorization: 'Bearer snip', 'Content-Type': 'application/x-www-form-urlencoded', 'Content-Length': Buffer.byteLength(postData), } }); req.write(postData);req.end(); Publishing: Node.js

Slide 24

Slide 24 text

@dunglas - mercure.rocks Publishing: Symfony (PHP)

Slide 25

Slide 25 text

@dunglas - mercure.rocks Subscribing

Slide 26

Slide 26 text

@dunglas - mercure.rocks Subscribing: Basic Usage const es = new EventSource( 'https://demo.mercure.rocks/.well-known/mercure?topic=https://example.com/books/1' ); // The callback will be called every time an update is published es.onmessage = function (e) { console.log(JSON.parse(e.data)); }

Slide 27

Slide 27 text

@dunglas - mercure.rocks Subscribing: Several Topics const url = new URL('https://demo.mercure.rocks/.well-known/mercure'); url.searchParams.append('topic', 'https://example.com/books/1') url.searchParams.append('topic', 'https://example.com/reviews/21') const es = new EventSource(url); es.onmessage = function (e) { console.log(JSON.parse(e.data)); }

Slide 28

Slide 28 text

@dunglas - mercure.rocks Subscribing: using patterns const url = new URL('https://demo.mercure.rocks/.well-known/mercure'); // URI Template (RFC 6560) url.searchParams.append('topic', 'https://example.com/books/{id}') // Both https://example.com/books/1 and https://example.com/books/33 will match const es = new EventSource(url); es.onmessage = function (e) { console.log(JSON.parse(e.data)); }

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

@dunglas - mercure.rocks The Discovery Mechanism

Slide 31

Slide 31 text

@dunglas - mercure.rocks Discovery Mechanism Web Linking: RFC 5988

Slide 32

Slide 32 text

@dunglas - mercure.rocks Authorization

Slide 33

Slide 33 text

@dunglas - mercure.rocks Authorization ❏ Uses JSON Web Token (JWT - RFC 7519) ❏ An update can be intended for one or several targets ❏ Publisher: must be authenticated ❏ Subscriber: ❏ Can be anonymous (if allowed by the config) ❏ Must be authenticated to receive private updates ❏ Two transports: cookie and Authorization header

Slide 34

Slide 34 text

@dunglas - mercure.rocks Authorization

Slide 35

Slide 35 text

@dunglas - mercure.rocks The Mercure Hub

Slide 36

Slide 36 text

@dunglas - mercure.rocks Reference Implementation ❏ Implements 100% of the Mercure protocol ❏ Fast, written in Go ❏ Works everywhere: static binaries and Docker ❏ Automatic HTTP/2 and HTTPS (Let’s Encrypt) ❏ CORS support, CSRF protection ❏ Cloud Native (12Factor App) ❏ Open source (AGPL) ❏ Optional: a server can implement directly the protocol

Slide 37

Slide 37 text

@dunglas - mercure.rocks Reference Implementation https://mercure.rocks

Slide 38

Slide 38 text

@dunglas - mercure.rocks Starting the Hub https://mercure.rocks

Slide 39

Slide 39 text

@dunglas - mercure.rocks On a Kubernetes Cluster

Slide 40

Slide 40 text

@dunglas - mercure.rocks Alternative Implementations ❏ No hubs: the hub is optional ❏ Go library (AGPL) ❏ Node.js library (GPL) ❏ HA version (managed and on premise)

Slide 41

Slide 41 text

@dunglas - mercure.rocks iGraal / Glory4Gamers Benchmarks ❏ Open Source version (EC2 t3.micro) ❏ 40k concurrent connections ❏ HA version (on premise) ❏ 200k concurrent connections

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

@dunglas - mercure.rocks Integrations

Slide 44

Slide 44 text

@dunglas - mercure.rocks Integrations ❏ Official integrations: ❏ Symfony, API Platform ❏ Libs: ❏ Go, JS, Python, Java, Dart, PHP, Laravel

Slide 45

Slide 45 text

@dunglas - mercure.rocks Example using  API Platform & React

Slide 46

Slide 46 text

@dunglas - mercure.rocks Create a Mercure-enabled Web API api-platform.com

Slide 47

Slide 47 text

You get a fully-featured API supporting: ❏ JSON-LD + Hydra + schema.org ❏ GraphQL, JSON:API, HAL ❏ OpenAPI ❏ auth, pagination, filters, validation, HTTP/2 push, test framework… ❏ and Mercure support! MIT license

Slide 48

Slide 48 text

@dunglas - mercure.rocks Scaffold Mercure-Enabled Clients api-platform.com

Slide 49

Slide 49 text

@dunglas - mercure.rocks Try it! bit.ly/2H17hN3

Slide 50

Slide 50 text

@dunglas - mercure.rocks Unleashing the Power of AsyncAPI servers: production: url: https://demo.mercure.rocks/.well-known/mercure protocol: mercure channels: 'https://example.com/books/{id}': parameters: id: schema: type: integer subscribe: message: $ref: '#/components/messages/book' publish: message: $ref: '#/components/messages/book'

Slide 51

Slide 51 text

@dunglas les-tilleuls.coop .rocks