Mercure: Real-Time for PHP Made Easy

by Kévin Dunglas

Slide 1

Slide 1 text

Real-Time for PHP and Beyond

Slide 2

Slide 2 text

@dunglas - mercure.rocks Kévin Dunglas ❏ Founder of Les-Tilleuls.coop ❏ Creator of Mercure.rocks, Vulcain.rocks, API Platform… ❏ Symfony Core Team Member @dunglas

Slide 3

Slide 3 text

@dunglas - mercure.rocks PHP, API and cloud experts ✊ Self-managed, 100% employee-owned 45 people, 1,000% growth in 6 years Come to our booth!  [email protected] Les-Tilleuls.coop

Slide 4

Slide 4 text

@dunglas - mercure.rocks Real-Time APIs?!

Slide 5

Slide 5 text

@dunglas - mercure.rocks Mercure: Push from Server to Clients ❏ Push notifications ❏ Synchronize connected devices in real-time ❏ Collaborative editing (Google Docs-like) ❏ Notify users when an async task has finished Modern and high level alternative to WebSocket

Slide 6

Slide 6 text

@dunglas - mercure.rocks

Slide 7

Slide 7 text

@dunglas - mercure.rocks Why a New Protocol?

Slide 8

Slide 8 text

Slide 9

Slide 9 text

@dunglas - mercure.rocks WebSocket (RFC 6455) Benefits over SSE: ❏ Full-duplex communication ❏ Low level: full control Drawbacks over SSE: ❏ Low level: no native ❏ auth ❏ re-connection ❏ state reconciliation ❏ events history ❏ Obsoleted by HTTP/2 & 3 ❏ Hard to secure

Slide 10

Slide 10 text

@dunglas - mercure.rocks The Persistent Connections Problem ❏ WebSocket and SSE rely on persistent connections ❏ PHP but also Serverless platforms, FastCGI…  are designed for short-lived connections ❏ Persistent, long-lived, concurrent connections are better handled by  dedicated and optimized software and hardware

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

@dunglas - mercure.rocks The Mercure Protocol

Slide 13

Slide 13 text

@dunglas - mercure.rocks

Slide 14

Slide 14 text

@dunglas - mercure.rocks Mercure, at a Glance ❏ Full-duplex, but plain old HTTP ❏ Publish: HTTP POST ❏ Subscribe: SSE ❏ Built-in: reconnection, retrieving of lost messages, history ❏ Auto-discoverable: designed for REST and GraphQL ❏ JWT-based authorization mechanism (private updates) ❏ Designed for serverless, PHP, FastCGI… ❏ End-2-End encryption support

Slide 15

Slide 15 text

@dunglas - mercure.rocks Stream Events to Clients

Slide 16

Slide 16 text

@dunglas - mercure.rocks Internet Draft: draft-dunglas-mercure

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

@dunglas - mercure.rocks Mercure and HTTP/2+

Slide 19

Slide 19 text

@dunglas - mercure.rocks © Narayan Prusty - What is Multiplexing in HTTP/2? HTTP/2: Multiplexing, and more ❏ Headers compression ❏ Headers deduplication ❏ No per-host connections limits

Slide 20

Slide 20 text

@dunglas - mercure.rocks HTTP/2 support: 92% of all users ...and SSEs also work with HTTP/1.x

Slide 21

Slide 21 text

@dunglas - mercure.rocks SSE support: 93% of all users

Slide 22

Slide 22 text

@dunglas - mercure.rocks No IE? No Edge?!

Slide 23

Slide 23 text

@dunglas - mercure.rocks There is a Polyfill for That! SSEs global support (with polyfill): ~100%

Slide 24

Slide 24 text

@dunglas - mercure.rocks Publishing

Slide 25

Slide 25 text

@dunglas - mercure.rocks Publishing

Slide 26

Slide 26 text

@dunglas - mercure.rocks Publishing: PHP

Slide 27

Slide 27 text

@dunglas - mercure.rocks Subscribing

Slide 28

Slide 28 text

@dunglas - mercure.rocks Subscribing: Basic Usage

Slide 29

Slide 29 text

@dunglas - mercure.rocks Subscribing: Several Topics

Slide 30

Slide 30 text

@dunglas - mercure.rocks Subscribing: URI Templates

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

@dunglas - mercure.rocks The Discovery Mechanism

Slide 33

Slide 33 text

@dunglas - mercure.rocks Discovery Mechanism Web Linking: RFC 5988

Slide 34

Slide 34 text

@dunglas - mercure.rocks Authorization

Slide 35

Slide 35 text

@dunglas - mercure.rocks Authorization ❏ Uses JSON Web Token (JWT - RFC 7519) ❏ An update can be intended for one or several targets ❏ Publisher: must be authenticated ❏ Subscriber: ❏ Can be anonymous (if allowed by the config) ❏ Must be authenticated to receive private updates ❏ Two transports: cookie and Authorization header

Slide 36

Slide 36 text

@dunglas - mercure.rocks

Slide 37

Slide 37 text

@dunglas - mercure.rocks JSON Web Token and Targets

Slide 38

Slide 38 text

@dunglas - mercure.rocks Publishing or Subscribing to All Targets

Slide 39

Slide 39 text

@dunglas - mercure.rocks Cookie-based Authorization ❏ Set by the app server during the discovery ❏ The app server and the Mercure hub must share the same domain (or subdomain) ❏ Supported by EventSource ❏ Recommended for web browsers

Slide 40

Slide 40 text

@dunglas - mercure.rocks HTTP Header-based Authorization ❏ Set by the client ❏ Not supported by native EventSource ❏ Supported by the polyfill ❏ Recommended for servers

Slide 41

Slide 41 text

@dunglas - mercure.rocks Publishing to Specific Targets

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

@dunglas - mercure.rocks The Mercure Hub

Slide 44

Slide 44 text

@dunglas - mercure.rocks Reference Implementation ❏ Implements 100% of the Mercure protocol ❏ Fast, written in Go ❏ Works everywhere: static binaries and Docker ❏ Automatic HTTP/2 and HTTPS (Let’s Encrypt) ❏ CORS support, CSRF protection ❏ Cloud Native (12Factor App) ❏ Open source (AGPL) ❏ Optional: a server can implement directly the protocol

Slide 45

Slide 45 text

@dunglas - mercure.rocks Reference Implementation https://mercure.rocks

Slide 46

Slide 46 text

@dunglas - mercure.rocks Starting the Hub https://mercure.rocks

Slide 47

Slide 47 text

@dunglas - mercure.rocks

Slide 48

Slide 48 text

@dunglas - mercure.rocks Using Docker

Slide 49

Slide 49 text

@dunglas - mercure.rocks On a Kubernetes Cluster

Slide 50

Slide 50 text

@dunglas - mercure.rocks Alternative Implementations ❏ No hubs: the hub is optional ❏ Go library (AGPL) ❏ Node.js library (GPL) ❏ HA version (managed and on premise)  in private beta: [email protected]

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

@dunglas - mercure.rocks Case Study:  Mercure on a  Large Scale at iGraal

Slide 53

Slide 53 text

@dunglas - mercure.rocks Quick intro about iGraal ❏ leader of cashback in France (5 millions members) ❏ cashback: earn money on your online purchases ❏ tech team of 30 persons ❏ available on website, mobile apps and web extensions

Slide 54

Slide 54 text

@dunglas - mercure.rocks What are iGraal Web Extensions ? ❏ tool to detect merchants & activate cashback ❏ demo

Slide 55

Slide 55 text

@dunglas - mercure.rocks About iGraal Web Extensions ❏ detect merchant cashback compatible ❏ activate cashback ❏ user account

Slide 56

Slide 56 text

@dunglas - mercure.rocks About iGraal Web Extensions

Slide 57

Slide 57 text

@dunglas - mercure.rocks Large Scale to Handle ❏ 500,000 extensions currently installed ❏ 1 browser « opened » = 1 extension « connected » ❏ average of 100 000 extensions « connected » simultaneously

Slide 58

Slide 58 text

@dunglas - mercure.rocks Architecture without Mercure

Slide 59

Slide 59 text

@dunglas - mercure.rocks Architecture without Mercure ❏ poll our API every 4H to sync user data in background

Slide 60

Slide 60 text

@dunglas - mercure.rocks Architecture With Mercure

Slide 61

Slide 61 text

@dunglas - mercure.rocks On Premise Configuration

Slide 62

Slide 62 text

@dunglas - mercure.rocks Developper Feedbacks ❏ quick and simple to integrate (online docs are enough) ❏ sending « update me » messages is easier   than generating data diff

Slide 63

Slide 63 text

@dunglas - mercure.rocks Devops Feedbacks ❏ backend resources savings €€€ ❏ small resources needs: ❏ 2 instances t3.medium for Mercure Nodes ❏ 1 instance t3.small for Kafka ❏ Load Balancing Tips ❏ enable Mercure’s heartbeat mechanism for Amazon’s Application Load Balancer (ALB) compatibility ❏ set a low « TTL » for Mercure connections to spread the load

Slide 64

Slide 64 text

@dunglas - mercure.rocks Future usages ❏ mobile applications?

Slide 65

Slide 65 text

@dunglas - mercure.rocks Study Conclusion « T’as mis des paillettes dans ma vie Kevin ! »

Slide 66

Slide 66 text

@dunglas - mercure.rocks Integrations

Slide 67

Slide 67 text

@dunglas - mercure.rocks Integrations ❏ Libs: PHP, amphp, JS, Python, Dart ❏ Official integrations: ❏ Symfony, API Platform ❏ Community integrations: ❏ Laravel, Yii framework, GitHub Actions ❏ Official examples: ❏ PHP, JS, Go, Ruby, Python

Slide 68

Slide 68 text

@dunglas - mercure.rocks Example using  API Platform & React

Slide 69

Slide 69 text

@dunglas - mercure.rocks Create a Mercure-enabled Web API api-platform.com

Slide 70

Slide 70 text

You get a fully-featured API supporting: ❏ JSON-LD + Hydra + schema.org ❏ GraphQL, JSON:API, HAL ❏ OpenAPI ❏ auth, pagination, filters, validation, HTTP/2 push, test framework… ❏ and Mercure support! MIT license