Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mercure: Real-Time for PHP Made Easy

Mercure: Real-Time for PHP Made Easy

Mercure is a protocol allowing to push data updates to web browsers and other HTTP clients in a convenient, fast, reliable and battery-efficient way. It is especially useful to publish real-time updates of resources served through web APIs, to reactive web and mobile apps. It is provided with an open source implementation written in Go.

It provides:

* native browser support, no lib nor SDK required (built on top of HTTP and server-sent events)
* compatible with all existing servers, even those who don't support persistent connections (PHP, serverless architecture, , FastCGI...)
* built-in connection re-establishment and state reconciliation
* JWT-based authorization mechanism (securely dispatch an update to some selected subscribers)
* performant, leverages HTTP/2 multiplexing
* designed with hypermedia in mind, also supports GraphQL
* auto-discoverable through web linking
* message encryption support
* can work with old browsers (IE7+) using an EventSource polyfill
* connection-less push in controlled environments (e.g. browsers on mobile handsets tied to specific carriers)

Kévin Dunglas

October 24, 2019
Tweet

More Decks by Kévin Dunglas

Other Decks in Programming

Transcript

  1. Real-Time for PHP and Beyond

    View Slide

  2. @dunglas - mercure.rocks
    Kévin Dunglas
    ❏ Founder of Les-Tilleuls.coop
    ❏ Creator of Mercure.rocks, Vulcain.rocks, API Platform…
    ❏ Symfony Core Team Member
    @dunglas

    View Slide

  3. @dunglas - mercure.rocks
    PHP, API and cloud experts
    ✊ Self-managed, 100% employee-owned
    45 people, 1,000% growth in 6 years
    Come to our booth!

    [email protected]
    Les-Tilleuls.coop

    View Slide

  4. @dunglas - mercure.rocks
    Real-Time APIs?!

    View Slide

  5. @dunglas - mercure.rocks
    Mercure: Push from Server to Clients
    ❏ Push notifications
    ❏ Synchronize connected devices in real-time
    ❏ Collaborative editing (Google Docs-like)
    ❏ Notify users when an async task has finished
    Modern and high level alternative to WebSocket

    View Slide

  6. @dunglas - mercure.rocks

    View Slide

  7. @dunglas - mercure.rocks
    Why a New Protocol?

    View Slide

  8. © Ilya Grigorik
    High Performance Browser Networking (O'Reilly)

    View Slide

  9. @dunglas - mercure.rocks
    WebSocket (RFC 6455)
    Benefits over SSE:
    ❏ Full-duplex
    communication
    ❏ Low level: full control
    Drawbacks over SSE:
    ❏ Low level: no native
    ❏ auth
    ❏ re-connection
    ❏ state reconciliation
    ❏ events history
    ❏ Obsoleted by HTTP/2 & 3
    ❏ Hard to secure

    View Slide

  10. @dunglas - mercure.rocks
    The Persistent Connections Problem
    ❏ WebSocket and SSE rely on persistent connections
    ❏ PHP but also Serverless platforms, FastCGI…

    are designed for short-lived connections
    ❏ Persistent, long-lived, concurrent connections are better
    handled by

    dedicated and optimized software and hardware

    View Slide

  11. View Slide

  12. @dunglas - mercure.rocks
    The Mercure Protocol

    View Slide

  13. @dunglas - mercure.rocks

    View Slide

  14. @dunglas - mercure.rocks
    Mercure, at a Glance
    ❏ Full-duplex, but plain old HTTP
    ❏ Publish: HTTP POST
    ❏ Subscribe: SSE
    ❏ Built-in: reconnection, retrieving of lost messages, history
    ❏ Auto-discoverable: designed for REST and GraphQL
    ❏ JWT-based authorization mechanism (private updates)
    ❏ Designed for serverless, PHP, FastCGI…
    ❏ End-2-End encryption support

    View Slide

  15. @dunglas - mercure.rocks
    Stream Events to Clients

    View Slide

  16. @dunglas - mercure.rocks
    Internet Draft: draft-dunglas-mercure

    View Slide

  17. View Slide

  18. @dunglas - mercure.rocks
    Mercure and HTTP/2+

    View Slide

  19. @dunglas - mercure.rocks
    © Narayan Prusty - What is Multiplexing in HTTP/2?
    HTTP/2: Multiplexing, and more
    ❏ Headers compression
    ❏ Headers deduplication
    ❏ No per-host connections limits

    View Slide

  20. @dunglas - mercure.rocks
    HTTP/2 support: 92% of all users
    ...and SSEs also work with HTTP/1.x

    View Slide

  21. @dunglas - mercure.rocks
    SSE support: 93% of all users

    View Slide

  22. @dunglas - mercure.rocks
    No IE? No Edge?!

    View Slide

  23. @dunglas - mercure.rocks
    There is a Polyfill for That!
    SSEs global support (with polyfill): ~100%

    View Slide

  24. @dunglas - mercure.rocks
    Publishing

    View Slide

  25. @dunglas - mercure.rocks
    Publishing

    View Slide

  26. @dunglas - mercure.rocks
    Publishing: PHP

    View Slide

  27. @dunglas - mercure.rocks
    Subscribing

    View Slide

  28. @dunglas - mercure.rocks
    Subscribing: Basic Usage

    View Slide

  29. @dunglas - mercure.rocks
    Subscribing: Several Topics

    View Slide

  30. @dunglas - mercure.rocks
    Subscribing: URI Templates

    View Slide

  31. View Slide

  32. @dunglas - mercure.rocks
    The Discovery Mechanism

    View Slide

  33. @dunglas - mercure.rocks
    Discovery Mechanism
    Web Linking: RFC 5988

    View Slide

  34. @dunglas - mercure.rocks
    Authorization

    View Slide

  35. @dunglas - mercure.rocks
    Authorization
    ❏ Uses JSON Web Token (JWT - RFC 7519)
    ❏ An update can be intended for one or several targets
    ❏ Publisher: must be authenticated
    ❏ Subscriber:
    ❏ Can be anonymous (if allowed by the config)
    ❏ Must be authenticated to receive private updates
    ❏ Two transports: cookie and Authorization header

    View Slide

  36. @dunglas - mercure.rocks

    View Slide

  37. @dunglas - mercure.rocks
    JSON Web Token and Targets

    View Slide

  38. @dunglas - mercure.rocks
    Publishing or Subscribing to All Targets

    View Slide

  39. @dunglas - mercure.rocks
    Cookie-based Authorization
    ❏ Set by the app server during the discovery
    ❏ The app server and the Mercure hub must share
    the same domain (or subdomain)
    ❏ Supported by EventSource
    ❏ Recommended for web browsers

    View Slide

  40. @dunglas - mercure.rocks
    HTTP Header-based Authorization
    ❏ Set by the client
    ❏ Not supported by native EventSource
    ❏ Supported by the polyfill
    ❏ Recommended for servers

    View Slide

  41. @dunglas - mercure.rocks
    Publishing to Specific Targets

    View Slide

  42. View Slide

  43. @dunglas - mercure.rocks
    The Mercure Hub

    View Slide

  44. @dunglas - mercure.rocks
    Reference Implementation
    ❏ Implements 100% of the Mercure protocol
    ❏ Fast, written in Go
    ❏ Works everywhere: static binaries and Docker
    ❏ Automatic HTTP/2 and HTTPS (Let’s Encrypt)
    ❏ CORS support, CSRF protection
    ❏ Cloud Native (12Factor App)
    ❏ Open source (AGPL)
    ❏ Optional: a server can implement directly the protocol

    View Slide

  45. @dunglas - mercure.rocks
    Reference Implementation
    https://mercure.rocks

    View Slide

  46. @dunglas - mercure.rocks
    Starting the Hub
    https://mercure.rocks

    View Slide

  47. @dunglas - mercure.rocks

    View Slide

  48. @dunglas - mercure.rocks
    Using Docker

    View Slide

  49. @dunglas - mercure.rocks
    On a Kubernetes Cluster

    View Slide

  50. @dunglas - mercure.rocks
    Alternative Implementations
    ❏ No hubs: the hub is optional
    ❏ Go library (AGPL)
    ❏ Node.js library (GPL)
    ❏ HA version (managed and on premise)

    in private beta: [email protected]

    View Slide

  51. View Slide

  52. @dunglas - mercure.rocks
    Case Study:

    Mercure on a

    Large Scale at iGraal

    View Slide

  53. @dunglas - mercure.rocks
    Quick intro about iGraal
    ❏ leader of cashback in France (5 millions members)
    ❏ cashback: earn money on your online purchases
    ❏ tech team of 30 persons
    ❏ available on website, mobile apps and web
    extensions

    View Slide

  54. @dunglas - mercure.rocks
    What are iGraal Web Extensions ?
    ❏ tool to detect merchants & activate cashback
    ❏ demo

    View Slide

  55. @dunglas - mercure.rocks
    About iGraal Web Extensions
    ❏ detect merchant cashback compatible
    ❏ activate cashback
    ❏ user account

    View Slide

  56. @dunglas - mercure.rocks
    About iGraal Web Extensions

    View Slide

  57. @dunglas - mercure.rocks
    Large Scale to Handle
    ❏ 500,000 extensions currently installed
    ❏ 1 browser « opened » = 1 extension « connected »
    ❏ average of 100 000 extensions « connected »
    simultaneously

    View Slide

  58. @dunglas - mercure.rocks
    Architecture without Mercure

    View Slide

  59. @dunglas - mercure.rocks
    Architecture without Mercure
    ❏ poll our API every 4H to sync user data in background

    View Slide

  60. @dunglas - mercure.rocks
    Architecture With Mercure

    View Slide

  61. @dunglas - mercure.rocks
    On Premise Configuration

    View Slide

  62. @dunglas - mercure.rocks
    Developper Feedbacks
    ❏ quick and simple to integrate (online docs are enough)
    ❏ sending « update me » messages is easier 

    than generating data diff

    View Slide

  63. @dunglas - mercure.rocks
    Devops Feedbacks
    ❏ backend resources savings €€€
    ❏ small resources needs:
    ❏ 2 instances t3.medium for Mercure Nodes
    ❏ 1 instance t3.small for Kafka
    ❏ Load Balancing Tips
    ❏ enable Mercure’s heartbeat mechanism for Amazon’s
    Application Load Balancer (ALB) compatibility
    ❏ set a low « TTL » for Mercure connections to spread the load

    View Slide

  64. @dunglas - mercure.rocks
    Future usages
    ❏ mobile applications?

    View Slide

  65. @dunglas - mercure.rocks
    Study Conclusion
    « T’as mis des paillettes dans ma vie Kevin ! »

    View Slide

  66. @dunglas - mercure.rocks
    Integrations

    View Slide

  67. @dunglas - mercure.rocks
    Integrations
    ❏ Libs: PHP, amphp, JS, Python, Dart
    ❏ Official integrations:
    ❏ Symfony, API Platform
    ❏ Community integrations:
    ❏ Laravel, Yii framework, GitHub Actions
    ❏ Official examples:
    ❏ PHP, JS, Go, Ruby, Python

    View Slide

  68. @dunglas - mercure.rocks
    Example using

    API Platform & React

    View Slide

  69. @dunglas - mercure.rocks
    Create a Mercure-enabled Web API
    api-platform.com

    View Slide

  70. You get a fully-featured API supporting:
    ❏ JSON-LD + Hydra + schema.org
    ❏ GraphQL, JSON:API, HAL
    ❏ OpenAPI
    ❏ auth, pagination, filters, validation,
    HTTP/2 push, test framework…
    ❏ and Mercure support!
    MIT license

    View Slide

  71. @dunglas - mercure.rocks
    Scaffold Mercure-Enabled Clients
    api-platform.com

    View Slide

  72. @dunglas - mercure.rocks
    Try it!
    bit.ly/2H17hN3

    View Slide

  73. https://mercure.rocks

    View Slide

  74. @dunglas les-tilleuls.coop
    .rocks

    View Slide