Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Mercure: Real-Time for PHP Made Easy

Mercure: Real-Time for PHP Made Easy

Mercure is a protocol allowing to push data updates to web browsers and other HTTP clients in a convenient, fast, reliable and battery-efficient way. It is especially useful to publish real-time updates of resources served through web APIs, to reactive web and mobile apps. It is provided with an open source implementation written in Go.

It provides:

* native browser support, no lib nor SDK required (built on top of HTTP and server-sent events)
* compatible with all existing servers, even those who don't support persistent connections (PHP, serverless architecture, , FastCGI...)
* built-in connection re-establishment and state reconciliation
* JWT-based authorization mechanism (securely dispatch an update to some selected subscribers)
* performant, leverages HTTP/2 multiplexing
* designed with hypermedia in mind, also supports GraphQL
* auto-discoverable through web linking
* message encryption support
* can work with old browsers (IE7+) using an EventSource polyfill
* connection-less push in controlled environments (e.g. browsers on mobile handsets tied to specific carriers)

Kévin Dunglas

October 24, 2019
Tweet

More Decks by Kévin Dunglas

Other Decks in Programming

Transcript

  1. Real-Time for PHP and Beyond

    View full-size slide

  2. @dunglas - mercure.rocks
    Kévin Dunglas
    ❏ Founder of Les-Tilleuls.coop
    ❏ Creator of Mercure.rocks, Vulcain.rocks, API Platform…
    ❏ Symfony Core Team Member
    @dunglas

    View full-size slide

  3. @dunglas - mercure.rocks
    PHP, API and cloud experts
    ✊ Self-managed, 100% employee-owned
    45 people, 1,000% growth in 6 years
    Come to our booth!

    [email protected]
    Les-Tilleuls.coop

    View full-size slide

  4. @dunglas - mercure.rocks
    Real-Time APIs?!

    View full-size slide

  5. @dunglas - mercure.rocks
    Mercure: Push from Server to Clients
    ❏ Push notifications
    ❏ Synchronize connected devices in real-time
    ❏ Collaborative editing (Google Docs-like)
    ❏ Notify users when an async task has finished
    Modern and high level alternative to WebSocket

    View full-size slide

  6. @dunglas - mercure.rocks

    View full-size slide

  7. @dunglas - mercure.rocks
    Why a New Protocol?

    View full-size slide

  8. © Ilya Grigorik
    High Performance Browser Networking (O'Reilly)

    View full-size slide

  9. @dunglas - mercure.rocks
    WebSocket (RFC 6455)
    Benefits over SSE:
    ❏ Full-duplex
    communication
    ❏ Low level: full control
    Drawbacks over SSE:
    ❏ Low level: no native
    ❏ auth
    ❏ re-connection
    ❏ state reconciliation
    ❏ events history
    ❏ Obsoleted by HTTP/2 & 3
    ❏ Hard to secure

    View full-size slide

  10. @dunglas - mercure.rocks
    The Persistent Connections Problem
    ❏ WebSocket and SSE rely on persistent connections
    ❏ PHP but also Serverless platforms, FastCGI…

    are designed for short-lived connections
    ❏ Persistent, long-lived, concurrent connections are better
    handled by

    dedicated and optimized software and hardware

    View full-size slide

  11. @dunglas - mercure.rocks
    The Mercure Protocol

    View full-size slide

  12. @dunglas - mercure.rocks

    View full-size slide

  13. @dunglas - mercure.rocks
    Mercure, at a Glance
    ❏ Full-duplex, but plain old HTTP
    ❏ Publish: HTTP POST
    ❏ Subscribe: SSE
    ❏ Built-in: reconnection, retrieving of lost messages, history
    ❏ Auto-discoverable: designed for REST and GraphQL
    ❏ JWT-based authorization mechanism (private updates)
    ❏ Designed for serverless, PHP, FastCGI…
    ❏ End-2-End encryption support

    View full-size slide

  14. @dunglas - mercure.rocks
    Stream Events to Clients

    View full-size slide

  15. @dunglas - mercure.rocks
    Internet Draft: draft-dunglas-mercure

    View full-size slide

  16. @dunglas - mercure.rocks
    Mercure and HTTP/2+

    View full-size slide

  17. @dunglas - mercure.rocks
    © Narayan Prusty - What is Multiplexing in HTTP/2?
    HTTP/2: Multiplexing, and more
    ❏ Headers compression
    ❏ Headers deduplication
    ❏ No per-host connections limits

    View full-size slide

  18. @dunglas - mercure.rocks
    HTTP/2 support: 92% of all users
    ...and SSEs also work with HTTP/1.x

    View full-size slide

  19. @dunglas - mercure.rocks
    SSE support: 93% of all users

    View full-size slide

  20. @dunglas - mercure.rocks
    No IE? No Edge?!

    View full-size slide

  21. @dunglas - mercure.rocks
    There is a Polyfill for That!
    SSEs global support (with polyfill): ~100%

    View full-size slide

  22. @dunglas - mercure.rocks
    Publishing

    View full-size slide

  23. @dunglas - mercure.rocks
    Publishing

    View full-size slide

  24. @dunglas - mercure.rocks
    Publishing: PHP

    View full-size slide

  25. @dunglas - mercure.rocks
    Subscribing

    View full-size slide

  26. @dunglas - mercure.rocks
    Subscribing: Basic Usage

    View full-size slide

  27. @dunglas - mercure.rocks
    Subscribing: Several Topics

    View full-size slide

  28. @dunglas - mercure.rocks
    Subscribing: URI Templates

    View full-size slide

  29. @dunglas - mercure.rocks
    The Discovery Mechanism

    View full-size slide

  30. @dunglas - mercure.rocks
    Discovery Mechanism
    Web Linking: RFC 5988

    View full-size slide

  31. @dunglas - mercure.rocks
    Authorization

    View full-size slide

  32. @dunglas - mercure.rocks
    Authorization
    ❏ Uses JSON Web Token (JWT - RFC 7519)
    ❏ An update can be intended for one or several targets
    ❏ Publisher: must be authenticated
    ❏ Subscriber:
    ❏ Can be anonymous (if allowed by the config)
    ❏ Must be authenticated to receive private updates
    ❏ Two transports: cookie and Authorization header

    View full-size slide

  33. @dunglas - mercure.rocks

    View full-size slide

  34. @dunglas - mercure.rocks
    JSON Web Token and Targets

    View full-size slide

  35. @dunglas - mercure.rocks
    Publishing or Subscribing to All Targets

    View full-size slide

  36. @dunglas - mercure.rocks
    Cookie-based Authorization
    ❏ Set by the app server during the discovery
    ❏ The app server and the Mercure hub must share
    the same domain (or subdomain)
    ❏ Supported by EventSource
    ❏ Recommended for web browsers

    View full-size slide

  37. @dunglas - mercure.rocks
    HTTP Header-based Authorization
    ❏ Set by the client
    ❏ Not supported by native EventSource
    ❏ Supported by the polyfill
    ❏ Recommended for servers

    View full-size slide

  38. @dunglas - mercure.rocks
    Publishing to Specific Targets

    View full-size slide

  39. @dunglas - mercure.rocks
    The Mercure Hub

    View full-size slide

  40. @dunglas - mercure.rocks
    Reference Implementation
    ❏ Implements 100% of the Mercure protocol
    ❏ Fast, written in Go
    ❏ Works everywhere: static binaries and Docker
    ❏ Automatic HTTP/2 and HTTPS (Let’s Encrypt)
    ❏ CORS support, CSRF protection
    ❏ Cloud Native (12Factor App)
    ❏ Open source (AGPL)
    ❏ Optional: a server can implement directly the protocol

    View full-size slide

  41. @dunglas - mercure.rocks
    Reference Implementation
    https://mercure.rocks

    View full-size slide

  42. @dunglas - mercure.rocks
    Starting the Hub
    https://mercure.rocks

    View full-size slide

  43. @dunglas - mercure.rocks

    View full-size slide

  44. @dunglas - mercure.rocks
    Using Docker

    View full-size slide

  45. @dunglas - mercure.rocks
    On a Kubernetes Cluster

    View full-size slide

  46. @dunglas - mercure.rocks
    Alternative Implementations
    ❏ No hubs: the hub is optional
    ❏ Go library (AGPL)
    ❏ Node.js library (GPL)
    ❏ HA version (managed and on premise)

    in private beta: [email protected]

    View full-size slide

  47. @dunglas - mercure.rocks
    Case Study:

    Mercure on a

    Large Scale at iGraal

    View full-size slide

  48. @dunglas - mercure.rocks
    Quick intro about iGraal
    ❏ leader of cashback in France (5 millions members)
    ❏ cashback: earn money on your online purchases
    ❏ tech team of 30 persons
    ❏ available on website, mobile apps and web
    extensions

    View full-size slide

  49. @dunglas - mercure.rocks
    What are iGraal Web Extensions ?
    ❏ tool to detect merchants & activate cashback
    ❏ demo

    View full-size slide

  50. @dunglas - mercure.rocks
    About iGraal Web Extensions
    ❏ detect merchant cashback compatible
    ❏ activate cashback
    ❏ user account

    View full-size slide

  51. @dunglas - mercure.rocks
    About iGraal Web Extensions

    View full-size slide

  52. @dunglas - mercure.rocks
    Large Scale to Handle
    ❏ 500,000 extensions currently installed
    ❏ 1 browser « opened » = 1 extension « connected »
    ❏ average of 100 000 extensions « connected »
    simultaneously

    View full-size slide

  53. @dunglas - mercure.rocks
    Architecture without Mercure

    View full-size slide

  54. @dunglas - mercure.rocks
    Architecture without Mercure
    ❏ poll our API every 4H to sync user data in background

    View full-size slide

  55. @dunglas - mercure.rocks
    Architecture With Mercure

    View full-size slide

  56. @dunglas - mercure.rocks
    On Premise Configuration

    View full-size slide

  57. @dunglas - mercure.rocks
    Developper Feedbacks
    ❏ quick and simple to integrate (online docs are enough)
    ❏ sending « update me » messages is easier 

    than generating data diff

    View full-size slide

  58. @dunglas - mercure.rocks
    Devops Feedbacks
    ❏ backend resources savings €€€
    ❏ small resources needs:
    ❏ 2 instances t3.medium for Mercure Nodes
    ❏ 1 instance t3.small for Kafka
    ❏ Load Balancing Tips
    ❏ enable Mercure’s heartbeat mechanism for Amazon’s
    Application Load Balancer (ALB) compatibility
    ❏ set a low « TTL » for Mercure connections to spread the load

    View full-size slide

  59. @dunglas - mercure.rocks
    Future usages
    ❏ mobile applications?

    View full-size slide

  60. @dunglas - mercure.rocks
    Study Conclusion
    « T’as mis des paillettes dans ma vie Kevin ! »

    View full-size slide

  61. @dunglas - mercure.rocks
    Integrations

    View full-size slide

  62. @dunglas - mercure.rocks
    Integrations
    ❏ Libs: PHP, amphp, JS, Python, Dart
    ❏ Official integrations:
    ❏ Symfony, API Platform
    ❏ Community integrations:
    ❏ Laravel, Yii framework, GitHub Actions
    ❏ Official examples:
    ❏ PHP, JS, Go, Ruby, Python

    View full-size slide

  63. @dunglas - mercure.rocks
    Example using

    API Platform & React

    View full-size slide

  64. @dunglas - mercure.rocks
    Create a Mercure-enabled Web API
    api-platform.com

    View full-size slide

  65. You get a fully-featured API supporting:
    ❏ JSON-LD + Hydra + schema.org
    ❏ GraphQL, JSON:API, HAL
    ❏ OpenAPI
    ❏ auth, pagination, filters, validation,
    HTTP/2 push, test framework…
    ❏ and Mercure support!
    MIT license

    View full-size slide

  66. @dunglas - mercure.rocks
    Scaffold Mercure-Enabled Clients
    api-platform.com

    View full-size slide

  67. @dunglas - mercure.rocks
    Try it!
    bit.ly/2H17hN3

    View full-size slide

  68. https://mercure.rocks

    View full-size slide

  69. @dunglas les-tilleuls.coop
    .rocks

    View full-size slide