to bring real-time communication to browsers — Sub-second latency; the de facto standard for video conferencing But... — Not built for large-scale delivery (no cheap CDN; you have to coordinate your own SFUs) — Deeper optimization means touching libwebrtc (hard)
deliver live video over HTTP — Video is just files, so existing CDNs work as-is But... — Latency is seconds to tens of seconds — LL-HLS / LHLS have narrowed the gap, but still within HTTP
RFC 9000 — The protocol behind HTTP/3, running in production on CDNs worldwide — Rebuilds what TCP did, on top of UDP Legacy QUIC HTTP/1.1, HTTP/2 HTTP/3, WebTransport TLS QUIC (TLS built in) TCP UDP IP IP — Usable from the browser (WebTransport)
was set up at the IETF and standardization began — Defines a Pub/Sub protocol for carrying media over QUIC — Major streaming, conferencing and CDN companies are involved
spec or blog posts: where you actually get stuck — Share what we chose at each point, and why → Even without building it yourself, you'll be able to say "this is where MoQ stands today"
a relay — The publisher publishes a Track — The relay learns about the Track * The diagrams are simplified (Namespace, control message details, and so on)
a relay — The publisher publishes a Track — The relay learns about the Track — The subscriber subscribes to the Track * The diagrams are simplified (Namespace, control message details, and so on)
a relay — The publisher publishes a Track — The relay learns about the Track — The subscriber subscribes to the Track — Data flows from the publisher to the subscriber * The diagrams are simplified (Namespace, control message details, and so on)
approach) — When a broadcast starts, record “this Track is on relay A” — When a subscribe arrives, look it up and forward to relay A — It works and it is simple, but it becomes a SPOF, so it needs redundancy 1. record 3. look up 4. forward subscribe
on the origin relay — Only “where the origin is” gets recorded — A relay that receives a subscribe connects straight to it — With 50 relays, the origin relay sends the same video 50 times Sends the same video once per relay
route? — Record “who is upstream” instead of “the origin is A” — A subscribe is forwarded upstream, and following it reaches the origin Store (route of a Track) (B's upstream is A) data But... — Who decides this route, and how? — Every time relays come and go, it has to be rebuilt subscribe
approach) — Tell neighbours about a publication, and they tell theirs (any relay topology assumed) — Each relay records “which neighbour this publication came from” — When a subscribe arrives, forward it to the right neighbour → repeat, and it reaches the publisher has the track But... from A (hop 1) from A (hop 1) ✓ from B (hop 2) ✓ from D (hop 2) from E (hop 3) from C (hop 3) — It needs a custom protocol (MoQLite) — There is a lag before the information propagates
moq-wasm (us) ◦ ◦ moq.dev Cloudflare Decentralized approach ◦ — The point of a PoC is to learn by running it. It is not the stage for building out scale — We understood the options, then picked the lightest one — At larger scale — multi-region, for instance — the call would be different
only WebTransport — Non-browser clients (native) can use raw QUIC directly — Underneath, both are QUIC, so ALPN (which announces the protocol name at connect time) can tell them apart Handle as raw QUIC Browser Browser → No need for separate listening ports Handle as WebTransport (over HTTP/3)
— We implemented MoQ on top of raw QUIC Adopted quinn (the de facto QUIC implementation in Rust) — Then we moved on to WebTransport support Adopted wtransport (a WebTransport implementation built on quinn) → Same QUIC implementation (quinn), so sharing a port should be straightforward
hand a QUIC Connection through let server = quinn::Endpoint::server(config, addr)?; loop { let conn = server.accept().await?.await ?; match conn.alpn () { // handle as raw QUIC // handle as WebTransport b"moqt" b"h3" => { ... } => { let session = wtransport::from_quinn (conn)?; // ↑ no such API exists ... } } } → Not a protocol problem — a library API problem
same problem — Existing MoQ implementations must be hitting the same problem — moq.dev had built its own library to pair with quinn (web-transport-quinn) — We switched to web-transport-quinn as well, and can now accept both on one port → Implementing a new protocol can fall outside what existing libraries assume
Clients should connect to the nearest region — Load should be spread across several machines within a region — Connections should survive an IP change (QUIC Connection Migration) Client Client Client Japa n United States Europe → Putting an LB in front is the obvious move — but what happens with QUIC?
L7 LBs assume HTTP, so raw QUIC can't be used — WebTransport over HTTP/3 works, but the LB terminates HTTP/3 — Between LB and relay it becomes HTTP/1.1 or HTTP/2, so QUIC is gone Client Terminated here → We want the LB to forward packets as they are, not terminate them
— It forwards packets as they are, so client and relay connect over QUIC directly — But an ordinary L4 LB distributes by 5-tuple hash — When the IP changes (Wi-Fi → LTE, etc.) the target changes too, and packets reach a different relay Client 5-tuple hash ② When it switches to LTE Client 5-tuple hash Doesn't know this connection → We want distribution per connection, not per 5-tuple
It distributes by the Connection ID inside the QUIC packet, not the 5-tuple — A draft is being developed at the IETF — Only AWS supports it (GCP and Azure do not) Client Looks at Connection ID ② Even when it switches to LTE Client Looks at Connection ID Arrives at the same relay → But our platform is GCP-based...
Cost Implement QUIC-LB ourselves — Hybrid AWS / GCP setup — Running two clouds Redirect approach — One extra RTT on connect, custom routing logic GeoDNS Load-aware distribution Relay IPs exposed directly, DNS ops as the fleet scales GCP L4 LB Connection Migration — Heavy to build and operate
up Connection Migration and adopted the GCP L4 LB — We chose shipping something that works over a more complete LB — The LB setup is independent of the application, so we can change it later → You want to get everything out of a new technology, but wait for what can wait
Sharing one port for QUIC/WebTransport 3 Infra Load balancing with QUIC-LB Centralized / decentralized → the implementer's call A new protocol can fall outside what existing libraries assume The ecosystem is not quite there yet If you take away “so this is where MoQ stands right now” from the spec, library and infrastructure angles, that is enough for me X: mti_daiki Mail: [email protected]
dmatsui
mclloyd
dougneiner
auna
baasie
thekraken
marketingsoph
axbom
oripsolob
notwaldorf
tamaranovitovic
smashingmag
lara
