When thousands of clients hang on the same live event, the hard part isn't sending updates fast. It's sending them at a rate every consumer can actually absorb — without letting the slowest socket drag the whole system into a memory spiral.
The failure mode nobody warns you about
The naive broadcast loop looks innocent. You get an update, you iterate over every open socket, you write. It works beautifully in a demo and falls apart the moment one client is on hotel Wi‑Fi.
A queue with no ceiling isn't a buffer. It's a slow-motion outage waiting for a busy enough night.
The write buffer for that one slow socket grows unbounded. Node happily keeps allocating. Latency for everyone climbs as the event loop spends its time managing backlog it can never drain.
Let the socket tell you when to stop
The fix is to stop treating write() as fire-and-forget and start reading its
return value — it tells you when the kernel buffer is full.
function broadcast(update: Update) {
const payload = encode(update);
for (const client of clients) {
// false → the buffer is full; back off instead of piling on.
const ok = client.socket.write(payload);
if (!ok) client.dropStale();
}
}
For a live score feed, correctness lives in the latest value, not every value. So a slow client should skip intermediate frames rather than replay a backlog. That single decision — coalesce, don't queue — is what keeps tail latency flat as concurrency climbs.
What actually moved the needle
- Per-client coalescing — keep only the newest pending frame per socket.
- A hard buffer ceiling — past it, drop the connection and let the client reconnect clean rather than nurse a zombie.
- Redis as the fan-out spine — so horizontal scaling doesn't mean every node holds every socket's state.
None of this is exotic. It's just taking the slowest consumer seriously as a first-class input to the design, instead of an edge case to patch later.