A streamed result has no backpressure: one slow consumer buffers it all in server memory

[frenzzy]

Describe the bug

A streamed result has no backpressure. The response stream is built with no pull and no queuing strategy, and every codec node is enqueued the moment it is parsed, so the producer runs as fast as it can resolve regardless of whether anyone is reading.

consumer read      : 6 chunks
producer yielded   : 17224 items
ran ahead by       : 17218 items
heap growth        : 18.8 MiB

(five runs: 17,180-17,333 items, 18.4-19.9 MiB)

That is one consumer reading six chunks over ~300 ms. On a large or infinite stream — a live source, a paginated read, anything backed by a cursor — a single slow client buffers the entire result in server memory. Application code cannot see it happening, and nothing bounds it.

Cancellation is fine: dropping the consumer does stop the producer. It is only the reading-too-slowly case that runs away.

Steps to reproduce

# Run against a build of the `next` BRANCH. The published `next` dist-tag is
# 2.0.0-rc.4, which predates the `<endpoint>/data/<id>` address (#3094) and
# answers 404 to every request below.
node repro.mjs

repro.mjs:

import { AsyncLocalStorage } from "node:async_hooks";
globalThis[Symbol.for("solid.RequestContext")] = new AsyncLocalStorage();
const srv = await import("@solidjs/web/server-functions/server");

let produced = 0;
srv.registerServerFunction("firehose", async function* () {
  while (produced < 100000) {
    produced++;
    yield { n: produced, payload: "x".repeat(200) };
    await new Promise(resolve => setImmediate(resolve));
  }
});

const response = await srv.handleServerFunctionRequest(
  new Request("http://localhost/_server/data/firehose", {
    method: "POST",
    body: "[]",
    headers: { "Sec-Fetch-Site": "same-origin", "X-Server-Function-Instance": "i" }
  })
);

const before = process.memoryUsage().heapUsed;
const reader = response.body.getReader();
for (let i = 0; i < 6; i++) {
  await reader.read();
  await new Promise(resolve => setTimeout(resolve, 50));   // a slow client
}
console.log("consumer read    :", 6, "chunks");
console.log("producer yielded :", produced, "items");
console.log("heap growth      :", ((process.memoryUsage().heapUsed - before) / 1048576).toFixed(1), "MiB");
await reader.cancel();
process.exit(0);

Output on next (e2b21041) — the numbers above.

Expected behavior

A producer that nobody is reading waits, the way a ReadableStream normally makes it wait.

Options

1. Gate the source pull on consumer demand — fix(web): pull a streamed result behind a demand gate #3124. I wrote the sentence below expecting this to be a question for the seroval seam rather than a small edit, and that was wrong: the runtime already installs an iterator wrapper described as "the only seam where a dropped consumer can stop the producer", and the same seam lets a slow one slow it. A read drives pull, pull releases one source pull, teardown releases a parked one. Measured over 200 idle event-loop turns, the producer advances by 1 instead of tracking the turn count.
2. Give the stream a queuing strategy and respect desiredSize. Smaller: keep the push shape but stop pulling the source while the queue is over its mark. Bounds the memory without inverting the pump.
3. Cap the queue and fail loudly past it. Turns an invisible memory leak into an error — worse for the well-behaved slow client, better than an OOM.
4. Document it: a streamed server-function result is buffered at the producer's pace, so do not stream something unbounded to a client you do not control. Honest, and it leaves the hazard in place.

(1) turned out to be both, so (2)-(4) are moot unless #3124 is the wrong shape.

Related

A failing test for this is in #3112 (counted in event-loop turns, so it means the same thing on any machine).

[frenzzy]

Fixed by #3124 (f06f7b1a) — verified on current next by re-running the repro above against a build of the branch:

after 200 idle turns, produced: 1        (was 200, and ~17,000 in the wall-clock shape)

One note for anyone reaching this from a search: the repro as written now answers 400, because #3130's fix refuses a body carrying no format tag. Add X-Server-Function-Format: 8 to run it against current next.

The nested shape — an async iterable one level inside the result, { items: rows() } — is not covered by that fix and is tracked separately in #3125.