Make the preview a pull channel the browser steers

docs/MIGRATING.md

@@ -22,6 +22,16 @@ projectMM ships **no migration code**: the persistence layer is robust by defaul
 
 ## Unreleased (`next-iteration`)
 
+### PreviewDriver's `fps` becomes `targetFps`, and now trades resolution (2026-08-25)
+
+The control is renamed and its meaning changed, so the rename is the point rather than cosmetic.
+
+**Before:** `fps` was a ceiling. The driver never exceeded it, but a link that could not sustain the rate simply delivered fewer frames and the control did nothing about it.
+
+**Now:** `targetFps` is the rate you *want*. The driver still never exceeds it, and when the link cannot keep up it **trades preview resolution** to get closer, lower it for full detail at a slower rate, raise it for a smoother but coarser preview. That makes the slider the place where you choose between detail and smoothness, which is what users were reaching for.
+
+**Action: none required.** The preview is a view, not output. A device that had a non-default `fps` saved falls back to the default 24 on first boot with this firmware, because the persisted key changed; set `targetFps` if you had tuned it. Mixed versions degrade soft: an old UI against new firmware sends no detail request and gets full detail (capped by memory); a new UI against old firmware sends an uplink message the device ignores.
+
 ### A module declares every control with `addControl` (2026-08-24)
 
 `addUint8`, `addUint16`, `addInt16`, `addInt32` and `addBool` are replaced by one overloaded

docs/architecture.md

@@ -342,7 +342,11 @@ Modules in the light pipeline can be added, replaced, or removed dynamically at
 - *Shared-struct (pull):* `Drivers` hands every child driver a `Buffer*` (source) plus a `Correction*` (shared brightness/reorder/white), and `Layer` exposes its pixel buffer to `Drivers` directly on the identity-mapping fast path: each consumer holds a `const`-pointer and reads it per frame. The pointers are **(re)bound on every rebuild**, not just at boot: `Drivers::prepare()` re-resolves the active `Layer` (`Effects::activeLayer()`) and calls `passBufferToDrivers()`, which re-runs `setSourceBuffer()`/`setLayer()` on each child (clearing them to `nullptr` when there is no active Layer). So a held pointer is valid only until the next rebuild — which is exactly why the consumers re-read it each frame and tolerate a null (the [robustness rule](#robustness)): a Layer add/delete/replace re-binds or clears it live, no dangling reference.
 - *Push to a core sink:* `PreviewDriver` owns the preview wire format (a one-time coordinate table + per-frame RGB point list) and pushes the bytes to a `BinaryBroadcaster` (the core HTTP server). The server broadcasts them over WebSocket without knowing they're a preview: the format and the light types stay entirely in the driver. See [PreviewDriver](moonmodules/light/moxygen/PreviewDriver.md).
 
-**Graceful degradation under transport backpressure.** The preview is the transport-side sibling of the memory-side [§ Degradation cascade](#degradation-cascade): when the browser can't keep up with a full-resolution frame (128² = ~49 KB), the producer sheds quality rather than stall the loop, in video-streaming order, frame rate then resolution. The frame streams from the driver buffer with no intermediate copy, a resumable memory-adaptive chunk per tick, and the next frame starts only once the previous drained, so the effective frame rate self-limits to what the link sustains. Only when a single frame can't drain promptly does it downsample via a spatial lattice (the adaptive-bitrate idea behind HLS/DASH, on a binary WebSocket). Each delivered frame is whole (a WebSocket message is atomic), the render loop is charged a bounded slice per tick, and a client blocked past the spin budget is closed and reconnects (a blip, not a freeze). The mechanism is payload-agnostic and lives in [PreviewDriver](moonmodules/light/moxygen/PreviewDriver.md) + `HttpServerModule`, so other bulky streams can ride the same transport.
+**Two WebSocket channels, by traffic class.** `/ws` carries the control plane (JSON state and patches); `/wsp` carries lossy binary streams (the preview). They are separate TCP connections on purpose: preview frames are large and droppable while state messages are small and latency-sensitive, and sharing one connection makes the small ones queue behind the big ones, head-of-line blocking, which surfaced as a flickering connection indicator and an unresponsive UI on large layouts. Separate connections is the standard remedy for that mixed-criticality case.
+
+`BinaryBroadcaster` stays domain-neutral through this: the core still only takes bytes and broadcasts them, with no knowledge that they are a preview. One query serves the producer, `subscriberCount()` for the status line; the work gate is the pull model itself (no standing request, no work). Inbound client frames are unmasked by the transport (framing is its job) and the payload bytes are handed opaquely to the registered `ClientMessageSink`; only the producer knows a `[0x51][stride][fps]` standing request or a `[0x52][stride]` table request from any other bytes. The two channels have separate caps (`MAX_WS_CLIENTS` 8, `MAX_PREVIEW_CLIENTS` 4) because both draw on one `CONFIG_LWIP_MAX_SOCKETS` budget of 16, shared with HTTP, mDNS, Art-Net, MQTT and OTA.
+
+**Graceful degradation under transport backpressure.** The preview is a PULL channel: a client posts a standing `[0x51][stride][fps]` request plus one-shot `[0x52]` table requests, and the device serves the most conservative standing request, building nothing at all when none stands. Every `/wsp` message rides ONE resumable per-client-cursor drain: each socket takes bytes at its own TCP pace on the transport tick, a frame offered while the slot still drains is dropped at the source, and a client is closed only on a real error or FIN, never for slowness. Congestion therefore costs preview frames, never LED time and never a disconnect. Each frame header reports the drops since the last delivered one, and the browser's controller (one pure function in `preview-adapt.js`, unit-tested) reads only that signal: persistent drops coarsen the lattice, drop-free windows refine it a rung at a time, and a refine that brings drops back is taken back with exponentially growing patience (the abandon-fast retry-slowly rule of adaptive-bitrate players). Geometry is cached client-side per (epoch, stride), so a stride change to a known rung costs no table traffic. There is no display cap: the bounds are device memory and the index type, and everything else degrades where it actually binds. The channel machinery is core and domain-neutral (opaque request bytes forwarded to a registered producer sink); `PreviewDriver` is one producer, so other bulky streams can ride the same transport.
 
 **Naming convention.** Capital `Layouts`, `Effects`, `Drivers` are class names (always capitalised when referring to the class). Lowercase "layouts", "layers", "drivers" is the English plural, used freely when context makes it clear. Singular "layout", "layer", "driver" is an individual instance.
 

docs/backlog/backlog-core.md

@@ -426,7 +426,7 @@ module by a path rather than a bare name.
 
 ### HTTP file serving blocks the render tick (backlog)
 
-`HttpServerModule::handleConnection()` serves large embedded files (`app.js`, `style.css`) with the blocking `TcpConnection::write` — a page load can briefly stall `loop20ms`. One-shot per load (lower priority than the per-tick preview issue, which is fixed). Fix: serve large HTTP responses with `writeChunks` (the same non-blocking path used for preview frames).
+`HttpServerModule::handleConnection()` serves large embedded files (`app.js`, `style.css`) with the blocking `TcpConnection::write` — a page load can briefly stall `loop20ms`. One-shot per load (lower priority than the per-tick preview issue, which is fixed). Fix: serve large HTTP responses through a resumable per-client cursor drained on tick20ms (the shape the preview and full-state sends use).
 
 ### Generic control + state topics over MQTT — the automation escape hatch (backlog)
 
@@ -510,8 +510,9 @@ work moved to a worker or made resumable, so each is a design change rather than
 Confirmed by external review (CodeRabbit, PR #56).
 
 **`tick()` — every frame, the sharp ones:**
-- `PreviewDriver::tick` — `sendFrame()` writes a socket synchronously; `buildAndSendCoordTable()`
-  resizes `keptIdx_`. Currently suppressed at the site with the reason.
+- `PreviewDriver::tick` — the socket half is FIXED (the pull-model transport only ARMS a message;
+  every socket byte moves on the transport tick). What remains: `buildCoordTable()` resizes
+  `keptIdx_` and the staging buffer on a rebuild/adopt tick. Suppressed at the site with the reason.
 - `ParallelLedDriver::tick` — `tickSync()`/`tickRing()` reach `busWaitIfBusy()`, which spins for
   the DMA peripheral. Deliberate (the driver owns the bus for the frame) but blocking. Suppressed.
 - `Drivers::tick` — joins/stops the render-split worker synchronously on the timed-out recovery
@@ -839,13 +840,6 @@ They are **not** CI failures (CI is Debug) and each one inspected so far is a fa
 
 Not done with the multi-destination/tab-UI merge because 17 warnings across four core files is its own change, not a tail on someone else's.
 
-## Preview stream: tail byte-phase desync under backpressure
-
-**Symptom (board B, 2026-07-14):** on a large grid the preview's bottom region (the frame tail) shows per-pixel noise; with a SOLID effect it flickers pure R / pure G / pure B at max brightness. Pure primaries from solid content = the frame bytes read at an offset that is not a multiple of 3 — tearing alone cannot discolor identical frames, so the resumable sender's offset accounting slips. Survives a browser refresh; occurs "occasionally" (per-frame), worst when the device is network-starved.
-
-**Suspect:** `HttpServerModule::sendBufferedFrame` (the zero-copy resumable send whose body is the live producer buffer, draining across transport ticks) — the resume-after-partial-socket-write path. Verify by logging the resume offset vs bytes actually written on EWOULDBLOCK; the fix is offset accounting, plus a resync rule (a client that missed bytes gets a fresh frame header, not a phase-shifted tail — the *robust to any input* bar).
-
-**Not** the shift-register transport bug and **not** buffer corruption: the composite buffer is proven correct (Solid writes every light; the preview alone garbles).
 
 ## MoonLive core/platform layering + JIT sdkconfig scoping (CodeRabbit #29, 4 findings)
 

docs/backlog/backlog-light.md

@@ -228,16 +228,6 @@ The migrated 1D PacMan and 1D Ant effects were removed — a chase rendered on a
 
 Only **NoiseEffect**, **PlasmaEffect** and **RipplesEffect** have z-aware math. The other honest-D2 effects use `Layer::extrude` to duplicate the z=0 plane, so every z-slice is identical on 3D layers. Candidates for genuine D3 promotion: Metaballs/GlowParticles (add z to blob coordinates), Plasma palette/Spiral (add z-driven phase term), Fire (z-drift heat grid), Rings/LavaLamp/Checkerboard/Particles (add z to each element). Prioritise after seeing real 3D installations; each promoted effect also needs its `dynamicBytes` budget for the full 3D buffer.
 
-### Full-density interpolated preview for large layouts (backlog)
-
-The preview index-downsamples a large layout to fit the WS send budget (e.g. 128×128 = 16384 lights → ~1639 sent at stride 10), so the UI shows a sparse sample, not every light. To show **all** lights at their real positions with **interpolated** colors for the unsent ones:
-
-- Decouple the `0x03` coordinate-table density from the per-frame `0x02` stride. Positions are static and sent once, so the table can carry **all** light coordinates (16384 × 3 = ~48 KB one-time — acceptable off the per-frame path, possibly chunked) while the per-frame RGB stays strided to protect ArtNet/the link.
-- The browser holds the full position set and, per frame, interpolates each unsent light's color from its nearest sent neighbours (the sent indices are known from the stride). True positions, guessed colors — better than the removed dense-box block-replicate because positions are exact.
-- Open questions: 48 KB one-time table vs `MAX_WRITE_CHUNKS` / send-buffer (needs chunked send or a raised cap, with the same partial-write care as `writeChunks`' drain); interpolation cost on a 16384-point cloud each frame in JS; whether nearest-neighbour or weighted is worth it.
-
-Not simple — own planning pass. Until then the preview is a faithful strided *sample* (correct shape/color/motion, not per-pixel). A cheap interim (point-size scaled by stride to fatten samples into their cells) was tried and reverted as not what's wanted — it filled the volume but didn't add real points.
-
 ### Self-describing preview frame header (mid term)
 
 The preview wire format is a private opcode protocol: `0x02` per-frame channels, `0x03` coordinate table, each a hand-rolled byte layout, and the color payload is **always RGB** regardless of the buffer's `channelsPerLight`. Every new data kind (RGBW display, beam direction, …) means inventing another opcode and another fixed layout by hand. The minimal fix that stops that sprawl: a small **typed header** — `[type][format][count][stride]` where `format` enumerates `{RGB, RGBW, …}` — so one message kind carries any per-light channel layout and the browser shader reads `format` to interpret the payload. Do it concrete-first, when RGBW *display* (below) is actually wanted, not speculatively. Prereq for both items below.
@@ -256,16 +246,6 @@ Today a "light" is a point at a static coordinate with a color. A **moving head*
 
 Today each layout child describes one light type (all LED strips, or all par lights), and the current model is one Layouts container per light type. Whether a single Layouts should hold mixed types (LED strips + par lights together), and how the per-channel layout would reconcile across them, isn't designed. Deferred until a concrete need forces it; it's adjacent to the fixture model above (a real fixture/attribute model may reframe how mixed types are expressed). (Moved from architecture.md § What we leave undesigned; a deferred design decision, not a settled 🚧 one.)
 
-### Extract the resumable backpressure transport as a domain-neutral channel (long term)
-
-The preview's transport — resumable cross-tick send from a stable buffer + newest-wins backpressure drop + adaptive graceful degradation (see [architecture.md § graceful degradation under transport backpressure](../architecture.md)) — is **payload-agnostic**: any bulky throttled stream (a future MJPEG/video preview, fixture-state streams, fleet telemetry) could ride it. The *payload* model (count/stride/RGB) is light-specific; the *byte-pump* is not. When a second consumer for this transport appears, promote the pump into a domain-neutral core primitive (a `ThrottledChannel`-style sink) that PreviewDriver becomes *a* producer on, rather than owning the protocol. Concrete-first: extract on the second use, not before — until then the seam stays inside HttpServerModule/PreviewDriver.
-
-### PreviewDriver `resumableFrames` default OFF: fix the tearing, then un-skip the dynamicBytes test
-
-`resumableFrames` (the downsampled-frame transport A/B) now defaults **OFF** because the resumable path visibly **tears the preview**: it shares the single-occupancy WS send slot with the ~1 Hz full-state push and the next preview frame, so a preempted mid-drain frame reaches the browser spliced (top rows new, the rest stale — PO saw it on the wall). OFF uses the proven-correct synchronous transport. Two follow-ups, both small:
-
-1. **The tear itself (the reason to eventually want ON again).** Give the preview send its OWN send slot instead of sharing `previewSend_` with the state push, OR make a preempted drain drop cleanly (versioned frame → the browser discards a spliced one) rather than splicing. Only then is ON safe to default. The off-thread send exists to avoid the ~17 ms render hitch at very large grids with the preview open, so this matters mainly for big walls; until fixed, synchronous is correct.
-2. **The skipped test.** `unit_PreviewDriver.cpp`'s "reports its resumable-path buffers in dynamicBytes" is `doctest::skip()`'d: it was written when `resumableFrames` defaulted ON and its first assertion relied on the rig constructor's `applyState()` allocating the staging buffer via that default. With the flag OFF, toggling it ON post-construction + `prepare()` did not re-allocate the buffers in the test rig the way the constructor path did (the "ON" reads dropped to 0). The *accounting* (`driverHeapBytes` sums `stageCap_` + `keptIdxCap_`) is unchanged and correct; only the test's default assumption broke. Un-skip by rebuilding the rig so it can deterministically allocate the resumable-path buffers with the flag OFF-by-default — likely wiring the flag ON into the rig BEFORE its first `drivers.applyState()` so the same acquire path production uses runs (the post-construction toggle route resisted several attempts; the constructor-time route is the one to nail). Small test-only work.
 
 ## LCD / DMA driver work
 

docs/history/lessons.md

@@ -604,3 +604,19 @@ evidence which *looks* most authoritative here is the evidence that lies.
   emitted x86-64 instructions for real, so the tests that only exist on that host actually execute.
   Worth doing on any change to a backend, an encoder, or the register allocator — it is minutes,
   and it is the difference between finding these locally and finding them in CI.
+
+## A lossy channel never closes a client for slowness (2026-08-25)
+
+The preview transport spent a bench day being patched before the pattern surfaced: every
+symptom (reconnect storms, blank refreshes, renderWait spikes charged to the LEDs) traced to a
+synchronous send path that treated a slow socket as a fault, spinning up to a stall budget on
+the output core and then **closing the client**. That converts ordinary congestion into
+disconnects, disconnects into re-primed state, and re-primed state into more congestion, a
+feedback loop.
+
+The rule that replaced it, and holds generally: **on a lossy stream, drop at the source and let
+TCP pace the rest.** Write only what the socket takes now (`writeSome`, never a wait), skip the
+frame when the previous one has not drained, report the skip to the receiver (its one honest
+congestion signal), and close only on a real error or FIN. The receiver steers quality from the
+drop reports. Every give-up budget that remains must bound *lack of progress*, never elapsed
+total, or slow-but-healthy transfers get truncated.