Integrate negotiated Heft child reporting

[Sean Larkin (TheLarkInn)]

Part of #5980

Stack

Parent: #5997 (copilot/reporter-r5b-demo-reporters)

This PR is based directly on the latest R5B demo tip and intentionally excludes the parallel R6 #5993 bootstrap work. Auto-merge remains disabled while stack ancestors are open.

Negotiated Heft reporting

Rush now allocates a pair of private inherited descriptors for operation children: one carries child-to-parent NDJSON events and the other returns the parent hello acknowledgement. The handshake negotiates protocol major, optional capabilities, required features, and parent-owned reporter context (reporter, log level, color, and terminal width).

Compatible Heft versions:

• advertise heft-child-events-v1 and reporter-context-v1;
• emit bounded externalOutput and structured diagnostics without selecting or constructing the parent's reporters;
• preserve child session, parent request/session/operation correlation, privacy classification, and child sourceSequence;
• propagate structured warnings back into Rush operation status calculation.

The Heft-side bootstrap uses a node-only frozen wire client rather than a runtime dependency on @rushstack/rush-reporter, avoiding a Heft/self-hosting dependency cycle. It activates only when Rush supplies both private descriptors; direct standalone Heft behavior is unchanged.

Fallback and problem matchers

Older Heft and opaque external children continue through ordered raw stdout/stderr. Missing or capability-incompatible negotiation falls back without losing output. Unsupported protocol majors or required features produce RUSH_PROTOCOL_UPDATE_REQUIRED; malformed, oversized, or truncated streams produce RUSH_PROTOCOL_INVALID_CHILD_STREAM.

Raw output is published before matcher diagnostics. Matchers consume ANSI-normalized copies keyed independently by operation, child record, and stdout/stderr stream, retain bounded partial lines, cap duplicates, and never mutate, suppress, or reorder the source output. They are disabled only after heft-child-events-v1 is actually accepted, preventing duplicate recovery when structured diagnostics exist.

Validation

• rush build --only @rushstack/heft --only @rushstack/rush-reporter --only @microsoft/rush-lib --only @microsoft/rush --verbose
• reporter protocol/matcher/Heft integration suite: 317 tests
• Rush child-process cross-version fixtures: 5 tests
• Rush operation stream/sharding/raw-order suite: 15 tests
• Heft child bootstrap fixtures: 3 tests
• apps/rush frontend suite: 28 tests
• node apps/rush/src/test/sandbox/reporter-demo/run.mjs
• rush check
• rush change --verify --no-fetch

Coverage includes new Rush + new Heft, old/capability-less child fallback, unsupported major/required features, unknown optional and required events, malformed/truncated/oversized records, descriptor close and child crash, stdout/stderr interleaving, UTF-8 chunk limits, sourceSequence, request/session/operation correlation, privacy, matcher recovery and duplicate caps, sharded operations, warning status propagation, and parent context.

Non-goals

• no install-run-rush bootstrap handoff or R6 ownership ([rush reporter][R6A] Add bootstrap handoff and compatibility adapters #5993);
• no agent auto-selection;
• no Rush 6 default flip or terminal API removal;
• no child authority to select parent reporters.

[Sean Larkin (TheLarkInn)]

Independent /pr-agentmerge review completed.

Findings fixed in 1f910c2

• Structured Heft externalOutput bypassed Rush project logs and build-cache log chunks. Structured output now enters the archival pipeline through a no-reemit terminal provider, preserving child correlation while avoiding duplicate reporter output and problem matching.
• Heft trusted inherited descriptor numbers and could write to or close unrelated descriptors when an intermediate launcher preserved environment variables but dropped fd mappings. Heft now requires pipe/socket descriptors before negotiation; Windows lifecycle commands explicitly retain safe raw stdout/stderr fallback because the shell launch path does not preserve Node fd mappings.
• The parent host now rejects child-session changes, non-monotonic source sequence, malformed externalOutput payloads, and chunks above the 64 KiB protocol limit.
• The reporter demo now accepts the valid fromCache operation terminal state when comparing rendered output with raw events.

Validation

• rush test --only @rushstack/rush-reporter --only @microsoft/rush-lib --only @microsoft/rush: 6/6 operations passed.
• Focused Heft HeftChildReporter suite: 4/4 passed.
• Reporter demo passed, covering negotiated current-Heft events, parent correlation/sourceSequence order, chunk bounds, legacy rollback parity, raw grouping, and duplicate suppression.
• rush check, rush change --verify, API Extractor, lint, and the focused post-fix review passed.
• The full Heft suite reached four unrelated pre-existing GitUtilities.checkIgnoreAsync failures caused by this hidden worktree path; the changed Heft suite passes directly.

There are no review threads to resolve. #6000 remains stacked on open #5997 with auto-merge disabled. Merge is blocked until the ancestor chain reaches main, the PR is retargeted/rebased one step at a time with an R7-only diff, and full required CI/review gates pass.

[Sean Larkin (TheLarkInn)]

Combined deep review of current head 1f910c26e8.

1. HIGH extra reporter descriptors are attached to every Unix shell operation. Forked grandchildren can retain the pipes and hang operation completion. Allocate descriptors only for negotiated children.
2. MEDIUM the acknowledgement stream lacks an error handler. A closed child pipe can raise an unhandled EPIPE.
3. MEDIUM forwarded child envelopes retain child-chosen source and privacy. Override both from trusted parent context before telemetry or reporter fan-out.
4. MEDIUM handshake/context exceptions can escape the stream data handler and terminate Rush. Validate once and reject attachAsync() instead.

[Sean Larkin (TheLarkInn)]

Combined review findings are fixed at 80f5307976.

• Rebased the seven R7 commits onto exact Add direct Rush reporter demo path #5997 head 97f807ed6549387b6aab2eccec197123196b725d and verified the diff is R7-only.
• Reporter pipes are allocated only for direct, syntactically safe Heft commands. Unknown shell commands, wrappers, control operators, and grandchild launch forms stay on raw stdout and stderr.
• Ack error and close handlers are installed before reads can trigger writes. Callback EPIPE followed by an error event settles once without an uncaught exception or hang. Premature ack failure remains a nonfatal raw fallback when the child succeeds.
• Child source, operation scope, iteration, and privacy floor are parent-owned. Spoofed public privacy and producer identity do not reach reporter or telemetry fan-out.
• Parent context is validated once, zero or invalid terminal width falls back safely, and stream callback failures reject attachAsync() with bounded descriptor cleanup.
• Structured output and diagnostics are archived without duplicate reporter output or problem matching.

Validation passed for reporter, rush-lib, Rush, focused Heft, cross-version and malformed fixtures, EPIPE and cleanup regressions, the reporter demo, rush check, and stacked rush change --verify. The full Heft run still has only the four existing GitUtilities.checkIgnoreAsync failures caused by the hidden worktree path.

Head is pending review and CI. Auto-merge remains disabled.

[Sean Larkin (TheLarkInn)]

Round-three restack and final trust-boundary review are complete at 928b60ee57.

• The eight R7 commits are based directly on finalized Add direct Rush reporter demo path #5997 head 97f807ed6549387b6aab2eccec197123196b725d; the merge-base is exact and the range is R7-only.
• Child descriptors are limited to direct, shell-safe Heft commands. Windows, wrappers, control operators, substitutions, and opaque external commands retain raw fallback without inherited reporter pipes.
• Ack transport failures emit a warning diagnostic and cannot fail an exit-0 child or mask its signal/nonzero exit status.
• The host reconstructs envelopes, scope, source, output payloads, diagnostics, parameters, sources, and identifier arrays from validated declared fields only.
• Parent source/privacy/operation/iteration identities are authoritative. Secret output is omitted from archival logs, secret parameters are redacted, and child remediation commands are removed.
• Only externalOutput and diagnosticEmitted may cross the Heft child capability; optional presentation events are dropped and required parent lifecycle events are rejected.
• Structured diagnostics are archived with readable tool/code/location/message context without duplicate reporter events or problem matching.

Final validation against 97f807ed65:

• targeted Heft, reporter, rush-lib, and Rush builds passed twice around the test phase;
• reporter: 339 tests;
• rush-lib: 819 tests;
• focused Heft child suite: 6 tests;
• apps/rush: 71 tests;
• direct reporter demo passed;
• rush check and rush change --verify --no-fetch passed;
• final independent review reported no remaining findings.

Auto-merge remains disabled while #5997 and its ancestors are open.

[Sean Larkin (TheLarkInn)]

Final combined review of 928b60ee57 found no critical, high, or medium actionable issue. Descriptor allocation, fallback, acknowledgement lifecycle, parent-owned identity and privacy, strict event and diagnostic reconstruction, secret handling, archival, matchers, bounds, and cleanup are review-complete.

Dismissible low follow-on: reconstruct scope.commandName, projectName, and phaseName from parent-owned operation context so external JSON consumers receive the same complete scope as parent events. Current first-party consumers use the trusted operation ID and are unaffected. This head is low-only.

[Sean Larkin (TheLarkInn)]

Deep review of current head b3ae871 found no critical, high, or medium actionable issues. Heft child trust, resolved iteration cleanup, diagnostics, and transport fallback remain intact. Merge remains gated by #5997 and maintainer approval.