Skip to content

test(ios): run the shared AX recovery fixture through the runner and characterize its depth memory - #2428

Merged
thymikee merged 1 commit into
claude/agent-device-issue-2424-conformancefrom
claude/agent-device-issue-2424-runner-adapter
Sep 9, 2026
Merged

test(ios): run the shared AX recovery fixture through the runner and characterize its depth memory#2428
thymikee merged 1 commit into
claude/agent-device-issue-2424-conformancefrom
claude/agent-device-issue-2424-runner-adapter

Conversation

@thymikee

@thymikee thymikee commented Sep 9, 2026

Copy link
Copy Markdown
Member

Summary

Runs contracts/fixtures/ios-ax-recovery-conformance.json (#2425) through the XCTest runner's private AX bridge. A Swift unit test replays every recovery case through a fake AX client against the real bridge capture, depth ladder, frontier extension, and completeness verdict, asserting outcome, request accounting, deepest level, and the delivered tree's canonical signature and node count. To make that possible RunnerAXSnapshotBridge gains a resolved client/target capture seam and the ladder loop becomes a platform-neutral function; behaviour is unchanged.

The fixture gains its eight hint cases, executed here as the runner's accepted-depth memory characterization: each step is a real ladder capture (a chain that fits the accepted rung, a longer chain under a node budget of that rung when the step is bounded, or a world that rejects every depth when the step fails) against a client that rejects above the accepted rung; the observed rejections, accepted rung, and boundedness are asserted, and the runner's own learning step runs only after a successful ladder, exactly as in production. The runner keeps learning a lower rung from bounded recoveries and keeps its memory untouched across a failed capture.

Second step of #2424, stacked on #2425; restores the README/ADR wording that the runner adapter exists. Five files, 970 lines.

Validation

Tested head: 0f44265a58.

  • pnpm check:affected --run --base <#2425 head>: every check passed except three packages/host-kit/src/code-signature.test.ts cases, which fail identically on a clean origin/main checkout on this Mac (the fixture root is not realpath'd while the walker realpaths manifests under the /tmp → /private/tmp symlink); unrelated to this diff, CI's Linux lane is authoritative. pnpm check:xctest-selection passes (both tests reach the host lane).
  • testPrivateAXRecoveryMatchesSharedFixture and testPrivateAXAcceptedDepthMemoryMatchesSharedFixture pass on an iOS 26.2 simulator through xcodebuild test-without-building on the unit-test build; the macOS host lane is CI-verified (it cannot run on this machine).
  • Planted red in the runner, each restored: serialize children in reverse order → multi-branch-recovery, host-request-budget, runner-extension-budget red; stop counting a vanished frontier element as missed → missing-live-element red. The adapter also caught one mis-simulated expectation on its first run (host-request-budget: four rejected rungs before the 12-level rung).

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
Installed (including dependencies) 4.52 MB 4.53 MB +2.9 kB
Package (unpacked) 4.52 MB 4.53 MB +2.9 kB
Package (download) 1.34 MB 1.34 MB +401 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 28.2 ms 28.6 ms +0.4 ms
CLI --help 78.5 ms 79.2 ms +0.8 ms

@thymikee

thymikee commented Sep 9, 2026

Copy link
Copy Markdown
Member Author

The production extraction at bf0be60 preserves the existing ladder and gives the shared fixture a real runner capture path; no new runtime regression found. The runner memory case characterizes existing bounded-recovery learning rather than changing that policy.

Resolve the conflict with the updated #2425 and rerun the fixture tests before merging. Merge #2414, then #2425 first. iOS smoke is still running.

…characterize its depth memory

Replay every recovery case of contracts/fixtures/ios-ax-recovery-conformance.json
through the runner's real private AX bridge, depth ladder, frontier extension,
and completeness verdict, asserting the delivered tree's canonical signature
and node count alongside outcome and request accounting. To make that possible
RunnerAXSnapshotBridge gains a resolved client/target capture seam and the
ladder loop becomes a platform-neutral function; behaviour is unchanged.

Add the fixture's hint cases as the runner's accepted-depth memory
characterization: each step is a real ladder capture against a client that
rejects above the accepted rung, bounded by node budget when the step is not
complete and rejected at every depth when it fails, whose observed rejections,
accepted rung, and boundedness are asserted before the runner's own learning
step records or skips it.
@thymikee
thymikee force-pushed the claude/agent-device-issue-2424-runner-adapter branch from bf0be60 to 0f44265 Compare September 9, 2026 15:31
@thymikee

thymikee commented Sep 9, 2026

Copy link
Copy Markdown
Member Author

The conflict resolution at 0f44265 leaves the production capture path unchanged. The shared runner fixture now covers failed recovery without teaching a new depth; no remaining code finding. Merge #2414 and #2425 first. iOS smoke and coverage are still running.

@thymikee
thymikee merged commit 75c3185 into main Sep 9, 2026
18 checks passed
@thymikee
thymikee deleted the claude/agent-device-issue-2424-runner-adapter branch September 9, 2026 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant