Skip to content

feat: clarify baseline timing, spec checks, and repository setup - #38

Merged
indykish merged 4 commits into
mainfrom
feat/chore-open-baseline-and-carryover
Sep 8, 2026
Merged

feat: clarify baseline timing, spec checks, and repository setup#38
indykish merged 4 commits into
mainfrom
feat/chore-open-baseline-and-carryover

Conversation

@indykish

@indykish indykish commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Baseline measurement happens before the Pull Request (PR), against the recorded comparison revision. Section checks run during implementation; the final gate runs every declared verification command itself and requires committed spec evidence.

Specs now validate required sections, existing references, Dimension-to-test mappings, and exact configured commands using Bun's parsed Markdown tree. Initialization reports incomplete command configuration, documentation repositories can declare verify.docs, and generated hooks print installation guidance safely.

Session notes 1

The rule and its checker moved together: spec.baseline was presence-only, which
would have let a header reading pending ship. It now requires a count or an
explicit n/a, tested both ways. The questionnaire is the invariance spec for
the ruleset, so 6.9/6.10 were rewritten and 6.9a, 6.9b and 18.4 added — leaving
them describing the old rule would have read as a regression at the next
invariance run rather than as the change it is.

  CHORE(open)                       pre-PR boundary
  ───────────                       ───────────────
  declare  Test Baseline: pending   base checkout ──► unit=<N>  (branch point)
  carry    base stash ──► worktree  worktree      ──► unit=<M>  (HEAD)
  commit   4 steps, no code yet     Test Delta = M − N
                                    orly gate pr: header with no count → red

No visual surface changed; this is rule prose, one criterion and its tests.

Review

Self-review of the diff, adversarial pass — the one finding worth recording:

  • P2 · false-pass risk in the criterion, resolved before commit. The old
    specBaseline used .some() over every line of the spec, so a tightened
    regex would have matched unit= prose anywhere in the body. Rewrote to
    .find() the header line and test only that line. Verified at
    src/criteria_spec.ts:129specLines returns the whole file, so the
    narrowing is load-bearing, not cosmetic.

gstack /review not run as a separate route: the session's working directory is
the consuming repository, not this one, and the diff is 4 prose files plus one
criterion with tests. Rerun before merge if you want the second pass.

Make

$ bun run typecheck                  exit 0   tsc --noEmit, no output
$ bun test src                       exit 0   185 pass, 0 fail, 502 expect() calls
$ make audit                         exit 0   43 passed / 10 passed / 23 passed, 0 failed
$ bin/orly verify                    exit 0   render.local, render.all, generated.root, packs.sources all 🟢
$ gitleaks detect --no-banner        exit 0   601 commits scanned, no leaks found
$ git push (pre-push governance)     exit 0   evidence written to .oracle/evidence.json

Two new tests, both in src/gates_spec.test.ts:
a Test Baseline still reading \pending` is red at the Pull Request gateanda branch carrying no code records n/a and spec.baseline is green`.

🤖 Generated with Claude Code

https://claude.ai/code/session_018ZTshuX7LqgMRvdRxMFNmU

Session notes 2

The baseline now names an immutable ancestor revision and an evidence report; a pending count cannot pass the final boundary. Spec parsing uses the runtime's Markdown tree, so quoted headings and fenced examples cannot supply required sections.
Repository setup requires conform and a named verify.* command, with Markdown and Markdown JSX (MDX) repositories covered by real hook and command tests. These notes supersede the earlier baseline recipe: use matching evidence or an isolated checkout of the recorded revision, not whichever revision the base checkout currently contains.

Review

  • P1 FIXED — src/install.ts: shell backticks in a hook error message executed an installation command. The message is now literal and tested with an installer sentinel.
  • P1 FIXED — audits/spec-template.ts: custom fence scanning accepted a required heading inside a code block. Sections, lists, and tables now come from Bun's parsed Markdown tree.
  • P1 FIXED — src/criteria.ts: the final gate excluded spec edits from its cleanliness check. Both staged and unstaged spec changes now require a commit.
  • P2 FIXED — src/validation.ts: shared setup validation rejects missing command roles while accepting documentation verification without application test suites.
  • P2 FIXED — audits/spec-template.ts: the pre-push naming audit found repeated values after the new file became tracked. Shared constants and a type predicate satisfy the audit and preserve TypeScript narrowing.

Broader tool review findings remain open; no deferral is recorded:

  • src/criteria.ts:60: installed audit scripts are not connected automatically to the work gate. Invoke applicable audits through one consistent path.
  • src/surfaces.ts:58: an unknown Git comparison base becomes an empty change set. Report an unavailable comparison instead of treating it as no changes.
  • core/operating-model.md:261 and src/criteria_spec.ts:31: shared behavior still assumes a personal docs checkout and an Indy acknowledgement. Move personal/product requirements into optional packs.
  • src/config.ts:323: detected package scripts always use Bun. Respect the repository's declared package manager.
  • audits/ufs.sh: local file discovery missed the new untracked spec checker. Include unignored new source files in local verification.

Make

Local verification:

make audit                         exit 0   263 pass, 0 fail; all audit checks passed
make install-evals                 exit 0   23 passed / 0 failed
make llmevals CHECK=1              exit 0   55 fixtures valid; no live calls
bun run typecheck                  exit 0   tsc --noEmit
bash audits/ufs.sh --all           exit 0   no violations across 43 files

The first push was blocked by the naming audit; the follow-up commit fixes its three findings. The successful retry passed the complete audit through the unchanged pre-push hook and advanced the remote branch to b8dc3a9. The secret scan covered 605 commits and found no leaks.
Live comprehension remains unverified: the earlier smoke run could not authenticate Claude because its session expired. No live-comprehension pass is claimed.
The regenerated M192 draft passes the spec checker locally; it has not been written to the separate product repository.

Remote Continuous Integration (CI) results at b8dc3a9: audit, secret scan, coverage job, and Snyk checks succeeded. The Codecov report still flags 14 uncovered changed lines (93.33333% patch coverage); passing tests do not imply full coverage. Greptile has posted no review or inline comments at this poll.

indykish and others added 2 commits September 8, 2026 14:31
… tree comes with

Two rules Indy set on Sep 08, 2026, both about what CHORE(open) is for.

The Test Baseline stops being measured at the open. The declared suites cost
minutes, and an open that spends them buys a number the boundary has to read
again anyway. CHORE(open) now DECLARES the header; the counts land before the
Pull Request, which is the gate that grades them — `spec.baseline` fails a
header still carrying no number, so the measurement can be late but not
forgotten.

Measuring later exposed something the old rule hid: a baseline read off the
same tree as the final count makes VERIFY's Test Delta compare a number
against itself. So the rule names whose count it is — the BRANCH POINT's,
read from the default branch's last green run of the declared lanes, or
measured in the base checkout, which is already sitting there. A branch that
adds no code records `n/a` and the Delta says it had nothing to measure.

The second rule: a worktree cut from a dirty base strands that work on the
default branch, where it is neither committed, nor reviewed, nor in the tree
the stream is about to gate. CHORE(open) now moves it across — `git stash` is
per-repository, so the entry the base pushes is the entry the worktree pops.
Move, never copy; name what came over; a conflicting pop stops the open.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ZTshuX7LqgMRvdRxMFNmU
The pre-Pull-Request baseline and the base-tree carry-over reach every
consuming repository on its next `orly update`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ZTshuX7LqgMRvdRxMFNmU
@codecov-commenter

codecov-commenter commented Sep 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.33333% with 14 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
audits/spec-template.ts 93.37% 10 Missing ⚠️
src/criteria.ts 84.00% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

@indykish indykish changed the title feat(lifecycle): the baseline is measured before the PR, and the base tree comes with feat: clarify baseline timing, spec checks, and repository setup Sep 8, 2026
@indykish
indykish merged commit 384f260 into main Sep 8, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants