Skip to content

Add evidence contract and agentic evaluation loop - #2

Merged
MicroMilo merged 1 commit into
mainfrom
agent/evidence-contract-loop
Aug 5, 2026
Merged

Add evidence contract and agentic evaluation loop#2
MicroMilo merged 1 commit into
mainfrom
agent/evidence-contract-loop

Conversation

@MicroMilo

Copy link
Copy Markdown
Owner

What changed

  • Added an explicit report contract (schema_version 1.1) with measurement status, coverage ledger, evidence-linked findings, confidence, and an action gate.
  • Changed incomplete size probes to remain null with timeout/error/unknown status instead of being interpreted as zero.
  • Marked artifact parent relationships and whether artifact bytes are already included in a measured target.
  • Added validate_report.py, deterministic timeout contract tests, and CI coverage.
  • Added a maintainer agentic evaluation loop reference using aggregated redacted evidence packets and independent role reviews.
  • Updated the README, skill instructions, schema reference, and synthetic report example.

Why

A real local baseline showed a timed-out Desktop probe, 251 Git repositories with dirty work, 40 artifact rows, and app-managed storage. The previous report could prioritize review but did not preserve enough evidence state for another Agent to safely reason about exact actions.

Evaluation

  • Ran a real local deep audit with no cleanup or file mutation.
  • Used independent audit-operator, first-time-user, and loop-architect subagents against an aggregated redacted packet.
  • Recorded the privacy-guardian role as incomplete after timeout; it was not treated as approval.
  • python3 tests/smoke_test.py
  • python3 tests/contract_test.py
  • python3 audit-local-files/scripts/validate_report.py on the real local report
  • skill-creator quick_validate.py passed

Raw local reports and role packets are not included.

@MicroMilo
MicroMilo marked this pull request as ready for review August 5, 2026 15:13
@MicroMilo
MicroMilo merged commit e278961 into main Aug 5, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant