Skip to content

Comments

feat: copilot-cli provider (ACP) + nested git workspace support#293

Merged
christso merged 10 commits intomainfrom
feat/workspace-lifecycle
Feb 22, 2026
Merged

feat: copilot-cli provider (ACP) + nested git workspace support#293
christso merged 10 commits intomainfrom
feat/workspace-lifecycle

Conversation

@christso
Copy link
Collaborator

@christso christso commented Feb 21, 2026 •
edited
Loading

Summary

  • Copilot CLI provider (ACP): New copilot-cli provider that spawns copilot --acp --stdio and communicates via @agentclientprotocol/sdk NDJSON. Bypasses the @github/copilot-sdk 60s session.idle timeout, enabling long-running agent tasks. Follows the vibe-kanban ACP pattern.
  • Nested git workspace support: captureFileChanges() now uses git diff --submodule=diff to expand nested repo changes into individual file diffs (instead of opaque gitlink hashes). Agents get full git log / git blame access in workspace templates.
  • Shared copilot utilities: Extracted copilot-utils.ts with shared code between copilot-cli and copilot-sdk providers (~96 line reduction). Fixed timer resource leak (missing clearTimeout in finally) and cost accumulation bug (overwrite → accumulate) in both providers.
  • System prompt fix: Moved system prompt delivery from --system-prompt CLI flag (unsupported) to ACP prompt messages array.
  • Bug fix: file_changes missing from rubric/score-range LLM judge prompts: buildRubricPrompt() and buildScoreRangePrompt() in llm-judge.ts (and counterparts in llm-judge-prompt.ts) were not including the [[ ## file_changes ## ]] section — only the freeform path did. Fixed all four functions.
  • Rename agent_judge evaluator judge_target → target: The evaluator is already type: agent_judge, so judge_target was semantically redundant. judge_target is now exclusively a target-level field (in targets.yaml), while evaluators use target to reference a configured provider.
  • Workers override warning: Logs a warning when workers > 1 is overridden to 1 for shared workspaces.
  • New example: examples/features/file-changes-judges/ — minimal eval proving file_changes works with all three judge types (Azure LLM rubrics, built-in agent judge, copilot-cli agent judge), all scoring 1.0.

Key files

File Change
packages/core/src/evaluation/providers/copilot-cli.ts New ACP-based provider
packages/core/src/evaluation/providers/copilot-utils.ts Shared utilities (extracted)
packages/core/src/evaluation/providers/copilot-sdk.ts Refactored to use shared utils
packages/core/src/evaluation/workspace/file-changes.ts --submodule=diff + stageNestedRepoChanges()
packages/core/src/evaluation/evaluators/llm-judge.ts file_changes in rubric/score-range prompts
packages/core/src/evaluation/evaluators/llm-judge-prompt.ts file_changes in checklist/score-range assembly
packages/core/src/evaluation/types.ts agent_judge config: judge_target → target
packages/core/src/evaluation/orchestrator.ts Workers warning + target rename
examples/features/file-changes-judges/ New example eval
examples/showcase/cross-repo-sync/ Updated for copilot-cli + nested git

Test plan

  • All 678 unit tests pass
  • Pre-push hooks pass (Build, Typecheck, Lint, Test)
  • End-to-end: cross-repo-sync eval with copilot-cli target (score 0.75, 297K chars file_changes)
  • End-to-end: file-changes-judges eval — Azure rubrics judge, built-in agent judge, copilot-cli agent judge all score 1.0
  • Verified [[ ## file_changes ## ]] section present in all judge prompts (rubrics, freeform, agent_judge built-in, agent_judge delegated)
  • No judge_target references remain in any eval YAML — only in targets.yaml where it belongs

🤖 Generated with Claude Code

@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Feb 21, 2026 •
edited
Loading

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 100aa0c
Status: ✅  Deploy successful!
Preview URL: https://abb5e87a.agentv.pages.dev
Branch Preview URL: https://feat-workspace-lifecycle.agentv.pages.dev

View logs

@christso christso force-pushed the feat/workspace-lifecycle branch 2 times, most recently from 00b5edc to 4f57253 Compare February 21, 2026 12:42
@christso @claude
feat: workspace lifecycle hooks + cross-repo-sync showcase
8286fcc 
Replace setup/teardown with before_all/after_all/before_each/after_each
lifecycle hooks (bun:test/Vitest naming). Shared workspace across tests
in a suite with after_each reset. Remove workspaceFingerprint (YAGNI).

Add cross-repo-sync showcase demonstrating the full workspace config
surface with real ground truth diffs from agentevals commit history.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@christso christso force-pushed the feat/workspace-lifecycle branch from 4f57253 to 8286fcc Compare February 21, 2026 12:50
christso and others added 3 commits February 21, 2026 21:09
@christso @claude
feat(core): add copilot-cli provider using ACP protocol
442ca3c 
Add a new `copilot-cli` provider that spawns the Copilot CLI binary
directly and communicates via the Agent Client Protocol (ACP), bypassing
the @github/copilot-sdk's 60s session.idle timeout for long-running agents.

- New `CopilotCliProvider` using @agentclientprotocol/sdk
- `copilot-cli` is now its own ProviderKind (no longer aliases to copilot)
- CopilotCliResolvedConfig with executable, model, args, cwd, etc.
- Symbol-based log tracker matching copilot-sdk pattern

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@christso @claude
fix(core): replace non-null assertions with runtime check in copilot-cli
c2c64d6 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@christso @claude
feat(core): add nested git support and copilot-cli provider tests
e93ca10 
- Update file-changes.ts to use --submodule=diff for nested repo diffs
- Add stageNestedRepoChanges() helper to stage files in child repos
- Preserve .git in setup.ts so agents have full git history access
- Add copilot-cli provider resolution tests in targets.test.ts
- Update showcase targets.yaml to use copilot-cli provider
- Add nested git repo tests with clean env to avoid GIT_DIR leaks

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@christso christso changed the title feat: workspace lifecycle hooks + cross-repo-sync showcase feat: copilot-cli provider (ACP) + nested git workspace support Feb 21, 2026
christso and others added 6 commits February 21, 2026 21:57
@christso @claude
refactor(core): extract shared copilot utils, fix timer leak and cost…
e1fd132 
… tracking

- Extract resolvePlatformCliPath, buildLogFilename, sanitizeForFilename,
  formatElapsed, killProcess, CopilotStreamLogger, isLogStreamingDisabled
  into copilot-utils.ts shared by both copilot-cli.ts and copilot-sdk.ts
- Fix timer leak in raceWithTimeout/sendWithTimeout: clear timeout in
  finally block so it doesn't fire after promise resolves (both providers)
- Fix cost accumulation in copilot-cli: usage_update cost is now
  accumulated instead of overwritten across multiple events
- Fix usage_update summarize event type (was 'usage', now 'usage_update')
- Log warning when shared workspace overrides workers count to 1

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@christso @claude
fix(core): lint fixes for copilot-utils import order and formatting
e355f2a 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@christso @claude
fix(core): pass system prompt via ACP prompt messages, not CLI flag
aa12ffe 
The copilot CLI binary does not support --system-prompt. Move system
prompt delivery to the ACP prompt messages array instead. Also add
azure_judge target to showcase for LLM judge testing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@christso @claude
fix(core): biome format for resolveSystemPrompt return
18ae09a 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@christso @claude
fix(core): include file_changes in rubric/score-range LLM judge prompts
46592f2 
The rubric (checklist) and score-range prompt builders in both
llm-judge.ts and llm-judge-prompt.ts were missing the file_changes
section — only the freeform path included it. This meant rubrics
evaluators could not see workspace diffs when grading agent output.

Also adds examples/features/file-changes-judges/ — a minimal eval
proving file_changes works with all three judge types: Azure LLM
rubrics judge, built-in agent judge, and copilot-cli agent judge.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@christso @claude
refactor(core): rename agent_judge judge_target to target
100aa0c 
The evaluator is already type: agent_judge — having judge_target on a
judge is semantically redundant. Rename to just target on the evaluator
config surface. The target-level judge_target (in targets.yaml) is
unchanged as it serves a different purpose there.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@christso christso merged commit 3f6e2a6 into main Feb 22, 2026
1 check passed
@christso christso deleted the feat/workspace-lifecycle branch February 22, 2026 00:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@christso