Skip to content

Add /review-pr skill and wire it into the Claude Code Review workflow#4351

Open
AryanGodara wants to merge 14 commits intomainfrom
aryan/claude-pr-review-skill
Open

Add /review-pr skill and wire it into the Claude Code Review workflow#4351
AryanGodara wants to merge 14 commits intomainfrom
aryan/claude-pr-review-skill

Conversation

@AryanGodara
Copy link
Copy Markdown
Member

@AryanGodara AryanGodara commented Apr 21, 2026

What

Adds /review-pr, a single Claude Code skill for reviewing PRs in cowprotocol/services. Three modes — locally vs your current branch (no PR yet), locally vs an existing PR, or in CI — share the same reference doc. Only the diff source and the report sink differ.

Also re-introduces the Claude Code Review workflow that #4381 dropped a few days ago. That deletion was a stop-gap because the off-the-shelf code-review@claude-code-plugins was burning too many tokens. This PR replaces it with a workflow that calls a focused in-repo skill (~400 LOC), so the prompt is bounded.

Design

  • Single source of truth: docs/COW_PR_REVIEW_SKILL.md. All three modes follow it.
  • diff mode covers "review before I open the PR". Runs against git merge-base HEAD origin/main, no gh calls.
  • Six universal guardrails — minimal public API, no rightward drift, single responsibility, split big files, no argument bloat, errors carry context — replace the older alloy/openapi sibling docs. The only kept sibling is database-migrations.md, because k8s pod overlap and staging/prod cadence are genuinely CoW-specific.
  • Anti-nit rule is mandatory. rustfmt-fixable issues are never findings. LGTM is a valid verdict.
  • Read-only in local modes. No auto-stash, no auto-rebase, no auto-comments. The user owns what lands on the PR.
  • git blame runs before flagging unusual code, in case it was a deliberate prior fix. Workflow uses fetch-depth: 0 for this.

Files

File Role
.claude/commands/review-pr.md Slash-command entry for /review-pr. Mode detection, preflights, handoff.
.claude/commands/blame-context.md Slash command for /blame-context. Tracked standalone in #4373.
.claude/commands/pr-synthesis.md Slash command for /pr-synthesis. Tracked standalone in #4374.
docs/COW_PR_REVIEW_SKILL.md Reference doc Claude follows. Guardrails, execution flow, severity rubric, report templates.
docs/skills/git-blame-historic-context.md Sibling skill — used before flagging unusual code. Tracked standalone in #4373.
docs/skills/pr-context-synthesis.md Sibling skill — produces the CONTEXT block. Tracked standalone in #4374.
docs/skills/pr-blame-walk.md Companion incident-investigation skill (N→1 suspect ranking).
docs/review-context/database-migrations.md Loaded when database/sql/** is touched.
.github/workflows/claude-code-review.yml CI wiring. fetch-depth: 0 for git blame.

How to try it

/review-pr                                # diff mode — current branch vs main
/review-pr 4243                           # pr-local mode
/review-pr cowprotocol/services#4243      # explicit form

pr-ci runs automatically when a PR is marked Ready for review.

CI behaviour to expect

The Claude Code Review job on this PR will fail with Workflow validation failed. That's GitHub's security rule against workflow self-modification — a PR modifying a workflow can't run its own modified version. Ignore the failure on this PR; every subsequent PR runs /review-pr as expected.

External-fork PRs silently no-op (secrets aren't available, pull-requests: write is downgraded). Same as the previous workflow.

Feedback

False positives become counter-examples for the anti-nit rule. Send me anything the report flags that you'd close as "not a real finding".

@AryanGodara AryanGodara self-assigned this Apr 21, 2026
@AryanGodara AryanGodara force-pushed the aryan/claude-pr-review-skill branch from adddaf4 to 91236b2 Compare April 21, 2026 21:26
@AryanGodara AryanGodara marked this pull request as ready for review April 21, 2026 21:36
@AryanGodara AryanGodara requested a review from a team as a code owner April 21, 2026 21:36
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a structured local PR review skill for Claude, including command definitions, settings, and comprehensive review context documentation for Rust, database migrations, and OpenAPI specifications. Two high-severity issues were identified: the inclusion of personal plugin configurations in the settings file which could conflict with other team members' environments, and a broken setup instruction that references a non-existent documentation file.

Comment thread .claude/commands/review-pr.md Outdated
Comment thread .claude/settings.json Outdated
Copy link
Copy Markdown
Contributor

@MartinquaXD MartinquaXD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR feels well intentioned but misguided.
Code reviews rely a lot on context and historic knowledge. Assembling a list of highly specific things to look out for will always be incomplete and hard to keep up-to-date.
IMO the play would be to agree as a team on a set of high level goals / rules-of-thumb to apply in the code base and let the AI judge if it there were followed.

I think we should probably only add highly specific things if we care deeply about them and the AI always misses them.

One thing we should probably have in this workflow is to tell the AI when to use git blame to uncover historic context. Often time when code looks very unusual it's like that for a reason so a PR that seemingly cleans up some mess could re-introduce very subtle errors that were hard to identify and fix in the first place.

Comment thread .claude/commands/review-pr.md Outdated
Comment thread .claude/commands/review-pr.md Outdated
Comment thread .claude/commands/review-pr.md Outdated
Comment thread docs/review-context/alloy-rs.md Outdated
Comment thread docs/review-context/alloy-rs.md Outdated
Comment thread docs/review-context/openapi.md Outdated
Comment thread docs/COW_PR_REVIEW_SKILL.md Outdated
Comment thread docs/COW_PR_REVIEW_SKILL.md Outdated
Comment thread docs/COW_PR_REVIEW_SKILL.md Outdated
Comment thread docs/COW_PR_REVIEW_SKILL.md Outdated
@AryanGodara AryanGodara changed the title Adding claude skill for PR review (WIP) Adding claude skill for PR review Apr 24, 2026
@AryanGodara AryanGodara marked this pull request as draft April 24, 2026 10:49
@AryanGodara AryanGodara changed the title Adding claude skill for PR review Add /review-pr skill and wire it into the Claude Code Review workflow Apr 25, 2026
@AryanGodara AryanGodara force-pushed the aryan/claude-pr-review-skill branch from 91236b2 to 71773bf Compare April 25, 2026 16:12
@AryanGodara AryanGodara marked this pull request as ready for review April 25, 2026 21:16
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a structured PR review skill for the CoW Protocol services repository, including an entry-point command, a core review instruction document, and specialized context for database migrations. The review feedback identifies several critical issues regarding the reliability of the git merge-base detection, inconsistencies in severity levels that would cause important findings to be filtered out by existing automation, and a failure to account for 'degraded static-diff' mode when using local filesystem tools. Suggestions were provided to use 'origin/main' for diffs, elevate breaking API and database documentation changes to 'High' severity, and explicitly restrict tool usage when a PR is not checked out locally.

Comment thread .claude/commands/review-pr.md Outdated
Comment thread docs/COW_PR_REVIEW_SKILL.md Outdated
Comment thread docs/COW_PR_REVIEW_SKILL.md
Comment thread docs/COW_PR_REVIEW_SKILL.md Outdated
Comment thread docs/review-context/database-migrations.md
@AryanGodara
Copy link
Copy Markdown
Member Author

Note on CI
The Claude Code Review job on this PR will fail with Workflow validation failed. The workflow file must exist and have identical content to the version on the repository's default branch.

GitHub's own error message says to ignore this on the first PR. After this lands, every subsequent PR will run /review-pr from the repo as expected.

Copy link
Copy Markdown
Contributor

@jmg-duarte jmg-duarte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great initiative, I'm just worried that merging this is a Sisyphean task

For the more complicated notes, you can follow up after, no need to separate things now, but do keep track of them or open an issue tracking it

Comment on lines +15 to +18
- If the environment variable `$GITHUB_ACTIONS == "true"` → `mode = "pr-ci"`.
- `$ARGUMENTS` MUST be a PR number, URL, or `owner/repo#N` form (the workflow passes it).
- Else if `$ARGUMENTS` is non-empty → `mode = "pr-local"`. Parse the argument (see [step 2](#2-parse-arguments-pr-modes-only)).
- Else (`$ARGUMENTS` empty, not in CI) → `mode = "diff"`. No `gh` calls needed; source is `git diff $(git merge-base HEAD origin/main)..HEAD` (the actual command runs in step 3 below, after fetching `origin/main`).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of 3 modes, why not 3 skills?

Even if they're just then called from there instead of having modes that add entropy to the agent

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considered this. The three modes share a lot of logic, ie, codemap, severity rubric, universal guardrails, anti-nit rule, finding shape, optional-tool probe. Splitting into three skills means three copies of all that. Might even need to be kept in sync by hand.

The mode-specific code localizes to four narrow places:

  • where the diff comes from (git diff merge-base vs gh pr diff)
  • whether to fetch PR metadata
  • whether to gh pr checkout
  • where the report goes (terminal vs the CI comment posted by the action).

Those are conditionals, not different review logics. So moving them into three skills doesn't remove the entropy, just relocates it.

Happy to revisit if mode-specific divergence grows past two or three places. Right now I'd rather keep one source of truth for the review content.

prompt: '/code-review:code-review ${{ github.repository }}/pull/${{ github.event.pull_request.number }} --comment'

# Invoke the in-repo slash command. The skill detects
# $GITHUB_ACTIONS=true and switches to `pr-ci` mode, posting
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

w.r.t. my previous comment, it really seems to me that we could have a CI skill specifically

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the CI mode is one of those four-place conditionals, not a separate review logic.

Also for context, the decision originated from initial reviews, where consensus was that we can re-purpose the first draft of this PR (which was only to self review/ review already ready for review PRs), into a single source of logic/rules.
And then also use the same skill in the CI, instead of having 2 separate review tools


## Usually worth flagging as **Medium**

- Migration scripts without comments when the *what* is non-obvious.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not very clear and I'd expect this to lead to extra review notes

Comment thread docs/review-context/database-migrations.md Outdated
Comment thread docs/review-context/database-migrations.md Outdated
Comment thread docs/COW_PR_REVIEW_SKILL.md Outdated
Comment thread docs/COW_PR_REVIEW_SKILL.md Outdated
Comment thread docs/COW_PR_REVIEW_SKILL.md Outdated
Comment thread docs/COW_PR_REVIEW_SKILL.md Outdated
@AryanGodara AryanGodara marked this pull request as draft April 27, 2026 20:13
@AryanGodara AryanGodara force-pushed the aryan/claude-pr-review-skill branch 2 times, most recently from ab1e843 to 75f4d32 Compare April 29, 2026 08:53
@AryanGodara AryanGodara force-pushed the aryan/claude-pr-review-skill branch 2 times, most recently from cfe1b6b to a869194 Compare May 5, 2026 11:41
@AryanGodara AryanGodara marked this pull request as ready for review May 5, 2026 13:43
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a comprehensive framework for automated PR reviews and historical context analysis, including new Claude commands and detailed skill documentation for database migrations and incident investigation. Feedback was provided to address a missing metadata field required for commit tracking and to clarify that concurrent index creation must be executed outside of transaction blocks to prevent migration failures.


1. **Reversibility.** State whether the migration is reversible. If yes, include or link the rollback script. If no, the PR explains *why* irreversibility is acceptable. Missing either → **High**.
2. **n-1 schema compatibility.** The previous app version must still function against the new schema. A migration that drops or renames a column, narrows a type, or adds a `NOT NULL` constraint without code already tolerating both shapes → **High** until the rollout plan is spelled out (typically: ship change in three releases — add → migrate code → drop).
3. **Blocking index creation on hot tables.** Use `CREATE INDEX CONCURRENTLY` on anything in the auction/settlement critical path (`orders`, `trades`, `auctions`, `settlements`, `order_events`, `auction_orders`, `quotes`, `order_quotes`). A blocking `CREATE INDEX` on one of these → **High**.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

PostgreSQL does not allow CREATE INDEX CONCURRENTLY to be executed within a transaction block. Since Flyway migrations (referenced in line 13) are executed within a transaction by default, any migration using this command will fail unless it is explicitly configured to run non-transactionally (e.g., using Flyway's non-transactional configuration).

Suggested change
3. **Blocking index creation on hot tables.** Use `CREATE INDEX CONCURRENTLY` on anything in the auction/settlement critical path (`orders`, `trades`, `auctions`, `settlements`, `order_events`, `auction_orders`, `quotes`, `order_quotes`). A blocking `CREATE INDEX` on one of these **High**.
3. Blocking index creation on hot tables. Use CREATE INDEX CONCURRENTLY on anything in the auction/settlement critical path (orders, trades, auctions, settlements, order_events, auction_orders, quotes, order_quotes). Note that concurrent index creation cannot run inside a transaction; ensure the migration is configured to run non-transactionally. A blocking CREATE INDEX on one of these -> High.

Comment thread docs/COW_PR_REVIEW_SKILL.md Outdated

```bash
gh pr view <PR_NUMBER> -R <owner>/<repo> \
--json title,body,author,labels,files,baseRefName,headRefName,additions,deletions,commits,reviewDecision,isDraft,state
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The gh pr view command is missing the headRefOid field in the --json argument. This field is necessary for the logic in section 2.5 (line 104), which compares the latest review's commit_id with the current head SHA to determine if new commits have been pushed since the last review round.

Suggested change
--json title,body,author,labels,files,baseRefName,headRefName,additions,deletions,commits,reviewDecision,isDraft,state
--json title,body,author,labels,files,baseRefName,headRefName,headRefOid,additions,deletions,commits,reviewDecision,isDraft,state

@AryanGodara AryanGodara force-pushed the aryan/claude-pr-review-skill branch from 9cfd4c3 to eaa6b40 Compare May 5, 2026 14:26
@AryanGodara AryanGodara force-pushed the aryan/claude-pr-review-skill branch from eaa6b40 to 6f58e13 Compare May 6, 2026 02:23
@squadgazzz
Copy link
Copy Markdown
Contributor

The Claude Code PR review CI job was removed as being very expensive due to our codebase size.

… second worked example, typo + cosmetic fixes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants