Add /review-pr skill and wire it into the Claude Code Review workflow#4351
Add /review-pr skill and wire it into the Claude Code Review workflow#4351AryanGodara wants to merge 14 commits intomainfrom
Conversation
adddaf4 to
91236b2
Compare
There was a problem hiding this comment.
Code Review
This pull request introduces a structured local PR review skill for Claude, including command definitions, settings, and comprehensive review context documentation for Rust, database migrations, and OpenAPI specifications. Two high-severity issues were identified: the inclusion of personal plugin configurations in the settings file which could conflict with other team members' environments, and a broken setup instruction that references a non-existent documentation file.
MartinquaXD
left a comment
There was a problem hiding this comment.
This PR feels well intentioned but misguided.
Code reviews rely a lot on context and historic knowledge. Assembling a list of highly specific things to look out for will always be incomplete and hard to keep up-to-date.
IMO the play would be to agree as a team on a set of high level goals / rules-of-thumb to apply in the code base and let the AI judge if it there were followed.
I think we should probably only add highly specific things if we care deeply about them and the AI always misses them.
One thing we should probably have in this workflow is to tell the AI when to use git blame to uncover historic context. Often time when code looks very unusual it's like that for a reason so a PR that seemingly cleans up some mess could re-introduce very subtle errors that were hard to identify and fix in the first place.
91236b2 to
71773bf
Compare
There was a problem hiding this comment.
Code Review
This pull request introduces a structured PR review skill for the CoW Protocol services repository, including an entry-point command, a core review instruction document, and specialized context for database migrations. The review feedback identifies several critical issues regarding the reliability of the git merge-base detection, inconsistencies in severity levels that would cause important findings to be filtered out by existing automation, and a failure to account for 'degraded static-diff' mode when using local filesystem tools. Suggestions were provided to use 'origin/main' for diffs, elevate breaking API and database documentation changes to 'High' severity, and explicitly restrict tool usage when a PR is not checked out locally.
|
Note on CI GitHub's own error message says to ignore this on the first PR. After this lands, every subsequent PR will run /review-pr from the repo as expected. |
jmg-duarte
left a comment
There was a problem hiding this comment.
Great initiative, I'm just worried that merging this is a Sisyphean task
For the more complicated notes, you can follow up after, no need to separate things now, but do keep track of them or open an issue tracking it
| - If the environment variable `$GITHUB_ACTIONS == "true"` → `mode = "pr-ci"`. | ||
| - `$ARGUMENTS` MUST be a PR number, URL, or `owner/repo#N` form (the workflow passes it). | ||
| - Else if `$ARGUMENTS` is non-empty → `mode = "pr-local"`. Parse the argument (see [step 2](#2-parse-arguments-pr-modes-only)). | ||
| - Else (`$ARGUMENTS` empty, not in CI) → `mode = "diff"`. No `gh` calls needed; source is `git diff $(git merge-base HEAD origin/main)..HEAD` (the actual command runs in step 3 below, after fetching `origin/main`). |
There was a problem hiding this comment.
Instead of 3 modes, why not 3 skills?
Even if they're just then called from there instead of having modes that add entropy to the agent
There was a problem hiding this comment.
Considered this. The three modes share a lot of logic, ie, codemap, severity rubric, universal guardrails, anti-nit rule, finding shape, optional-tool probe. Splitting into three skills means three copies of all that. Might even need to be kept in sync by hand.
The mode-specific code localizes to four narrow places:
- where the diff comes from (git diff merge-base vs gh pr diff)
- whether to fetch PR metadata
- whether to gh pr checkout
- where the report goes (terminal vs the CI comment posted by the action).
Those are conditionals, not different review logics. So moving them into three skills doesn't remove the entropy, just relocates it.
Happy to revisit if mode-specific divergence grows past two or three places. Right now I'd rather keep one source of truth for the review content.
| prompt: '/code-review:code-review ${{ github.repository }}/pull/${{ github.event.pull_request.number }} --comment' | ||
|
|
||
| # Invoke the in-repo slash command. The skill detects | ||
| # $GITHUB_ACTIONS=true and switches to `pr-ci` mode, posting |
There was a problem hiding this comment.
w.r.t. my previous comment, it really seems to me that we could have a CI skill specifically
There was a problem hiding this comment.
the CI mode is one of those four-place conditionals, not a separate review logic.
Also for context, the decision originated from initial reviews, where consensus was that we can re-purpose the first draft of this PR (which was only to self review/ review already ready for review PRs), into a single source of logic/rules.
And then also use the same skill in the CI, instead of having 2 separate review tools
|
|
||
| ## Usually worth flagging as **Medium** | ||
|
|
||
| - Migration scripts without comments when the *what* is non-obvious. |
There was a problem hiding this comment.
This is not very clear and I'd expect this to lead to extra review notes
ab1e843 to
75f4d32
Compare
cfe1b6b to
a869194
Compare
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive framework for automated PR reviews and historical context analysis, including new Claude commands and detailed skill documentation for database migrations and incident investigation. Feedback was provided to address a missing metadata field required for commit tracking and to clarify that concurrent index creation must be executed outside of transaction blocks to prevent migration failures.
|
|
||
| 1. **Reversibility.** State whether the migration is reversible. If yes, include or link the rollback script. If no, the PR explains *why* irreversibility is acceptable. Missing either → **High**. | ||
| 2. **n-1 schema compatibility.** The previous app version must still function against the new schema. A migration that drops or renames a column, narrows a type, or adds a `NOT NULL` constraint without code already tolerating both shapes → **High** until the rollout plan is spelled out (typically: ship change in three releases — add → migrate code → drop). | ||
| 3. **Blocking index creation on hot tables.** Use `CREATE INDEX CONCURRENTLY` on anything in the auction/settlement critical path (`orders`, `trades`, `auctions`, `settlements`, `order_events`, `auction_orders`, `quotes`, `order_quotes`). A blocking `CREATE INDEX` on one of these → **High**. |
There was a problem hiding this comment.
PostgreSQL does not allow CREATE INDEX CONCURRENTLY to be executed within a transaction block. Since Flyway migrations (referenced in line 13) are executed within a transaction by default, any migration using this command will fail unless it is explicitly configured to run non-transactionally (e.g., using Flyway's non-transactional configuration).
| 3. **Blocking index creation on hot tables.** Use `CREATE INDEX CONCURRENTLY` on anything in the auction/settlement critical path (`orders`, `trades`, `auctions`, `settlements`, `order_events`, `auction_orders`, `quotes`, `order_quotes`). A blocking `CREATE INDEX` on one of these → **High**. | |
| 3. Blocking index creation on hot tables. Use CREATE INDEX CONCURRENTLY on anything in the auction/settlement critical path (orders, trades, auctions, settlements, order_events, auction_orders, quotes, order_quotes). Note that concurrent index creation cannot run inside a transaction; ensure the migration is configured to run non-transactionally. A blocking CREATE INDEX on one of these -> High. |
|
|
||
| ```bash | ||
| gh pr view <PR_NUMBER> -R <owner>/<repo> \ | ||
| --json title,body,author,labels,files,baseRefName,headRefName,additions,deletions,commits,reviewDecision,isDraft,state |
There was a problem hiding this comment.
The gh pr view command is missing the headRefOid field in the --json argument. This field is necessary for the logic in section 2.5 (line 104), which compares the latest review's commit_id with the current head SHA to determine if new commits have been pushed since the last review round.
| --json title,body,author,labels,files,baseRefName,headRefName,additions,deletions,commits,reviewDecision,isDraft,state | |
| --json title,body,author,labels,files,baseRefName,headRefName,headRefOid,additions,deletions,commits,reviewDecision,isDraft,state |
9cfd4c3 to
eaa6b40
Compare
… exception, tighten db-migrations sibling
…tation and remove obsolete skill
…p unused -R flags, align drift rule
eaa6b40 to
6f58e13
Compare
…etch+inputs, cosmetic fixes
|
The Claude Code PR review CI job was removed as being very expensive due to our codebase size. |
… second worked example, typo + cosmetic fixes
What
Adds
/review-pr, a single Claude Code skill for reviewing PRs in cowprotocol/services. Three modes — locally vs your current branch (no PR yet), locally vs an existing PR, or in CI — share the same reference doc. Only the diff source and the report sink differ.Also re-introduces the Claude Code Review workflow that #4381 dropped a few days ago. That deletion was a stop-gap because the off-the-shelf
code-review@claude-code-pluginswas burning too many tokens. This PR replaces it with a workflow that calls a focused in-repo skill (~400 LOC), so the prompt is bounded.Design
docs/COW_PR_REVIEW_SKILL.md. All three modes follow it.diffmode covers "review before I open the PR". Runs againstgit merge-base HEAD origin/main, noghcalls.database-migrations.md, because k8s pod overlap and staging/prod cadence are genuinely CoW-specific.git blameruns before flagging unusual code, in case it was a deliberate prior fix. Workflow usesfetch-depth: 0for this.Files
.claude/commands/review-pr.md/review-pr. Mode detection, preflights, handoff..claude/commands/blame-context.md/blame-context. Tracked standalone in #4373..claude/commands/pr-synthesis.md/pr-synthesis. Tracked standalone in #4374.docs/COW_PR_REVIEW_SKILL.mddocs/skills/git-blame-historic-context.mddocs/skills/pr-context-synthesis.mddocs/skills/pr-blame-walk.mddocs/review-context/database-migrations.mddatabase/sql/**is touched..github/workflows/claude-code-review.ymlfetch-depth: 0for git blame.How to try it
pr-ciruns automatically when a PR is marked Ready for review.CI behaviour to expect
The
Claude Code Reviewjob on this PR will fail withWorkflow validation failed. That's GitHub's security rule against workflow self-modification — a PR modifying a workflow can't run its own modified version. Ignore the failure on this PR; every subsequent PR runs/review-pras expected.External-fork PRs silently no-op (secrets aren't available,
pull-requests: writeis downgraded). Same as the previous workflow.Feedback
False positives become counter-examples for the anti-nit rule. Send me anything the report flags that you'd close as "not a real finding".