Skip to content

Fix #5979: Enforce guardrail re-validation on retry and reject negative max_retries#5980

Open
devin-ai-integration[bot] wants to merge 2 commits into
mainfrom
devin/1780129450-fix-guardrail-enforcement
Open

Fix #5979: Enforce guardrail re-validation on retry and reject negative max_retries#5980
devin-ai-integration[bot] wants to merge 2 commits into
mainfrom
devin/1780129450-fix-guardrail-enforcement

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

Summary

Fixes two bugs in the task guardrail enforcement system where guardrails could be silently bypassed, treating them as suggestions rather than hard constraints.

Bug 1: Multiple guardrails skip re-validation of earlier guardrails after retry.
When guardrail N failed and the agent retried, the new output was only checked against guardrail N onward — guardrails 0..N-1 were never re-evaluated. A retry output that violated an earlier guardrail would pass through unchecked.

# Before (buggy): g0 passes original, g1 fails → retry → new output skips g0
for idx, guardrail in enumerate(self._guardrails):
    task_output = self._invoke_guardrail_function(...)  # each guardrail retries independently

# After (fixed): any failure → retry → re-run ALL guardrails from g0
task_output = self._run_guardrails(task_output, agent, tools)

Refactored _invoke_guardrail_function / _ainvoke_guardrail_function into _run_guardrails / _arun_guardrails which run all guardrails in a single loop — on any failure the task is re-executed and validation restarts from the first guardrail.

Bug 2: Negative guardrail_max_retries bypassed guardrails entirely.
Setting guardrail_max_retries = -1 made max_attempts = 0, causing the validation loop to never execute. Added ge=0 constraint to the field.

Tests added:

  • test_retry_revalidates_earlier_guardrails — core scenario where retry output violates an earlier guardrail
  • test_retry_restart_catches_earlier_guardrail_violation — regression test with interleaved failures
  • test_negative_guardrail_max_retries_rejected — validation of negative values
  • test_zero_guardrail_max_retries_no_retry — zero retries fails immediately
  • test_single_guardrail_retry_still_works — backward compat for single guardrail

Link to Devin session: https://app.devin.ai/sessions/8b6d443fc3e4401cb126c03c8cf037e9

…_retries

Fixes #5979 — guardrails are now enforced as hard constraints.

Two bugs fixed:

1. Multiple guardrails skipped re-validation of earlier guardrails after retry.
   When guardrail N failed and the agent retried, the new output was only
   checked against guardrail N onward. Earlier guardrails (0..N-1) were not
   re-evaluated, allowing retry outputs to silently violate them.

   Fix: refactored _invoke_guardrail_function into _run_guardrails which
   runs ALL guardrails from the beginning on each retry attempt.

2. Negative guardrail_max_retries bypassed guardrails entirely.
   Setting guardrail_max_retries to a negative value made the guardrail
   loop execute zero iterations, letting output pass without validation.

   Fix: added ge=0 constraint to the guardrail_max_retries field.

Co-Authored-By: João <joao@crewai.com>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Co-Authored-By: João <joao@crewai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants