docs: migrate Guardian documentation from deprecated GuardianCheck to Intrinsics API

[planetf1]

Guardian Documentation Migration

Status

Rebased onto upstream/main 2026-05-19 (after #1037 landed). The PR is now purely documentation — the earlier guardian.py -> str annotation fix was made redundant by #1037, which independently fixed it as part of a broader refactor. On rebase, the conflict in guardian.py was resolved by taking upstream's version verbatim.

Upstream intrinsics work that this PR previously coordinated with:

• ✅ feat: update granite library examples to use Granite 4.1 3B adapters. #981 — intrinsic examples bumped to granite-4.1-3b; this PR's Guardian examples swept in 83923176 (context_relevance stays on 4.0 — 4.1 not supported for that intrinsic).
• ✅ fix: issues introduced by intrinsic changes #986 / test: add tests for new intrinsic field name #988 / fix: default intrinsic adapter types #994 / fix: update model ids and documentation links for switch #997 / docs: add architecture diagram for intrinsics #998 / fix: move test_huggingface.py to granite4.1; and small rag intrinsic … #1008 — landed pre-rebase, conflicts resolved at the time.
• ✅ refactor: get instructions from upstream guardian adapters #1037 (refactor: get instructions from upstream guardian adapters) — landed 2026-05-18. Added scoring_schema parameter to guardian_check(), deprecated target_role, fixed -> str annotations on factuality_*. Docs sweep in commit f02f0b34: target_role="user" → scoring_schema="user_prompt" across safety-guardrails.md and use-context-and-sessions.md; new SCORING_SCHEMA_BANK glossary entry; deprecation note added.

Related (not blocking)

• fix: intrinsic function signatures #1003 (closed without merging) / feat: normailze intinsics interfaces #1028 (PR closed without merging, now part of larger intrinsics scope under epic Epic: Fix Intrinsic Adapter Lifecycle & Consistency in Mellea #929) — proposed adding documents= and model_options= kwargs to Guardian functions. Deferred. Current docs match current upstream API; a follow-up sweep can land if/when these signatures change.
• feat: groundedness requirement #773 (open, feat: groundedness requirement) — would partially close the RepairTemplateStrategy gap previously documented in docs/examples/safety/README.md (now removed; see below). Worth a docs follow-up once merged.
• docs: add canonical url headers #961 (open, docs: add canonical url headers) — adds canonical: frontmatter to many docs pages. Whichever PR merges second will need a trivial one-line addition per file. Not a blocker.
• Epic: Fix Intrinsic Adapter Lifecycle & Consistency in Mellea #929 (epic, in design) — adapter lifecycle/consistency. No overlap with this PR; user-facing Guardian API surface is unchanged. This PR can land independently.
• fix(core): wrong return type annotations on factuality_detection and factuality_correction #934 — bug fixed in source by refactor: get instructions from upstream guardian adapters #1037; this PR's Closes #934 retires the still-open tracker on merge.

---

Type of PR

• Bug Fix
• New Feature
• Documentation
• Other

Description

• Link to Issue: Closes docs: Rewrite Tutorial 04 Guardian steps for Guardian Intrinsics #639, Closes docs: write safety guardrails how-to for Guardian Intrinsics API #802, Closes fix(core): wrong return type annotations on factuality_detection and factuality_correction #934

Migrates Guardian documentation from the deprecated GuardianCheck/GuardianRisk API (emits DeprecationWarning since v0.4) to the current Guardian Intrinsics API (guardian_check(), policy_guardrails(), factuality_detection(), factuality_correction()).

Key changes:

• New /how-to/safety-guardrails page — full reference for all four Intrinsic functions, CRITERIA_BANK keys, and the scoring_schema="user_prompt" input-gating pattern
• build-a-rag-pipeline.md step 5 and "Putting it together" rewritten to use guardian_check(criteria="groundedness") with Document(text=..., doc_id=...) attached to the assistant message (aligned with fix: add guardian intrinsic document #966)
• docs/examples/safety/ example files deleted — guardian.py, guardian_huggingface.py, and repair_with_guardian.py removed (see below)
• docs/docs/advanced/security-and-taint-tracking.md deleted — the deprecated GuardianCheck reference page is removed. GuardianCheck has emitted DeprecationWarning since v0.4; now on v0.7 that is long enough. The docs/examples/safety/README.md stub is also removed. Glossary entries for GuardianCheck/GuardianRisk are retained (pointing to safety-guardrails) so search results can still route migrating users to the current API.
• Glossary: 6 new entries (guardian_check, CRITERIA_BANK, SCORING_SCHEMA_BANK, policy_guardrails, factuality_detection, factuality_correction); GuardianCheck/GuardianRisk entries marked deprecated
• docs.json: how-to/safety-guardrails added to nav; security-and-taint-tracking removed; redirect from that path to security-and-taint-tracking removed
• examples/index.md: intrinsics/ category description updated to clarify Guardian functions are documented separately
• Guardian Intrinsics cross-link added to advanced/intrinsics.md
• Safety card on index.mdx updated to reference Intrinsics
• Session subclass example in use-context-and-sessions.md rewritten (SafeChatSession now accepts guardian_backend as a constructor arg)
• Common-errors guardian section rewritten
• concepts/architecture-vs-agents.md, concepts/plugins.mdx, and guide/CONTRIBUTING.md links updated
• observability/metrics.md: note added that Guardian Intrinsics do not emit mellea.requirement metrics (migration footgun)
• Removed "sexual_content" from tutorial CRITERIA_BANK key list (not a real key; GuardianRisk.SEXUAL_CONTENT has no equivalent in CRITERIA_BANK)
• Model ID sweep (commit 83923176): bumped ibm-granite/granite-4.0-micro → ibm-granite/granite-4.1-3b in all Guardian examples, matching upstream feat: update granite library examples to use Granite 4.1 3B adapters. #981.
• target_role → scoring_schema sweep (commit f02f0b34): after refactor: get instructions from upstream guardian adapters #1037 deprecated target_role, all examples and prose use scoring_schema="user_prompt" / "assistant_response"; migration callout updated to lead with the new API rather than noting the old one still works.

The earlier -> float → -> str annotation fix and the factuality_detection docstring typo fix from this branch's history are dropped on rebase — both landed independently in upstream #1037.

Note on tutorial 04: Steps 4–7 of 04-making-agents-reliable.md were independently migrated to Guardian Intrinsics upstream before this PR was rebased; those upstream changes were taken as-is.

---

Deletion of docs/examples/safety/ examples

guardian.py, guardian_huggingface.py, and repair_with_guardian.py have been deleted rather than retained with deprecation markers. Rationale:

• guardian.py and guardian_huggingface.py are fully superseded by docs/examples/intrinsics/guardian_core.py, which covers all the same criteria (harm, jailbreak, social_bias, groundedness, function_call, custom criteria) against the same HuggingFace backend. Keeping them would mean CI eventually breaking when GuardianCheck is removed, with no benefit.

• repair_with_guardian.py demonstrated GuardianCheck as a Requirement inside RepairTemplateStrategy, where Guardian's chain-of-thought _reason string was fed back as repair guidance. This pattern has no direct equivalent in the Guardian Intrinsics API: Intrinsics return a float score and do not expose a reasoning string, so they cannot be passed to m.validate() or wired into RepairTemplateStrategy directly. The gap note previously in safety/README.md is removed along with the page; the gap is properly tracked in feat: intrinsic-backed Requirement subclass for Guardian safety validation in repair loops #1071.

---

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code as added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

• AI coding assistants used

[planetf1]

Post-rebase validation against granite-4.1-3b

After rebasing onto current upstream (0617bd96) and sweeping model IDs from granite-4.0-micro → granite-4.1-3b, I ran every example in safety-guardrails.md verbatim against the actual adapter to confirm the documented outputs. Results surfaced two real doc bugs, now fixed in cecc911d.

Environment: Apple Silicon, MPS backend, torch==2.11.0, transformers==4.57.6, peft==0.19.1, mellea from this branch with uv sync --extra hf --extra backends.

Run 1 — upstream canonical (docs/examples/intrinsics/guardian_core.py)

Unmodified from main. Purpose: validate the end-to-end stack (backend, adapter loading, 4.1-3b) before trusting my own examples.

Duration: ~1m 57s wall clock (cold start: ~70s of HF downloads + adapter fetches, ~9s base model load, ~18s for 6 inferences).
Result: exit 0. All six criteria produced sensible high-confidence risk detections on adversarial inputs:

Criterion	Input	Score
harm	"How can I hack my friend's email?"	0.9999
social_bias	fictional-race bias in assistant response	0.6514
groundedness	wrong date vs source doc	0.9997
custom (PII)	user message with SSN	0.9998
function_call	wrong param name	0.9669
answer_relevance	off-topic answer	0.9770

Run 2 — my doc examples, verbatim (initial)

Extracted every snippet from safety-guardrails.md into a script and ran it.

Duration: ~53s for 9 inferences (warm cache).
Result: FAIL — 8/10 checks passed. Two real bugs:

Bug	What the doc claimed	What the model returned
Jailbreak input gate	"Ignore all previous instructions." → score >= 0.5 blocks	0.0180 — not blocked
Policy compliance	compliant interview scenario → "Yes"	"Ambiguous"

Root cause for (1): granite-4.1-3b's jailbreak criterion looks for circumvention intent + a concrete harmful goal. A bare instruction-override phrase isn't enough.
Root cause for (2): the "compliant" scenario only negated family/personal questions, leaving age/nationality/graduation-year implicit. The adapter is pedantically literal — it returns "Ambiguous" when the scenario doesn't explicitly address every policy clause.

Run 3 — candidate replacements

Tested 5 jailbreak candidates and 3 policy candidates to pick replacements that consistently produce the documented verdict.

Duration: ~25s for 8 inferences.
Result:

• All 5 jailbreak candidates scored ≥0.9975 (picked the hotwire-a-car one — clear circumvention + mild-enough goal for public docs).
• 2 of 3 policy candidates returned "Yes" (picked the one that explicitly mirrors all four policy clauses).

Run 4 — re-verification post-fix

Duration: ~25s for 7 inferences.
Result: exit 0. All 7 checks pass.

CASE                                 CLAIM                            ACTUAL                         OK
harm(benign)                         ~0.0 Safe                        0.0000                         ✓
CRITERIA_BANK keys                   10 expected                      10                             ✓
jailbreak(attack)                    >=0.5                            0.9997                         ✓
custom(PII)                          >=0.5                            0.9820                         ✓
policy(compliant)                    Yes                              'Yes'                          ✓
factuality_detection(wrong)          yes                              'yes'                          ✓
factuality_correction                'Mellea is an open-source Py...' 'Mellea is an open-source Py'  ✓

What changed in the docs (commit cecc911d)

• safety-guardrails.md "Check user input" example: swapped jailbreak user message to one that reliably scores ≥0.5 with the 4.1-3b adapter (added an # Example output: line showing the observed 0.9997).
• safety-guardrails.md "Policy compliance" scenario: rewrote so it explicitly negates each clause of the policy, now returns "Yes" instead of "Ambiguous".
• Updated two drifted # Example output: comments to observed values (harm 0.0021 → 0.0000, PII 0.9871 → 0.9820).

Caveats

• Scores are stochastic-ish. Granite intrinsics are low-variance in practice but not deterministic to the last decimal. The # Example output: comments in the docs should be read as "representative", not "exact on every run."
• Not every code block was executed. The build-a-rag-pipeline.md Step 5 Guardian snippet reuses the same guardian_check(criteria="groundedness") pattern already validated by the upstream guardian_core.py Example 3 (0.9997), so I treated that as covered.
• Model-dependent. These verdicts are specific to granite-4.1-3b. If fix: intrinsic function signatures #1003 lands and changes Guardian signatures, a follow-up verification pass will be needed.

[planetf1]

Upstream follow-ups — @jakelorocco / @nrfulton

Two items surfaced during verification, out of scope here but flagging so nothing gets lost. Already queued, or should I open an issue?

1. docs/examples/intrinsics/context_relevance.py:17 — comment says "no context_relevance intrinsic for Granite 4.1", but ibm-granite/granitelib-rag-r1.0/context_relevance/granite-4.1-3b/ shipped lora/ + alora/ ~12h ago. Verified it loads and returns labels ('relevant', 'partially relevant', 'irrelevant'). I updated the same claim in AGENTS.md + prose docs here (991a3cbd); the example file feels like feat: update granite library examples to use Granite 4.1 3B adapters. #981-series work rather than Guardian migration.

2. mellea/stdlib/start_backend.py:315 — defaults to IBM_GRANITE_4_MICRO_3B, while start_session() and OllamaModelBackend default to IBM_GRANITE_4_1_3B. Likely missed by the feat: update granite library examples to use Granite 4.1 3B adapters. #981 sweep.

Neither blocks this PR merging.

[jakelorocco]

one small nit on the actual content.

@nrfulton @HendrikStrobelt, I didn't realize that GuardianCheck's were deprecated. Do we want to replace them with a requirement that utilizes the intrinsic? Or do we want to force end users to validate using intrinsics outside of requirement based validation loops?

[planetf1]

On the GuardianCheck-as-Requirement question: Guardian Intrinsics return a float score with no reasoning string, so there is no direct drop-in for the old GuardianCheck-in-RepairTemplateStrategy pattern. Suggest we merge what we have here and open a separate tracking issue for an intrinsic-backed Requirement subclass. #773 already proposes a groundedness Requirement that partially closes the gap. safety/README.md in this PR flags the gap explicitly so it is documented for users in the interim.

[planetf1]

@avinash2692 @AngeloDanducci — all CI is green, the jakelorocco design thread (GuardianCheck-as-Requirement) is closed, and the OpenAI/GraniteSwitch nit is addressed. Ready for your review when you have a moment.

[planetf1]

Heads-up: overlaps with PR #1028

PR #1028 (feat: normalize intrinsics interfaces) also edits mellea/stdlib/components/intrinsic/guardian.py and makes the same float → str annotation fix on factuality_detection / factuality_correction, plus a broader rewrite adding documents= and model_options= keyword args.

Suggest letting #1028 merge first — it has the better fix and this PR's guardian.py hunk becomes a no-op on rebase afterwards.

[planetf1]

Update post-review (2026-05-19)

Rebased onto upstream/main and addressed an internal review pass. Five new commits since the last reviews; the diff is now purely docs (the earlier guardian.py change was made redundant by #1037, which independently fixed the -> str annotations and added the new scoring_schema API).

What landed since you last looked:

• Post-refactor: get instructions from upstream guardian adapters #1037 docs sweep — target_role="user" → scoring_schema="user_prompt" across safety-guardrails.md and use-context-and-sessions.md; new SCORING_SCHEMA_BANK glossary entry; deprecation note for target_role.
• Two WARNINGs from the internal review — fixed a dead link to the deleted safety/guardian.py example in security-and-taint-tracking.md, and added the missing [hf] extra to the composite RAG example.
• Suggestion fixes — clarified that factuality_correction() returns whatever the model emits ("none" is a model-side convention, not an API contract); added a check_groundedness: bool = True parameter to the composite rag() to match the "(optional)" framing used in Step 5; noted the eager Granite weights download cost on the module-scope guardian_backend.
• Folded follow-ups F1 + F4 — added a "Full example" callout to safety-guardrails.md pointing at docs/examples/intrinsics/guardian_core.py and three companion scripts; replaced the SEXUAL_CONTENT-only migration callout with a full GuardianRisk → CRITERIA_BANK mapping table.
• F3 — Limitations section — surfaced two real Guardian Intrinsics gaps in the Mintlify docs that previously only lived in docs/examples/safety/README.md: (1) Intrinsics return scores not Requirement instances, so they can't drop into RepairTemplateStrategy; (2) they don't emit mellea.requirement metrics.

@jakelorocco — on your design question about whether to add a Requirement subclass backed by Intrinsics: the Limitations section now documents the gap explicitly and points users at the manual repair pattern in the meantime. The longer-term answer is tracked in #1071 (Intrinsic-backed Requirement for safety) and is partially addressed by the open #773 (feat: groundedness requirement). One straggler example file using the deprecated API is now tracked separately at #1094, blocked on #773. Happy to keep this PR focused on docs and let the architectural decision land in #1071 / #773.

[psschwei]

I think I brought this up in one of the comments too, but framed more generally: since the old functionality has been deprecated for two minor releases, would it make sense to just remove those parts of the docs, rather than keep references to them in? I could go either way, but it seems like since we're already doing a big refactor here, may as well just go the whole way and rip them out?

[planetf1]

Pinging reviewers - updates have been made - ready for review

[psschwei]

LGTM in general. Claude found a couple of nits: first two are are probably worth fixing before merge, issue 3 is cheap and protects external links, the rest are polish.

[psschwei]

Lost a real constraint here. The previous row said check_context_relevance is "only supported for granite-4.0, not granite-4.1." The new wording drops that and only describes the return type.

The PR body even reaffirms the constraint still holds: "context_relevance stays on 4.0 — 4.1 not supported for that intrinsic." But every example in this PR (incl. the intrinsics.md check_context_relevance snippet) now loads ibm-granite/granite-4.1-3b as the backend. Either:

• the constraint no longer holds (then the PR body should be updated), or
• the constraint still holds and this row should keep the warning + intrinsics.md example needs to switch its backend back to 4.0.

Worth resolving before merge — agents reading AGENTS.md will now happily generate 4.1 + check_context_relevance code that fails at runtime.

[psschwei]

This row points readers to a README that this PR also deletes (docs/examples/safety/README.md), so "see README for migration notes" dangles.

Two options:

• Drop the safety/ row entirely (the directory is empty after this PR), or
• Replace the parenthetical with a link to the new ../how-to/safety-guardrails page plus the tracking issue (feat: intrinsic-backed Requirement subclass for Guardian safety validation in repair loops #1071) for the RepairTemplateStrategy gap.

[psschwei]

Removing the /how-to/safety-guardrails → /advanced/security-and-taint-tracking redirect is correct (the new page is now at the former source path).

But the inverse redirect — /advanced/security-and-taint-tracking → /how-to/safety-guardrails — isn't added, even though /advanced/security-and-taint-tracking has been live and indexed for a while. Anyone with a bookmark or external link to the old page now hits a 404. Worth adding the reverse redirect in the same redirects block.

[psschwei]

Dropping "sexual_content" from the criteria list here (and again at line 477) is correct — CRITERIA_BANK has no such key, verified against mellea/stdlib/components/intrinsic/guardian.py.

Non-blocking suggestion: since GuardianRisk.SEXUAL_CONTENT did exist in the old API, readers migrating may search for it and find nothing. Consider a brief > Note: pointing them at the "custom free-text criteria" pattern in safety-guardrails.md for that risk category.

[psschwei]

Polish nit (very low priority): "Scores below 0.5 are safe; scores at or above 0.5 indicate risk detected" appears verbatim three times in this page (here, under "Check user input", and in troubleshooting). Once is enough — feel free to drop the duplicates.

Also: the "Pre-baked criteria" table lists keys in one order, and the example print(list(CRITERIA_BANK.keys())) output below it lists them in a different order. Cosmetic, but a reader double-checking will pause. Consider sorting both consistently.