remove structured output#1846
Conversation
tim-inkeep
commented
Feb 9, 2026
tim-inkeep
commented
- Removed structuredOutput from validation schemas, model resolution, and CLI defaults
- Removed structuredOutput from UI forms and components
- Fixed broken model examples in docs
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: b56df64 The changes in this PR will be included in the next version bump. This PR includes changesets to release 10 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Removes the Structured Output column from the CLI Defaults table to reflect the removal of the structuredOutput model type in PR #1846.
Remove the Structured Output column from the CLI Defaults table in models.mdx to reflect the removal of the structuredOutput model type in PR #1846.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Review Summary
7 Key Findings | Risk: High
This PR removes the structuredOutput model configuration — a good simplification in principle — but the removal is incomplete. The runtime execution path was updated correctly, but numerous type definitions, SDK interfaces, CLI code generators, documentation, and example files still reference structuredOutput. This creates a "split world" where TypeScript allows configurations that are silently ignored at runtime.
🔴❗ Critical (3) ❗🔴
🔴 1) packages/agents-core/src/client-exports.ts Client-facing schema still has structuredOutput
Issue: The FullAgentDefinitionSchema Zod schema (lines 72-77) still includes structuredOutput in the models object definition. This is a client-facing export used by the manage-ui and other consumers.
Why: The schema contract exposed to users still accepts structuredOutput, which will be silently stripped or ignored at runtime. This inconsistency between the accepted schema and actual behavior is a contract violation that could confuse API consumers.
Fix: Remove the structuredOutput object definition from lines 72-77. The models object should only contain base and summarizer.
Refs:
🔴 2) packages/create-agents/src/utils.ts Scaffolding still generates structuredOutput config
Issue: The default model configurations (defaultGoogleModelConfigurations, defaultOpenaiModelConfigurations, defaultAnthropicModelConfigurations) at lines 46-79 and Azure configuration at lines 340-348 all still include structuredOutput definitions.
Why: New projects created with create-agents will be scaffolded with structuredOutput configuration that is silently ignored at runtime. This creates confusion for new users following the generated code and contradicts the migration guidance in the changeset.
Fix: Remove structuredOutput from all default model configurations in packages/create-agents/src/utils.ts.
Refs:
🔴 3) packages/agents-sdk/src/agent.ts Model inheritance code still propagates structuredOutput
Issue: The applyModelInheritance() method at lines 998-999 still propagates structuredOutput from project to agent. Since structuredOutput was removed from the core ModelSchema, this code path references a property that no longer exists in the validated schema.
Why: Creates dead code that could confuse maintainers. If a consumer sets structuredOutput locally on the agent before inheritance runs, it would remain but have no runtime effect, creating subtle behavioral inconsistencies.
Fix: Remove the structuredOutput inheritance logic:
// Remove:
if (!this.models.structuredOutput && projectModels.structuredOutput) {
this.models.structuredOutput = projectModels.structuredOutput;
}Refs:
Inline comments:
- 🔴 Critical:
templates.test.ts:256-265Duplicate summarizer key in test fixture - 🔴 Critical:
types.ts:263structuredOutput still in SDK types - 🔴 Critical:
project.ts:30structuredOutput still in ProjectConfig types
🟠⚠️ Major (3) 🟠⚠️
🟠 1) agents-docs/content/typescript-sdk/models.mdx Documentation still references structuredOutput
Issue: The "CLI Defaults" table at lines 308-312 still lists a "Structured Output" column with model recommendations for each provider. Additionally, the snippet at agents-docs/_snippets/multi-agent-framework/data-components-model-recommendations.mdx advises configuring "the structured output model" which no longer exists.
Why: Users following this documentation will expect a structuredOutput model configuration option that no longer exists. When they try to configure it per these docs, their configuration will be silently ignored.
Fix: Remove the "Structured Output" column from the CLI Defaults table. Update the data-components snippet to explain that data components now use the base model.
Refs:
🟠 2) .changeset/ Breaking change uses patch bump instead of minor
Issue: The changeset in excess-emerald-kite.md explicitly states "Breaking Change" and provides migration guidance, but uses patch version bumps for all affected packages. Per AGENTS.md: "Minor: Schema changes requiring migration, significant behavior changes."
Why: Users relying on semver to avoid breaking changes may be caught off guard. A patch version should not contain breaking changes.
Fix: Change the version bumps from patch to minor for all affected packages in both changeset files.
Refs:
🟠 3) .changeset/ Duplicate changesets covering same packages
Issue: There are two changeset files for this change: chatty-cobras-rhyme.md (terse message, 6 packages) and excess-emerald-kite.md (detailed message, 3 packages). Three packages (agents-core, agents-sdk, agents-cli) appear in both, which will result in duplicate changelog entries.
Why: Creates confusing changelog with redundant entries. The terse changeset also doesn't follow AGENTS.md style guidelines ("removed structured output" should be "Remove structuredOutput model configuration option").
Fix: Consolidate into a single changeset covering all 6 affected packages with the detailed migration guidance.
Refs:
Inline comments:
- 🟠 Major:
lib/types/project.ts:8structuredOutput still in UI types
🟡 Minor (1) 🟡
🟡 1) packages/agents-core/src/data-access/manage/projectFull.ts DEFAULT_MODELS still has structuredOutput
Issue: The DEFAULT_MODELS constant at line 1342 still includes structuredOutput: { model: 'claude-sonnet-4-5' }. This constant is used as a fallback when projects don't have models configured.
Why: Internal inconsistency. The default models still include a field that has been removed from the public API. New projects without explicit model configuration will have structuredOutput in their resolved models that is never used.
Fix: Remove structuredOutput from DEFAULT_MODELS.
Refs:
Inline comments:
- 🟡 Minor:
model-templates.ts:11-14Orphaned structuredOutputModelProviderOptionsTemplate - 🟡 Minor:
agent-generator.ts:138CLI still references structuredOutput in model comparison
💭 Consider (3) 💭
💭 1) packages/agents-sdk/example-project.ts:18 Example project still shows structuredOutput
Developers copying this example will configure a removed option. Remove the line.
💭 2) packages/agents-sdk/src/builderFunctions.ts:56 JSDoc example still shows structuredOutput
API documentation examples show a removed option. Update the JSDoc.
💭 3) agents-api/__snapshots__/openapi.json OpenAPI snapshot not regenerated
The snapshot still includes structuredOutput in Model schemas. Should be regenerated after schema changes are complete.
💡 APPROVE WITH SUGGESTIONS
Summary: The intent of this PR — simplifying the model configuration by removing structuredOutput — is good. However, the removal is incomplete. Multiple type definitions, SDK interfaces, code generators, and documentation still reference structuredOutput, creating a situation where configurations are accepted by TypeScript but silently ignored at runtime. The critical issues (SDK types, client-exports schema, create-agents scaffolding, inheritance code) should be addressed before merging. The changeset should also use minor instead of patch for a breaking change.
Discarded (5)
| Location | Issue | Reason Discarded |
|---|---|---|
Agent.test.ts:1500-1534 |
Test expectation may not match new implementation | Investigated — test uses mocks that make it pass; test name is misleading but test logic is valid |
model-options.tsx:8,12,16 |
Unused DEFAULT_*_STRUCTURED_OUTPUT_MODEL constants | Low impact — dead exports that will be tree-shaken; not blocking |
Agent.test.ts:1536 |
Test name misleading after removal | Naming inconsistency only — test logic is valid |
| Product semantic impact | Cost implications of always using base model | Informational — product decision already made, not actionable in this PR |
| OpenAPI snapshot regeneration | Snapshot contains stale structuredOutput | Marked as Consider — should be regenerated but blocked by completing other schema fixes first |
Reviewers (9)
| Reviewer | Returned | Main Findings | Consider | Inline Comments | Pending Recs | Discarded |
|---|---|---|---|---|---|---|
pr-review-consistency |
6 | 3 | 0 | 0 | 0 | 0 |
pr-review-breaking-changes |
14 | 2 | 0 | 2 | 0 | 0 |
pr-review-types |
13 | 1 | 2 | 2 | 0 | 1 |
pr-review-devops |
4 | 2 | 0 | 1 | 0 | 0 |
pr-review-product |
5 | 1 | 0 | 0 | 0 | 1 |
pr-review-docs |
1 | 0 | 0 | 0 | 0 | 0 |
pr-review-tests |
4 | 0 | 0 | 0 | 0 | 3 |
pr-review-standards |
1 | 0 | 0 | 1 | 0 | 0 |
pr-review-frontend |
3 | 0 | 0 | 0 | 0 | 0 |
| Total | 51 | 9 | 2 | 6 | 0 | 5 |
There was a problem hiding this comment.
PR Review Summary (Delta Re-review)
0 New Findings | 7 Pending from Previous Review | Risk: High (unchanged)
🆕 Delta Changes Reviewed
The only change since the last review is:
✅ docker-compose.yml — Port mapping corrected from 3003:3003 to 3002:3002
Assessment: This is a valid consistency fix. The API URL environment variables already reference port 3002 (lines 7-8), so the port mapping should match. No issues.
🕐 Pending Recommendations (7)
The following issues were raised in the previous review and remain unaddressed:
From previous review — Critical:
- 🔴
packages/agents-core/src/client-exports.ts:72-77— Client-facingFullAgentDefinitionSchemastill includesstructuredOutput - 🔴
packages/create-agents/src/utils.ts:46-79, 340-348— Scaffolding templates still generatestructuredOutputconfig - 🔴
packages/agents-sdk/src/agent.ts:998-999— Model inheritance code still propagatesstructuredOutput
From previous review — Major:
- 🟠
agents-docs/content/typescript-sdk/models.mdx:308-312— Docs CLI Defaults table still references structuredOutput (partially addressed: main table updated, but snippets may remain) - 🟠
.changeset/— Breaking change usespatchbump instead ofminor - 🟠
.changeset/— Duplicate changesets covering overlapping packages
From previous review — Minor:
- 🟡
packages/agents-core/src/data-access/manage/projectFull.ts:1342—DEFAULT_MODELSstill includesstructuredOutput
Additional locations still containing structuredOutput (verified):
packages/agents-sdk/src/types.ts:263— SDK type definitionpackages/agents-sdk/src/project.ts:30, 80, 102— Project type and JSDoc examplespackages/agents-sdk/src/agent.ts:54— Agent type definition
🚫 REQUEST CHANGES
Summary: The delta change (docker-compose port fix) is fine and can be merged. However, the core purpose of this PR — removing structuredOutput — remains incomplete. The runtime execution path was updated correctly, but type definitions, SDK interfaces, code generators, and some documentation still reference structuredOutput. This creates a "split world" where TypeScript accepts configurations that are silently ignored at runtime.
The previous review's critical findings should be addressed before merging to avoid shipping an inconsistent API contract.
Reviewers (0)
| Reviewer | Returned | Main Findings | Consider | Inline Comments | Pending Recs | Discarded |
|---|---|---|---|---|---|---|
| No subagents dispatched | — | — | — | — | — | — |
Delta-scoped re-review: Only 1 file changed (docker-compose.yml), no code review subagents needed.
There was a problem hiding this comment.
PR Review Summary
5 Key Findings | Risk: Medium
This is a delta re-review following previous feedback. The author addressed most critical issues from the first review — client-exports.ts, create-agents/utils.ts, SDK inheritance code, and docs were all updated correctly. However, several locations were missed and new issues were introduced during the fix.
🔴❗ Critical (1) ❗🔴
- 🔴 Critical:
templates.test.ts:259-264Duplicatesummarizerkey introduced when removingstructuredOutput— invalid test fixture
🟠⚠️ Major (3) 🟠⚠️
- 🟠 Major:
agent.ts:52-56Private field still includesstructuredOutputproperty - 🟠 Major:
project.ts:99-103Private field still includesstructuredOutputproperty - 🟠 Major:
project.ts:77-80JSDoc example shows removedstructuredOutputconfig
🟡 Minor (1) 🟡
- 🟡 Minor:
sub-agent-generator.ts:30Model comparison array still includesstructuredOutput
🕐 Pending Recommendations (1)
These issues were raised in the previous review and remain unaddressed:
- 🔴
.changeset/chatty-cobras-rhyme.md— Changeset usespatchbumps for a breaking change. Per AGENTS.md: "Minor: Schema changes requiring migration, significant behavior changes."
💡 APPROVE WITH SUGGESTIONS
Summary: The structuredOutput removal is ~90% complete. The critical test fixture bug (duplicate key) should be fixed, and the remaining SDK private fields and JSDoc example should be updated to match the public API. The changeset semver issue from the prior review also remains unaddressed — a breaking change should use minor version bumps, not patch. All findings have inline comments with 1-click suggestions where applicable.
Discarded (2)
| Location | Issue | Reason Discarded |
|---|---|---|
project-comparator.ts:587 |
CLI change detection still references structuredOutput | Pre-existing issue not introduced by this PR; file not in diff |
| MCP server packages | May contain stale structuredOutput in generated schemas | Auto-generated from OpenAPI; out of scope for manual PR changes |
Reviewers (4)
| Reviewer | Returned | Main Findings | Consider | While You're Here | Inline Comments | Pending Recs | Discarded |
|---|---|---|---|---|---|---|---|
pr-review-consistency |
3 | 0 | 0 | 0 | 3 | 0 | 0 |
pr-review-breaking-changes |
6 | 0 | 0 | 0 | 2 | 1 | 1 |
pr-review-tests |
1 | 0 | 0 | 0 | 1 | 0 | 0 |
pr-review-devops |
0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Total | 10 | 0 | 0 | 0 | 6 | 1 | 1 |
Note: All 5 main findings were posted as inline comments with 1-click suggestions where applicable. The inline comment count (6) includes one duplicate that was merged.
| summarizer: { | ||
| model: 'gpt-4o-mini', | ||
| }, | ||
| summarizer: { | ||
| model: 'gpt-4o-mini', | ||
| } |
There was a problem hiding this comment.
🔴 CRITICAL: Duplicate summarizer key in test fixture
Issue: The removal of structuredOutput appears to have accidentally introduced a duplicate summarizer key. Lines 259-261 add a new summarizer block (with wrong indentation), while lines 262-264 contain the original summarizer block.
Why: In JavaScript objects, duplicate keys silently override earlier occurrences. This creates invalid test data that could cause silent test failures or false positives. The template replacement logic being tested may behave unpredictably when encountering this malformed fixture.
Fix:
| summarizer: { | |
| model: 'gpt-4o-mini', | |
| }, | |
| summarizer: { | |
| model: 'gpt-4o-mini', | |
| } | |
| summarizer: { | |
| model: 'gpt-4o-mini', | |
| } |
Refs:
- agent-generator.ts:138 — correctly updated model types array
There was a problem hiding this comment.
PR Review Summary
0 New Findings | 2 Pending from Prior Reviews | Risk: Medium
This is a delta re-review scoped to changes since the last automated review. The delta consists of:
✅ agents-cli/src/commands/pull-v3/components/project-generator.ts — Improved validation logic for models.base check (now only validates when models exists, avoiding redundant error messages)
✅ agents-cli/src/commands/pull-v3/components/__tests__/project-generator.test.ts — Updated test expectations to match the corrected behavior
Assessment: The delta changes are valid. The validation logic fix is correct and the test updates appropriately align with the implementation change. No issues in the delta.
🕐 Pending Recommendations (2)
The following issues were raised in prior reviews and remain unaddressed:
-
🔴
packages/create-agents/src/__tests__/templates.test.ts:259-264— Critical: Duplicatesummarizerkey in test fixture. An inline comment with a 1-click suggestion was posted but not applied. -
🟠
.changeset/chatty-cobras-rhyme.md— Changeset usespatchbumps for a breaking change. Per AGENTS.md semver guidance: "Minor: Schema changes requiring migration, significant behavior changes."
💡 APPROVE WITH SUGGESTIONS
Summary: The delta changes since the last review are solid — the validation logic improvement in project-generator.ts is a correct bug fix. However, two prior issues remain unaddressed: (1) the critical duplicate summarizer key bug in templates.test.ts which creates an invalid test fixture, and (2) the semver concern about using patch instead of minor for a breaking change. The inline comment on the test file has a 1-click suggestion ready to apply.
Discarded (0)
No findings discarded in this delta review.
Reviewers (2)
| Reviewer | Returned | Main Findings | Consider | While You're Here | Inline Comments | Pending Recs | Discarded |
|---|---|---|---|---|---|---|---|
pr-review-tests |
0 | 0 | 0 | 0 | 0 | 0 | 0 |
pr-review-standards |
0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Total | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Note: Delta-scoped re-review with only 2 files changed. Both reviewers confirmed the delta changes are appropriate.
2 participants
