Skip to content

feat: reasoning, web search, and thinking event support#62

Open
erezcor wants to merge 1 commit intoCopilotKit:mainfrom
erezcor:feat/reasoning-and-web-search-events
Open

feat: reasoning, web search, and thinking event support#62
erezcor wants to merge 1 commit intoCopilotKit:mainfrom
erezcor:feat/reasoning-and-web-search-events

Conversation

@erezcor
Copy link
Copy Markdown

@erezcor erezcor commented Mar 30, 2026

Summary

Adds support for reasoning and web search events in fixture responses across multiple providers:

  • OpenAI Responses API: Optional reasoning field on TextResponse emits response.reasoning_summary_text.delta events before text content. Optional webSearches field emits response.web_search_call events.
  • Anthropic Claude: reasoning field emits thinking content blocks before text in /v1/messages.
  • Both HTTP SSE and WebSocket transports supported.
  • Backward compatible — existing fixtures work unchanged.

Closes #60

Changes

  • src/types.ts: TextResponse extended with optional reasoning?: string and webSearches?: string[]
  • src/responses.ts: buildTextStreamEvents accepts optional reasoning/webSearches and prepends the appropriate events with correct output_index adjustment. Private helpers buildReasoningStreamEvents and buildWebSearchStreamEvents handle individual event sequences. Non-streaming buildTextResponse includes reasoning and web search output items.
  • src/messages.ts: Emits thinking content blocks when reasoning is present (both streaming and non-streaming)
  • src/ws-responses.ts: Passes reasoning/webSearches through to buildTextStreamEvents (no duplicated logic)
  • src/fixture-loader.ts: Validates reasoning (string) and webSearches (array of strings)
  • src/stream-collapse.ts: Extracts reasoning from response.reasoning_summary_text.delta and web searches from response.output_item.done web_search_call events. Claude collapse extracts reasoning from thinking_delta events.
  • src/index.ts: Removed unexported builders from public API

Example

mock.addFixture({
  match: { userMessage: "hello" },
  response: {
    content: "Based on my research, here's what I found.",
    reasoning: "Let me think about what the user is asking...",
    webSearches: ["latest greeting conventions 2025"],
  },
});

Test plan

  • Reasoning events emitted correctly in Responses API SSE
  • Reasoning deltas reconstruct full text
  • Web search events emitted before text events
  • Web search items contain query strings
  • Combined reasoning + web search + text streaming
  • Non-streaming responses include reasoning and web search output items
  • Anthropic thinking blocks (streaming and non-streaming)
  • Backward compatibility — plain text fixtures unchanged
  • All 1322 existing tests pass
  • Prettier, ESLint clean

Copy link
Copy Markdown

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

Copy link
Copy Markdown
Contributor

@jpr5 jpr5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — 7-Agent Standard CR

878 lines across 7 files reviewed by 7 specialized agents.

Overall the PR is well-structured — the data flows are correctly traced, output indices are managed properly, and backward compatibility is maintained. Two bugs and three critical test gaps need addressing before merge.


Bugs

1. webSearches silently dropped for Claude Messages API

handleMessages() in src/messages.ts passes response.reasoning but never response.webSearches. A fixture with webSearches served via /v1/messages silently drops them — zero warning, zero log, zero error.

A user who defines a fixture with webSearches and tests via the Responses API will see it work correctly, then switch to Claude Messages API and see the web searches vanish with no indication anything was omitted.

Fix (minimum): Log a warning in handleMessages when response.webSearches is present:

if (response.webSearches?.length) {
  logger.warn("webSearches in fixture response are not supported for Claude Messages API — ignoring");
}

Or add a fixture-loader validation warning that webSearches only applies to the Responses API.

2. reasoning: "" silently treated as no reasoning

Both buildTextStreamEvents and buildClaudeTextStreamEvents use if (reasoning) — empty string is falsy in JS, so reasoning: "" produces zero events. Meanwhile, fixture-loader.ts validates it as a legal string. Gap between validation and runtime.

Fix: Add a validation warning for reasoning: "":

if (response.reasoning === "") {
  results.push({ severity: "warning", fixtureIndex: i, message: "reasoning is empty string — no reasoning events will be emitted" });
}

Same pattern as existing content: "" validation.


Missing Test Coverage (critical gaps)

3. No stream-collapse tests for reasoning/webSearch extraction

collapseOpenAISSE gained reasoning delta accumulation + web search query extraction from response.output_item.done. collapseAnthropicSSE gained thinking_delta extraction. Zero tests cover any of this. These functions power record-and-replay — wrong extraction means silent data loss in recorded fixtures.

Need tests for:

  • collapseOpenAISSE with reasoning summary events → result.reasoning
  • collapseOpenAISSE with web search output_item.done events → result.webSearches
  • collapseAnthropicSSE with thinking_delta events → result.reasoning

4. No fixture-loader validation tests

27 lines of new validation logic (reasoning type check, webSearches array check, element type check) with zero tests.

Need tests for:

  • reasoning: 123 → error
  • webSearches: "not-array" → error
  • webSearches: ["valid", 42] → error on element
  • Valid values → no errors

5. No WebSocket reasoning passthrough test

ws-responses.ts was modified to pass reasoning/webSearches through to buildTextStreamEvents — untested plumbing.


PR Description Fix

The PR body lists src/index.ts as changed with "Removed unexported builders from public API" — but index.ts is not in the diff. buildTextStreamEvents is still exported. Please update the description.


Non-blocking Notes

  • buildReasoningStreamEvents hardcodes output_index: 0 while buildWebSearchStreamEvents takes startOutputIndex — asymmetric but correct today since reasoning is always first
  • webSearches: string[] could be Array<{ query: string }> for extensibility — noted for future
  • Consider webSearches: [] and webSearches: [""] validation warnings (empty array / empty string elements)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support reasoning and web search events in Responses API fixtures

2 participants