Skip to content

test(samples/js): add Responses web service sample and integration coverage#671

Open
MaanavD wants to merge 11 commits intomainfrom
agents/update-responses-api-sdk-vision-support
Open

test(samples/js): add Responses web service sample and integration coverage#671
MaanavD wants to merge 11 commits intomainfrom
agents/update-responses-api-sdk-vision-support

Conversation

@MaanavD
Copy link
Copy Markdown
Collaborator

@MaanavD MaanavD commented Apr 23, 2026 •
edited
Loading

Summary

Reworks this PR to avoid changing the JavaScript SDK API surface. The PR now focuses on using an existing Foundry Local OpenAI-compatible web service for Responses API scenarios, similar to the existing chat-completions web-server sample.

Changes

  • Restored the JS SDK Responses client/types/factory surface to the main-branch baseline.
  • Added samples/js/web-server-responses, a JavaScript sample that uses the OpenAI JS SDK against a running Foundry Local /v1 web service and demonstrates:
    • openai.responses.create() non-streaming calls,
    • streaming Responses events,
    • Responses tool calling with function_call_output follow-up.
  • Added sdk/js/test/openai/responsesWebService.test.ts integration coverage that calls /v1/responses directly with fetch for:
    • non-streaming response creation,
    • SSE streaming events,
    • function calling over the web service.
  • The web-service integration tests do not instantiate FoundryLocalManager or the JS native addon. They require an already-running service and are enabled with:
    • FOUNDRY_LOCAL_RESPONSES_ENDPOINT or FOUNDRY_LOCAL_ENDPOINT
    • FOUNDRY_LOCAL_RESPONSES_MODEL or FOUNDRY_LOCAL_MODEL
  • Updated sample READMEs to list the new JavaScript sample.

Local validation

npm run build: passed
npx mocha --import=tsx test/openai/responsesClient.test.ts test/openai/responsesWebService.test.ts --grep "Integration" --invert: 29 passing
npx mocha --import=tsx test/openai/responsesWebService.test.ts --grep "Integration": 3 pending (skipped because no external web-service env vars were set)

@MaanavD
feat(sdk/js): add vision support, list(), and missing types to Respon…
e1ad530 
…ses API client

- Add InputImageContent and InputFileContent content part types
- Expand ContentPart union to include new types
- Add 7 new reasoning/annotation streaming event interfaces and expand StreamingEvent union
- Add ListResponsesResult type
- Add list() method to ResponsesClient (GET /v1/responses)
- Default store to true in ResponsesClientSettings._serialize()
- Add vision.ts with createImageContentFromFile and createImageContentFromUrl helpers
- Export vision helpers from index.ts
- Add unit tests: vision helpers, list(), reasoning event types, store default
- Add integration tests for list() and vision (skipped when addon unavailable)
- Add IS_NATIVE_ADDON_AVAILABLE guard to testUtils to skip integration tests gracefully
- Update responses.ts example with list() and vision examples

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 23, 2026 16:54
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 23, 2026 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
foundry-local Ready Ready Preview, Comment May 1, 2026 10:04pm

Request Review

@MaanavD MaanavD mentioned this pull request Apr 23, 2026
Closed
Copilot AI reviewed Apr 23, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the JS/TS SDK’s Responses API support by adding multimodal (vision) input helpers, a list() endpoint wrapper, and additional missing streaming event/type definitions to better align with the service contract.

Changes:

  • Added new content-part and streaming event types (image/file inputs + reasoning/annotation events) and a ListResponsesResult type.
  • Added ResponsesClient.list() (GET /v1/responses) and changed ResponsesClientSettings._serialize() so store defaults to true.
  • Introduced createImageContentFromFile() / createImageContentFromUrl() helpers, exported via src/index.ts, and expanded tests + examples accordingly.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
sdk/js/test/testUtils.ts Adds IS_NATIVE_ADDON_AVAILABLE detection to help skip integration tests when the native addon isn’t present.
sdk/js/test/openai/responsesClient.test.ts Adds unit/integration tests for vision helpers, list(), new streaming event types, and updates store-default expectations.
sdk/js/src/types.ts Adds image/file content parts, reasoning/annotation streaming event interfaces, extends unions, and defines ListResponsesResult.
sdk/js/src/openai/vision.ts New helper module to build InputImageContent parts from files or URLs.
sdk/js/src/openai/responsesClient.ts Adds list() and changes serialization default for store.
sdk/js/src/index.ts Exports the new vision helper functions from the public SDK surface.
sdk/js/examples/responses.ts Adds examples demonstrating listing stored responses and sending vision input.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sdk/js/test/testUtils.ts Outdated
Comment thread sdk/js/test/openai/responsesClient.test.ts Outdated
Comment thread sdk/js/test/openai/responsesClient.test.ts Outdated
Comment thread sdk/js/src/openai/responsesClient.ts Outdated
apsonawane
apsonawane reviewed Apr 23, 2026
View reviewed changes
Comment thread sdk/js/src/openai/vision.ts Outdated
apsonawane
apsonawane reviewed Apr 23, 2026
View reviewed changes
Comment thread sdk/js/src/openai/vision.ts Outdated
apsonawane
apsonawane reviewed Apr 23, 2026
View reviewed changes
Comment thread sdk/js/src/openai/vision.ts Outdated
apsonawane
apsonawane reviewed Apr 23, 2026
View reviewed changes
Comment thread sdk/js/src/types.ts Outdated
apsonawane
apsonawane reviewed Apr 23, 2026
View reviewed changes
Comment thread sdk/js/src/openai/vision.ts Outdated
@MaanavD
fix(sdk/js): address PR review comments
e831369 
- Make media_type optional in InputImageContent (server can infer)
- Add file existence check in createImageContentFromFile
- Add bmp support to MEDIA_TYPE_MAP
- Add optional maxDimension resize via soft-peer sharp dependency
- Omit media_type in createImageContentFromUrl (server infers from URL)
- Add JSDoc on ResponsesClientSettings.store documenting default=true
- Replace FoundryLocalManager.create() in checkNativeAddonAvailable with
  file-existence checks (avoids side effects)
- Replace unreachable-server list() test with globalThis.fetch mock
- Use fs.mkdtempSync for unique temp dirs in file-based tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@apsonawane apsonawane requested a review from Copilot April 24, 2026 00:00
Copilot AI reviewed Apr 24, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sdk/js/test/testUtils.ts Outdated
Comment thread sdk/js/src/openai/vision.ts Outdated
Comment thread sdk/js/src/openai/vision.ts Outdated
Comment thread sdk/js/src/openai/vision.ts Outdated
apsonawane
apsonawane reviewed Apr 24, 2026
View reviewed changes
Comment thread sdk/js/src/openai/vision.ts Outdated
@MaanavD
fix(sdk/js): address second round of PR review comments
e092c2b 
- vision.ts: use fs.promises.readFile (async) instead of sync existsSync/readFileSync
- vision.ts: validate maxDimension is a finite positive integer before use
- vision.ts: update JSDoc to reflect overloaded options param (object or detail string)
- vision.ts: resizeImage returns { buffer, mediaType } so media_type is explicit post-resize
- vision.ts: pass fallbackMediaType into resizeImage to avoid hardcoded 'image/png'
- testUtils.ts: check all 4 candidate addon paths (sdk/js/prebuilds, sdk/js/native,
  sdk/js/dist/prebuilds, sdk/js/dist/native) matching CoreInterop.loadAddon() logic

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
baijumeswani
baijumeswani reviewed Apr 27, 2026
View reviewed changes
Comment thread sdk/js/src/types.ts Outdated
baijumeswani
baijumeswani reviewed Apr 27, 2026
View reviewed changes
Comment thread sdk/js/examples/responses.ts Outdated
baijumeswani
baijumeswani reviewed Apr 27, 2026
View reviewed changes
Comment thread sdk/js/src/openai/responsesClient.ts Outdated
baijumeswani
baijumeswani reviewed Apr 27, 2026
View reviewed changes
Comment thread sdk/js/src/types.ts Outdated
baijumeswani
baijumeswani reviewed Apr 27, 2026
View reviewed changes
Comment thread sdk/js/test/testUtils.ts Outdated
baijumeswani
baijumeswani reviewed Apr 27, 2026
View reviewed changes
Comment thread sdk/js/test/openai/responsesClient.test.ts Outdated
@MaanavD
fix(sdk/js): address latest responses API review feedback
88f0a0d 
- Add list pagination options and response metadata fields
- Align streaming event types with server DTOs for reasoning, annotations, refusals, content parts, and function calls
- Make responses example self-contained by generating a temporary PNG
- Document Responses API store default, vision formats including BMP, and Foundry Local image_data contract
- Remove native-addon availability auto-skip from Responses integration tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
baijumeswani
baijumeswani previously approved these changes Apr 27, 2026
View reviewed changes
@MaanavD MaanavD mentioned this pull request Apr 28, 2026
Draft
@MaanavD
feat(sdk/js): add responses FFI fallback
a707ee4 
Use the existing chat_completions native command for Responses create and streaming calls when a CoreInterop transport is available, with HTTP fallback for server-backed operations. Keep FFI-created stored responses available through the same client instance and cover the behavior with unit tests.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MaanavD
refactor(sdk/js): focus responses PR on web service usage
edf1db7 
Revert the SDK API surface changes from this branch and add a JavaScript web-service Responses sample plus integration coverage for direct /v1/responses calls, streaming, and tool calling.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MaanavD MaanavD changed the title feat(sdk/js): add vision support, list(), and missing types to Responses API client test(samples/js): add Responses web service sample and integration coverage Apr 30, 2026
@MaanavD
test(sdk/js): use external responses web service
aba3b03 
Make the JavaScript Responses web-service sample and integration tests call an already-running OpenAI-compatible Foundry Local web server without using the JS native addon.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MaanavD
fix(samples/js): use FoundryLocalManager for setup; OpenAI SDK for re…
092d4c4 
…sponses calls

Match the C# web-server sample pattern: FoundryLocalManager handles SDK init, EP download, model download/load, and startWebService. The OpenAI JS SDK (openai npm package) makes all Responses API calls against the running endpoint.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MaanavD
fix: use baseUrl in responsesWebService integration test fetch calls
719d25f 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@MaanavD
docs: add Responses web service sample README
785399f 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
MaanavD pushed a commit that referenced this pull request May 3, 2026
Align Responses sample/tests with JS PR #671 patterns; all tests pass…
39c8488 
… locally

Cross-references the JS Responses sample/tests (PR #671) to keep the C# pattern consistent.

Sample (samples/cs/responses-foundry-local-web-server):
- Added README.md mirroring the JS sample (prereqs, run, expected output, troubleshooting)
- Tool now uses an empty-params schema (matches JS PR), which the small qwen2.5-0.5b reliably calls
- Single ResponseTool reused on the follow-up call; deterministic options (Temperature=0, MaxOutputTokenCount=64)
- Cleanup wrapped in try/finally so StopWebService/Unload run even on exceptions

Integration tests (sdk/cs/test/FoundryLocal.Tests/ResponsesIntegrationTests.cs):
- Mirrors the JS suite responsesWebService.test.ts (NonStreaming, Streaming, FunctionCalling)
- Skips when Utils.IsRunningInCI() is true and when qwen2.5-0.5b is not pre-cached
- Streaming asserts response.created, response.output_text.delta, and response.completed events (parity with JS)
- Tool-calling test reuses the same get_weather empty-params definition
- Streaming options include StreamingEnabled = true so the official ResponsesClient allows the call

Pre-existing fix (test infra only):
- Utils.GetRepoRoot() previously failed in git worktrees because .git is a file, not a directory; now accepts either form. This unblocked test execution in worktree checkouts.

Validation:
- dotnet build samples/cs/responses-foundry-local-web-server -c Release: 0 warnings, 0 errors
- dotnet build sdk/cs/test/FoundryLocal.Tests -c Release: 0 errors
- dotnet test --filter ResponsesIntegration: all 3 Responses tests pass end-to-end against a real local model
- The 10 remaining failures across the project are pre-existing EmbeddingClientTests infra (different model not cached), unrelated to this PR

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@apsonawane apsonawane apsonawane left review comments

Copilot code review Copilot Copilot left review comments

@baijumeswani baijumeswani baijumeswani left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@MaanavD @baijumeswani @apsonawane