Skip to content

Added embedding API to C++ SDK#668

Open
phanindraraja wants to merge 9 commits intomainfrom
user/rchava/embedding-support-cpp-sdk
Open

Added embedding API to C++ SDK#668
phanindraraja wants to merge 9 commits intomainfrom
user/rchava/embedding-support-cpp-sdk

Conversation

@phanindraraja
Copy link
Copy Markdown
Member

@phanindraraja phanindraraja commented Apr 23, 2026

Summary

Adds embedding support to the C++ SDK and aligns the embedding samples across all SDK languages to a consistent initialization sequence. Includes optimizations to the existing embedding client implementations.

Changes

C++ SDK — embedding client

Public OpenAIEmbeddingClient exposing GenerateEmbedding (single input) and GenerateEmbeddings (batch input). Empty and whitespace-only inputs are rejected client-side. Responses parse into strongly-typed EmbeddingCreateResponse / EmbeddingObject / EmbeddingUsage structs via nlohmann/json.

  • Tests: 13 unit tests with a mocked core (no native DLL required) and one E2E test that exercises the full download → load → embed → unload flow against a real model.
  • In-tree sample: sdk/cpp/sample/main.cpp gains an embeddings example. Manager initialization uses a Configuration object that supports passing core-level settings such as AzureCatalogFilter.
  • Standalone sample: samples/cpp/embeddings/{main.cpp,README.md} provides a top-level entry point for the embeddings demo.
  • Build: /EHsc is enabled globally so the wil exception-based APIs used by core.h resolve correctly.

Embedding sample alignment

Every embedding sample runs the same seven-step sequence: create the manager → discover execution providers (prints a table) → download and register execution providers → get the catalog → get the model → download the model → load the model. The model alias is qwen3-embedding-0.6b across all samples.

Optimizations to existing embedding clients

  • Materialize the batch input collection once before validation and serialization, avoiding multiple enumerations.
  • Rename a setup-method local that shadowed a class-level static.
  • Remove a test util that modified test data model config.
  • Replace a value-discarding map call with an idiomatic insert-if-absent pattern when injecting missing response fields.
  • Drop redundant length checks on collections already covered by truthiness.
  • Remove a stale TOC entry from the API docs.

Testing

  • C++ unit tests: 13/13 pass under ctest --preset x64-debug.
  • C++ E2E: runs locally with the native Core DLL staged alongside the test binary and --gtest_also_run_disabled_tests.
  • All embedding integration surfaces are unchanged from the user's perspective.

@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 23, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
foundry-local Ready Ready Preview, Comment May 5, 2026 8:44pm

Request Review

@phanindraraja phanindraraja changed the title Add embedding support for cpp SDK Added embedding API to C++ SDK Apr 23, 2026
@phanindraraja phanindraraja marked this pull request as ready for review April 23, 2026 19:42
Copilot AI review requested due to automatic review settings April 23, 2026 19:42
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an OpenAI-compatible embeddings client to the C++ SDK and aligns surrounding SDK/sample code to support embedding usage consistently across languages.

Changes:

  • Introduces OpenAIEmbeddingClient (C++), including JSON parsing for embedding responses and CMake integration.
  • Adds C++ unit tests + disabled E2E coverage for embeddings; updates the C++ sample to demonstrate single and batch embeddings.
  • Minor cleanups/alignments in Rust/Python/C# embedding clients/tests/docs and updates samples to use the newer embedding alias and to download/register execution providers before running.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated no comments.

Show a summary per file
File Description
sdk/rust/src/openai/embedding_client.rs Simplifies/clarifies response “patching” before deserializing into async_openai types.
sdk/rust/docs/api.md Removes outdated TOC entry for an embedding response type.
sdk/python/src/openai/embedding_client.py Simplifies empty-input check (if not inputs).
sdk/cs/test/FoundryLocal.Tests/Utils.cs Removes the model-config max_length patch helper (no longer used).
sdk/cs/test/FoundryLocal.Tests/EmbeddingClientTests.cs Removes max_length patch usage; minor variable rename/cleanup.
sdk/cs/src/OpenAI/EmbeddingClient.cs Avoids multiple enumeration by materializing inputs once and validating the list.
sdk/cpp/test/e2e_test.cpp Adds disabled E2E test for downloading/loading/unloading and generating embeddings (single + batch).
sdk/cpp/test/client_test.cpp Adds unit tests validating request format, response parsing, and input/core-error handling for embeddings.
sdk/cpp/src/parser.h Adds from_json parsers for embedding response structs.
sdk/cpp/src/openai_embedding_client.cpp Implements the C++ embeddings client over the native embeddings command.
sdk/cpp/sample/main.cpp Adds a runnable embeddings example (single + batch).
sdk/cpp/include/openai/openai_embedding_client.h New public C++ header exposing embedding client + response types.
sdk/cpp/include/model.h Grants embedding client access to IModel::CoreAccess (friend + forward decl).
sdk/cpp/include/foundry_local.h Exposes embeddings client via umbrella header include.
sdk/cpp/CMakeLists.txt Enables /EHsc on MSVC and adds the new embeddings source file to the static library.
samples/rust/embeddings/src/main.rs Updates embedding alias and adds EP discovery + download/register with progress output.
samples/python/embeddings/src/app.py Updates embedding alias and adds EP discovery + download/register with progress output.
samples/js/embeddings/app.js Updates embedding alias and adds EP discovery + download/register with progress output.
samples/cs/embeddings/Program.cs Updates embedding alias and adds EP discovery + download/register with progress output.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants