Fix KFS multi-dimensional string tensor deserialization by atobiszei · Pull Request #4358 · openvinotoolkit/model_server

atobiszei · 2026-07-08T08:30:08Z

Both convertStringRequestToOVTensor (contents path) and convertBinaryExtensionStringFromBufferToNativeOVTensor (raw_input_contents path) hardcoded a 1D ov::Shape{N}, ignoring the shape field in the request.

For models with >1 string inputs this caused inference to fail.

Fix: read shape from request, use it when its product matches element count.

Add two new unit tests (StringInputsConversionKFSTest):

native_ov_string_2d_shape (contents path)
rawInputContents_native_ov_string_2d_shape (raw_input_contents path)

Problem originally detected on TFS API but since it is being removed it appears fix is needed for KFS as well.

CVS-187884

Both convertStringRequestToOVTensor (contents path) and convertBinaryExtensionStringFromBufferToNativeOVTensor (raw_input_contents path) hardcoded a 1D ov::Shape{N}, ignoring the shape field in the request. For models with string inputs of rank > 1 (e.g. [batch, seq_len]) this caused inference to fail with: 'model input (shape=[?,?]) and the tensor (shape=(N)) are incompatible' Fix: read shape from request, use it when its product matches element count; fall back to 1D otherwise to preserve backward compatibility. Add two new unit tests (StringInputsConversionKFSTest): - native_ov_string_2d_shape (contents path) - rawInputContents_native_ov_string_2d_shape (raw_input_contents path) Resolves CVS-187884

Copilot

Pull request overview

Note

Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.

Fixes KFS string tensor deserialization so multi-dimensional request shapes are preserved (when consistent with element count), preventing inference failures for models with >1D string inputs.

Changes:

Read and apply shape from KFSRequest::InferInputTensor for string inputs (contents + raw_input_contents) when shape product matches the parsed element count.
Fall back to 1D shape on mismatch to preserve backward compatibility.
Add unit tests verifying 2D string tensor shape handling for both KFS paths.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
src/test/tensor_conversion_test.cpp	Adds KFS-focused unit tests validating 2D string shape handling for both contents and raw buffer paths.
src/kfs_frontend/tensor_conversion.cpp	Updates KFS string conversion to honor request-provided shapes (with mismatch fallback).
src/kfs_frontend/kfs_utils.cpp	Updates raw buffer string conversion to honor request-provided shapes (with mismatch fallback).

+    int numElements = getBinaryInputsSize(src);
+    // Build ov::Shape from request shape, preserving multi-dimensional shapes (e.g. [1,5]).
+    ov::Shape shape;
+    if (src.shape_size() > 0) {
+        size_t shapeElements = 1;
+        for (int i = 0; i < src.shape_size(); i++) {
+            shape.push_back(static_cast<size_t>(src.shape().at(i)));
+            shapeElements *= static_cast<size_t>(src.shape().at(i));
+        }
+        if (static_cast<int>(shapeElements) != numElements) {
+            SPDLOG_DEBUG("String input shape product {} != contents size {}; falling back to 1D", shapeElements, numElements);
+            shape = ov::Shape{static_cast<size_t>(numElements)};
+        }
+    } else {
+        shape = ov::Shape{static_cast<size_t>(numElements)};
+    }


+    if (src.shape_size() > 0) {
+        size_t shapeElements = 1;
+        for (int i = 0; i < src.shape_size(); i++) {
+            shape.push_back(static_cast<size_t>(src.shape().at(i)));
+            shapeElements *= static_cast<size_t>(src.shape().at(i));
+        }
+        if (shapeElements != batchSize) {
+            SPDLOG_DEBUG("Input string shape mismatch: shape product {} != parsed string count {}", shapeElements, batchSize);
+            shape = ov::Shape{batchSize};
+        }
+    } else {
+        shape = ov::Shape{batchSize};
+    }


+        if (static_cast<int>(shapeElements) != numElements) {
+            SPDLOG_DEBUG("String input shape product {} != contents size {}; falling back to 1D", shapeElements, numElements);
+            shape = ov::Shape{static_cast<size_t>(numElements)};
+        }


Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

+template <>
+Status convertStringRequestToOVTensor<::KFSRequest::InferInputTensor>(const ::KFSRequest::InferInputTensor& src, ov::Tensor& tensor, const std::string* buffer) {
+    OVMS_PROFILE_FUNCTION();
+    if (buffer != nullptr) {
+        return convertBinaryExtensionStringFromBufferToNativeOVTensor(src, tensor, buffer);
+    }
+    int numElements = getBinaryInputsSize(src);


+        if (!validShape || shapeElements != static_cast<size_t>(numElements)) {
+            SPDLOG_DEBUG("String input shape product {} != contents size {}; rejecting request", shapeElements, numElements);
+            return StatusCode::INVALID_STRING_INPUT;
+        }
+    } else {
+        shape = ov::Shape{static_cast<size_t>(numElements)};
+    }


+    // Use shape from request if provided and its element count matches the parsed string count.
+    // This preserves multi-dimensional shapes (e.g. [1, 5] for batch=1, seq_len=5).
+    ov::Shape shape;
+    if (src.shape_size() > 0) {
+        size_t shapeElements = 1;
+        bool validShape = true;
+        for (int i = 0; i < src.shape_size() && validShape; i++) {
+            int64_t dim = src.shape().at(i);
+            if (dim < 0) {
+                validShape = false;
+                break;
+            }
+            size_t dimSize = static_cast<size_t>(dim);
+            if (dimSize != 0 && shapeElements > std::numeric_limits<size_t>::max() / dimSize) {
+                validShape = false;  // multiplication would overflow
+                break;
+            }
+            shape.push_back(dimSize);
+            shapeElements *= dimSize;
+        }
+        if (!validShape || shapeElements != batchSize) {
+            SPDLOG_DEBUG("Input string shape mismatch: shape product {} != parsed string count {}; rejecting request", shapeElements, batchSize);
+            return StatusCode::INVALID_STRING_INPUT;
+        }
+    } else {
+        shape = ov::Shape{batchSize};
+    }


+TEST(StringInputsConversionKFSTest, native_ov_string_shape_mismatch_invalid) {
+    // Shape product [2,5]=10 does not match 3 provided strings - request must be rejected
+    ::KFSRequest::InferInputTensor requestTensor;
+    requestTensor.set_datatype("BYTES");
+    requestTensor.add_shape(2);
+    requestTensor.add_shape(5);
+    std::vector<std::string> strings = {"aa", "bbb", "c"};
+    for (const auto& s : strings) {
+        auto bytes_val = requestTensor.mutable_contents()->mutable_bytes_contents()->Add();
+        bytes_val->append(s.data(), s.size());
+    }
+    ov::Tensor tensor;
+    ASSERT_EQ(convertStringRequestToOVTensor(requestTensor, tensor, nullptr), ovms::StatusCode::INVALID_STRING_INPUT);
+}


+TEST(StringInputsConversionKFSTest, rawInputContents_native_ov_string_shape_mismatch_invalid) {
+    // Shape product [2,5]=10 does not match 3 strings in buffer - request must be rejected
+    ::KFSRequest::InferInputTensor requestTensor;
+    requestTensor.set_datatype("BYTES");
+    requestTensor.add_shape(2);
+    requestTensor.add_shape(5);
+    std::vector<std::string> strings = {"a", "bb", "ccc"};
+    std::string rawInputContents;
+    size_t dataSize = 0;
+    for (const auto& s : strings) {
+        dataSize += s.size() + sizeof(uint32_t);
+    }
+    rawInputContents.resize(dataSize);
+    size_t offset = 0;
+    for (const auto& s : strings) {
+        uint32_t inputSize = s.size();
+        std::memcpy(rawInputContents.data() + offset, &inputSize, sizeof(uint32_t));
+        offset += sizeof(uint32_t);
+        std::memcpy(rawInputContents.data() + offset, s.data(), s.size());
+        offset += s.size();
+    }
+    ov::Tensor tensor;
+    ASSERT_EQ(convertStringRequestToOVTensor(requestTensor, tensor, &rawInputContents), ovms::StatusCode::INVALID_STRING_INPUT);
+}


Add forward declaration of inference::ModelInferRequest_InferInputTensor and explicit specialization declaration for convertStringRequestToOVTensor in tensor_conversion.hpp so TUs that include the header suppress primary template instantiation for KFSRequest::InferInputTensor instead of using the old 1D-only body.

atobiszei added 2 commits July 7, 2026 15:15

Self-review

a5b1c09

atobiszei requested a review from Copilot July 8, 2026 08:40

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Copilot started reviewing on behalf of atobiszei July 8, 2026 09:05 View session

Review fixes

4ec1280

atobiszei requested a review from Copilot July 8, 2026 12:54

Copilot started reviewing on behalf of atobiszei July 8, 2026 12:55 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix KFS multi-dimensional string tensor deserialization#4358

Fix KFS multi-dimensional string tensor deserialization#4358
atobiszei wants to merge 4 commits into
mainfrom
atobisze_fix_2d_string

atobiszei commented Jul 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

atobiszei commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

atobiszei commented Jul 8, 2026 •

edited

Loading