Skip to content

Update Milvus benchmark results (v2.6.14) and add SQ4U support#746

Merged
liliu-z merged 4 commits intozilliztech:mainfrom
CLiqing:update-milvus-benchmark
Apr 3, 2026
Merged

Update Milvus benchmark results (v2.6.14) and add SQ4U support#746
liliu-z merged 4 commits intozilliztech:mainfrom
CLiqing:update-milvus-benchmark

Conversation

@CLiqing
Copy link
Copy Markdown
Contributor

@CLiqing CLiqing commented Apr 3, 2026

Summary

  • Add SQ4U scalar quantization type support for Milvus HNSW index
  • Update Milvus benchmark results with v2.6.14 (16c64g, force_merge)
  • Add ElasticCloud and ZillizCloud benchmark results

Milvus Benchmark Details

Environment: 16 vCPU, 64GB RAM, Intel Xeon 8375C @ 2.9GHz
Version: Milvus 2.6.14
Index: HNSW_SQ with force_merge (single segment)
Dataset: Cohere 768-dim, 1M and 10M

Configurations (32 total)

Dataset Index Variable Values
1M SQ4U+FP16 refine_k 1.0, 1.2, 1.5, 2.0, 2.5, 3.0, 4.0, 5.0
1M SQ8 ef 100, 120, 150, 200, 250, 300, 400, 500
10M SQ4U+FP16 ef 100, 120, 150, 200, 250, 300, 400, 500
10M SQ8 ef 100, 120, 150, 200, 250, 300, 400, 500

Key Results

Dataset Config Best QPS Recall Range
1M SQ4U+FP16 10,663 0.84 - 0.98
1M SQ8 5,973 0.92 - 0.98
10M SQ4U+FP16 3,917 0.92 - 0.98
10M SQ8 2,747 0.92 - 0.97

🤖 Generated with Claude Code

Ubuntu and others added 3 commits April 3, 2026 09:34
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Milvus results (16c64g, force_merge, v2.6.14):
- 1M Cohere: SQ4U+FP16 (sweep refine_k) + SQ8 (sweep ef), 8 points each
- 10M Cohere: SQ4U+FP16 + SQ8 (sweep ef), 8 points each
- Total 32 benchmark configurations

ElasticCloud and ZillizCloud results from standard benchmark runs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rename ElasticCloud and ZillizCloud result files from 20260209 to 20260403
and update task_label to standard_20260403 for consistency with Milvus results.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@CLiqing CLiqing force-pushed the update-milvus-benchmark branch from 40dcbf3 to 6edf702 Compare April 3, 2026 09:36
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@CLiqing CLiqing force-pushed the update-milvus-benchmark branch from 6edf702 to e16cb6c Compare April 3, 2026 10:31
Copy link
Copy Markdown
Collaborator

@liliu-z liliu-z left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@sre-ci-robot
Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: CLiqing, liliu-z
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@liliu-z liliu-z merged commit 8f7d6bb into zilliztech:main Apr 3, 2026
4 checks passed
@CLiqing CLiqing deleted the update-milvus-benchmark branch April 8, 2026 06:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants