Skip to content

[WIP] feat(minimaxm3): add B200 disaggregated configs #2230

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
xinli-sw wants to merge 5 commits into
base: main
Choose a base branch
from
Open

[WIP] feat(minimaxm3): add B200 disaggregated configs #2230

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f345aa7
feat(minimaxm3): add B200 disaggregated configs
xinli-sw Jul 16, 2026
df27c5a
Merge branch 'main' into feat/minimaxm3-b200-vllm-disagg
xinli-sw Jul 16, 2026
9bdbbdd
newline
xinli-sw Jul 16, 2026
f3ae7fe
newline
xinli-sw Jul 16, 2026
f0a7050
Merge remote-tracking branch 'origin/main' into feat/minimaxm3-b200-v…
xinli-sw Jul 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions ...multi_node/srt-slurm-recipes/vllm/minimax-m3/b200-fp4/8k1k/1p1d-dep2-tp4-c1-c16-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
name: "minimax-m3-vllm-disagg-b200-1p1d-fp4-dep2-tp4-8k1k"
model:
path: "nvidia/MiniMax-M3-NVFP4"
container: "vllm/vllm-openai:cu129-nightly-8e981630c9336233ca9de91452f68918bddbc4e2"
precision: "fp4"
resources:
gpu_type: "b200"
gpus_per_node: 8
prefill_nodes: 1
decode_nodes: 1
prefill_workers: 1
decode_workers: 1
gpus_per_prefill: 2
gpus_per_decode: 4
dynamo: {install: true, version: 1.2.1}
frontend: {type: dynamo, enable_multiple_frontends: false}
backend:
type: vllm
connector: null
allow_prefill_decode_colocation: true
prefill_environment: {VLLM_FLOAT32_MATMUL_PRECISION: high, UCX_TLS: "cuda_ipc,cuda_copy,rc"}
decode_environment: {VLLM_FLOAT32_MATMUL_PRECISION: high, UCX_TLS: "cuda_ipc,cuda_copy,rc"}
vllm_config:
prefill:
served-model-name: nvidia/MiniMax-M3-NVFP4
no-enable-flashinfer-autotune: true
tensor-parallel-size: 1
data-parallel-size: 2
data-parallel-rpc-port: 13345
enable-expert-parallel: true
trust-remote-code: true
no-enable-prefix-caching: true
kv-transfer-config: '{"kv_connector": "NixlConnector", "kv_role": "kv_both"}'
attention-config: '{"backend": "FLASHINFER", "use_trtllm_attention": true, "indexer_kv_dtype": "fp8"}'
kv-cache-dtype: fp8
block-size: 128
gpu-memory-utilization: 0.90
max-model-len: 9472
language-model-only: true
stream-interval: 32
max-cudagraph-capture-size: 2048
max-num-batched-tokens: 16384
decode:
served-model-name: nvidia/MiniMax-M3-NVFP4
no-enable-flashinfer-autotune: true
tensor-parallel-size: 4
enable-expert-parallel: false
trust-remote-code: true
no-enable-prefix-caching: true
kv-transfer-config: '{"kv_connector": "NixlConnector", "kv_role": "kv_both"}'
attention-config: '{"backend": "FLASHINFER", "use_trtllm_attention": true, "indexer_kv_dtype": "fp8"}'
kv-cache-dtype: fp8
block-size: 128
gpu-memory-utilization: 0.90
max-model-len: 9472
language-model-only: true
stream-interval: 32
max-num-seqs: 1024
max-num-batched-tokens: 16384
max-cudagraph-capture-size: 2048
health_check: {max_attempts: 360, interval_seconds: 10}
benchmark: {type: "sa-bench", isl: 8192, osl: 1024, random_range_ratio: 0.8, concurrencies: "1x4x8x16", req_rate: "inf"}
127 changes: 127 additions & 0 deletions ...ks/multi_node/srt-slurm-recipes/vllm/minimax-m3/b200-fp4/8k1k/1p2d-dep2-tp4-c64-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
name: minimax-m3-vllm-disagg-b200-1p2d-fp4-dep2-tp4-c64-8k1k
model:
path: nvidia/MiniMax-M3-NVFP4
container: vllm/vllm-openai:cu129-nightly-8e981630c9336233ca9de91452f68918bddbc4e2
precision: fp4
resources:
gpu_type: b200
prefill_nodes: 1
prefill_workers: 1
gpus_per_prefill: 2
decode_nodes: 2
decode_workers: 2
spread_workers: true
gpus_per_decode: 4
gpus_per_node: 8
backend:
type: vllm
connector: null
prefill_environment:
VLLM_USE_DEEP_GEMM: '1'
VLLM_SKIP_P2P_CHECK: '1'
VLLM_RANDOMIZE_DP_DUMMY_INPUTS: '1'
NVIDIA_GDRCOPY: '1'
PYTHONUNBUFFERED: '1'
VLLM_LOG_STATS_INTERVAL: '1'
NVSHMEM_IB_ENABLE_IBGDA: '1'
NCCL_CUMEM_ENABLE: '1'
NCCL_MNNVL_ENABLE: '1'
NCCL_NVLS_ENABLE: '1'
NCCL_TIMEOUT: '1800'
TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: '1800'
VLLM_USE_FLASHINFER_MOE_FP4: '1'
VLLM_USE_TRTLLM_RAGGED_DEEPSEEK_PREFILL: '0'
VLLM_USE_NCCL_SYMM_MEM: '1'
UCX_IB_ROCE_REACHABILITY_MODE: local_subnet
VLLM_NIXL_SIDE_CHANNEL_PORT: '5600'
VLLM_NIXL_ABORT_REQUEST_TIMEOUT: '300'
VLLM_FLOAT32_MATMUL_PRECISION: high
UCX_TLS: cuda_ipc,cuda_copy,rc
decode_environment:
VLLM_USE_DEEP_GEMM: '1'
VLLM_SKIP_P2P_CHECK: '1'
VLLM_RANDOMIZE_DP_DUMMY_INPUTS: '1'
NVIDIA_GDRCOPY: '1'
PYTHONUNBUFFERED: '1'
VLLM_LOG_STATS_INTERVAL: '1'
NVSHMEM_IB_ENABLE_IBGDA: '1'
NCCL_CUMEM_ENABLE: '1'
NCCL_MNNVL_ENABLE: '1'
NCCL_NVLS_ENABLE: '1'
NCCL_TIMEOUT: '1800'
TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: '1800'
VLLM_USE_FLASHINFER_MOE_FP4: '1'
VLLM_USE_TRTLLM_RAGGED_DEEPSEEK_PREFILL: '0'
VLLM_USE_NCCL_SYMM_MEM: '1'
UCX_IB_ROCE_REACHABILITY_MODE: local_subnet
VLLM_NIXL_SIDE_CHANNEL_PORT: '5600'
VLLM_NIXL_ABORT_REQUEST_TIMEOUT: '300'
VLLM_MOE_DP_CHUNK_SIZE: '384'
VLLM_SHARED_EXPERTS_STREAM_TOKEN_THRESHOLD: '8192'
VLLM_FLOAT32_MATMUL_PRECISION: high
UCX_TLS: cuda_ipc,cuda_copy,rc
vllm_config:
prefill:
tensor-parallel-size: 1
pipeline-parallel-size: 1
enable-expert-parallel: true
data-parallel-size: 2
data-parallel-rpc-port: 13345
data-parallel-hybrid-lb: true
max-model-len: 9472
max-num-seqs: 16
enforce-eager: true
gpu-memory-utilization: 0.9
max-num-batched-tokens: 16384
no-enable-chunked-prefill: true
kv-cache-dtype: fp8
async-scheduling: true
no-enable-prefix-caching: true
trust-remote-code: true
served-model-name: nvidia/MiniMax-M3-NVFP4
no-enable-flashinfer-autotune: true
kv-transfer-config: '{"kv_connector": "NixlConnector", "kv_role": "kv_both"}'
attention-config: '{"backend": "FLASHINFER", "use_trtllm_attention": true, "indexer_kv_dtype": "fp8"}'
block-size: 128
language-model-only: true
stream-interval: 32
max-cudagraph-capture-size: 2048
decode:
tensor-parallel-size: 4
pipeline-parallel-size: 1
enable-expert-parallel: false
max-model-len: 9472
max-num-seqs: 32
gpu-memory-utilization: 0.9
max-num-batched-tokens: 16384
max-cudagraph-capture-size: 2048
compilation-config: '{"cudagraph_mode":"FULL_DECODE_ONLY","custom_ops":["+rms_norm"],"pass_config":{}}'
kv-cache-dtype: fp8
all2all-backend: deepep_low_latency
async-scheduling: true
stream-interval: 32
enable-dbo: true
dbo-decode-token-threshold: 32
no-enable-prefix-caching: true
trust-remote-code: true
served-model-name: nvidia/MiniMax-M3-NVFP4
no-enable-flashinfer-autotune: true
kv-transfer-config: '{"kv_connector": "NixlConnector", "kv_role": "kv_both"}'
attention-config: '{"backend": "FLASHINFER", "use_trtllm_attention": true, "indexer_kv_dtype": "fp8"}'
block-size: 128
language-model-only: true
benchmark:
type: sa-bench
isl: 8192
osl: 1024
concurrencies: '64'
req_rate: inf
frontend:
type: dynamo
enable_multiple_frontends: false
args:
dyn-chat-processor: vllm
reasoning-parser: minimax_m3
dynamo:
install: false
version: 1.2.1
127 changes: 127 additions & 0 deletions ...s/multi_node/srt-slurm-recipes/vllm/minimax-m3/b200-fp4/8k1k/2p2d-dep2-tp4-c128-8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
name: minimax-m3-vllm-disagg-b200-2p2d-fp4-dep2-tp4-c128-8k1k
model:
path: nvidia/MiniMax-M3-NVFP4
container: vllm/vllm-openai:cu129-nightly-8e981630c9336233ca9de91452f68918bddbc4e2
precision: fp4
resources:
gpu_type: b200
prefill_nodes: 1
prefill_workers: 2
gpus_per_prefill: 2
decode_nodes: 2
decode_workers: 2
spread_workers: true
gpus_per_decode: 4
gpus_per_node: 8
backend:
type: vllm
connector: null
prefill_environment:
VLLM_USE_DEEP_GEMM: '1'
VLLM_SKIP_P2P_CHECK: '1'
VLLM_RANDOMIZE_DP_DUMMY_INPUTS: '1'
NVIDIA_GDRCOPY: '1'
PYTHONUNBUFFERED: '1'
VLLM_LOG_STATS_INTERVAL: '1'
NVSHMEM_IB_ENABLE_IBGDA: '1'
NCCL_CUMEM_ENABLE: '1'
NCCL_MNNVL_ENABLE: '1'
NCCL_NVLS_ENABLE: '1'
NCCL_TIMEOUT: '1800'
TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: '1800'
VLLM_USE_FLASHINFER_MOE_FP4: '1'
VLLM_USE_TRTLLM_RAGGED_DEEPSEEK_PREFILL: '0'
VLLM_USE_NCCL_SYMM_MEM: '1'
UCX_IB_ROCE_REACHABILITY_MODE: local_subnet
VLLM_NIXL_SIDE_CHANNEL_PORT: '5600'
VLLM_NIXL_ABORT_REQUEST_TIMEOUT: '300'
VLLM_FLOAT32_MATMUL_PRECISION: high
UCX_TLS: cuda_ipc,cuda_copy,rc
decode_environment:
VLLM_USE_DEEP_GEMM: '1'
VLLM_SKIP_P2P_CHECK: '1'
VLLM_RANDOMIZE_DP_DUMMY_INPUTS: '1'
NVIDIA_GDRCOPY: '1'
PYTHONUNBUFFERED: '1'
VLLM_LOG_STATS_INTERVAL: '1'
NVSHMEM_IB_ENABLE_IBGDA: '1'
NCCL_CUMEM_ENABLE: '1'
NCCL_MNNVL_ENABLE: '1'
NCCL_NVLS_ENABLE: '1'
NCCL_TIMEOUT: '1800'
TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: '1800'
VLLM_USE_FLASHINFER_MOE_FP4: '1'
VLLM_USE_TRTLLM_RAGGED_DEEPSEEK_PREFILL: '0'
VLLM_USE_NCCL_SYMM_MEM: '1'
UCX_IB_ROCE_REACHABILITY_MODE: local_subnet
VLLM_NIXL_SIDE_CHANNEL_PORT: '5600'
VLLM_NIXL_ABORT_REQUEST_TIMEOUT: '300'
VLLM_MOE_DP_CHUNK_SIZE: '384'
VLLM_SHARED_EXPERTS_STREAM_TOKEN_THRESHOLD: '8192'
VLLM_FLOAT32_MATMUL_PRECISION: high
UCX_TLS: cuda_ipc,cuda_copy,rc
vllm_config:
prefill:
tensor-parallel-size: 1
pipeline-parallel-size: 1
enable-expert-parallel: true
data-parallel-size: 2
data-parallel-rpc-port: 13345
data-parallel-hybrid-lb: true
max-model-len: 9472
max-num-seqs: 16
enforce-eager: true
gpu-memory-utilization: 0.9
max-num-batched-tokens: 16384
no-enable-chunked-prefill: true
kv-cache-dtype: fp8
async-scheduling: true
no-enable-prefix-caching: true
trust-remote-code: true
served-model-name: nvidia/MiniMax-M3-NVFP4
no-enable-flashinfer-autotune: true
kv-transfer-config: '{"kv_connector": "NixlConnector", "kv_role": "kv_both"}'
attention-config: '{"backend": "FLASHINFER", "use_trtllm_attention": true, "indexer_kv_dtype": "fp8"}'
block-size: 128
language-model-only: true
stream-interval: 32
max-cudagraph-capture-size: 2048
decode:
tensor-parallel-size: 4
pipeline-parallel-size: 1
enable-expert-parallel: false
max-model-len: 9472
max-num-seqs: 64
gpu-memory-utilization: 0.9
max-num-batched-tokens: 16384
max-cudagraph-capture-size: 2048
compilation-config: '{"cudagraph_mode":"FULL_DECODE_ONLY","custom_ops":["+rms_norm"],"pass_config":{}}'
kv-cache-dtype: fp8
all2all-backend: deepep_low_latency
async-scheduling: true
stream-interval: 32
enable-dbo: true
dbo-decode-token-threshold: 32
no-enable-prefix-caching: true
trust-remote-code: true
served-model-name: nvidia/MiniMax-M3-NVFP4
no-enable-flashinfer-autotune: true
kv-transfer-config: '{"kv_connector": "NixlConnector", "kv_role": "kv_both"}'
attention-config: '{"backend": "FLASHINFER", "use_trtllm_attention": true, "indexer_kv_dtype": "fp8"}'
block-size: 128
language-model-only: true
benchmark:
type: sa-bench
isl: 8192
osl: 1024
concurrencies: '128'
req_rate: inf
frontend:
type: dynamo
enable_multiple_frontends: false
args:
dyn-chat-processor: vllm
reasoning-parser: minimax_m3
dynamo:
install: false
version: 1.2.1
Comment on lines +124 to +127

@claude claude Bot Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 All 4 new B200 MiniMax-M3 recipes set dynamo.install: false (version 1.2.1) under a frontend.type: dynamo, but the byte-identical container image is used by the existing B300-fp4 recipes, which all require dynamo.install: true (version 1.3.0.dev20260614) to run. If dynamo isn't pre-bundled in this image (which the B300 requirement implies), the disaggregated dynamo frontend/router will fail to start on B200, producing no benchmark results for any of the 4 new configs. Same issue on lines ~129-132 of the other 3 new recipe files (3p2d-c256, 3p2d-c512, 4p2d-c1024).

Extended reasoning...

All 4 new B200 recipes (2p2d-dep2-tp4-c128, 3p2d-dep2-tp4-c256, 3p2d-dep2-tp4-c512, 4p2d-dep2-tp4-c1024) set:

dynamo:
  install: false
  version: 1.2.1

with frontend.type: dynamo (a disaggregated dynamo frontend/router). The exact same container image, vllm/vllm-openai:vllm-minimax-m3-perf-x86_64-13.0.1-8b00f41, is already used by all 8 existing MiniMax-M3 B300-fp4/8k1k recipes, and every one of those sets dynamo.install: true with version: 1.3.0.dev20260614. A repo-wide grep confirms these 4 new files are the only recipes anywhere under srt-slurm-recipes that combine a dynamo frontend with install: false (158 recipes use install: true; the rest omit the key entirely — none use false). A comment on an unrelated sglang/deepseek-v4 recipe (disagg-gb300-*.yaml) explicitly documents the rationale for this field elsewhere in the tree: dynamo.install: true is needed because "the container ... has no dynamo baked in." Since the vLLM MiniMax-M3 image is byte-identical between B200 and B300, the natural reading is that it likewise lacks a pre-installed dynamo runtime, and disabling install on B200 would leave the disaggregated frontend/router with nothing to launch — causing the job to fail at startup and the sweep to produce zero results.

I checked runners/launch_b200-dgxc.sh for the new B200 branch this PR adds (elif [[ $FRAMEWORK == "dynamo-vllm" && $MODEL_PREFIX == "minimaxm3" ... ]]): it only clones srt-slurm and copies the recipe directory in; there is no separate, out-of-band dynamo install step, so the recipe's own dynamo.install field is what governs whether the runtime gets installed for this launch path (same as for every other multinode recipe in the tree).

Addressing the refutation: one reviewer pointed out that version: 1.2.1 is not a stray leftover — it exactly matches the router: { name: dynamo-router, version: "1.2.1" } entry newly added to configs/nvidia-master.yaml for minimaxm3-fp4-b200-dynamo-vllm, and argued this reflects a deliberate, coordinated change (both install and version moved together) rather than a copy/paste mistake, possibly because B200 provisions dynamo through a different path tied to that master-config router entry. I looked into this: the same router: { name: dynamo-router, version: X } field exists on dozens of other master-config entries (e.g. qwen3.5-fp4-gb300-dynamo-sglang → version 1.1.0, matching that recipe's dynamo: version: "1.1.0", which also has no install key at all — not install: false). This is the established pattern across the repo: the master-config router.version simply mirrors whatever version is declared in the recipe, and it is not shown anywhere in this repo (launch scripts, utils/matrix_logic, ingest code) to trigger a separate dynamo-provisioning mechanism that would make install: false safe. So while the version pin is clearly deliberate rather than accidental, that doesn't establish that flipping install to false is also correct — it's equally consistent with the author having intentionally set the version to match the new master-config entry while mistakenly (or unverifiedly) also disabling the install step, not realizing this image needs it just like its B300 counterpart.

Step-by-step, the likely failure mode:

  1. launch_b200-dgxc.sh hits the new elif branch for dynamo-vllm/minimaxm3/fp4, clones srt-slurm, copies in recipes/vllm/minimax-m3/b200-fp4, and later runs uv pip install -e . (installs srtctl only — not dynamo).
  2. srtctl apply -f "$CONFIG_FILE" ... reads the recipe YAML, sees dynamo.install: false, and skips installing the dynamo runtime package/binaries into the container/venv used to launch the job.
  3. The recipe's frontend.type: dynamo still expects to start a dynamo frontend/router process for the disaggregated prefill/decode topology.
  4. Because the image itself apparently doesn't bundle dynamo (as evidenced by the B300 recipes on the identical image needing install: true to function), the frontend/router process fails to start (missing binary/import error).
  5. The job fails at launch for all 4 new configs, and the PR's stated goal — adding validated B200 disaggregated benchmark points — produces no usable results.

Fix: set dynamo.install: true for all 4 new recipes (mirroring the B300 recipes on the same image), and confirm whether version: 1.2.1 is actually the correct/compatible version to install for this image, or whether it should match the validated 1.3.0.dev20260614 used on B300. Given the residual uncertainty raised by the refutation, it would also be worth a quick sanity check against srt-slurm's handling of dynamo.install (or a smoke-test launch) before merge, since the intent behind the coordinated version change is plausible even though the install: false value itself looks like the actual mistake.

Loading