[DNM] Validate MI355X AITER + vLLM backend fixes / 联合验证 MI355X 后端修复

MODELS.md

@@ -155,6 +155,7 @@ Other offloading tiers, including NVMe KV cache offloading, are outside the init
 | Kimi-K3 | `kimik3` | 2026-07-27 ([#2391](https://github.com/SemiAnalysisAI/InferenceX/pull/2391)) | Agentic coding (DSpark only) | Agentic coding non-DSpark arm (deprecated from day 0) |
 | GLM-5.2 | `glm5.2` | 2026-07-18 ([#2268](https://github.com/SemiAnalysisAI/InferenceX/pull/2268)) | Agentic coding (the non-MTP arm still runs while the MTP-only transition remains pending, as explained in the Deprecation Notice) | |
 | MiniMax-M3 | `minimaxm3` | 2026-06-12 ([#1724](https://github.com/SemiAnalysisAI/InferenceX/pull/1724)) | Agentic coding | Single-turn 1k1k, Single-turn 8k1k (removed 2026-08-04, [#2493](https://github.com/SemiAnalysisAI/InferenceX/pull/2493)) |
+| DeepSeek-V4.1-Flash | `dsv41flash` | Pending | Agentic coding on MI355X (draft; GPU validation pending) | — |
 | DeepSeek-V4-Pro | `dsv4` | 2026-04-24 ([#1130](https://github.com/SemiAnalysisAI/InferenceX/pull/1130)) | Single-turn 8k1k (last day 2026-09-08, see [Deprecation Notice](#deprecation-notice)), Agentic coding (the non-MTP arm still runs while the MTP-only transition remains pending, as explained in the Deprecation Notice) | Single-turn 1k1k |
 | GLM-5 / GLM-5.1 | `glm5`, `glm5.1` | 2026-03-06 ([#762](https://github.com/SemiAnalysisAI/InferenceX/pull/762)), with GLM-5.1 added 2026-04-21 ([#1098](https://github.com/SemiAnalysisAI/InferenceX/pull/1098)) | None (retired 2026-07-18, [#2276](https://github.com/SemiAnalysisAI/InferenceX/pull/2276)) | Single-turn 1k1k, Single-turn 1k8k (GLM-5 only), Single-turn 8k1k |
 | MiniMax-M2.5/2.7 | `minimaxm2.5` | 2026-02-18 ([#755](https://github.com/SemiAnalysisAI/InferenceX/pull/755)) | None (retired 2026-06-20, [#1874](https://github.com/SemiAnalysisAI/InferenceX/pull/1874)) | Single-turn 1k1k, Single-turn 1k8k, Single-turn 8k1k |

MODELS_zh.md

@@ -155,6 +155,7 @@ InferenceX 支持 SGLang 和 vLLM 双方的维护者，并响应 AI 实验室和
 | Kimi-K3 | `kimik3` | 2026-07-27 ([#2391](https://github.com/SemiAnalysisAI/InferenceX/pull/2391)) | 智能体编码（仅 DSpark） | 智能体编码非 DSpark 分支（自第 0 天起弃用） |
 | GLM-5.2 | `glm5.2` | 2026-07-18（[#2268](https://github.com/SemiAnalysisAI/InferenceX/pull/2268)） | 智能体编码（非 MTP 分支仍在运行，「仅 MTP」转换仍待执行，见弃用公告） | |
 | MiniMax-M3 | `minimaxm3` | 2026-06-12（[#1724](https://github.com/SemiAnalysisAI/InferenceX/pull/1724)） | 智能体编码 | 单轮 1k1k、单轮 8k1k（2026-08-04 移除，[#2493](https://github.com/SemiAnalysisAI/InferenceX/pull/2493)） |
+| DeepSeek-V4.1-Flash | `dsv41flash` | 待验证 | MI355X 上的 Agentic coding（草案；等待 GPU 验证） | — |
 | DeepSeek-V4-Pro | `dsv4` | 2026-04-24（[#1130](https://github.com/SemiAnalysisAI/InferenceX/pull/1130)） | 单轮 8k1k、智能体编码（非 MTP 分支仍在运行，「仅 MTP」转换仍待执行，见弃用公告） | 单轮 1k1k |
 | GLM-5 / GLM-5.1 | `glm5`、`glm5.1` | 2026-03-06（[#762](https://github.com/SemiAnalysisAI/InferenceX/pull/762)），GLM-5.1 于 2026-04-21 加入（[#1098](https://github.com/SemiAnalysisAI/InferenceX/pull/1098)） | 无（2026-07-18 退役，[#2276](https://github.com/SemiAnalysisAI/InferenceX/pull/2276)） | 单轮 1k1k、单轮 1k8k（仅 GLM-5）、单轮 8k1k |
 | MiniMax-M2.5/2.7 | `minimaxm2.5` | 2026-02-18（[#755](https://github.com/SemiAnalysisAI/InferenceX/pull/755)） | 无（2026-06-20 退役，[#1874](https://github.com/SemiAnalysisAI/InferenceX/pull/1874)） | 单轮 1k1k、单轮 1k8k、单轮 8k1k |

benchmarks/single_node/agentic/dsv41flash_fp4_mi355x_vllm_mtp.sh

@@ -0,0 +1,107 @@
+#!/usr/bin/env bash
+set -eo pipefail
+
+# DeepSeek-V4.1-Flash on MI355X: native DSpark and GPU-resident KV.
+# Follow upstream AMD defaults for Engram; storage behavior needs verification.
+# Image: vllm/vllm-openai-rocm:deepseekv41-flash-0909 (configured in amd-master.yaml); GPU validation is pending.
+# https://github.com/vllm-project/recipes/blob/main/models/deepseek-ai/DeepSeek-V4.1-Flash.yaml
+source "$(dirname "$0")/../../benchmark_lib.sh"
+check_env_vars MODEL TP CONC KV_OFFLOADING TOTAL_CPU_DRAM_GB RESULT_DIR DURATION
+require_agentic_kv_offload_none
+export GPU_COUNT="$TP"
+
+# Patch both installed libraries inside this serving container, before downloads.
+python3 "$(dirname "${BASH_SOURCE[0]}")/../../../runners/patch_aiter_quiet_auto_backend.py"
+python3 "$(dirname "${BASH_SOURCE[0]}")/../../../runners/patch_vllm_w4a16_backend.py"
+
+# Complete/resume partial downloads instead of trusting nonempty directories.
+if [[ -n "${MODEL_PATH:-}" && "$MODEL_PATH" != "$MODEL" ]]; then
+    hf download "$MODEL" --local-dir "$MODEL_PATH"
+else
+    hf download "$MODEL"
+    export MODEL_PATH="$MODEL"
+fi
+
+if [[ -n "${ROCR_VISIBLE_DEVICES:-}" ]]; then
+    export HIP_VISIBLE_DEVICES="$ROCR_VISIBLE_DEVICES"
+fi
+export VLLM_ROCM_USE_AITER=1
+export VLLM_ROCM_USE_AITER_MOE=1
+# DeepseekV41ForCausalLM is not torch-compiled upstream, so the default
+# cudagraph_mode=FULL_AND_PIECEWISE aborts at engine init with "piecewise CUDA
+# graphs unavailable" (run 34566727564). The model is built for the breakable
+# cudagraph path -- amd/attention.py uses eager_break_during_capture.
+export VLLM_USE_BREAKABLE_CUDAGRAPH=1
+export OMP_NUM_THREADS=1
+# Pin the full-context corpus for this 1M-context recipe.
+export WEKA_LOADER_OVERRIDE=semianalysis_cc_traces_weka_062126
+resolve_trace_source
+install_agentic_deps
+mkdir -p "$RESULT_DIR"
+SERVER_LOG="$RESULT_DIR/server.log"
+export VLLM_ENGINE_READY_TIMEOUT_S=3600
+export VLLM_USE_RUST_FRONTEND=1
+export PYTHONUNBUFFERED=1
+
+# Match the sibling's scheduler headroom for AgentX subagent fan-out.
+MAX_NUM_SEQS=$((2 * CONC))
+NUM_SPEC_TOKENS=5
+CAPTURE_SIZE=1
+while (( CAPTURE_SIZE < MAX_NUM_SEQS * (1 + NUM_SPEC_TOKENS) && CAPTURE_SIZE < 2048 )); do
+    CAPTURE_SIZE=$((CAPTURE_SIZE * 2))
+done
+
+# Use the runner-specific port assigned by launch_mi355x-amds.sh.
+export AIPERF_SERVER_URL="http://localhost:${PORT}"
+export AIPERF_SERVER_METRICS_URLS="${AIPERF_SERVER_URL}/metrics"
+export AIPERF_REQUIRED_SERVER_METRIC_PREFIX="vllm:"
+echo "Using vLLM endpoint ${AIPERF_SERVER_URL}"
+
+# Golden AL: golden_al_distribution/dsv41flash_dspark.yaml, thinking_on, five draft tokens.
+# Accuracy evals keep real block rejection; throughput fixes acceptance to AL 3.51.
+# Adaptive verification stays off in both modes on ROCm: it trims verification
+# requests on device, which DeepseekV4IndexerBackend does not support, so the
+# engine refused to start with it enabled (run 34651830283, eval-only c32).
+if [[ "${EVAL_ONLY:-false}" == true ]]; then
+    SPEC_CONFIG='{"method":"dspark","num_speculative_tokens":5,"draft_sample_method":"probabilistic","rejection_sample_method":"block","enable_adaptive_verification":false}'
+else
+    SPEC_CONFIG='{"method":"dspark","num_speculative_tokens":5,"draft_sample_method":"probabilistic","rejection_sample_method":"synthetic","synthetic_acceptance_length":3.51,"enable_adaptive_verification":false}'
+fi
+VLLM_CMD=(
+    vllm serve "$MODEL_PATH" --served-model-name "$MODEL"
+    --host 0.0.0.0 --port "$PORT" --tensor-parallel-size "$TP"
+    --language-model-only
+    --tokenizer-mode deepseek_v41
+    --tool-call-parser deepseek_v41 --enable-auto-tool-choice
+    --reasoning-parser deepseek_v41
+    --moe-backend aiter_triton_mxfp4_bf16
+    --gpu-memory-utilization 0.9
+    --speculative-config "$SPEC_CONFIG"
+    --max-model-len 1048576
+    --max-num-seqs "$MAX_NUM_SEQS"
+    --max-cudagraph-capture-size "$CAPTURE_SIZE"
+    --max-num-batched-tokens 16384
+    --disable-uvicorn-access-log
+)
+printf '%q ' "${VLLM_CMD[@]}" | tee "$RESULT_DIR/vllm_command.txt"
+printf '\n' | tee -a "$RESULT_DIR/vllm_command.txt"
+SERVER_PID=""
+cleanup_server() {
+    local rc=$?
+    trap - EXIT INT TERM
+    stop_background_process_tree "$SERVER_PID" "vLLM server" 60
+    exit "$rc"
+}
+trap cleanup_server EXIT
+trap 'exit 130' INT
+trap 'exit 143' TERM
+"${VLLM_CMD[@]}" > "$SERVER_LOG" 2>&1 &
+SERVER_PID=$!
+wait_for_server_ready --port "$PORT" --server-log "$SERVER_LOG" --server-pid "$SERVER_PID"
+
+if [[ "${EVAL_ONLY:-false}" == true ]]; then
+    run_eval --port "$PORT"
+else
+    build_replay_cmd "$RESULT_DIR"
+    run_agentic_replay_and_write_outputs "$RESULT_DIR"
+fi

configs/amd-master.yaml

@@ -1752,3 +1752,19 @@ dsv4-fp4-mi355x-sglang-agentic-mtp:
       - { tp: 8, ep: 1, dp-attn: false, kv-offloading: none, conc-list: [1, 4, 16], spec-decoding: mtp }
       - { tp: 8, ep: 1, dp-attn: false, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [32, 48], spec-decoding: mtp }
       - { tp: 8, ep: 1, dp-attn: true, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [128, 256], spec-decoding: mtp }
+
+# Official upstream ROCm image from the vLLM recipe; MI355X runtime validation is pending.
+dsv41flash-fp4-mi355x-vllm-agentic-dspark:
+  image: vllm/vllm-openai-rocm:deepseekv41-flash-0909
+  model: deepseek-ai/DeepSeek-V4.1-Flash
+  model-prefix: dsv41flash
+  runner: cluster:mi355x-amds
+  precision: fp4
+  framework: vllm
+  multinode: false
+  scenarios:
+    agentic-coding:
+    - dram-utilization: 0.60
+      search-space:
+      # Follow upstream AMD Engram defaults; omit the CUDA-only config flag.
+      - { tp: 4, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 2, 4, 8, 16, 32] }

docs/configuration-procedures.md

@@ -396,3 +396,13 @@ Stop before dispatching GPU work or claiming the configuration complete when any
 - YAML, Bash, strict schema, exact-key generation, launcher simulation, or recipe validation fails.
 
 A configuration is ready for sweep only when the executable files agree, the exact key generates, the runtime route exists, the changelog selects it, and all layer-specific checks above pass.
+
+## DeepSeek-V4.1-Flash on MI355X
+
+The draft `dsv41flash-fp4-mi355x-vllm-agentic-dspark` recipe extends [#2958](https://github.com/SemiAnalysisAI/InferenceX/pull/2958) to MI355X AgentX: TP4, concurrency 1–32, native five-token DSpark. Throughput uses the [committed golden AL](../golden_al_distribution/dsv41flash_dspark.yaml) of 3.51 for thinking on and five draft tokens, with synthetic rejection sampling and adaptive verification disabled. Accuracy evals retain real block rejection but, unlike the CUDA arms, also keep adaptive verification disabled: it trims verification requests on device, which the ROCm `DeepseekV4IndexerBackend` does not support, and the engine refused to start with it enabled ([run 34651830283](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/34651830283)). FP4 describes the MXFP4 experts; the checkpoint also contains MXFP8 weights.
+
+Follow the AMD overrides in [upstream recipe #946](https://github.com/vllm-project/recipes/pull/946): `VLLM_ROCM_USE_AITER=1`, `VLLM_ROCM_USE_AITER_MOE=1`, and `--moe-backend aiter_triton_mxfp4_bf16`. The recipe pins `semianalysis_cc_traces_weka_062126` (the unfiltered corpus) via `WEKA_LOADER_OVERRIDE`. KV stays GPU-resident; Engram follows upstream AMD defaults. Do not copy the NVIDIA `--engram-config` option: upstream currently rejects it on ROCm. The MI355X launcher uses the shared HF cache and mounts this model's repository at `/ix`, and exports `INFMAX_CONTAINER_WORKSPACE=/ix` so AgentX dependencies and outputs resolve inside that mount.
+
+Before model download or server startup, this recipe applies two patches inside the existing serving image. [`patch_aiter_quiet_auto_backend.py`](../runners/patch_aiter_quiet_auto_backend.py), equivalent to [AITER #5471](https://github.com/ROCm/aiter/pull/5471), keeps automatic Gluon-to-Triton fallback quiet and warns once per process for an unsupported explicit `backend="gluon"` request. [`patch_vllm_w4a16_backend.py`](../runners/patch_vllm_w4a16_backend.py) makes vLLM explicitly request `backend="triton"` for both W4A16 GEMMs on gfx942/gfx950; gfx1250 and other architectures retain automatic selection, and the MX scale swizzle is unchanged. Neither patch changes log levels. Both patchers reject partial/unknown source before writing, accept already-patched files, and print the target path plus before/after SHA256 hashes for provenance.
+
+**GPU validation is pending:** The recipe uses `vllm/vllm-openai-rocm:deepseekv41-flash-0909`, the image referenced by the [upstream vLLM recipe](https://recipes.vllm.ai/deepseek-ai/DeepSeek-V4.1-Flash?hardware=mi355x). The tag was published to Docker Hub on 2026-09-11 after AMD verification; it returned HTTP 404 in run 34466680355 before publication. Engram behavior and serving compatibility still require GPU validation. Follow the [AgentX procedure](./eval-agentx-procedures.md#7-run-agentx-fast-feedback-versus-canonical-evidence) for runtime evidence; local generation and registry metadata are not GPU proof.

docs/configuration-procedures_zh.md

@@ -374,3 +374,13 @@ python -m pytest utils/matrix_logic/ -v
 - YAML、Bash、严格 schema、精确 key 生成、launcher 模拟或配方验证失败。
 
 只有当所有可执行文件一致、精确 key 能生成、运行时路由存在、changelog 能选择该 key，且以上各层检查全部通过时，配置才可以进入 sweep。
+
+## MI355X 上的 DeepSeek-V4.1-Flash
+
+草案配方 `dsv41flash-fp4-mi355x-vllm-agentic-dspark` 将 [#2958](https://github.com/SemiAnalysisAI/InferenceX/pull/2958) 扩展至 MI355X AgentX：TP4、并发 1–32、原生五 token DSpark。吞吐测试使用[已提交的黄金 AL](../golden_al_distribution/dsv41flash_dspark.yaml)：thinking 开启、五个草稿 token 对应 3.51，采用合成拒绝采样并关闭自适应验证。准确率 eval 保留真实块拒绝采样，但与 CUDA 分支不同，同样关闭自适应验证：它会在设备端裁剪验证请求，而 ROCm 的 `DeepseekV4IndexerBackend` 不支持该操作，启用后引擎拒绝启动（[运行 34651830283](https://github.com/SemiAnalysisAI/InferenceX/actions/runs/34651830283)）。FP4 表示 MXFP4 专家权重；检查点还包含 MXFP8 权重。
+
+遵循[上游配方 #946](https://github.com/vllm-project/recipes/pull/946) 的 AMD 设置：`VLLM_ROCM_USE_AITER=1`、`VLLM_ROCM_USE_AITER_MOE=1` 和 `--moe-backend aiter_triton_mxfp4_bf16`。配方通过 `WEKA_LOADER_OVERRIDE` 固定使用完整语料 `semianalysis_cc_traces_weka_062126`。KV 驻留 GPU；Engram 沿用上游 AMD 默认设置。不要复制 NVIDIA 的 `--engram-config` 选项：上游目前在 ROCm 上拒绝该选项。MI355X launcher 使用共享 HF 缓存，并将此模型的仓库挂载至 `/ix`，同时导出 `INFMAX_CONTAINER_WORKSPACE=/ix`，确保 AgentX 依赖与输出路径位于该挂载中。
+
+在下载模型或启动服务前，此配方在现有服务镜像内应用两个补丁。[`patch_aiter_quiet_auto_backend.py`](../runners/patch_aiter_quiet_auto_backend.py) 与 [AITER #5471](https://github.com/ROCm/aiter/pull/5471) 等效：自动从 Gluon 回退至 Triton 时不再告警，显式请求不受支持的 `backend="gluon"` 时每个进程只告警一次。[`patch_vllm_w4a16_backend.py`](../runners/patch_vllm_w4a16_backend.py) 让 vLLM 在 gfx942/gfx950 上对两个 W4A16 GEMM 显式传入 `backend="triton"`；gfx1250 和其他架构仍采用自动选择，MX scale swizzle 保持不变。两个补丁均不调整日志级别；脚本在写入前拒绝部分应用或未知源码，允许重复运行，并输出目标路径及修改前后的 SHA256，便于追溯。
+
+**GPU 验证尚待完成：** 配方使用 `vllm/vllm-openai-rocm:deepseekv41-flash-0909`，即[上游 vLLM 配方](https://recipes.vllm.ai/deepseek-ai/DeepSeek-V4.1-Flash?hardware=mi355x)引用的镜像。该标签经 AMD 验证后于 2026-09-11 发布到 Docker Hub；发布前在运行 34466680355 中返回 HTTP 404。Engram 行为与服务兼容性仍需 GPU 验证。请按 [AgentX 流程](./eval-agentx-procedures_zh.md) 获取运行时证据；本地矩阵生成和镜像元数据不等于 GPU 验证。