[DNM] Validate MI355X AITER + vLLM backend fixes / 联合验证 MI355X 后端修复 - #3031
[DNM] Validate MI355X AITER + vLLM backend fixes / 联合验证 MI355X 后端修复#3031functionstackx wants to merge 11 commits into
Conversation
新增 MI355X 的 DS V4.1 Flash AgentX 草案配方：TP4、并发 1–32、原生 DSpark 和 AITER，并补充 launcher、测试及中英文文档。ROCm 镜像发布与 Engram 兼容性仍待验证。
将 MI355X DS V4.1 Flash 固定至 semianalysis_cc_traces_weka_062126，避免使用 256k 语料，并同步更新文档与性能变更记录。
DSv4.1 Flash 吞吐测试使用 thinking 开启、五个草稿 token 对应的黄金合成 AL 3.51，关闭自适应验证；准确率评测保留真实验证。同步中英文文档并追加性能变更记录。
使用指定的 ECR ROCm release 镜像替代不可用的 Docker Hub 标签，保留完整上下文语料与 DSpark 设置，并同步文档及性能变更记录。
为 DS V4.1 Flash 的 /ix 挂载设置 INFMAX_CONTAINER_WORKSPACE，修复 AgentX 依赖与输出路径，并验证共享 helper 的实际路径解析。
Upstream vLLM does not apply @support_torch_compile to
DeepseekV41ForCausalLM, so the default cudagraph_mode=FULL_AND_PIECEWISE
aborts at engine init ("piecewise CUDA graphs unavailable, model is not
torch-compiled and breakable CUDA graph is off"). All concurrencies
failed this way in run 34566727564.
The model targets the breakable cudagraph path -- amd/attention.py
imports eager_break_during_capture -- so enable it explicitly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…Sv4.1 Flash MI355X The upstream vLLM recipe image is now public on Docker Hub (AMD-verified, pushed 2026-09-11). Replace the interim ECR release-repo image and update docs/changelog to match.
The EVAL_ONLY speculative config enabled adaptive verification, which trims verification requests on device. The ROCm DeepseekV4IndexerBackend does not support that, so the eval-only engine refused to start (run 34651830283, c32). Keep real block rejection for evals; throughput settings are unchanged. EVAL_ONLY 的推测解码配置启用了自适应验证，它会在设备端裁剪验证请求；ROCm 的 DeepseekV4IndexerBackend 不支持该操作，导致仅评测引擎拒绝启动（运行 34651830283，c32）。评测仍保留真实块拒绝采样；吞吐设置不变。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
在 MI355X 配方启动前应用 AITER 静默自动后端补丁，保留后端选择和日志级别，并添加幂等性及分派测试。 Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
将此验证分支新增的八条 changelog 关联到 PR #3031，保留主分支历史内容。 Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34661165439 |
联合应用 AITER 静默自动选择和 vLLM gfx942/gfx950 显式 Triton 补丁，保留日志级别并验证补丁来源与幂等性。 Signed-off-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
functionstackx
changed the title
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34661567802 |
2 participants
Purpose / 目的
Test both patches together in the existing MI355X DeepSeek-V4.1-Flash image:
在现有 MI355X DeepSeek-V4.1-Flash 镜像中联合测试两个补丁：
Both patchers run against the installed packages before model download and server startup. They reject partial or unknown source states, accept already-patched files without rewriting, and print paths and before/after SHA256 values. Log levels, scale swizzling, the image and benchmark settings are unchanged.
两个补丁均在下载模型及启动服务前应用于已安装的包。补丁程序拒绝部分应用或未知源码状态，已应用的文件不会被重写，并输出路径及修改前后的 SHA256。日志级别、scale swizzling、镜像及 benchmark 设置不变。
This branch includes the still-unmerged recipe from #2962 so the PR sweep targeting
maincan run it. Keep DNM; do not merge independently of that recipe. Changelog scope:dsv41flash-fp4-mi355x-vllm-agentic-dspark.此分支包含 #2962 中尚未合并的配方，以便面向
main的 PR sweep 能够运行。保留 DNM；请勿脱离原配方单独合并。Changelog 范围：dsv41flash-fp4-mi355x-vllm-agentic-dspark。Validation / 验证
Final GPU result: Combined fail-fast sweep 34661567802 passed on commit
1d894b495b32ff8ed15c76c9d97853d90ee2f742. All six MI355X TP4 benchmark points and the eval job succeeded. All seven complete server logs show readiness and inference: 44,516 lines, zero occurrences ofGLUON backend not available. Using TRITON backend!!!.**最终 GPU 结果：**联合 fail-fast sweep 已通过，六个 MI355X TP4 benchmark 点及评测作业全部成功。七份完整服务日志均确认服务就绪并执行推理，共 44,516 行，原始 Gluon 回退警告为零。
All seven job logs confirm both patches applied with matching final hashes: AITER
31c68dda0122a4865d6d55e8a7bf6bab9a81c67856f75921d342980a6584e02f, vLLM44eccaaff89ff8d0acdfd18043500c317ccfa7c63ffdf11ae772fb540603c643. GSM8K results: 1,319 questions, strict-match 0.9734647460, flexible-extract 0.9727065959; both passed the configured 0.9 gate.七份作业日志均确认两个补丁已应用，最终哈希与上游草稿一致。GSM8K 共 1,319 题，strict-match 为 0.9734647460，flexible-extract 为 0.9727065959，均通过配置的 0.9 门槛。
Caveat: concurrency 32 recorded 2,003 successful profiling requests and one
ClientOSErrorwhile writing a request body (0.05% error rate); the job passed its configured gate. Profiling also logged cancellation/grace-period messages at its duration limit. This is not a claim that every log is error-free. There is no controlled baseline accuracy/performance comparison, and the combined run does not isolate each patch.注意：concurrency 32 的 profiling 请求中有 2,003 次成功、1 次写入请求体时的
ClientOSError，错误率为 0.05%；作业通过了配置的门槛。达到运行时限后，profiling 还记录了取消及宽限期消息。因此不声称所有日志均无错误。此次没有进行受控的基线准确率或性能对比，联合运行也无法单独归因于某个补丁。56 runner tests passed with
uv run --no-project --exclude-newer PT12H --python 3.12 --with pytest --with pydantic --with pyyaml python -m pytest runners/test_slurm_utils.py -q.Both runtime transformations are byte-identical to their upstream drafts. Repeated application preserves bytes and modification time.
CPU checks with dependency stubs preserve the expected dispatch across gfx942/gfx950/gfx1250/gfx1100, scale-swizzle behavior and the gfx1250 SILU bypass.
Six-point MI355X matrix generation, changelog validation,
bash -nandgit diff --checkpassed.上述 uv 命令运行的 56 项 runner 测试已通过。
两个运行时补丁的输出均与上游草稿逐字节一致；重复应用不会修改内容或修改时间。
使用依赖桩的 CPU 检查确认四种架构的预期后端选择、scale-swizzle 行为及 gfx1250 SILU 旁路保持正确。
六点 MI355X 矩阵生成、changelog 验证、
bash -n及git diff --check已通过。Validation used
full-sweep-fail-fast, both in-container patch markers/hashes, server readiness, actual benchmark execution, and completeserver.logartifacts. The checks above verify the combined configuration; broader correctness and performance comparisons remain outside this run.验证使用
full-sweep-fail-fast，检查了容器内两个补丁的标记及哈希、服务就绪、实际 benchmark 执行和完整服务日志。上述检查验证了联合配置；更广泛的正确性与性能对比不在此次运行范围内。Baseline: the original job has 56,264 warning lines in its full server-log artifact. The superseded AITER-only sweep was cancelled when the requested scope changed.
基线：原作业的完整服务日志包含 56,264 行警告。需求变更后，已取消被替代的仅 AITER sweep。
AI assistance was used for implementation and local verification. Human review and GPU CI evidence are required before merging; no performance improvement is claimed.
实现及本地验证使用了 AI 辅助。合并前仍需人工审核及 GPU CI 证据；不声称有性能提升。