Skip to content

[PowerX] audit power tests and remove the one tautological file, keeping B200 exporter coverage / 审计功耗测试,删除唯一重言文件并保留 B200 覆盖 - #3089

Merged
edwingao28 merged 2 commits into
mainfrom
chore/remove-tautological-power-tests
Sep 14, 2026
Merged

edwingao28 merged 2 commits into
mainfrom
chore/remove-tautological-power-tests

Conversation

@edwingao28

@edwingao28 edwingao28 commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Description

Audited all 15 test files I added since June: three were already removed in #2799, ten exercise real code with hand-worked expectations and stay, one was tautological. runners/test_kimik3_bh_power.py asserted its own stubs (bundle_server_logs wrote server evidence, the test asserted it back) and compared a dumped yaml fixture to itself. Deleted. Its one real invariant, the B200 dcgm-exporter URI carrying exactly one #, moves into the shared contract test beside GB300 and H200; one stub-echo assertion on the enroot fixture there is also removed.

中文

审计了我自六月以来新增的全部 15 个测试文件:3 个已在 #2799 中删除,10 个跑的是真实代码且期望值为手工推导,予以保留,1 个为重言测试。runners/test_kimik3_bh_power.py 断言的是自己的 stub(bundle_server_logs 写入 server evidence,测试再断言该字符串),并把导出的 yaml fixture 与自身比较。已删除。其中唯一真实的不变量,即 B200 dcgm-exporter URI 只含一个 #,已并入共享契约测试,与 GB300、H200 同列;该文件中一条断言 enroot stub 自身输出的语句同时删除。

Related Issue

None.

Type of Change

  • Bug fix
  • New feature
  • Configuration change
  • Documentation update
  • Other (please describe): test cleanup

Checklist

  • I have tested my changes locally
  • I have updated documentation if necessary
  • For every change that can affect benchmark performance and every recipe addition or modification, I have appended a new entry to the physical end of perf-changelog.yaml and have not edited historical entries — n/a
  • Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /reuse-sweep-run on this PR — n/a

Verification

CI pytest for this path: 476 passed. Breaking the B200 launcher's # separator fails the migrated case.


Note

Low Risk
Test-only cleanup and consolidation; no production launcher or runtime behavior changes.

Overview
Removes runners/test_kimik3_bh_power.py, which mostly asserted stub behavior and round-tripped YAML fixtures, and drops it from the test-process-result workflow path filters and pytest command.

The meaningful B200 invariant—dcgm-exporter cold import must use docker://nvcr.io#... with exactly one #—moves into test_exporter_cold_import_uses_nvidia_registry alongside GB300 and H200. That test now parametrizes launch_b200-nscale-slurm.sh, optionally skips cache-path rewriting, pulls in enroot_uri_for_image when present, and stops asserting the stub enroot fixture’s file contents (only the import reference is checked).

Reviewed by Cursor Bugbot for commit 589d162. Bugbot is set up for automated code reviews on this repo. Configure here.

@edwingao28
edwingao28 requested a review from a team September 14, 2026 00:52
@edwingao28 edwingao28 changed the title [PowerX] delete tautological Kimi-K3 power test and keep B200 exporter coverage / 删除重言测试并保留 B200 exporter 覆盖 [PowerX] audit my power tests and remove the one tautological file, keeping B200 exporter coverage / 审计功耗测试,删除唯一重言文件并保留 B200 覆盖 Sep 14, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Additional findings (outside the current diff — GitHub can't attach inline comments there):

  • 🟡 runners/test_kimik3_bh_power.py — Deleting this file removes the only coverage for b200/h200 Kimi-K3 power runtime selection (wrong-commit detection, power-pin stamping, power on/off routing to the correct git repo) and for evidence bundling on a failed power run; the replacement in utils/test_gb300_power_official_contract.py only exercises the exporter cold-import '#'-separator invariant, not runtime selection. Fix: add or keep parametrized coverage of the b200/h200 wrong-head/power-routing and failure-evidence-bundling behavior (e.g. alongside the existing gb300-only test_kimi_power_gate_rejects_unsupported_route/test_kimi_power_route_rejects_wrong_commit tests) before removing this file, since gb300 is covered but b200/h200 are not.

    Extended reasoning...

    utils/test_gb300_power_official_contract.py has test_kimi_agentx_route_uses_recipe_power_gate, test_kimi_power_gate_rejects_unsupported_route, and test_kimi_power_route_rejects_wrong_commit, but each is parametrized only on launcher_name='launch_gb300-nv.sh' (lines 398-434). No test anywhere now sources launch_b200-nscale-slurm.sh or launch_h200-dgxc-slurm.sh to check that a wrong git HEAD aborts the run, that the AGENTX_POWER_SRT_SLURM_PIN stamp is written only when power is enabled, or that bundle_server_logs is invoked before a non-zero exit on a failed power run for those two launchers. A future change that breaks wrong-head detection or evidence bundling specifically in launch_b200-nscale-slurm.sh or launch_h200-dgxc-slurm.sh would pass CI silently, whereas before this diff it was caught by test_kimi_power_selects_verified_runtime and test_kimi_failed_power_stages_evidence_before_exit.

    Verification: nit (coverage gap, no production break). The candidate's factual claim holds. The deleted runners/test_kimik3_bh_power.py::test_kimi_power_selects_verified_runtime provided real (not merely tautological) b200/h200 coverage: wrong-head abort ("assert result.returncode != 0; assert not stamp.exists()"), pin stamping gated on power ("stamp.read_text().strip() == 'a'*40" only when power), and…

Comment thread utils/test_gb300_power_official_contract.py
@edwingao28 edwingao28 changed the title [PowerX] audit my power tests and remove the one tautological file, keeping B200 exporter coverage / 审计功耗测试,删除唯一重言文件并保留 B200 覆盖 [PowerX] audit power tests and remove the one tautological file, keeping B200 exporter coverage / 审计功耗测试,删除唯一重言文件并保留 B200 覆盖 Sep 14, 2026
@edwingao28
edwingao28 merged commit fb3198b into main Sep 14, 2026
8 checks passed
@edwingao28
edwingao28 deleted the chore/remove-tautological-power-tests branch September 14, 2026 04:39
@edwingao28

Copy link
Copy Markdown
Collaborator Author

Re the b200/h200 route-coverage suggestion in the review summary: not extending it here. The shared harness slices on USES_AGENTX_POWER / Error: dcgm-power requires; b200 enters via USES_DCGM_POWER with a different error string and h200 nests USES_KIMIK3_POWER with FRAMEWORK=vllm, so covering them means per-launcher bash-slicing branches, the pattern this PR removes. Both routes are verified end-to-end by the reused PR sweeps of #3043 (B200) and #3044 (H200) on main.

@adibarra adibarra mentioned this pull request Sep 16, 2026
52 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant