Skip to content

ci: persist ccache across pods in CUDA workflow - #7947

Merged
mohanchen merged 2 commits into
developfrom
MrLi000001-patch-1
Sep 9, 2026
Merged

ci: persist ccache across pods in CUDA workflow#7947
mohanchen merged 2 commits into
developfrom
MrLi000001-patch-1

Conversation

@MrLi000001

@MrLi000001 MrLi000001 commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

The CUDA test workflow still relies on a /tmp/ccache volume mount for
its compiler cache. ECI/ARC pods are ephemeral: /tmp is destroyed on
teardown, so every CUDA run compiles the full tree from scratch
(Configure & Build ~20-21 min per run, measured across recent runs).

This PR mirrors the approach merged in #7935 for the CPU workflow, adapted
for the CUDA side, and keeps the workflow layout otherwise untouched:

  • Persist ccache across pods via actions/cache, keyed on a fingerprint of
    c++ / nvcc / ccache versions (8 GiB cap on the ccache directory). The
    stale /tmp/ccache volume mount is removed.
  • Use a dedicated abacus-cuda-ccache-v1-* key prefix: nvcc objects are
    not interchangeable with the gcc objects cached by the CPU workflow, so
    the two caches stay isolated.
  • Add a push: develop trigger so the default branch seeds the shared
    GPU cache baseline (PR-scoped caches cannot be shared across PRs).
  • Report ccache --show-stats to the job summary for rollout verification.

No CMake changes are needed: the top-level CMakeLists.txt already sets
CMAKE_CUDA_COMPILER_LAUNCHER when ccache is detected, and
CMAKE_CUDA_ARCHITECTURES=70 (pinned in #7753) keeps the cached objects
uniform across runs.

Measured on this PR (identical code, i.e. a ~100% hit rate — the best
case): the restore step takes ~1 min and Configure & Build drops from
~21 min to ~1.5 min, with all 43 checks green. PRs that modify sources
will see partial misses; the exact hit rates and build times will be
collected from the reported stats after merge. Test execution times are
unaffected. GPU runners are a scarce resource, so shorter builds also
reduce queueing.

Note: GitHub's cache quota is 10 GiB per repository (LRU eviction),
shared by both workflows. The CPU cache measured 1.35 GiB in practice;
the CUDA cache size will be confirmed from the cache-save step log and
the reported ccache stats (nvcc objects are typically larger than gcc's,
so it may exceed 1.35 GiB, but both combined are expected to stay well
within the quota).

Reminder

  • I have read AGENTS.md and docs/developers_guide/agent_governance.md.
  • I have linked an issue or explained why this PR does not need one. — CI workflow change only; no user-visible code change, so no issue is linked.
  • I have added adequate unit tests and/or case tests, or explained why not. — the workflow itself runs the full CUDA test suite; cache stats are reported in the job summary for verification.
  • I have listed the exact verification commands run and their results. — verified on this PR: cold run Configure & Build 20.7 min (cache miss, within the no-cache baseline of 20.5-21.1 min); warm rerun: Restore ~1.1 min + Configure & Build ~1.3 min; all 43 checks green. ccache stats: <粘贴 "Report compiler cache" 步骤的实际输出>.
  • I have described user-visible behavior changes, including INPUT parameter changes. — none; CI infrastructure only.
  • I have explained core-module impact for ESolver, HSolver, ElecState, Hamilt, Operator, Psi, or other source/ changes. — none; .github/workflows/cuda.yml only.
  • I have requested any needed governance exception below. — not needed.

Added push trigger for develop branch and enhanced compiler cache management.
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Agent Governance Check

Severity Rule Location Reason Suggested action Exception
warning PR metadata completeness pull_request.body missing sections: Linked Issue, Unit Tests and/or Case Tests for my changes, What's changed? Fill the PR template with issue linkage, test evidence, and a concise change summary. allowed

@MrLi000001 MrLi000001 closed this Sep 9, 2026
@MrLi000001 MrLi000001 reopened this Sep 9, 2026
@mohanchen mohanchen added Compile & CICD & Docs & Dependencies Issues related to compiling ABACUS Refactor Refactor ABACUS codes labels Sep 9, 2026
@mohanchen
mohanchen merged commit 4c94b80 into develop Sep 9, 2026
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Compile & CICD & Docs & Dependencies Issues related to compiling ABACUS Refactor Refactor ABACUS codes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants