Skip to content

fix(xsysinfo): include AMD GTT in APU VRAM detection - #12094

Open
leilei3167 wants to merge 2 commits into
mudler:masterfrom
leilei3167:fix/issue-12058-amd-gtt-vram
Open

leilei3167 wants to merge 2 commits into
mudler:masterfrom
leilei3167:fix/issue-12058-amd-gtt-vram

Conversation

@leilei3167

Copy link
Copy Markdown
Contributor

Description

Fixes #12058

On AMD shared-memory APUs (Strix Halo / gfx1151 and similar), rocm-smi --showmeminfo vram only reports the BIOS UMA carve-out (e.g. 512 MiB). Usable HIP memory lives in the much larger GTT pool (often 100+ GiB). LocalAI then showed "512 MB GPU RAM", and auto-fit / MinPerGPUVRAM decisions were sized from that undercount.

This change:

  • Queries rocm-smi --showmeminfo all --csv (falls back to vram on older builds)
  • Parses VRAM and GTT columns from the CSV header
  • When the VRAM/GTT ratio matches a shared-memory APU (GTT >= 8×VRAM), folds GTT into reported Total/Used
  • Leaves dedicated dGPU accounting on VRAM only (host GTT aperture is not treated as extra VRAM)

Notes for Reviewers

  • Heuristic follows the maint note on GPU RAM detection with incorrect VRAM value #12058: account for usable shared memory without treating all AMD GTT as dedicated VRAM.
  • LOCALAI_VRAM_BUDGET remains an allocation cap on the detected total; it still cannot raise a bad detection by itself.

Evidence (old red → new green)

# Before (VRAM-only parse of --showmeminfo all CSV):
go test ./pkg/xsysinfo/ -ginkgo.focus='folds GTT into totals'
# FAIL: Expected <uint64>: 536870912 to equal <uint64>: 133680857088

# After:
go test ./pkg/xsysinfo/ -ginkgo.focus='folds GTT into totals'   # PASS
go test ./pkg/xsysinfo/ -count=1                               # PASS

Signed commits

  • Yes, I signed my commits.
  • Documentation updated (docs/content/) for user-facing changes, or not applicable

On shared-memory AMD APUs (e.g. Strix Halo), rocm-smi VRAM is only the
BIOS UMA carve-out while usable HIP memory lives in GTT. Query
--showmeminfo all and fold GTT when the VRAM/GTT ratio matches an APU,
preserving dedicated dGPU accounting.

Fixes mudler#12058

Signed-off-by: lei_lei <imleilei123@gmail.com>
@leilei3167
leilei3167 force-pushed the fix/issue-12058-amd-gtt-vram branch from a9eb813 to 903e0a4 Compare September 16, 2026 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GPU RAM detection with incorrect VRAM value

1 participant