Vulkan GPUs: real device names and selectable ordinals (rebase of #7356 onto #7476) (#7498)

* Vulkan GPUs: real device names and selectable ordinals

Rebases the durable half of #7356 onto the inference_gpu transport #7476
landed on main. Those two PRs solve an overlapping problem and disagree on
the data model, so merging #7356 as-is would ship two parallel Vulkan
device concepts with different index semantics. This keeps main's transport
and adds what #7356 had that #7476 does not.

- _vulkan_probe.py emits a 5th column, ggml's device description, sanitized
  for the tab protocol and UTF-8 safe. Reader tolerates 4- or 5-column
  output so an older probe still parses.
- llama_cpp gains _run_vulkan_probe (shared parse) and
  vulkan_device_inventory (names + is_igpu + real totals).
- get_vulkan_inference_gpu_info reports the real name and an explicit
  is_igpu instead of "Vulkan<i>" and a total == 0 guess.
- index_kind becomes "vulkan", not "relative", and gpu_ids picks are
  supported on Vulkan builds once the probe enumerated ordinals. The XPU ban
  no longer applies to them: a Vulkan pick is a ggml ordinal, not a torch-xpu
  index, so it works on an Intel host too.
- Frontend picker reads the Vulkan inventory as the pickable set.

Memory deliberately still comes from _get_gpu_memory, not the inventory.
That path applies _apply_igpu_host_reserve_mib and zeroes a shared total;
budgeting an APU off its raw shared total would hand out the whole machine's
RAM with no OS headroom. Identity is joined onto it by ordinal, so a probe
failure degrades to Vulkan<i> names with the memory readings intact.

Dropped from #7356 as superseded: validate_vulkan_gpu_ids (main's
resolve_requested_gpu_ids already rejects duplicates and
_resolve_gguf_gpu_ids_for_request already probes for existence), the
gguf_devices transport, and the iGPU budget fallback in 71619891e, which
main's aggregateGpuMemoryTotalGb handles better by counting a shared pool
once.

Also keeps #7356's removal of the late diffusion raise, so the graceful
gpu_ids drop stays reachable for a GGUF only classified as diffusion after
download. #7415's real guard, _reject_vulkan_diffusion_gpu_ids_before_
teardown, is untouched.

Verified on Windows + Strix Halo: backend Vulkan/GPU-selection suites at the
same 4 pre-existing failures as main, tests/studio 1671 passed with no new
failures, frontend typecheck clean. Hardware confirmation of the underlying
behavior is on #7356 from @Bebiv24 (RX 9070 XT + RX 480).

Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Daniel Han 2026-07-27 05:21:48 -07:00 committed by GitHub
commit 74295d93d8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 297 additions and 70 deletions

View file

@ -578,3 +578,24 @@ def test_legacy_migration_is_idempotent_and_non_destructive():
# Layer 3: non-overwriting merge skips an existing (or default) key, so even a
# forced re-run cannot duplicate or clobber a user's config.
assert "if (isDefaultConfig(migrated) || Object.hasOwn(map, key)) {" in src
def test_vulkan_inference_devices_are_the_pickable_set():
"""GGUF loads run through llama-server, so on a Vulkan build the picker must
offer the inference inventory (ggml ordinals, the space `--device Vulkan<i>`
pins) rather than the torch view, which can miss cards llama-server drives.
The XPU ban must not apply there: it is about torch-xpu ordinals no
applicator speaks, and a Vulkan pick does not use them.
"""
src = " ".join(_read("hooks/use-gpu-info.ts").split())
# The Vulkan inventory is consulted first, and only when it has devices.
assert (
"const inference = data?.inference_gpu; "
'if (inference?.backend === "vulkan" && (inference.devices ?? []).length) {' in src
)
# Pinnable on the ggml ordinal space, gated on the backend's own support flag.
assert "const picksAccepted = inference.gguf_gpu_ids_supported !== false;" in src
assert 'physicalIndex: picksAccepted && d.index_kind === "vulkan",' in src
# The torch fallback keeps its physical-only gate and the XPU ban.
assert 'data?.device_backend !== "xpu" &&' in src
assert 'physicalIndex: pinnableBackend && d.index_kind === "physical",' in src