* Vulkan GPUs: real device names and selectable ordinals
Rebases the durable half of #7356 onto the inference_gpu transport #7476
landed on main. Those two PRs solve an overlapping problem and disagree on
the data model, so merging #7356 as-is would ship two parallel Vulkan
device concepts with different index semantics. This keeps main's transport
and adds what #7356 had that #7476 does not.
- _vulkan_probe.py emits a 5th column, ggml's device description, sanitized
for the tab protocol and UTF-8 safe. Reader tolerates 4- or 5-column
output so an older probe still parses.
- llama_cpp gains _run_vulkan_probe (shared parse) and
vulkan_device_inventory (names + is_igpu + real totals).
- get_vulkan_inference_gpu_info reports the real name and an explicit
is_igpu instead of "Vulkan<i>" and a total == 0 guess.
- index_kind becomes "vulkan", not "relative", and gpu_ids picks are
supported on Vulkan builds once the probe enumerated ordinals. The XPU ban
no longer applies to them: a Vulkan pick is a ggml ordinal, not a torch-xpu
index, so it works on an Intel host too.
- Frontend picker reads the Vulkan inventory as the pickable set.
Memory deliberately still comes from _get_gpu_memory, not the inventory.
That path applies _apply_igpu_host_reserve_mib and zeroes a shared total;
budgeting an APU off its raw shared total would hand out the whole machine's
RAM with no OS headroom. Identity is joined onto it by ordinal, so a probe
failure degrades to Vulkan<i> names with the memory readings intact.
Dropped from #7356 as superseded: validate_vulkan_gpu_ids (main's
resolve_requested_gpu_ids already rejects duplicates and
_resolve_gguf_gpu_ids_for_request already probes for existence), the
gguf_devices transport, and the iGPU budget fallback in 71619891e, which
main's aggregateGpuMemoryTotalGb handles better by counting a shared pool
once.
Also keeps #7356's removal of the late diffusion raise, so the graceful
gpu_ids drop stays reachable for a GGUF only classified as diffusion after
download. #7415's real guard, _reject_vulkan_diffusion_gpu_ids_before_
teardown, is untouched.
Verified on Windows + Strix Halo: backend Vulkan/GPU-selection suites at the
same 4 pre-existing failures as main, tests/studio 1671 passed with no new
failures, frontend typecheck clean. Hardware confirmation of the underlying
behavior is on #7356 from @Bebiv24 (RX 9070 XT + RX 480).
Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
parent
da447d47ba
commit
74295d93d8
8 changed files with 297 additions and 70 deletions
|
|
@ -578,3 +578,24 @@ def test_legacy_migration_is_idempotent_and_non_destructive():
|
|||
# Layer 3: non-overwriting merge skips an existing (or default) key, so even a
|
||||
# forced re-run cannot duplicate or clobber a user's config.
|
||||
assert "if (isDefaultConfig(migrated) || Object.hasOwn(map, key)) {" in src
|
||||
|
||||
|
||||
def test_vulkan_inference_devices_are_the_pickable_set():
|
||||
"""GGUF loads run through llama-server, so on a Vulkan build the picker must
|
||||
offer the inference inventory (ggml ordinals, the space `--device Vulkan<i>`
|
||||
pins) rather than the torch view, which can miss cards llama-server drives.
|
||||
The XPU ban must not apply there: it is about torch-xpu ordinals no
|
||||
applicator speaks, and a Vulkan pick does not use them.
|
||||
"""
|
||||
src = " ".join(_read("hooks/use-gpu-info.ts").split())
|
||||
# The Vulkan inventory is consulted first, and only when it has devices.
|
||||
assert (
|
||||
"const inference = data?.inference_gpu; "
|
||||
'if (inference?.backend === "vulkan" && (inference.devices ?? []).length) {' in src
|
||||
)
|
||||
# Pinnable on the ggml ordinal space, gated on the backend's own support flag.
|
||||
assert "const picksAccepted = inference.gguf_gpu_ids_supported !== false;" in src
|
||||
assert 'physicalIndex: picksAccepted && d.index_kind === "vulkan",' in src
|
||||
# The torch fallback keeps its physical-only gate and the XPU ban.
|
||||
assert 'data?.device_backend !== "xpu" &&' in src
|
||||
assert 'physicalIndex: pinnableBackend && d.index_kind === "physical",' in src
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue