* Vulkan GPUs: real device names and selectable ordinals
Rebases the durable half of #7356 onto the inference_gpu transport #7476
landed on main. Those two PRs solve an overlapping problem and disagree on
the data model, so merging #7356 as-is would ship two parallel Vulkan
device concepts with different index semantics. This keeps main's transport
and adds what #7356 had that #7476 does not.
- _vulkan_probe.py emits a 5th column, ggml's device description, sanitized
for the tab protocol and UTF-8 safe. Reader tolerates 4- or 5-column
output so an older probe still parses.
- llama_cpp gains _run_vulkan_probe (shared parse) and
vulkan_device_inventory (names + is_igpu + real totals).
- get_vulkan_inference_gpu_info reports the real name and an explicit
is_igpu instead of "Vulkan<i>" and a total == 0 guess.
- index_kind becomes "vulkan", not "relative", and gpu_ids picks are
supported on Vulkan builds once the probe enumerated ordinals. The XPU ban
no longer applies to them: a Vulkan pick is a ggml ordinal, not a torch-xpu
index, so it works on an Intel host too.
- Frontend picker reads the Vulkan inventory as the pickable set.
Memory deliberately still comes from _get_gpu_memory, not the inventory.
That path applies _apply_igpu_host_reserve_mib and zeroes a shared total;
budgeting an APU off its raw shared total would hand out the whole machine's
RAM with no OS headroom. Identity is joined onto it by ordinal, so a probe
failure degrades to Vulkan<i> names with the memory readings intact.
Dropped from #7356 as superseded: validate_vulkan_gpu_ids (main's
resolve_requested_gpu_ids already rejects duplicates and
_resolve_gguf_gpu_ids_for_request already probes for existence), the
gguf_devices transport, and the iGPU budget fallback in 71619891e, which
main's aggregateGpuMemoryTotalGb handles better by counting a shared pool
once.
Also keeps #7356's removal of the late diffusion raise, so the graceful
gpu_ids drop stays reachable for a GGUF only classified as diffusion after
download. #7415's real guard, _reject_vulkan_diffusion_gpu_ids_before_
teardown, is untouched.
Verified on Windows + Strix Halo: backend Vulkan/GPU-selection suites at the
same 4 pre-existing failures as main, tests/studio 1671 passed with no new
failures, frontend typecheck clean. Hardware confirmation of the underlying
behavior is on #7356 from @Bebiv24 (RX 9070 XT + RX 480).
Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
parent
da447d47ba
commit
74295d93d8
8 changed files with 297 additions and 70 deletions
|
|
@ -100,13 +100,29 @@ function toGpuInfo(
|
|||
}
|
||||
|
||||
function toGpuDevices(data: SystemInfoResponse | null): SystemGpuDevice[] {
|
||||
// Unpinnable configurations must hide every pick surface: XPU indices are
|
||||
// torch-xpu ordinals no applicator speaks, and Vulkan-only builds pin ggml's
|
||||
// own ordinals -- /load and /validate 400 picks on both, so the backend
|
||||
// reports gpu.gguf_gpu_ids_supported and every gate keyed on physicalIndex
|
||||
// (picker, persisted-pick reconcile) follows it. The device flavor lives on
|
||||
// the TOP-LEVEL device_backend field; absent support info defaults to
|
||||
// pinnable (older backend).
|
||||
// GGUF loads run through llama-server, so on a Vulkan build the pickable set
|
||||
// is the inference inventory, not the torch view: it can see cards torch
|
||||
// cannot, and its indices are the ggml ordinals `--device Vulkan<i>` pins.
|
||||
// The XPU ban does not apply there, it is about torch-xpu ordinals that no
|
||||
// applicator speaks; a Vulkan pick does not use them.
|
||||
const inference = data?.inference_gpu;
|
||||
if (inference?.backend === "vulkan" && (inference.devices ?? []).length) {
|
||||
const picksAccepted = inference.gguf_gpu_ids_supported !== false;
|
||||
return (inference.devices ?? [])
|
||||
.filter((d) => typeof d.index === "number")
|
||||
.map((d) => ({
|
||||
index: d.index as number,
|
||||
name: d.name ?? `GPU ${d.index}`,
|
||||
memoryTotalGb: d.memory_total_gb ?? 0,
|
||||
memoryFreeGb: d.vram_free_gb ?? 0,
|
||||
physicalIndex: picksAccepted && d.index_kind === "vulkan",
|
||||
}));
|
||||
}
|
||||
// Otherwise the torch view is the pickable set. Unpinnable configurations
|
||||
// must hide every pick surface: XPU indices are torch-xpu ordinals no
|
||||
// applicator speaks, so /load and /validate 400 them, and the backend reports
|
||||
// gpu.gguf_gpu_ids_supported. Absent support info defaults to pinnable
|
||||
// (older backend).
|
||||
const pinnableBackend =
|
||||
data?.device_backend !== "xpu" &&
|
||||
data?.gpu?.gguf_gpu_ids_supported !== false;
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue