unsloth/studio/backend/tests/test_install_resolve_prebuilt.py
Souravrajvi0 7917c7828c
Installer: opt-in Vulkan llama.cpp backend (and fallback when no AMD card is HIP-supported) (#7373)
* feat(install): opt-in Vulkan llama.cpp backend and HIP gfx fallback (#7357)

Add UNSLOTH_LLAMA_BACKEND=vulkan and --llama-backend vulkan to force the
upstream Vulkan prebuilt on any host, persist llama_backend in the install
marker, and re-assert it during Studio updates.

On Windows AMD, auto-fallback to Vulkan when no detected gfx arch is in the
upstream win-hip-radeon GPU_TARGETS set (e.g. gfx803 / RX 480). Mixed setups
where at least one card is HIP-supported still default to HIP unless opted in.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(install): address Codex P2s on Vulkan gfx routing (#7357)

Honor ROCm family tokens (gfx110X), include fork-supported gfx1103, require
a known active gfx before auto-Vulkan, and base the HIP floor check on the
visible-device target instead of every physical GPU in hipinfo.

* Address Codex review: env namespace, physical-NVIDIA guard, test kwarg

- llama_backend_from_env: stop reading UNSLOTH_LLAMA_CPP_BACKEND. That is a
  separate pre-existing setup variable meaning auto/cpu; setup.sh/setup.ps1
  warn and ignore other values, so reading it here forced Vulkan behind that
  warning. Vulkan opt-in stays on UNSLOTH_LLAMA_BACKEND / UNSLOTH_FORCE_VULKAN.
- _should_auto_vulkan_for_amd_windows: gate on not has_physical_nvidia (not
  merely has_usable_nvidia). A CUDA-masked NVIDIA card keeps has_physical_nvidia
  while has_usable_nvidia goes False; Vulkan ignores CUDA_VISIBLE_DEVICES and
  could enumerate the reserved card. Mirrors the Intel auto path. Explicit
  opt-in still overrides.
- test fakes: validate_prebuilt_attempts/validate_prebuilt_choice gained a
  llama_backend kwarg; the four fake signatures in the fallback tests now
  accept it, clearing the TypeError that reddened Backend CI / Repo tests (CPU).

Tests: UNSLOTH_LLAMA_CPP_BACKEND=vulkan no longer triggers Vulkan; hidden
physical NVIDIA suppresses AMD auto-Vulkan while explicit opt-in overrides.

* Keep gfx1034 on the ROCm path (fork gfx103X bundle covers it)

The WINDOWS_HIP_PREBUILT_GFX_TARGETS allow-list omitted gfx1034, so
_route_to_vulkan_prebuilt downgraded RX 6500/6400-class hosts to the upstream
Vulkan prebuilt before published_rocm_choice_for_host could match the fork
windows-rocm gfx103X bundle (whose members include gfx1034). Add gfx1034 to the
allow-list and a regression test asserting it stays on the fork ROCm asset.

* Fix auto-Vulkan stealing fork windows-rocm gfx908/gfx90a hosts for PR #7373

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix Vulkan marker claiming a backend that was never installed for PR #7373

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the Vulkan backend routing comments for PR #7373

* Keep the visible-device-aware gfx when setup forwards --rocm-gfx

setup.ps1 resolves the gfx arch from its own probe, and that pick is not
fully visible-device aware: neither the hipinfo nor the amd-smi branch
reads CUDA_VISIBLE_DEVICES, and the amd-smi branch matches a bare integer
only, so a comma-separated HIP/ROCR mask such as 1,0 also falls back to
GPU 0. The resulting arch was then forwarded through --rocm-gfx and
replaced the arch detect_host() had already resolved for the
runtime-visible GPU.

On a mixed-AMD Windows host that flipped the auto-Vulkan decision: with
GPU 0 gfx1100 and a masked-in gfx1010, the forward reinstated gfx1100,
_should_auto_vulkan_for_amd_windows() saw a HIP-supported arch and the
HIP bundle was installed for a GPU that cannot run it.

Fold the forward in as a fill rather than a replacement: it still supplies
the arch on amd-smi-only, driver-only and name-inferred hosts where the
probe reports none, which is what --rocm-gfx exists for, but no longer
overwrites a successfully detected active arch. An explicit
UNSLOTH_ROCM_GFX_ARCH stays authoritative, since it is the documented
manual override for hosts whose arch the probes get wrong.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scope the Windows AMD Vulkan fallback per device and per repo

Three follow-ups on the auto-Vulkan routing for #7357.

Keep an explicit --rocm-gfx authoritative. The previous round stopped a
forwarded gfx from replacing an arch detect_host() had already resolved,
but --rocm-gfx is also the documented operator override for hosts whose
probe is wrong or stale, and both arrive as the same argv. Narrow the
advisory case to the two shapes setup can actually be describing: an arch
the probe saw on this host (setup picked a different physical GPU of the
same box), or a family label such as gfx110X, which is a bundle name the
update path derives from the marker asset rather than a real GPU arch.
Any other value is an override for an arch no probe reported and stays
authoritative. Keeping family labels advisory also preserves the rule that
an in-generation-but-unbuilt arch (gfx1033) is never upgraded into the
gfx103X bundle.

Do not auto-route to Vulkan from a HIP-only device mask. HIP_VISIBLE_DEVICES,
ROCR_VISIBLE_DEVICES and CUDA_VISIBLE_DEVICES select the active arch, but the
Vulkan runtime honours none of them: it enumerates through
GGML_VK_VISIBLE_DEVICES and Vulkan ordinals in
LlamaCppBackend._get_gpu_free_memory_vulkan. Masking down to a below-floor
card therefore used to install a backend that could still enumerate the
HIP-capable card the user deliberately hid, possibly one reserved for another
workload. Require every physical AMD gfx to be below the floor, matching the
has_physical_nvidia gate right above it. So the per-GPU list survives to that
check, a forward that agrees with the probe no longer collapses
rocm_gfx_targets to a single entry.

Make the HIP support predicate repository-specific. The floor constant is a
union of ggml-org's windows-hip gpu_targets and the fork's windows-rocm
bundles, so it only answers "is this arch served" for the fork. With
--published-repo ggml-org/llama.cpp, direct_upstream_release_plan() offers
win-hip-radeon then CPU and never Vulkan, so the four fork-only archs
(gfx908, gfx90a, gfx1034, gfx1103) were declared supported and fell through
to CPU instead of the Vulkan bundle that would actually run. Add
UPSTREAM_WINDOWS_HIP_GFX_TARGETS and select the set from the planned repo.

* Keep probe-confirmed AMD GPUs in the physical list when a gfx is forwarded

rocm_gfx_targets is the physical inventory _should_auto_vulkan_for_amd_windows()
reads, so a forwarded --rocm-gfx that the probe never reported was deleting cards
the probe had confirmed. On a mixed Windows AMD box whose active device is masked
down to a below-floor card, a stale UNSLOTH_ROCM_GFX_ARCH or a name-inferred arch
for the other GPU collapsed the list to that one arch, the floor check concluded no
AMD GPU on the host reaches the Windows HIP prebuilt, and the install auto-fell back
to Vulkan, which honours no HIP mask and would enumerate the reserved HIP-capable
card. Add the forwarded arch to the list instead of replacing it: it selects the HIP
target, it does not redefine what hardware is present.

An empty probe still yields a single-entry list, so the driver-only Windows AMD host
the forward exists for keeps its automatic Vulkan fallback, and an explicit
--llama-backend vulkan is unaffected.

* Do not auto-fall back to Vulkan when a HIP device mask filtered the probe

hipinfo is itself a HIP application, and AMD documents HIP_VISIBLE_DEVICES as
"only devices whose index is present in the sequence are visible to HIP", with
that spelling recommended on Windows. Under a mask the Windows probe therefore
enumerates the visible devices, so rocm_gfx_targets is what survived the mask
rather than the physical inventory the auto-Vulkan floor check assumes. A
masked-out gfx1100 next to a visible gfx803 made the check conclude that no AMD
GPU on the box reaches the Windows HIP prebuilt and route the install to Vulkan,
which honours none of these masks and would enumerate the reserved card.

Decline to guess when a mask is set: the physical inventory is unknowable from a
masked probe, so keep the HIP / fork / source path. This only ever turns the
automatic fallback off, never on. The driver-only single-GPU host the fallback
exists for sets no mask, an all-hiding "" / -1 mask is still handled as no active
target rather than a partial view, and an explicit --llama-backend vulkan or
UNSLOTH_LLAMA_BACKEND=vulkan is unaffected.

Reading the physical inventory through an unmasked re-probe would also correct
_pick_rocm_gfx_target, which indexes the token list by the mask value and so
already assumes an unmasked probe. That is pre-existing behaviour on main and is
left alone here.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Treat an all-hiding HIP device mask as suppressing the Vulkan fallback too

The mask guard exempted an empty or -1 value on the grounds that the probe reports
no active target under it, but that only holds for the probe: a forwarded
--rocm-gfx still reconstructs an active arch, and setup infers that arch from the
display-adapter name, which no HIP mask touches. A user who hid every AMD GPU from
HIP could therefore still be auto-routed to Vulkan, which honours none of these
masks and would then use all of them. That is the strongest form of the hazard the
guard exists for, not an exemption from it.

Presence of any of the three variables is now the whole test, which also removes
the value parsing. An explicit --llama-backend vulkan or UNSLOTH_LLAMA_BACKEND is
still unaffected.

* Grant the fork-only Windows HIP coverage to the fork, not to every mirror

The floor set is a union of the fork's windows-rocm bundles and only the fork is
planned from its manifest: resolve_simple_install_release_plans() compares
== DEFAULT_PUBLISHED_REPO and sends every other --published-repo through
direct_upstream_release_plan(), whose AMD branch offers win-hip-radeon then CPU
and never Vulkan. Exempting only the exact ggml-org spelling therefore told a
mirror carrying upstream-standard assets that fork-only archs such as gfx1034,
gfx1103 and gfx908 were HIP-served, landing them on HIP or CPU instead of the
Vulkan bundle that would actually run. Gate on the fork instead.

Matching the dispatch exactly, spelling included, also fixes a differently cased
repo: that really does take the upstream path, so it must be answered with
upstream coverage rather than the fork superset. An empty repo still defaults to
the fork, as the resolver does.

* Derive the Windows HIP gfx floor guard from the published manifest

The guard compared WINDOWS_HIP_PREBUILT_GFX_TARGETS against a second hardcoded
tuple in the same test file, so a windows-rocm arch newly published by the fork
passed both. Affected hosts would then be routed off the hash-approved fork ROCm
bundle onto an unhashed upstream Vulkan build with nothing failing.

Read the fork's llama-prebuilt-manifest.json through the installer's own
resolver instead, and assert the floor, the family labels, and the routing
tuple all still cover what it publishes. The manifest ships only as a release
asset, so an unreachable release skips with an explicit reason rather than
flaking. Both literals match the manifest as published today.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Compress the Vulkan backend routing comments and docstrings for PR #7373

* Correct the family-label rationale in the Windows HIP coverage check

The comment justified serving gfx103X / gfx110X against any repository by
claiming upstream's windows-hip targets build every member of those families.
The fork manifest maps gfx103X to gfx1030..1032 plus gfx1034 and gfx110X to
gfx1100..1102 plus gfx1103, and UPSTREAM_WINDOWS_HIP_GFX_TARGETS carries
neither gfx1034 nor gfx1103, so the stated reason is wrong even though the
answer is right.

State the real reason instead. A family label is a bundle name, not an arch,
so the concrete GPU is unknown at this point; answering unsupported to cover
the two uncovered members would move gfx1030..1032 and gfx1100..1102 off a
working HIP build onto Vulkan for a card the label cannot identify. Those two
archs still reach Vulkan through the concrete-arch branch below, which does
answer per repository.

Comment only. No behaviour change: the 5850-combination override sweep still
reports 0 rocm_gfx_target changes, 0 auto_vulkan False to True flips and 680
True to False flips all backed by a probe-confirmed HIP GPU, and both the
feature and override profile matrices are byte-identical.

* Pin that a deliberate CPU install outranks Vulkan for PR #7373

UNSLOTH_LLAMA_CPP_BACKEND (setup.sh / setup.ps1, "auto" or "cpu") and
UNSLOTH_LLAMA_BACKEND (this module, a backend name) are separate variables at
separate layers, and both accept "cpu". setup translates its own =cpu into
--force-cpu, which is what pins the CPU-only bundle on a GPU host and keeps
Intel iGPU Vulkan crashes away (#7213), so no trigger this PR adds may
outrank it.

_route_to_vulkan_prebuilt already gets this right, since force_cpu
short-circuits ahead of the forced, auto-Intel and auto-no-HIP triggers.
Cover it so it stays that way: the matrix runs [Linux, Windows, macOS] x
[NVIDIA, AMD, Intel, CPU only] x [unset, vulkan, hip, rocm, cpu] with the
legacy UNSLOTH_FORCE_VULKAN set as well, and asserts the published bundle
survives every one. WSL presents as Linux to this resolver, so it rides the
Linux row.

Also assert the guard is not vacuous: the same host still takes Vulkan once
the CPU pin is gone, so the matrix cannot pass on a resolver that had simply
stopped routing to Vulkan.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-27 06:57:19 -07:00

1616 lines
63 KiB
Python

# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""install_llama_prebuilt.py: the --resolve-prebuilt probe (plans against the fork
by default; --published-repo overrides).
These back the in-app update for source-build (markerless) installs: the backend
asks the installer whether an official prebuilt exists for this host without
downloading. Network and host detection are stubbed; no GPU or internet needed. The one
exception is the windows-rocm floor guard, which reads the fork's published manifest
because nothing in-tree mirrors it, and skips when that release is unreachable.
"""
from __future__ import annotations
import importlib
import json
import sys
from pathlib import Path
from types import SimpleNamespace
import pytest
_studio = Path(__file__).resolve().parent.parent.parent
if str(_studio) not in sys.path:
sys.path.insert(0, str(_studio))
ilp = importlib.import_module("install_llama_prebuilt")
if not hasattr(ilp, "resolve_simple_install_release_plans"):
pytest.skip("PR symbols not present - check branch", allow_module_level = True)
FORK = ilp.DEFAULT_PUBLISHED_REPO # unslothai/llama.cpp
UPSTREAM = ilp.UPSTREAM_REPO # ggml-org/llama.cpp
@pytest.fixture(autouse = True)
def _no_ambient_hip_device_mask(monkeypatch):
"""These tests describe hosts through HostInfo, not through the environment.
A mask inherited from the shell (ML boxes commonly export CUDA_VISIBLE_DEVICES) means
the arch probe saw only part of the GPUs, which the Windows auto-Vulkan guard treats as
an unknown physical inventory. Clear all three so a host is described by its fields
alone; the tests that are about the mask set it explicitly."""
for _env in ("HIP_VISIBLE_DEVICES", "ROCR_VISIBLE_DEVICES", "CUDA_VISIBLE_DEVICES"):
monkeypatch.delenv(_env, raising = False)
def _host(**kw):
base = dict(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = False,
is_macos = False,
is_x86_64 = False,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
has_rocm = False,
rocm_gfx_target = None,
macos_version = None,
)
base.update(kw)
return ilp.HostInfo(**base)
def test_force_cpu_clears_all_gpu_attributes_including_intel():
# --cpu-fallback is the "select the CPU prebuilt even when a GPU is present"
# escape hatch. It must drop EVERY GPU attribute, including has_intel_gpu, or
# the planner still prepends the Vulkan asset on an Intel-GPU host.
host = _host(
is_linux = True,
is_x86_64 = True,
has_usable_nvidia = True,
has_physical_nvidia = True,
has_rocm = True,
rocm_gfx_target = "gfx1100",
has_intel_gpu = True,
)
forced = ilp._apply_host_overrides(host, force_cpu = True)
assert forced.has_usable_nvidia is False
assert forced.has_physical_nvidia is False
assert forced.has_rocm is False
assert forced.rocm_gfx_target is None
assert forced.has_intel_gpu is False
def test_macos_upstream_pin_only_for_explicit_pre26_upstream():
pre26 = _host(
system = "Darwin",
is_macos = True,
is_arm64 = True,
machine = "arm64",
macos_version = (15, 5),
)
assert ilp.pinned_macos_release_tag(pre26, UPSTREAM) == "b9415"
assert ilp.pinned_macos_release_tag(pre26, FORK) is None
tahoe = _host(
system = "Darwin",
is_macos = True,
is_arm64 = True,
machine = "arm64",
macos_version = (26, 0),
)
assert ilp.pinned_macos_release_tag(tahoe, UPSTREAM) is None
def _run_resolve(monkeypatch, capsys, plans_or_exc):
monkeypatch.setattr(
ilp,
"detect_host",
lambda: _host(system = "Darwin", is_macos = True, is_arm64 = True, machine = "arm64"),
)
def _resolver(tag, host, repo, published_release_tag):
if isinstance(plans_or_exc, Exception):
raise plans_or_exc
return ("b9585", plans_or_exc)
monkeypatch.setattr(ilp, "resolve_simple_install_release_plans", _resolver)
monkeypatch.setattr(
sys,
"argv",
["install_llama_prebuilt.py", "--resolve-prebuilt", "latest", "--output-format", "json"],
)
rc = ilp.main()
assert rc == ilp.EXIT_SUCCESS
return json.loads(capsys.readouterr().out.strip().splitlines()[-1])
def test_resolve_prebuilt_available(monkeypatch, capsys):
plan = SimpleNamespace(
release_tag = "b9585",
llama_tag = "b9585",
attempts = [
SimpleNamespace(name = "llama-b9585-bin-macos-arm64.tar.gz", install_kind = "macos-arm64")
],
)
out = _run_resolve(monkeypatch, capsys, [plan])
assert out["prebuilt_available"] is True
assert out["repo"] == FORK
assert out["release_tag"] == "b9585"
assert out["asset"] == "llama-b9585-bin-macos-arm64.tar.gz"
assert out["install_kind"] == "macos-arm64"
def test_resolve_prebuilt_unavailable(monkeypatch, capsys):
out = _run_resolve(monkeypatch, capsys, ilp.PrebuiltFallback("no macOS asset"))
assert out["prebuilt_available"] is False
assert out["repo"] == FORK
def _run_resolve_capture_host(monkeypatch, capsys):
"""Drive --resolve-prebuilt and return the host the resolver was handed."""
seen = {}
def _resolver(tag, host, repo, published_release_tag):
seen["repo"] = repo
seen["host"] = host
raise ilp.PrebuiltFallback("no asset")
monkeypatch.setattr(ilp, "resolve_simple_install_release_plans", _resolver)
monkeypatch.setattr(
sys,
"argv",
["install_llama_prebuilt.py", "--resolve-prebuilt", "latest", "--output-format", "json"],
)
assert ilp.main() == ilp.EXIT_SUCCESS
out = json.loads(capsys.readouterr().out.strip().splitlines()[-1])
return seen, out
def test_resolve_prebuilt_cpu_linux_routes_to_fork(monkeypatch, capsys):
# CPU-only Linux host (no GPU): the dispatch routes to the fork, which now
# ships the CPU prebuilt -- it no longer falls back to ggml-org upstream.
monkeypatch.setattr(ilp, "detect_host", lambda: _host(is_linux = True, is_x86_64 = True))
seen, out = _run_resolve_capture_host(monkeypatch, capsys)
assert seen["repo"] == FORK
assert out["repo"] == FORK
def test_resolve_prebuilt_rocm_sdk_only_host_still_offered_cpu(monkeypatch, capsys):
# A CPU-only host that merely has ROCm/HIP SDK tools on PATH (no AMD GPU, so
# detect_host leaves has_rocm False) is a valid CPU-prebuilt target. The probe
# must NOT reclassify it as ROCm from tool presence alone and suppress the CPU
# bundle -- that would deny the fork CPU prebuilt to a legitimate CPU source
# build. The host is left CPU-only and resolves against the fork.
monkeypatch.setattr(ilp, "detect_host", lambda: _host(is_linux = True, is_x86_64 = True))
monkeypatch.setattr(
ilp.shutil, "which", lambda tool: "/opt/rocm/bin/hipconfig" if tool == "hipconfig" else None
)
seen, out = _run_resolve_capture_host(monkeypatch, capsys)
assert seen["repo"] == FORK
assert seen["host"].has_rocm is False
# Blackwell floor is sm_100 (data-center B100/B200, B300/GB300), below consumer
# sm_120 -- 120 wrongly excluded data-center hosts from the prebuilt selection.
def _gpu_linux_host(caps):
return _host(
is_linux = True,
is_x86_64 = True,
has_physical_nvidia = True,
has_usable_nvidia = True,
driver_cuda_version = (13, 1),
compute_caps = caps,
)
# _host_is_blackwell / _blackwell_min_toolkit_for_host are prebuilt_core
# re-exports; their value tables moved verbatim to
# tests/studio/install/test_prebuilt_core.py.
def _linux_cuda_artifact(runtime_line, supported_sms, min_sm, max_sm, profile):
return ilp.PublishedLlamaArtifact(
asset_name = f"app-b9739-linux-x64-{profile}.tar.gz",
install_kind = "linux-cuda",
runtime_line = runtime_line,
coverage_class = "newer",
supported_sms = supported_sms,
min_sm = min_sm,
max_sm = max_sm,
bundle_profile = profile,
rank = 50,
)
def test_linux_blackwell_override_prefers_cuda13_for_datacenter(monkeypatch):
# Both bundles cover sm_100 and torch reports cuda12, so coverage alone can't
# decide -- only the sm_100 Blackwell floor lifts cuda13 to the front.
cuda12 = _linux_cuda_artifact(
"cuda12", ["86", "89", "90", "100", "120"], 86, 120, "cuda12-newer"
)
cuda13 = _linux_cuda_artifact(
"cuda13", ["86", "89", "90", "100", "103", "120"], 86, 120, "cuda13-newer"
)
release = ilp.PublishedReleaseBundle(
repo = FORK,
release_tag = "b9739-mix",
upstream_tag = "b9739",
assets = {cuda12.asset_name: "https://x/cuda12", cuda13.asset_name: "https://x/cuda13"},
artifacts = [cuda12, cuda13],
)
monkeypatch.setattr(
ilp,
"detected_linux_runtime_lines",
lambda: (["cuda13", "cuda12"], {"cuda13": ["/usr/lib"], "cuda12": ["/usr/lib"]}),
)
selection = ilp.linux_cuda_choice_from_release(
_gpu_linux_host(["10.0"]), release, preferred_runtime_line = "cuda12"
)
assert selection is not None
assert selection.primary.runtime_line == "cuda13"
assert selection.primary.bundle_profile == "cuda13-newer"
def test_drop_blackwell_incapable_windows_cuda_applies_to_datacenter():
# B200 (sm_100) on Windows must drop the cuda-12.4 build and keep cuda13.
host = _host(
system = "Windows",
is_windows = True,
is_x86_64 = True,
has_physical_nvidia = True,
has_usable_nvidia = True,
compute_caps = ["10.0"],
)
cuda124 = ilp.AssetChoice(
repo = FORK,
tag = "b9739",
name = "llama-b9739-bin-win-cuda-12.4-x64.zip",
url = "https://x/124",
source_label = "published",
install_kind = "windows-cuda",
)
cuda13 = ilp.AssetChoice(
repo = FORK,
tag = "b9739",
name = "app-b9739-windows-x64-cuda13-newer.zip",
url = "https://x/13",
source_label = "published",
install_kind = "windows-cuda",
max_sm = 120,
)
kept = ilp._drop_blackwell_incapable_windows_cuda(host, [cuda124, cuda13])
assert [a.name for a in kept] == [cuda13.name]
def test_sm103_host_drops_cuda128_windows_build():
# B300 (sm_103) needs cuda-12.9: a legacy win-cuda-12.8 build must be dropped.
host = _host(
system = "Windows",
is_windows = True,
is_x86_64 = True,
has_physical_nvidia = True,
has_usable_nvidia = True,
compute_caps = ["10.3"],
)
cuda128 = ilp.AssetChoice(
repo = FORK,
tag = "b9739",
name = "llama-b9739-bin-win-cuda-12.8-x64.zip",
url = "https://x/128",
source_label = "published",
install_kind = "windows-cuda",
)
cuda129 = ilp.AssetChoice(
repo = FORK,
tag = "b9739",
name = "llama-b9739-bin-win-cuda-12.9-x64.zip",
url = "https://x/129",
source_label = "published",
install_kind = "windows-cuda",
)
kept = ilp._drop_blackwell_incapable_windows_cuda(host, [cuda128, cuda129])
assert [a.name for a in kept] == [cuda129.name]
# sm_100 stays on the 12.8 family floor and keeps the same 12.8 build.
b200 = _host(
system = "Windows",
is_windows = True,
is_x86_64 = True,
has_physical_nvidia = True,
has_usable_nvidia = True,
compute_caps = ["10.0"],
)
kept_b200 = ilp._drop_blackwell_incapable_windows_cuda(b200, [cuda128, cuda129])
assert [a.name for a in kept_b200] == [cuda128.name, cuda129.name]
def _upstream_release(tag, asset_names):
return {
"tag_name": tag,
"assets": [
{"name": n, "browser_download_url": f"https://example/{n}"} for n in asset_names
],
}
def test_direct_upstream_arm64_intel_prefers_vulkan():
# Auto-detected Intel GPU on Linux arm64 -> Vulkan prebuilt first, CPU
# second (mirrors the x86_64 branch; ggml-org ships the arm64 Vulkan asset).
host = _host(is_linux = True, is_arm64 = True, machine = "aarch64", has_intel_gpu = True)
rel = _upstream_release(
"b9925",
["llama-b9925-bin-ubuntu-vulkan-arm64.tar.gz", "llama-b9925-bin-ubuntu-arm64.tar.gz"],
)
plan = ilp.direct_upstream_release_plan(rel, host, UPSTREAM, "latest")
kinds = [a.install_kind for a in plan.attempts]
assert kinds[0] == "linux-vulkan", kinds
assert "linux-arm64" in kinds
assert plan.attempts[0].name == "llama-b9925-bin-ubuntu-vulkan-arm64.tar.gz"
def test_direct_upstream_intel_with_hidden_nvidia_is_cpu_only():
# A host with a physical NVIDIA hidden via CUDA_VISIBLE_DEVICES (physical
# True, usable False) + an Intel iGPU must NOT get the Vulkan archive even
# when planning directly against upstream: Vulkan ignores CUDA_VISIBLE_DEVICES
# and could grab the reserved card. It falls through to the CPU asset.
host = _host(
is_linux = True,
is_x86_64 = True,
has_intel_gpu = True,
has_physical_nvidia = True,
has_usable_nvidia = False,
)
rel = _upstream_release(
"b9925",
["llama-b9925-bin-ubuntu-vulkan-x64.tar.gz", "llama-b9925-bin-ubuntu-x64.tar.gz"],
)
plan = ilp.direct_upstream_release_plan(rel, host, UPSTREAM, "latest")
assert [a.install_kind for a in plan.attempts] == ["linux-cpu"]
def test_direct_upstream_arm64_without_intel_is_cpu_only():
host = _host(is_linux = True, is_arm64 = True, machine = "aarch64")
rel = _upstream_release(
"b9925",
["llama-b9925-bin-ubuntu-vulkan-arm64.tar.gz", "llama-b9925-bin-ubuntu-arm64.tar.gz"],
)
plan = ilp.direct_upstream_release_plan(rel, host, UPSTREAM, "latest")
assert [a.install_kind for a in plan.attempts] == ["linux-arm64"]
def test_direct_upstream_x86_intel_prefers_vulkan():
host = _host(is_linux = True, is_x86_64 = True, has_intel_gpu = True)
rel = _upstream_release(
"b9925",
["llama-b9925-bin-ubuntu-vulkan-x64.tar.gz", "llama-b9925-bin-ubuntu-x64.tar.gz"],
)
plan = ilp.direct_upstream_release_plan(rel, host, UPSTREAM, "latest")
kinds = [a.install_kind for a in plan.attempts]
assert kinds[0] == "linux-vulkan", kinds
assert "linux-cpu" in kinds
def test_linux_vulkan_health_glob_matches_bare_cpu_lib():
# The widened glob must cover both arch-suffixed (x64) and bare (arm64) CPU
# libs so a valid Vulkan install is not re-flagged unhealthy every check.
choice = ilp.AssetChoice(
repo = UPSTREAM,
tag = "b9925",
name = "llama-b9925-bin-ubuntu-vulkan-arm64.tar.gz",
url = "https://example/x",
source_label = "upstream",
install_kind = "linux-vulkan",
)
groups = ilp.runtime_payload_health_groups(choice)
assert ["libggml-cpu*.so*"] in groups
assert ["libggml-cpu-*.so*"] not in groups
def test_route_to_vulkan_prebuilt_auto_intel_goes_upstream_and_drops_fork_pin():
# Routing fork -> upstream also drops the fork release pin, which is in a
# different tag namespace and would make the upstream resolver miss.
host = _host(is_linux = True, is_x86_64 = True, has_intel_gpu = True)
routed, repo, tag, _persist = ilp._route_to_vulkan_prebuilt(
host, FORK, "b9596-mix-abc", force_cpu = False
)
assert repo == UPSTREAM
assert tag == ""
assert routed.has_intel_gpu is True
def test_route_to_vulkan_prebuilt_preserves_explicit_upstream_pin():
# A pin set WITH an explicit upstream repo is already on upstream -> kept.
host = _host(is_linux = True, is_x86_64 = True, has_intel_gpu = True)
_routed, repo, tag, _persist = ilp._route_to_vulkan_prebuilt(
host, UPSTREAM, "b9596", force_cpu = False
)
assert repo == UPSTREAM
assert tag == "b9596"
def test_route_to_vulkan_prebuilt_cpu_fallback_wins():
# --cpu-fallback suppresses Vulkan routing even for an Intel host.
host = _host(is_linux = True, is_x86_64 = True, has_intel_gpu = True)
routed, repo, tag, _persist = ilp._route_to_vulkan_prebuilt(
host, FORK, "b9596-mix-abc", force_cpu = True
)
assert repo == FORK
assert tag == "b9596-mix-abc"
assert routed is host
@pytest.mark.parametrize("cpu_flag", ["--cpu-fallback", "--force-cpu"])
def test_resolve_prebuilt_cpu_fallback_overrides_intel_vulkan(monkeypatch, capsys, cpu_flag):
"""Either CPU flag via CLI must suppress Vulkan even on an Intel GPU host: both
drop GPU detection (--force-cpu additionally persists, on the install path)."""
monkeypatch.setattr(
ilp,
"detect_host",
lambda: _host(is_linux = True, is_x86_64 = True, has_intel_gpu = True),
)
seen = {}
def _resolver(tag, host, repo, published_release_tag):
seen["host"] = host
seen["repo"] = repo
raise ilp.PrebuiltFallback("no asset")
monkeypatch.setattr(ilp, "resolve_simple_install_release_plans", _resolver)
monkeypatch.setattr(
sys,
"argv",
[
"install_llama_prebuilt.py",
"--resolve-prebuilt",
"latest",
cpu_flag,
"--output-format",
"json",
],
)
assert ilp.main() == ilp.EXIT_SUCCESS
# The CPU flag must suppress Intel GPU, route to fork (not upstream Vulkan)
assert seen["host"].has_intel_gpu is False
assert seen["repo"] == FORK
@pytest.mark.parametrize(
"flags, expect_force, expect_persist",
[
([], False, False),
# Automatic/transient last resort (arm64 GPU-build recovery): drops GPU but
# does NOT persist, so a later update heals to a GPU bundle (#6097).
(["--cpu-fallback"], True, False),
# Deliberate CPU-only (UNSLOTH_LLAMA_CPP_BACKEND=cpu): drops GPU AND persists so
# the updater re-asserts it and never revives the Intel iGPU crash (#7213).
(["--force-cpu"], True, True),
(["--cpu-fallback", "--force-cpu"], True, True),
],
)
def test_cli_cpu_flags_thread_force_and_persist(
monkeypatch, tmp_path, flags, expect_force, expect_persist
):
captured = {}
monkeypatch.setattr(ilp, "install_prebuilt", lambda **kw: captured.update(kw))
monkeypatch.setattr(
sys,
"argv",
["install_llama_prebuilt.py", "--install-dir", str(tmp_path / "llama.cpp"), *flags],
)
assert ilp.main() == ilp.EXIT_SUCCESS
assert captured["force_cpu"] is expect_force
assert captured["persist_force_cpu"] is expect_persist
@pytest.mark.parametrize(
"existing, requested, expected",
[
# A deliberate --force-cpu on top of a naturally-installed CPU bundle (same
# asset, install skipped) must still flip the marker to true (#7213).
(False, True, True),
(None, True, True),
# No spurious writes when already in sync, and a released force syncs down.
(True, True, True),
(False, False, False),
(True, False, False),
],
)
def test_sync_marker_force_cpu(tmp_path, existing, requested, expected):
marker = {"tag": "b9585", "asset": "llama-b9585-bin-ubuntu-x64.tar.gz"}
if existing is not None:
marker["force_cpu"] = existing
marker_path = tmp_path / "UNSLOTH_PREBUILT_INFO.json"
marker_path.write_text(json.dumps(marker))
ilp.sync_marker_force_cpu(tmp_path, requested)
written = json.loads(marker_path.read_text())
assert written["force_cpu"] is expected
# Unrelated fields are preserved.
assert written["asset"] == "llama-b9585-bin-ubuntu-x64.tar.gz"
def test_sync_marker_force_cpu_missing_marker_is_noop(tmp_path):
# No marker (or unreadable) must not crash the reuse path.
ilp.sync_marker_force_cpu(tmp_path, True)
assert not (tmp_path / "UNSLOTH_PREBUILT_INFO.json").exists()
def test_route_to_vulkan_prebuilt_hidden_nvidia_not_rerouted():
# A mixed NVIDIA+Intel host that hid NVIDIA (CUDA_VISIBLE_DEVICES=""/-1):
# physical NVIDIA present but not usable. Must NOT auto-route to Vulkan, or
# Vulkan (which ignores CUDA_VISIBLE_DEVICES) could grab the reserved GPU.
host = _host(
is_linux = True,
is_x86_64 = True,
has_intel_gpu = True,
has_physical_nvidia = True,
has_usable_nvidia = False,
)
_routed, repo, _tag, _persist = ilp._route_to_vulkan_prebuilt(host, FORK, "", force_cpu = False)
assert repo == FORK
def test_route_to_vulkan_prebuilt_rocm_host_not_rerouted():
# An Intel iGPU alongside a usable ROCm GPU stays on its ROCm/fork path.
host = _host(is_linux = True, is_x86_64 = True, has_intel_gpu = True, has_rocm = True)
_routed, repo, _tag, _persist = ilp._route_to_vulkan_prebuilt(host, FORK, "", force_cpu = False)
assert repo == FORK
def test_route_to_vulkan_prebuilt_non_intel_unchanged():
host = _host(is_linux = True, is_x86_64 = True)
routed, repo, _tag, _persist = ilp._route_to_vulkan_prebuilt(host, FORK, "", force_cpu = False)
assert repo == FORK
assert routed is host
def test_resolve_prebuilt_intel_host_routes_to_upstream(monkeypatch, capsys):
# The --resolve-prebuilt probe must agree with the install path: an
# auto-detected Intel host resolves against upstream (Vulkan), not the fork.
monkeypatch.setattr(
ilp, "detect_host", lambda: _host(is_linux = True, is_x86_64 = True, has_intel_gpu = True)
)
seen, out = _run_resolve_capture_host(monkeypatch, capsys)
assert seen["repo"] == UPSTREAM
assert out["repo"] == UPSTREAM
# ---------------------------------------------------------------------------
# windows_intel_gpu_in_registry: the in-process Windows Intel probe. A fake
# winreg module stands in for the real registry so the walk runs anywhere.
# ---------------------------------------------------------------------------
class _FakeRegKey:
def __init__(
self,
subkeys = None,
values = None,
denied = False,
):
self.subkeys = subkeys or {}
self.values = values or {}
self.denied = denied
def __enter__(self):
return self
def __exit__(self, *exc):
return False
class _FakeWinreg:
HKEY_LOCAL_MACHINE = object()
def __init__(self, root_key):
self._root_key = root_key
def OpenKey(self, parent, name):
if parent is self.HKEY_LOCAL_MACHINE:
# Pin the production constant: a typo'd class GUID must fail here,
# not silently return the fake tree.
if name != ilp._WINDOWS_DISPLAY_CLASS_KEY:
raise FileNotFoundError(name)
if self._root_key is None:
raise FileNotFoundError(name)
return self._root_key
key = parent.subkeys.get(name)
if key is None:
# Real winreg raises OSError, never KeyError, for a missing key.
raise FileNotFoundError(name)
if key.denied:
raise PermissionError(name)
return key
def QueryInfoKey(self, key):
return (len(key.subkeys), len(key.values), 0)
def EnumKey(self, key, index):
return list(key.subkeys)[index]
def QueryValueEx(self, key, value_name):
if value_name not in key.values:
raise FileNotFoundError(value_name)
return (key.values[value_name], 1)
def _probe_with_display_class(monkeypatch, adapters):
# The helper lazily does `import winreg`; plant the fake in sys.modules the
# same way unsloth_cli/tests/test_start.py fakes it for _refresh_windows_path.
monkeypatch.setitem(sys.modules, "winreg", _FakeWinreg(_FakeRegKey(subkeys = adapters)))
return ilp.windows_intel_gpu_in_registry()
def test_windows_intel_registry_matches_vendor_id(monkeypatch):
assert (
_probe_with_display_class(
monkeypatch,
{
"0000": _FakeRegKey(
values = {
"MatchingDeviceId": r"PCI\VEN_8086&DEV_56A0&SUBSYS_12345678",
"DriverDesc": "Intel(R) Arc(TM) A770 Graphics",
}
),
},
)
is True
)
def test_windows_intel_registry_matches_driver_desc_without_device_id(monkeypatch):
assert (
_probe_with_display_class(
monkeypatch,
{
"0000": _FakeRegKey(values = {"DriverDesc": "Intel(R) UHD Graphics 630"}),
},
)
is True
)
def test_windows_intel_registry_ignores_non_intel_adapters(monkeypatch):
assert (
_probe_with_display_class(
monkeypatch,
{
"0000": _FakeRegKey(
values = {
"MatchingDeviceId": r"PCI\VEN_10DE&DEV_2684",
"DriverDesc": "NVIDIA GeForce RTX 4090",
}
),
"0001": _FakeRegKey(
values = {
"MatchingDeviceId": r"PCI\VEN_1002&DEV_744C",
"DriverDesc": "AMD Radeon RX 7900 XTX",
}
),
},
)
is False
)
def test_windows_intel_registry_skips_restricted_properties_subkey(monkeypatch):
# The real class key carries an ACL-restricted "Properties" subkey and can
# deny access to individual adapter keys; neither may abort the walk.
assert (
_probe_with_display_class(
monkeypatch,
{
"Properties": _FakeRegKey(denied = True),
"0000": _FakeRegKey(denied = True),
"0001": _FakeRegKey(
values = {
"MatchingDeviceId": r"PCI\VEN_8086&DEV_56A0",
}
),
},
)
is True
)
def test_windows_intel_registry_missing_class_key_is_false(monkeypatch):
monkeypatch.setitem(sys.modules, "winreg", _FakeWinreg(None))
assert ilp.windows_intel_gpu_in_registry() is False
def _detect_windows_host(
monkeypatch,
winreg_fake,
powershell_stdout = "",
):
"""Drive the real detect_host() as a GPU-less Windows host with a fake
registry, recording every run_capture invocation. Pins the wiring the
unit tests above cannot see: registry-first, CIM only on a registry miss."""
monkeypatch.setitem(sys.modules, "winreg", winreg_fake)
monkeypatch.setattr(ilp.platform, "system", lambda: "Windows")
monkeypatch.setattr(ilp.platform, "machine", lambda: "AMD64")
for _env in (
"CUDA_VISIBLE_DEVICES",
"HIP_VISIBLE_DEVICES",
"ROCR_VISIBLE_DEVICES",
"HIP_PATH",
"ROCM_PATH",
):
monkeypatch.delenv(_env, raising = False)
monkeypatch.setattr(
ilp.shutil,
"which",
lambda name: "powershell" if name in ("powershell", "pwsh") else None,
)
captured = []
def _fake_run_capture(command, **kwargs):
captured.append(command[0])
if command[0] == "powershell":
return SimpleNamespace(returncode = 0, stdout = powershell_stdout, stderr = "")
return SimpleNamespace(returncode = 1, stdout = "", stderr = "")
monkeypatch.setattr(ilp, "run_capture", _fake_run_capture)
return ilp.detect_host(), captured
def test_detect_host_registry_intel_skips_cim_probe(monkeypatch):
winreg = _FakeWinreg(
_FakeRegKey(
subkeys = {
"0000": _FakeRegKey(values = {"MatchingDeviceId": r"PCI\VEN_8086&DEV_56A0"}),
}
)
)
host, captured = _detect_windows_host(monkeypatch, winreg)
assert host.has_intel_gpu is True
assert "powershell" not in captured
def test_detect_host_cim_fallback_fires_on_registry_miss(monkeypatch):
winreg = _FakeWinreg(
_FakeRegKey(
subkeys = {
"0000": _FakeRegKey(values = {"MatchingDeviceId": r"PCI\VEN_10DE&DEV_2684"}),
}
)
)
host, captured = _detect_windows_host(
monkeypatch, winreg, powershell_stdout = "Intel(R) Arc(TM) A770 Graphics"
)
assert host.has_intel_gpu is True
assert "powershell" in captured
def test_windows_intel_registry_unexpected_error_is_false(monkeypatch):
# The probe is advisory: even a non-OSError bug in the walk must return
# False (deferring to the CIM fallback), never crash detect_host.
class _ExplodingWinreg:
HKEY_LOCAL_MACHINE = object()
def OpenKey(self, parent, name):
raise TypeError(name)
monkeypatch.setitem(sys.modules, "winreg", _ExplodingWinreg())
assert ilp.windows_intel_gpu_in_registry() is False
def test_detect_host_cim_rescues_exploding_registry(monkeypatch):
class _ExplodingWinreg:
HKEY_LOCAL_MACHINE = object()
def OpenKey(self, parent, name):
raise TypeError(name)
host, captured = _detect_windows_host(
monkeypatch, _ExplodingWinreg(), powershell_stdout = "Intel(R) Arc(TM) A770 Graphics"
)
assert host.has_intel_gpu is True
assert "powershell" in captured
def _windows_amd_host(**overrides):
defaults = dict(
system = "Windows",
machine = "amd64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
has_rocm = True,
has_intel_gpu = False,
)
defaults.update(overrides)
return ilp.HostInfo(**defaults)
def test_route_to_vulkan_prebuilt_auto_fallback_for_legacy_amd_gfx():
host = _windows_amd_host(rocm_gfx_target = "gfx803", rocm_gfx_targets = ["gfx803"])
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert repo == UPSTREAM
assert persist == "vulkan"
assert routed.has_intel_gpu is True
assert routed.has_rocm is False
def test_route_to_vulkan_prebuilt_keeps_hip_when_one_gpu_is_supported():
host = _windows_amd_host(
rocm_gfx_target = "gfx1201",
rocm_gfx_targets = ["gfx1201", "gfx803"],
)
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert routed is host
assert repo == FORK
assert persist is None
def test_route_to_vulkan_prebuilt_auto_fallback_skips_hip_masked_hosts():
# A HIP mask can hide a HIP-capable dGPU, but the Vulkan runtime honours none of them,
# so auto-routing would let the installed backend grab the gfx1201 the user masked
# off.
host = _windows_amd_host(
rocm_gfx_target = "gfx803",
rocm_gfx_targets = ["gfx1201", "gfx803"],
)
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert repo == FORK
assert persist is None
assert routed is host
def test_route_to_vulkan_prebuilt_auto_fallback_when_no_amd_gpu_reaches_floor():
# Every physical AMD device is below the floor, so no card can be exposed to HIP and
# the #7357 auto-Vulkan fallback still fires.
host = _windows_amd_host(
rocm_gfx_target = "gfx900",
rocm_gfx_targets = ["gfx803", "gfx900"],
)
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert repo == UPSTREAM
assert persist == "vulkan"
assert routed.has_rocm is False
@pytest.mark.parametrize(
"mask_env", ["HIP_VISIBLE_DEVICES", "ROCR_VISIBLE_DEVICES", "CUDA_VISIBLE_DEVICES"]
)
def test_auto_vulkan_declines_when_a_hip_device_mask_filtered_the_probe(mask_env, monkeypatch):
# hipinfo is a HIP application, so under a mask rocm_gfx_targets is the VISIBLE set and
# a HIP-capable card can be hidden entirely. "No AMD GPU here reaches the floor" is then
# unprovable, and Vulkan honours none of these masks, so the auto fallback must decline
# rather than hand it the reserved card.
monkeypatch.setenv(mask_env, "1")
host = _windows_amd_host(rocm_gfx_target = "gfx803", rocm_gfx_targets = ["gfx803"])
assert ilp._should_auto_vulkan_for_amd_windows(host, FORK) is False
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert routed is host
assert repo == FORK
assert persist is None
@pytest.mark.parametrize("mask_value", ["", " ", "-1"])
def test_auto_vulkan_declines_when_the_mask_hides_every_amd_gpu(mask_value, monkeypatch):
# An all-hiding mask is the strongest form of the same signal, not an exemption:
# detect_host() resolves no arch under it, but a forwarded --rocm-gfx still reconstructs
# one (setup infers it from the display-adapter name, which no HIP mask touches), so
# auto-routing would hand Vulkan every AMD GPU the user hid from HIP.
monkeypatch.setenv("HIP_VISIBLE_DEVICES", mask_value)
host = _windows_amd_host(rocm_gfx_target = None, rocm_gfx_targets = [])
host = ilp._apply_host_overrides(host, override_rocm_gfx = "gfx803")
assert ilp._active_rocm_gfx_target(host) == "gfx803"
assert ilp._should_auto_vulkan_for_amd_windows(host, FORK) is False
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert routed is host
assert repo == FORK
assert persist is None
def test_hip_device_mask_check_is_presence_not_value(monkeypatch):
# Presence is the whole test: any value means the HIP view is not the physical one, and
# no value can be read as "the probe saw everything".
assert ilp._hip_visible_device_mask_set() is False
for value in ("", " ", "-1", "0", "1", "0,1"):
monkeypatch.setenv("HIP_VISIBLE_DEVICES", value)
assert ilp._hip_visible_device_mask_set() is True, value
monkeypatch.delenv("HIP_VISIBLE_DEVICES")
monkeypatch.setenv("ROCR_VISIBLE_DEVICES", "0")
assert ilp._hip_visible_device_mask_set() is True
monkeypatch.delenv("ROCR_VISIBLE_DEVICES")
monkeypatch.setenv("CUDA_VISIBLE_DEVICES", "0")
assert ilp._hip_visible_device_mask_set() is True
def test_masked_probe_suppression_does_not_touch_non_amd_auto_paths(monkeypatch):
# The mask says nothing about an Intel iGPU, whose Vulkan auto path is unrelated.
monkeypatch.setenv("HIP_VISIBLE_DEVICES", "1")
host = _host(
system = "Windows",
is_windows = True,
has_intel_gpu = True,
has_rocm = False,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
_routed, repo, _tag, _persist = ilp._route_to_vulkan_prebuilt(
host, FORK, "pin", force_cpu = False
)
assert repo == UPSTREAM
def test_route_to_vulkan_prebuilt_hip_masked_host_still_honours_explicit_optin(monkeypatch):
# The mask guard only suppresses the AUTOMATIC fallback; an explicit opt-in is the user
# taking responsibility for the Vulkan device mask themselves.
monkeypatch.delenv("UNSLOTH_FORCE_VULKAN", raising = False)
host = _windows_amd_host(
rocm_gfx_target = "gfx803",
rocm_gfx_targets = ["gfx1201", "gfx803"],
)
_routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(
host, FORK, "pin", force_cpu = False, llama_backend = "vulkan"
)
assert repo == UPSTREAM
assert persist == "vulkan"
def test_auto_vulkan_is_repository_specific_for_fork_only_gfx():
# gfx1034 is served only by the fork's gfx103X bundle: ggml-org's windows-hip radeon
# build does not target it and direct_upstream_release_plan() offers win-hip then CPU
# with no Vulkan branch, so the predicate must answer per repo.
host = _windows_amd_host(rocm_gfx_target = "gfx1034", rocm_gfx_targets = ["gfx1034"])
assert ilp._should_auto_vulkan_for_amd_windows(host, FORK) is False
assert ilp._should_auto_vulkan_for_amd_windows(host, UPSTREAM) is True
# An arch upstream really does build stays on HIP for both repos.
supported = _windows_amd_host(rocm_gfx_target = "gfx1100", rocm_gfx_targets = ["gfx1100"])
assert ilp._should_auto_vulkan_for_amd_windows(supported, FORK) is False
assert ilp._should_auto_vulkan_for_amd_windows(supported, UPSTREAM) is False
# A family label is a bundle name, not an arch: upstream builds every member but
# gfx1034 / gfx1103, and the label cannot say which card this is, so it stays on HIP
# rather than moving the covered members onto Vulkan.
family = _windows_amd_host(rocm_gfx_target = "gfx110X", rocm_gfx_targets = ["gfx110X"])
assert ilp._should_auto_vulkan_for_amd_windows(family, UPSTREAM) is False
@pytest.mark.parametrize(
"repo", ["acme/llama.cpp-mirror", "GGML-ORG/llama.cpp", "unslothAI/llama.cpp"]
)
def test_fork_only_gfx_coverage_is_not_granted_to_other_repos(repo):
# Only the fork is planned from a manifest: resolve_simple_install_release_plans()
# compares == DEFAULT_PUBLISHED_REPO and sends everything else, mirrors and differently
# cased spellings alike, to direct_upstream_release_plan(). Granting a fork-only arch
# coverage there lands it on win-hip-radeon or CPU instead of Vulkan, so the predicate
# must gate on the fork rather than exempt one name.
host = _windows_amd_host(rocm_gfx_target = "gfx1034", rocm_gfx_targets = ["gfx1034"])
assert ilp._should_auto_vulkan_for_amd_windows(host, repo) is True
supported = _windows_amd_host(rocm_gfx_target = "gfx1100", rocm_gfx_targets = ["gfx1100"])
assert ilp._should_auto_vulkan_for_amd_windows(supported, repo) is False
@pytest.mark.parametrize("repo", [None, ""])
def test_empty_published_repo_gets_fork_coverage(repo):
# Negative control: the resolver defaults an empty repo to the fork, so the predicate
# must too, or the default install path loses its fork-only archs.
host = _windows_amd_host(rocm_gfx_target = "gfx1034", rocm_gfx_targets = ["gfx1034"])
assert ilp._should_auto_vulkan_for_amd_windows(host, repo) is False
def test_upstream_windows_hip_targets_are_a_subset_of_the_combined_floor():
# The floor must stay a superset, else auto-Vulkan steals a host upstream builds for.
assert ilp.UPSTREAM_WINDOWS_HIP_GFX_TARGETS <= ilp.WINDOWS_HIP_PREBUILT_GFX_TARGETS
# The fork-only extras are exactly the archs that must route to Vulkan upstream.
assert ilp.WINDOWS_HIP_PREBUILT_GFX_TARGETS - ilp.UPSTREAM_WINDOWS_HIP_GFX_TARGETS == {
"gfx908",
"gfx90a",
"gfx1034",
"gfx1103",
}
def test_route_to_vulkan_prebuilt_unknown_gfx_does_not_auto_fallback():
host = _windows_amd_host(
has_rocm = True,
rocm_gfx_target = None,
rocm_gfx_targets = [],
)
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert routed is host
assert repo == FORK
assert persist is None
def test_route_to_vulkan_prebuilt_family_gfx_token_keeps_rocm():
host = _windows_amd_host(rocm_gfx_target = "gfx110X", rocm_gfx_targets = ["gfx110X"])
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert routed is host
assert repo == FORK
assert persist is None
def test_route_to_vulkan_prebuilt_gfx1103_keeps_rocm():
host = _windows_amd_host(rocm_gfx_target = "gfx1103", rocm_gfx_targets = ["gfx1103"])
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert routed is host
assert repo == FORK
assert persist is None
def test_route_to_vulkan_prebuilt_gfx1034_keeps_rocm():
# gfx1034 (RX 6500/6400-class) is covered by the fork's gfx103X bundle.
host = _windows_amd_host(rocm_gfx_target = "gfx1034", rocm_gfx_targets = ["gfx1034"])
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert routed is host
assert repo == FORK
assert persist is None
def test_route_to_vulkan_prebuilt_explicit_opt_in_on_mixed_amd(monkeypatch):
monkeypatch.setenv("UNSLOTH_LLAMA_BACKEND", "vulkan")
host = _windows_amd_host(
rocm_gfx_target = "gfx1201",
rocm_gfx_targets = ["gfx1201", "gfx803"],
)
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert repo == UPSTREAM
assert persist == "vulkan"
assert routed.has_rocm is False
def test_direct_upstream_windows_amd_legacy_gfx_routes_to_vulkan():
host = _windows_amd_host(rocm_gfx_target = "gfx803", rocm_gfx_targets = ["gfx803"])
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
rel = _upstream_release(
"b9925",
[
"llama-b9925-bin-win-hip-radeon-x64.zip",
"llama-b9925-bin-win-vulkan-x64.zip",
"llama-b9925-bin-win-cpu-x64.zip",
],
)
plan = ilp.direct_upstream_release_plan(rel, routed, repo, "latest")
assert persist == "vulkan"
assert plan.attempts[0].install_kind == "windows-vulkan"
def test_llama_backend_env_requests_vulkan(monkeypatch):
assert ilp.llama_backend_from_env() is None
monkeypatch.setenv("UNSLOTH_LLAMA_BACKEND", "vulkan")
assert ilp.llama_backend_from_env() == "vulkan"
assert ilp.force_vulkan_requested() is True
def test_llama_cpp_backend_env_does_not_trigger_vulkan(monkeypatch):
# UNSLOTH_LLAMA_CPP_BACKEND is a separate setup variable (auto/cpu) whose other values
# setup warns about and ignores, so reading it here would opt in behind that warning.
monkeypatch.delenv("UNSLOTH_LLAMA_BACKEND", raising = False)
monkeypatch.delenv("UNSLOTH_FORCE_VULKAN", raising = False)
monkeypatch.setenv("UNSLOTH_LLAMA_CPP_BACKEND", "vulkan")
assert ilp.llama_backend_from_env() is None
assert ilp.force_vulkan_requested() is False
def test_route_to_vulkan_prebuilt_hidden_physical_nvidia_amd_not_rerouted():
# Vulkan ignores CUDA_VISIBLE_DEVICES, so a CUDA-masked NVIDIA card next to a legacy
# AMD gfx must not auto-route: Vulkan could grab the reserved NVIDIA GPU.
host = _windows_amd_host(
rocm_gfx_target = "gfx803",
rocm_gfx_targets = ["gfx803"],
has_physical_nvidia = True,
has_usable_nvidia = False,
)
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert routed is host
assert repo == FORK
assert persist is None
def test_route_to_vulkan_prebuilt_explicit_opt_in_overrides_hidden_nvidia(monkeypatch):
# The physical-NVIDIA guard only gates the AMD auto path; an explicit opt-in wins.
monkeypatch.setenv("UNSLOTH_LLAMA_BACKEND", "vulkan")
host = _windows_amd_host(
rocm_gfx_target = "gfx803",
rocm_gfx_targets = ["gfx803"],
has_physical_nvidia = True,
has_usable_nvidia = False,
)
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert repo == UPSTREAM
assert persist == "vulkan"
# The gfx archs the fork's llama-prebuilt-manifest.json maps to a windows-rocm bundle.
# Static because parametrisation happens at import time and the routing tests below must
# stay offline; the guard further down re-derives it from the published manifest and fails
# on drift, so this is a checked mirror, not a second source of truth.
_FORK_WINDOWS_ROCM_GFX = (
"gfx908",
"gfx90a",
"gfx1030",
"gfx1031",
"gfx1032",
"gfx1034",
"gfx1100",
"gfx1101",
"gfx1102",
"gfx1103",
"gfx1150",
"gfx1151",
"gfx1200",
"gfx1201",
)
def _published_fork_windows_rocm_artifacts():
"""The fork's windows-rocm artifact records, read the way an install reads them.
_download_host_resolved_release is the path a default fork install takes first: it
resolves the latest release off the download host and hands llama-prebuilt-manifest.json
to parse_published_release_bundle, so these are the very records
published_rocm_choice_for_host later matches a host gfx against. No api.github.com call,
hence no shared rate-limit bucket to exhaust.
The manifest ships only as a release asset and nothing in-tree mirrors it, so this is
the one honest source. Only OSError and the release-side PrebuiltFallback become a skip,
so an offline run stays quiet while a manifest that fetches but no longer parses still
fails loudly."""
try:
resolved = ilp._download_host_resolved_release(FORK)
except OSError as exc:
pytest.skip(f"{FORK} release manifest unreachable: {exc}")
except ilp.PrebuiltFallback as exc:
pytest.skip(f"{FORK} latest release was rejected before its manifest parsed: {exc}")
if resolved is None:
pytest.skip(f"{FORK} published no resolvable latest release")
tag = resolved.bundle.release_tag
artifacts = [
artifact
for artifact in resolved.bundle.artifacts
if artifact.install_kind == "windows-rocm"
]
assert artifacts, f"{FORK}@{tag} manifest listed no windows-rocm artifacts"
return tag, artifacts
def test_windows_hip_gfx_floor_covers_every_fork_windows_rocm_bundle():
# Derived from the published manifest, not a second literal: a gfx the fork builds but
# the floor omits bypasses the fork manifest, downgrading a hash-approved windows-rocm
# bundle to an unhashed upstream Vulkan build. A newly published arch must redden here.
tag, artifacts = _published_fork_windows_rocm_artifacts()
# published_rocm_choice_for_host serves a bundle on a concrete mapped_targets entry or on
# the umbrella gfx_target itself, so both spellings must clear a floor. A gfx_target
# absent from its own mapped_targets is the family label (gfx110X); one present in it is
# a standalone bundle (gfx908) already counted as concrete.
concrete = {target.lower() for artifact in artifacts for target in artifact.mapped_targets}
labels = {
artifact.gfx_target.lower()
for artifact in artifacts
if artifact.gfx_target and artifact.gfx_target.lower() not in concrete
}
unfloored = sorted(concrete - ilp.WINDOWS_HIP_PREBUILT_GFX_TARGETS)
assert (
not unfloored
), f"auto-Vulkan would steal windows-rocm archs published in {FORK}@{tag}: {unfloored}"
unlabelled = sorted(labels - ilp.WINDOWS_ROCM_FAMILY_GFX_LABELS)
assert not unlabelled, (
f"update markers forward family labels {FORK}@{tag} publishes but "
f"WINDOWS_ROCM_FAMILY_GFX_LABELS omits: {unlabelled}"
)
# Keep the import-time tuple the offline routing tests parametrise on an exact mirror.
assert set(_FORK_WINDOWS_ROCM_GFX) == concrete, (
f"_FORK_WINDOWS_ROCM_GFX drifted from {FORK}@{tag}: "
f"gained {sorted(concrete - set(_FORK_WINDOWS_ROCM_GFX))}, "
f"lost {sorted(set(_FORK_WINDOWS_ROCM_GFX) - concrete)}"
)
@pytest.mark.parametrize("gfx", _FORK_WINDOWS_ROCM_GFX)
def test_route_to_vulkan_prebuilt_keeps_every_fork_windows_rocm_arch(gfx, monkeypatch):
# No ambient opt-in: this asserts the AUTO path leaves covered archs alone.
monkeypatch.delenv("UNSLOTH_LLAMA_BACKEND", raising = False)
monkeypatch.delenv("UNSLOTH_FORCE_VULKAN", raising = False)
host = _windows_amd_host(rocm_gfx_target = gfx, rocm_gfx_targets = [gfx])
routed, repo, tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert routed is host
assert (repo, tag) == (FORK, "pin")
assert persist is None
def test_forwarded_gfx_does_not_undo_visible_device_auto_vulkan(monkeypatch):
# Mixed-AMD Windows host: GPU 0 = gfx1100 (HIP prebuilt exists), GPU 1 = gfx1010 (none).
# Under CUDA_VISIBLE_DEVICES=1 setup.ps1 still resolves GPU 0 and forwards gfx1100, but
# detect_host() resolved the visible gfx1010, so folding the forward in must not
# reinstate gfx1100 and install a HIP bundle the visible GPU cannot run.
monkeypatch.delenv("UNSLOTH_LLAMA_BACKEND", raising = False)
monkeypatch.delenv("UNSLOTH_FORCE_VULKAN", raising = False)
monkeypatch.delenv("UNSLOTH_ROCM_GFX_ARCH", raising = False)
host = _windows_amd_host(rocm_gfx_target = "gfx1010", rocm_gfx_targets = ["gfx1100", "gfx1010"])
host = ilp._apply_host_overrides(host, override_rocm_gfx = "gfx1100")
assert ilp._active_rocm_gfx_target(host) == "gfx1010"
assert host.rocm_gfx_targets == ["gfx1100", "gfx1010"]
# gfx1100 is masked off, not absent, and Vulkan does not honour the HIP mask, so the
# automatic fallback stays off and the HIP / fork path is kept.
assert ilp._should_auto_vulkan_for_amd_windows(host, FORK) is False
_routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert repo == FORK
assert persist is None
def test_forwarded_gfx_absent_from_probe_keeps_the_physical_hip_card(monkeypatch):
# Mixed-AMD Windows host: GPU 0 = gfx1100 (HIP prebuilt exists), GPU 1 = gfx803 (below
# the floor). CUDA_VISIBLE_DEVICES=1 reserves the gfx1100, so detect_host() picks gfx803
# as active but still reports both cards, and setup forwards a third arch the probe never
# saw (a stale env var, or name inference reading the other card). That forward selects
# the HIP target but must not delete the probe's inventory, or the floor check concludes
# no AMD GPU here reaches HIP and auto-routes to Vulkan, which ignores the HIP mask and
# enumerates the reserved gfx1100.
monkeypatch.delenv("UNSLOTH_LLAMA_BACKEND", raising = False)
monkeypatch.delenv("UNSLOTH_FORCE_VULKAN", raising = False)
monkeypatch.delenv("UNSLOTH_ROCM_GFX_ARCH", raising = False)
host = _windows_amd_host(rocm_gfx_target = "gfx803", rocm_gfx_targets = ["gfx1100", "gfx803"])
host = ilp._apply_host_overrides(host, override_rocm_gfx = "gfx900")
assert ilp._active_rocm_gfx_target(host) == "gfx900"
assert host.rocm_gfx_targets == ["gfx1100", "gfx803", "gfx900"]
assert ilp._should_auto_vulkan_for_amd_windows(host, FORK) is False
_routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert repo == FORK
assert persist is None
def test_forwarded_gfx_absent_from_probe_keeps_a_single_probed_hip_card(monkeypatch):
# Same rule on a single-GPU box: a stale below-floor forward over a probe-confirmed
# gfx1100 must not auto-route that machine to Vulkan.
monkeypatch.delenv("UNSLOTH_LLAMA_BACKEND", raising = False)
monkeypatch.delenv("UNSLOTH_FORCE_VULKAN", raising = False)
monkeypatch.delenv("UNSLOTH_ROCM_GFX_ARCH", raising = False)
host = _windows_amd_host(rocm_gfx_target = "gfx1100", rocm_gfx_targets = ["gfx1100"])
host = ilp._apply_host_overrides(host, override_rocm_gfx = "gfx803")
assert ilp._active_rocm_gfx_target(host) == "gfx803"
assert host.rocm_gfx_targets == ["gfx1100", "gfx803"]
assert ilp._should_auto_vulkan_for_amd_windows(host, FORK) is False
def test_forwarded_gfx_absent_from_probe_still_allows_explicit_vulkan(monkeypatch):
# The physical-inventory rule gates the AUTO path only; naming the backend wins.
monkeypatch.setenv("UNSLOTH_LLAMA_BACKEND", "vulkan")
monkeypatch.delenv("UNSLOTH_ROCM_GFX_ARCH", raising = False)
host = _windows_amd_host(rocm_gfx_target = "gfx1100", rocm_gfx_targets = ["gfx1100"])
host = ilp._apply_host_overrides(host, override_rocm_gfx = "gfx803")
_routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert repo == UPSTREAM
assert persist == "vulkan"
def test_forwarded_gfx_on_unprobed_host_still_auto_vulkans(monkeypatch):
# Negative control: a driver-only AMD host runs no successful probe (no hipinfo, amd-smi
# suppressed), so --rocm-gfx is the ONLY source of the arch and there is no inventory to
# preserve. This is the #7357 path the feature exists for; it must still reach Vulkan.
monkeypatch.delenv("UNSLOTH_LLAMA_BACKEND", raising = False)
monkeypatch.delenv("UNSLOTH_FORCE_VULKAN", raising = False)
monkeypatch.delenv("UNSLOTH_ROCM_GFX_ARCH", raising = False)
host = _windows_amd_host(rocm_gfx_target = None, rocm_gfx_targets = [])
host = ilp._apply_host_overrides(host, override_rocm_gfx = "gfx803")
assert host.rocm_gfx_targets == ["gfx803"]
assert ilp._should_auto_vulkan_for_amd_windows(host, FORK) is True
_routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert repo == UPSTREAM
assert persist == "vulkan"
def test_forwarded_gfx_still_fills_an_unprobed_arch(monkeypatch):
# Negative control: on an amd-smi-only host detect_host() reports no arch, so the
# forward is the only source and must still apply.
monkeypatch.delenv("UNSLOTH_LLAMA_BACKEND", raising = False)
monkeypatch.delenv("UNSLOTH_FORCE_VULKAN", raising = False)
monkeypatch.delenv("UNSLOTH_ROCM_GFX_ARCH", raising = False)
host = _windows_amd_host(rocm_gfx_target = None, rocm_gfx_targets = [])
host = ilp._apply_host_overrides(host, override_rocm_gfx = "gfx1151")
assert ilp._active_rocm_gfx_target(host) == "gfx1151"
assert ilp._should_auto_vulkan_for_amd_windows(host) is False
_routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert repo == FORK
assert persist is None
def test_llama_backend_hip_opts_out_of_auto_vulkan(monkeypatch):
# hip names a backend, so it keeps the fork path even on an auto-fallback arch.
monkeypatch.setenv("UNSLOTH_LLAMA_BACKEND", "hip")
host = _windows_amd_host(rocm_gfx_target = "gfx803", rocm_gfx_targets = ["gfx803"])
routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert routed is host
assert repo == FORK
assert persist is None
assert ilp.force_vulkan_requested() is False
def test_explicit_backend_beats_legacy_force_vulkan(monkeypatch):
# A stale UNSLOTH_FORCE_VULKAN must not overrule UNSLOTH_LLAMA_BACKEND=rocm (== hip).
monkeypatch.setenv("UNSLOTH_FORCE_VULKAN", "1")
monkeypatch.setenv("UNSLOTH_LLAMA_BACKEND", "rocm")
assert ilp.resolved_llama_backend() == "hip"
assert ilp.force_vulkan_requested() is False
host = _windows_amd_host(rocm_gfx_target = "gfx1100", rocm_gfx_targets = ["gfx1100"])
_routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert repo == FORK
assert persist is None
def test_unknown_llama_backend_value_falls_through_to_legacy_flag(monkeypatch):
# An unrecognised value is ignored, not an error, so the legacy flag still works.
monkeypatch.setenv("UNSLOTH_LLAMA_BACKEND", "banana")
assert ilp.resolved_llama_backend() is None
assert ilp.force_vulkan_requested() is False
monkeypatch.setenv("UNSLOTH_FORCE_VULKAN", "1")
assert ilp.force_vulkan_requested() is True
def test_llama_backend_flag_beats_conflicting_env(monkeypatch):
# --llama-backend is the caller's explicit request and outranks the env.
monkeypatch.setenv("UNSLOTH_LLAMA_BACKEND", "hip")
assert ilp.force_vulkan_requested("vulkan") is True
host = _windows_amd_host(rocm_gfx_target = "gfx1100", rocm_gfx_targets = ["gfx1100"])
_routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(
host, FORK, "pin", force_cpu = False, llama_backend = "vulkan"
)
assert repo == UPSTREAM
assert persist == "vulkan"
def _windows_arm64_host(**overrides):
defaults = dict(
system = "Windows",
machine = "ARM64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = False,
is_arm64 = True,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
has_rocm = False,
has_intel_gpu = False,
)
defaults.update(overrides)
return ilp.HostInfo(**defaults)
@pytest.mark.parametrize(
"env, flag",
[
({"UNSLOTH_LLAMA_BACKEND": "vulkan"}, None),
({"UNSLOTH_FORCE_VULKAN": "1"}, None),
({}, "vulkan"),
],
)
def test_vulkan_opt_in_ignored_on_windows_arm64(monkeypatch, env, flag):
# Upstream builds win-vulkan for x64 only (arm64 gets CPU + opencl-adreno), so rewriting
# the host would only swap the published arm64 bundle for the upstream CPU one.
for name, value in env.items():
monkeypatch.setenv(name, value)
host = _windows_arm64_host()
routed, repo, tag, persist = ilp._route_to_vulkan_prebuilt(
host, FORK, "pin", force_cpu = False, llama_backend = flag
)
assert routed is host
assert (repo, tag) == (FORK, "pin")
assert persist is None
def test_vulkan_opt_in_still_routes_on_windows_x64(monkeypatch):
# Negative control for the arm64 guard: x64 keeps its Vulkan routing.
monkeypatch.setenv("UNSLOTH_LLAMA_BACKEND", "vulkan")
host = _windows_amd_host(rocm_gfx_target = "gfx1100", rocm_gfx_targets = ["gfx1100"])
_routed, repo, _tag, persist = ilp._route_to_vulkan_prebuilt(host, FORK, "pin", force_cpu = False)
assert repo == UPSTREAM
assert persist == "vulkan"
def _choice(install_kind, name = "asset.zip"):
return ilp.AssetChoice(
repo = UPSTREAM,
tag = "b9925",
name = name,
url = f"https://example/{name}",
source_label = "upstream",
install_kind = install_kind,
)
@pytest.mark.parametrize("kind", ["windows-vulkan", "linux-vulkan"])
def test_persisted_llama_backend_keeps_vulkan_for_a_vulkan_bundle(kind):
assert ilp.persisted_llama_backend("vulkan", _choice(kind)) == "vulkan"
@pytest.mark.parametrize("kind", ["windows-arm64", "windows-cpu", "linux-cpu", "windows-rocm"])
def test_persisted_llama_backend_drops_vulkan_for_a_non_vulkan_bundle(kind):
# _plan_llama_phase re-asserts the marker's backend on every later update, so a Vulkan
# request that fell through to CPU must not leave a marker claiming Vulkan.
assert ilp.persisted_llama_backend("vulkan", _choice(kind)) is None
def test_persisted_llama_backend_passes_none_through():
assert ilp.persisted_llama_backend(None, _choice("windows-vulkan")) is None
def test_marker_records_no_backend_when_vulkan_fell_back_to_cpu(tmp_path):
# End to end over write_prebuilt_metadata: describe the CPU attempt that actually won,
# so the next update re-detects instead of re-asserting Vulkan forever.
checksums = ilp.ApprovedReleaseChecksums(
repo = UPSTREAM,
release_tag = "b9925",
upstream_tag = "b9925",
source_repo = UPSTREAM,
source_repo_url = f"https://github.com/{UPSTREAM}",
)
cpu = _choice("windows-arm64", "llama-b9925-bin-win-cpu-arm64.zip")
ilp.write_prebuilt_metadata(
tmp_path,
requested_tag = "latest",
llama_tag = "b9925",
release_tag = "b9925",
choice = cpu,
approved_checksums = checksums,
prebuilt_fallback_used = False,
llama_backend = "vulkan",
)
marker = json.loads((tmp_path / "UNSLOTH_PREBUILT_INFO.json").read_text())
assert marker["asset"] == "llama-b9925-bin-win-cpu-arm64.zip"
assert marker["llama_backend"] is None
vulkan = _choice("windows-vulkan", "llama-b9925-bin-win-vulkan-x64.zip")
ilp.write_prebuilt_metadata(
tmp_path,
requested_tag = "latest",
llama_tag = "b9925",
release_tag = "b9925",
choice = vulkan,
approved_checksums = checksums,
prebuilt_fallback_used = False,
llama_backend = "vulkan",
)
marker = json.loads((tmp_path / "UNSLOTH_PREBUILT_INFO.json").read_text())
assert marker["llama_backend"] == "vulkan"
# UNSLOTH_LLAMA_CPP_BACKEND (setup.sh/setup.ps1, "auto"|"cpu") and
# UNSLOTH_LLAMA_BACKEND (this module, a backend name) are different variables at
# different layers, and both accept "cpu". setup translates its own =cpu into
# --force-cpu to pin the CPU-only bundle on a GPU host, which is what keeps Intel
# iGPU Vulkan crashes away (#7213). Vulkan is opt-in here, so no trigger it adds
# may outrank that flag on any host.
_SIM_PLATFORMS = {
# WSL presents as Linux to this resolver, so it rides the Linux row.
"Linux": dict(
system = "Linux",
is_windows = False,
is_linux = True,
is_macos = False,
machine = "x86_64",
is_x86_64 = True,
is_arm64 = False,
),
"Windows": dict(
system = "Windows",
is_windows = True,
is_linux = False,
is_macos = False,
machine = "amd64",
is_x86_64 = True,
is_arm64 = False,
),
"macOS": dict(
system = "Darwin",
is_windows = False,
is_linux = False,
is_macos = True,
machine = "arm64",
is_x86_64 = False,
is_arm64 = True,
),
}
_SIM_GPUS = {
"nvidia": dict(
has_physical_nvidia = True,
has_usable_nvidia = True,
has_rocm = False,
has_intel_gpu = False,
nvidia_smi = "/usr/bin/nvidia-smi",
driver_cuda_version = "12.4",
compute_caps = ["8.9"],
),
"amd": dict(
has_physical_nvidia = False,
has_usable_nvidia = False,
has_rocm = True,
has_intel_gpu = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
rocm_gfx_target = "gfx803",
rocm_gfx_targets = ["gfx803"],
),
"intel": dict(
has_physical_nvidia = False,
has_usable_nvidia = False,
has_rocm = False,
has_intel_gpu = True,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
),
"cpu_only": dict(
has_physical_nvidia = False,
has_usable_nvidia = False,
has_rocm = False,
has_intel_gpu = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
),
}
def _sim_host(platform_name, gpu_name):
base = dict(visible_cuda_devices = None)
base.update(_SIM_PLATFORMS[platform_name])
base.update(_SIM_GPUS[gpu_name])
return ilp.HostInfo(**base)
@pytest.mark.parametrize("platform_name", sorted(_SIM_PLATFORMS))
@pytest.mark.parametrize("gpu_name", sorted(_SIM_GPUS))
@pytest.mark.parametrize("backend_env", [None, "vulkan", "hip", "rocm", "cpu"])
def test_forced_cpu_outranks_every_vulkan_trigger(
monkeypatch, platform_name, gpu_name, backend_env
):
"""A deliberate CPU install stays CPU on every host, whatever asks for Vulkan."""
monkeypatch.delenv("UNSLOTH_FORCE_VULKAN", raising = False)
if backend_env is None:
monkeypatch.delenv("UNSLOTH_LLAMA_BACKEND", raising = False)
else:
monkeypatch.setenv("UNSLOTH_LLAMA_BACKEND", backend_env)
# The legacy switch too, so a stale one cannot smuggle Vulkan past --force-cpu.
monkeypatch.setenv("UNSLOTH_FORCE_VULKAN", "1")
repo, tag = "unslothai/llama.cpp-prebuilt", "latest"
_, out_repo, _, persist = ilp._route_to_vulkan_prebuilt(
_sim_host(platform_name, gpu_name),
repo,
tag,
force_cpu = True,
llama_backend = "vulkan",
)
assert out_repo == repo, (platform_name, gpu_name, backend_env)
assert persist is None, (platform_name, gpu_name, backend_env)
def test_the_forced_cpu_guard_is_not_vacuous():
"""The same host DOES take Vulkan once the CPU pin is gone, or the check above
would pass on a resolver that had stopped routing to Vulkan entirely."""
repo, tag = "unslothai/llama.cpp-prebuilt", "latest"
_, out_repo, _, persist = ilp._route_to_vulkan_prebuilt(
_sim_host("Linux", "amd"),
repo,
tag,
force_cpu = False,
llama_backend = "vulkan",
)
assert out_repo != repo or persist == "vulkan"