ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Source llama.cpp prebuilts from unslothai/llama.cpp (CUDA, ROCm, macOS) (#5963)

Browse source 
* Studio: route arm64 Linux CUDA hosts to linux-arm64-cuda prebuilts

* Studio: SM-aware selection for windows-cuda app bundles

* Studio: select published ROCm bundles by gfx target (linux + windows)

* Studio: route macOS installs to the fork's prebuilt bundles

* Studio: fix windows cuda13 driver-13.0 gate and ROCm gfx prefix overreach

* Fix Blackwell Windows pin shadowing native app-bundle (b9360 over b9457)

* Match Windows cuda12 driver floor to Linux (12.x minor-version compat)

* Fix Windows app-bundle dropped when runtime DLLs come from torch/lib

* Fold the manifest resolver into the simple-path resolver (one entry, no dormant full path)

* Remove unused UNSLOTH_LLAMA_PUBLISHED_REPO override

* Route Windows GPU hosts to the fork prebuilts in setup.ps1

* Document sm_103 path divergence and mark --simple-policy as a no-op

* Note sm_103 coverage now comes from the producer manifest

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove the now-vestigial --simple-policy flag (one resolver handles all hosts)

* Unify the fork onto the manifest path; drop the linux-x64 filename path and hardcoded coverage tables

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Strip whitespace from manifest gfx_target/mapped_targets when parsing

* Windows CUDA: sort coverage-unknown bundles last so they can't outrank targeted ones

* Share the SM-coverage sort key between the linux and windows selectors via _sm_range

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: reject approved releases with an exact source archive but no source repo to clone from

* Studio: accept the fork's windows-rocm kind in the Windows reinstall check

* Studio: accept a manifest-bundle source repo in the exact-source release check

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: route Linux hosts to the fork only when a usable GPU is present

* Fix Windows AMD lemonade tag resolution for PR #5963

The fork release scan passes each scanned release's upstream tag
(b9518, ...) to the lemonade lookup, but lemonade publishes its own tag
series (b1292, ...) that never contains upstream tag numbers. On a
Windows AMD host every scanned release therefore 404s the lemonade
fetch twice, the upstream HIP zip is dropped by the approved-hash gate,
and the scan walks the whole release history until it dies on the
unauthenticated GitHub rate limit or falls to a HIP source build. The
Linux path already passes the requested tag ("latest") and works.

Thread the requested tag through resolve_release_asset_choice ->
resolve_asset_choice -> resolve_upstream_asset_choice as lemonade_tag,
used only by the lemonade lookups. Upstream asset names keep the
concrete per-release tag and all new parameters default to the old
behavior.

Verified on a gfx1151 box: before, the native Windows install scanned
b9518..b8811 and aborted on rate limit; after, it selects
llama-b1292-windows-rocm-gfx1151-x64.zip (lemonade) from fork release
b9518, passes staged validation, and the installed llama-server
enumerates ROCm0. WSL keeps selecting the matching ubuntu bundle.
Adds a regression test pinning that the Windows fork path resolves
lemonade via /releases/latest, never /releases/tags/<fork-tag>.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Plan lemonade for Linux ROCm hosts on the ggml-org direct path for PR #5963

Audit follow-up to 72f32364 across the other selection pathways. The
ggml-org direct planner kept its lemonade attempt for Windows ROCm
hosts but planned only the CPU tarball for Linux ROCm hosts, so an AMD
Linux box routed to ggml-org (for example a --published-repo override)
silently installed the CPU build. That lemonade planning used to live
in the --simple-policy dispatcher this PR removed.

Add the lemonade attempt ahead of the CPU tarball in the Linux x86_64
branch, mirroring the Windows branch, with the lookup keyed to the
requested tag. Adds a regression test asserting lemonade is the first
attempt for a Linux ROCm host on the direct path.

Also re-verified the other pathways on a gfx1151 box: the fork-routed
flows pass the requested tag everywhere, repeat runs over an existing
lemonade install correctly skip with "already matches selected release
b9518" on both native Windows and WSL, and macOS, CUDA and CPU
selection are untouched. Suites: 328 passed on Linux, Windows matches
the pre-existing baseline.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
This commit is contained in:
oobabooga 2026-06-10 12:49:57 -03:00 • committed by GitHub
commit cc1a724efc
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 1311 additions and 1060 deletions
Download patch file Download diff file Expand all files Collapse all files

229
studio/backend/tests/test_lemonade_llamacpp_rocm_bins_mock.py
View file

@ -1,10 +1,11 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Validates that the installer resolves lemonade ROCm prebuilt assets.
"""Validates that the installer correctly resolves lemonade ROCm prebuilt assets.
Uses a faked HostInfo so no AMD GPU is needed. The lemonade GitHub API calls
are stubbed so the suite runs offline and isn't subject to rate limits.
Uses a faked HostInfo so no AMD GPU is needed. Network calls to the lemonade
GitHub API are stubbed out so the suite runs without internet access and is
not subject to rate limits.
"""
from __future__ import annotations
@ -32,7 +33,7 @@ if resolve_lemonade_rocm_choice is None or _LEMONADE_GFX_FAMILIES is None:
@pytest.fixture(autouse = True)
def _clear_lemonade_release_cache():
"""Prevent cross-test pollution of the lemonade release lru_cache when
tests vary the fetch_json mock return value."""
future tests vary the fetch_json mock return value."""
_cache = getattr(_mod, "_fetch_lemonade_release_cached", None)
if _cache is not None and hasattr(_cache, "cache_clear"):
_cache.cache_clear()
@ -89,7 +90,9 @@ def _lookup_family(gfx: str) -> str | None:
return None
# ---------------------------------------------------------------------------
# GPU family mapping
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
@ -111,7 +114,9 @@ def test_unknown_gpu_not_in_families():
assert _lookup_family("gfx999") is None
# ---------------------------------------------------------------------------
# Asset resolution - hits real lemonade GitHub API
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
@ -141,59 +146,71 @@ def test_unknown_gpu_falls_through_to_upstream():
assert result is None
# Simple-policy dispatcher must plan a lemonade ROCm attempt for AMD-only hosts.
# This is the path setup.sh invokes (via --simple-policy), so the lemonade
# integration is useless if it isn't wired in here.
# ---------------------------------------------------------------------------
# The Linux attempt builder must plan a lemonade ROCm attempt for AMD-only hosts.
# This is the path setup.sh actually invokes (fork hosts now select from the
# manifest), so the lemonade integration is useless if it isn't wired in here.
# ---------------------------------------------------------------------------
direct_linux_release_plan = getattr(_mod, "direct_linux_release_plan", None)
_linux_published_attempts = getattr(_mod, "_linux_published_attempts", None)
direct_upstream_release_plan = getattr(_mod, "direct_upstream_release_plan", None)
PublishedLlamaArtifact = _mod.PublishedLlamaArtifact
PublishedReleaseBundle = _mod.PublishedReleaseBundle
def _stub_unsloth_release(release_tag: str = "b9022") -> dict:
# Minimal payload parse_direct_linux_release_bundle accepts. It needs at
# least one `app-{label}-linux-x64*.tar.gz` asset to recognise the bundle;
# we ship a bare CPU one so the planner has a baseline non-ROCm fallback.
asset_name = f"app-{release_tag}-linux-x64.tar.gz"
return {
"tag_name": release_tag,
"name": release_tag,
"assets": [
{
"name": asset_name,
"browser_download_url": f"https://example.invalid/{asset_name}",
},
],
}
def _rocm_bundle(gfx_family: str, mapped_targets: list[str]) -> "PublishedReleaseBundle":
"""A fork manifest bundle exposing a per-gfx linux-rocm artifact, so
published_rocm_choice_for_host can match the host before the lemonade
fallback is appended."""
asset_name = f"app-b9457-linux-x64-rocm-{gfx_family}.tar.gz"
artifact = PublishedLlamaArtifact(
asset_name = asset_name,
install_kind = "linux-rocm",
runtime_line = None,
coverage_class = None,
supported_sms = [],
min_sm = None,
max_sm = None,
bundle_profile = None,
rank = 1000,
gfx_target = gfx_family,
mapped_targets = mapped_targets,
)
return PublishedReleaseBundle(
repo = "unslothai/llama.cpp",
release_tag = "v1.0",
upstream_tag = "b9457",
assets = {asset_name: f"https://example.invalid/{asset_name}"},
artifacts = [artifact],
)
@pytest.mark.skipif(
direct_linux_release_plan is None,
reason = "simple-policy dispatcher not present on this branch",
_linux_published_attempts is None,
reason = "Linux attempt builder not present on this branch",
)
def test_simple_policy_plans_lemonade_for_rocm_host():
def test_linux_attempts_include_fork_rocm_and_lemonade_for_rocm_host():
host = _make_rocm_host("gfx1151")
bundle = _rocm_bundle("gfx1151", ["gfx1151"])
with patch.object(_mod, "fetch_json", return_value = _stub_lemonade_release()):
plan = direct_linux_release_plan(
_stub_unsloth_release(),
host,
"unslothai/llama.cpp",
"latest",
)
assert plan is not None, "ROCm host should not be skipped by simple-policy planner"
kinds = [a.install_kind for a in plan.attempts]
assert (
"linux-rocm" in kinds
), f"simple-policy planner did not include a lemonade ROCm attempt; got {kinds}"
rocm_attempt = next(a for a in plan.attempts if a.install_kind == "linux-rocm")
assert rocm_attempt.source_label == "lemonade"
assert "gfx1151" in rocm_attempt.name
attempts = _linux_published_attempts(host, bundle, "latest")
kinds = [a.install_kind for a in attempts]
assert "linux-rocm" in kinds, f"builder did not include any linux-rocm attempt; got {kinds}"
sources = {a.source_label for a in attempts if a.install_kind == "linux-rocm"}
# The fork's own per-gfx bundle is preferred, with the lemonade prebuilt as
# the fallback -- both must be present for a covered ROCm host.
assert "published" in sources, f"fork ROCm bundle missing; got {sources}"
assert "lemonade" in sources, f"lemonade ROCm fallback missing; got {sources}"
lemonade_attempt = next(a for a in attempts if a.source_label == "lemonade")
assert "gfx1151" in lemonade_attempt.name
@pytest.mark.skipif(
direct_upstream_release_plan is None,
reason = "simple-policy dispatcher not present on this branch",
reason = "direct release planners not present on this branch",
)
def test_simple_policy_plans_lemonade_for_windows_hip_host():
def test_direct_upstream_plan_includes_lemonade_for_windows_hip_host():
host = _make_rocm_host("gfx1151", windows = True)
release = {
"tag_name": "b9022",
@ -204,16 +221,14 @@ def test_simple_policy_plans_lemonade_for_windows_hip_host():
plan = direct_upstream_release_plan(release, host, "ggml-org/llama.cpp", "latest")
assert plan is not None, "Windows ROCm host should plan a lemonade HIP attempt"
kinds = [a.install_kind for a in plan.attempts]
assert (
"windows-hip" in kinds
), f"simple-policy planner did not include a lemonade HIP attempt; got {kinds}"
assert "windows-hip" in kinds, f"planner did not include a lemonade HIP attempt; got {kinds}"
@pytest.mark.skipif(
direct_upstream_release_plan is None,
reason = "simple-policy dispatcher not present on this branch",
reason = "direct release planners not present on this branch",
)
def test_simple_policy_windows_hip_falls_back_to_upstream_when_lemonade_unavailable():
def test_windows_hip_falls_back_to_upstream_when_lemonade_unavailable():
"""If lemonade returns None (e.g. gfx999 or transient API failure), the planner
must still include the upstream HIP asset rather than silently downgrading to CPU."""
host = _make_rocm_host("gfx999", windows = True)
@ -247,8 +262,9 @@ def test_lemonade_release_api_url_pinned_tag():
def test_lemonade_release_api_url_encodes_tag():
"""Slashes / hashes in the tag must be URL-encoded so the URL can't be
reshaped (defence in depth -- tags should already be sanitised upstream)."""
"""Unexpected slashes / hashes in the tag must be URL-encoded so the URL
cannot be reshaped (defence in depth -- tags should already be sanitised
upstream)."""
url = _mod._lemonade_release_api_for("b1260/../latest")
assert "/releases/tags/b1260%2F..%2Flatest" in url
assert "//latest" not in url.split("/releases/tags/", 1)[1]
@ -263,9 +279,9 @@ def test_lemonade_resolver_skipped_by_opt_out_env(monkeypatch):
def test_lemonade_resolver_rejects_non_github_url(monkeypatch):
"""If the GitHub API response contained an off-host download URL, the
resolver must refuse it (lemonade assets aren't in the approved-hash
manifest)."""
"""If the GitHub API response somehow contained an off-host download URL,
the resolver must refuse to use it (lemonade assets are not in the
approved-hash manifest)."""
bad_release = {
"tag_name": _STUB_TAG,
"assets": [
@ -289,7 +305,7 @@ def test_lemonade_resolver_rejects_http_scheme():
def test_lemonade_resolver_accepts_github_cdn():
# Real GitHub release CDN URLs carry the /github-production-release-asset- prefix
# Real GitHub release CDN URLs carry the /github-production-release-asset- prefix.
assert _mod._is_trusted_github_release_url(
"https://objects.githubusercontent.com/github-production-release-asset-abc123/456/789?token=x",
"lemonade-sdk/llamacpp-rocm",
@ -297,7 +313,7 @@ def test_lemonade_resolver_accepts_github_cdn():
def test_lemonade_resolver_rejects_arbitrary_cdn_path():
# A CDN URL without the release-asset path prefix must be rejected
# A CDN URL without the release-asset path prefix must be rejected.
assert not _mod._is_trusted_github_release_url(
"https://objects.githubusercontent.com/abc/def",
"lemonade-sdk/llamacpp-rocm",
@ -339,7 +355,7 @@ def test_lemonade_runtime_patterns_include_hip_runtime():
Lemonade ZIPs carry transitive deps (libamd_comgr, libLLVM, libclang-cpp,
...) whose names change across ROCm releases. A broad ``lib*.so*`` glob
avoids enumerating every transitive dependency by name.
avoids having to enumerate every transitive dependency by name.
"""
from install_llama_prebuilt import runtime_patterns_for_choice, AssetChoice
@ -353,7 +369,7 @@ def test_lemonade_runtime_patterns_include_hip_runtime():
)
pats = runtime_patterns_for_choice(choice)
# The broad glob must be present so every .so in the lemonade bundle
# (including future transitive deps) gets overlaid.
# (including transitive deps added in future ROCm releases) gets overlaid.
assert "lib*.so*" in pats, f"'lib*.so*' missing from linux-rocm patterns: {pats}"
@ -365,9 +381,9 @@ _pick_rocm_gfx_target = getattr(_mod, "_pick_rocm_gfx_target", None)
reason = "_pick_rocm_gfx_target not present on this branch",
)
def test_pick_rocm_gfx_target_honors_cuda_visible_devices(monkeypatch):
"""AMD HIP honours CUDA_VISIBLE_DEVICES like HIP_VISIBLE_DEVICES; on a
gfx1151 + gfx1100 mixed host, CUDA_VISIBLE_DEVICES=1 must select gfx1100."""
# Two GPUs; rocminfo reports each token twice (as in real tool output).
"""AMD HIP honours CUDA_VISIBLE_DEVICES identically to HIP_VISIBLE_DEVICES;
on a gfx1151 + gfx1100 mixed host, CUDA_VISIBLE_DEVICES=1 must select gfx1100."""
# Two GPUs; rocminfo reports each token twice (as in the real tool output).
probe_out = "gfx1151\ngfx1151\ngfx1100\ngfx1100"
monkeypatch.delenv("HIP_VISIBLE_DEVICES", raising = False)
monkeypatch.delenv("ROCR_VISIBLE_DEVICES", raising = False)
@ -396,7 +412,7 @@ def test_pick_rocm_gfx_target_same_arch_multi_gpu(monkeypatch):
"""Regression: [gfx1100, gfx1100, gfx1151] with HIP_VISIBLE_DEVICES=2 must
return gfx1151, not fall back to GPU 0 due to dict.fromkeys collapsing the
two gfx1100 entries into one and making index 2 out of range."""
# rocminfo output for 3 GPUs (2x gfx1100 dGPU + 1x gfx1151 APU).
# Simulate rocminfo output for 3 GPUs (2x gfx1100 dGPU + 1x gfx1151 APU).
# Each GPU gets its own Agent section with a few token mentions.
probe_out = (
"***\nAgent 1\n***\n gfx1100 some info\n gfx1100\n"
@ -407,3 +423,96 @@ def test_pick_rocm_gfx_target_same_arch_multi_gpu(monkeypatch):
monkeypatch.delenv("CUDA_VISIBLE_DEVICES", raising = False)
monkeypatch.setenv("HIP_VISIBLE_DEVICES", "2")
assert _pick_rocm_gfx_target(probe_out) == "gfx1151"
# ---------------------------------------------------------------------------
# Fork release scan: Windows ROCm resolves lemonade by the requested tag
# ---------------------------------------------------------------------------
_resolve_release_asset_choice = getattr(_mod, "resolve_release_asset_choice", None)
_ApprovedReleaseChecksums = getattr(_mod, "ApprovedReleaseChecksums", None)
@pytest.mark.skipif(
_resolve_release_asset_choice is None or _ApprovedReleaseChecksums is None,
reason = "fork release planner not present on this branch",
)
def test_fork_scan_windows_rocm_resolves_lemonade_by_requested_tag():
"""The fork release scan pins llama_tag to per-release upstream tags
(b9457, ...) that lemonade's own tag series never contains, so the
lemonade lookup must use the requested tag ("latest") instead. Pinning
lemonade to the per-release tag 404s on every scanned release and a
Windows ROCm host ends in a rate-limited fatal instead of the lemonade
prebuilt."""
host = _make_rocm_host("gfx1151", windows = True)
# No windows-rocm artifact in the bundle, matching current fork releases.
bundle = _rocm_bundle("gfx1151", ["gfx1151"])
checksums = _ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "v1.0",
upstream_tag = "b9457",
artifacts = {},
)
seen_urls: list[str] = []
def _fake_fetch(api_url, *args, **kwargs):
seen_urls.append(api_url)
if "lemonade-sdk" in api_url:
if api_url.endswith("/releases/latest"):
return _stub_lemonade_release()
raise RuntimeError(f"unexpected pinned lemonade fetch: {api_url}")
# ggml-org asset listing for the upstream HIP/CPU filename fallbacks.
return {"tag_name": "b9457", "assets": []}
with patch.object(_mod, "fetch_json", side_effect = _fake_fetch):
attempts = _resolve_release_asset_choice(
host,
"b9457", # concrete per-release upstream tag from the scan loop
bundle,
checksums,
requested_tag = "latest",
)
lemonade = [a for a in attempts if a.source_label == "lemonade"]
assert lemonade, f"lemonade attempt missing for Windows ROCm host; got {attempts}"
assert "gfx1151" in lemonade[0].name
assert any(
u.endswith("/releases/latest") for u in seen_urls
), f"lemonade was never resolved via /releases/latest; fetches: {seen_urls}"
assert not any(
"lemonade-sdk" in u and "/releases/tags/" in u for u in seen_urls
), f"lemonade lookup was pinned to the fork release tag: {seen_urls}"
@pytest.mark.skipif(
direct_upstream_release_plan is None,
reason = "direct release planners not present on this branch",
)
def test_direct_upstream_plan_includes_lemonade_for_linux_rocm_host():
"""A Linux ROCm host on the ggml-org direct path (e.g. a --published-repo
override) must plan lemonade before the CPU tarball, mirroring the Windows
branch. The lemonade planning previously lived in the removed
--simple-policy dispatcher, so without this leg such hosts silently
install the CPU build."""
host = _make_rocm_host("gfx1151")
release = {
"tag_name": "b9022",
"name": "b9022",
"assets": [
{
"name": "llama-b9022-bin-ubuntu-x64.tar.gz",
"browser_download_url": (
"https://github.com/ggml-org/llama.cpp/releases/download/"
"b9022/llama-b9022-bin-ubuntu-x64.tar.gz"
),
}
],
}
with patch.object(_mod, "fetch_json", return_value = _stub_lemonade_release()):
plan = direct_upstream_release_plan(release, host, "ggml-org/llama.cpp", "latest")
assert plan is not None, "Linux ROCm host should produce a direct plan"
kinds = [a.install_kind for a in plan.attempts]
sources = [a.source_label for a in plan.attempts]
assert "linux-rocm" in kinds, f"lemonade ROCm attempt missing; got {kinds}"
assert sources[0] == "lemonade", f"lemonade must be the first attempt; got {sources}"
assert "gfx1151" in plan.attempts[0].name

990
studio/install_llama_prebuilt.py
View file

File diff suppressed because it is too large Load diff

28
studio/setup.ps1
View file

@ -2556,7 +2556,9 @@ $LlamaCppDir = Join-Path $UnslothHome "llama.cpp"
$NeedLlamaSourceBuild = $false
$SkipPrebuiltInstall = $false
$RequestedLlamaTag = if ($env:UNSLOTH_LLAMA_TAG) { $env:UNSLOTH_LLAMA_TAG } else { $DefaultLlamaTag }
$HelperReleaseRepo = "ggml-org/llama.cpp"
# GPU Windows (CUDA / ROCm) installs the fork's app-* prebuilts; CPU-only stays
# on ggml-org (the fork ships no windows-cpu bundle). Mirrors setup.sh's routing.
$HelperReleaseRepo = if ($HasNvidiaSmi -or $HasROCm) { "unslothai/llama.cpp" } else { "ggml-org/llama.cpp" }
$LlamaPr = if ($env:UNSLOTH_LLAMA_PR) { $env:UNSLOTH_LLAMA_PR.Trim() } else { "" }
$LlamaPrForce = if ($env:UNSLOTH_LLAMA_PR_FORCE) { $env:UNSLOTH_LLAMA_PR_FORCE.Trim() } else { $DefaultLlamaPrForce }
@ -2655,20 +2657,25 @@ if ($env:UNSLOTH_LLAMA_FORCE_COMPILE -eq "1") {
if (Test-Path -LiteralPath $LlamaCppDir) {
substep "Existing llama.cpp install detected -- validating staged prebuilt update before replacement"
# If the existing install is the wrong kind (e.g. windows-cpu on a ROCm
# machine that should have windows-hip), remove it so the installer is
# machine that should have windows-rocm), remove it so the installer is
# forced to download the correct variant rather than skipping on tag match.
$existingMetaPath = Join-Path $LlamaCppDir "UNSLOTH_PREBUILT_INFO.json"
if (Test-Path $existingMetaPath) {
try {
$existingMeta = Get-Content $existingMetaPath -Raw | ConvertFrom-Json
$existingKind = $existingMeta.install_kind
# A name-inferred gfx arch (Adrenalin-only, no confirmed runtime)
# still wants the GPU (windows-hip) build -- the lemonade prebuilt
# bundles its own runtime. Treat a known arch as ROCm-capable here,
# mirroring the --rocm-gfx forward below.
$expectedKind = if ($HasROCm -or $script:ROCmGfxArch) { "windows-hip" } elseif ($HasNvidiaSmi) { "windows-cuda" } else { "windows-cpu" }
if ($existingKind -and $existingKind -ne $expectedKind) {
substep "Removing mismatched llama.cpp install (found '$existingKind', need '$expectedKind')..."
# A ROCm host may legitimately carry the fork's windows-rocm bundle
# or the upstream windows-hip fallback, so accept either and never
# treat a valid ROCm install as mismatched. A name-inferred gfx
# arch (Adrenalin-only, no confirmed runtime) still counts as
# ROCm-capable -- the lemonade prebuilt bundles its own runtime,
# mirroring the --rocm-gfx forward below. NOTE: this block is
# currently inert -- write_prebuilt_metadata does not persist an
# install_kind key, so $existingKind is always null. If that changes,
# add the remaining host kinds (e.g. windows-arm64) before relying on it.
$expectedKinds = if ($HasROCm -or $script:ROCmGfxArch) { @("windows-rocm", "windows-hip") } elseif ($HasNvidiaSmi) { @("windows-cuda") } else { @("windows-cpu") }
if ($existingKind -and ($existingKind -notin $expectedKinds)) {
substep "Removing mismatched llama.cpp install (found '$existingKind', need one of: $($expectedKinds -join ', '))..."
Remove-Item -Recurse -Force -LiteralPath $LlamaCppDir -ErrorAction SilentlyContinue
}
} catch {
@ -2687,8 +2694,7 @@ if ($env:UNSLOTH_LLAMA_FORCE_COMPILE -eq "1") {
"$PSScriptRoot\install_llama_prebuilt.py",
"--install-dir", $LlamaCppDir,
"--llama-tag", $RequestedLlamaTag,
"--published-repo", $HelperReleaseRepo,
"--simple-policy"
"--published-repo", $HelperReleaseRepo
)
if ($HasROCm) {
$prebuiltArgs += "--has-rocm"

50
studio/setup.sh
View file

@ -812,6 +812,7 @@ if ! command -v rocminfo >/dev/null 2>&1 && [ -x /opt/rocm/bin/rocminfo ]; then
PATH="$PATH:/opt/rocm/bin"
fi
_setup_amd_detected=false
_setup_nvidia_usable=false
_setup_gfx_all=""
_setup_mkt=""
if command -v rocminfo >/dev/null 2>&1 && \
@ -832,6 +833,7 @@ fi
if command -v nvidia-smi >/dev/null 2>&1 && \
nvidia-smi -L 2>/dev/null | awk '/^GPU[[:space:]]+[0-9]+:/{found=1} END{exit !found}'; then
_setup_nvidia_usable=true
step "gpu" "NVIDIA GPU detected"
elif [ "$_setup_amd_detected" = true ]; then
_setup_vis="${HIP_VISIBLE_DEVICES:-${ROCR_VISIBLE_DEVICES:-}}"
@ -910,34 +912,46 @@ _HOST_SYSTEM="$(uname -s 2>/dev/null || true)"
_HOST_MACHINE="$(uname -m 2>/dev/null || true)"
# Pick the release repo install_llama_prebuilt.py plans against.
# unslothai/llama.cpp ships only Linux CUDA bundles, so CPU-only Linux
# x86_64 routes to ggml-org for bin-ubuntu-x64.tar.gz. Anything with a
# GPU tool installed stays on unslothai (CUDA bundle / ROCm source build).
# The fork ships CUDA (Linux x64/arm64, Windows), ROCm (Linux/Windows) and
# macOS bundles. Only the plain CPU/Vulkan bundles still come from ggml-org, so
# CPU-only Linux (x86_64 and arm64) routes there; GPU Linux, Windows and macOS
# use unslothai.
_LINUX_HAS_GPU=false
for _GPU_TOOL in nvidia-smi rocminfo amd-smi hipconfig hipinfo; do
if command -v "$_GPU_TOOL" >/dev/null 2>&1; then
_LINUX_HAS_GPU=true
break
fi
done
# Route to the fork only for a usable GPU. NVIDIA counts only when a device is
# actually enumerated (_setup_nvidia_usable, from the nvidia-smi -L probe above)
# AND not hidden via CUDA_VISIBLE_DEVICES=-1 -- mirroring install_llama_prebuilt.py's
# has_usable_nvidia. Mere nvidia-smi presence (CPU-only CUDA-toolkit containers,
# broken drivers) or a hidden GPU therefore takes the ggml-org CPU prebuilt
# instead of a slow source build. AMD is deliberately left on tooling presence,
# not usability: an unusable NVIDIA host has a good CPU prebuilt to fall back to,
# whereas tightening AMD would regress ROCm hosts exposing only hipconfig/hipinfo
# into an unnecessary CPU build.
if [ "$_setup_nvidia_usable" = true ] && [ "${CUDA_VISIBLE_DEVICES:-}" != "-1" ]; then
_LINUX_HAS_GPU=true
else
for _GPU_TOOL in rocminfo amd-smi hipconfig hipinfo; do
if command -v "$_GPU_TOOL" >/dev/null 2>&1; then
_LINUX_HAS_GPU=true
break
fi
done
fi
if [ "$_HOST_SYSTEM" = "Darwin" ]; then
_HELPER_RELEASE_REPO="ggml-org/llama.cpp"
elif [ "$_HOST_SYSTEM" = "Linux" ] \
if [ "$_HOST_SYSTEM" = "Linux" ] \
&& [ "$_HOST_MACHINE" = "x86_64" ] \
&& [ "$_LINUX_HAS_GPU" = false ]; then
_HELPER_RELEASE_REPO="ggml-org/llama.cpp"
elif [ "$_HOST_SYSTEM" = "Linux" ] \
&& { [ "$_HOST_MACHINE" = "aarch64" ] || [ "$_HOST_MACHINE" = "arm64" ]; } \
&& [ "$_LINUX_HAS_GPU" = false ]; then
# Linux ARM64 (Ampere Altra, Raspberry Pi 5, GitHub `ubuntu-24.04-arm`,
# CPU-only Jetson rescue mode, ...). unslothai/llama.cpp only ships
# the Linux CUDA bundles, so without this branch the prebuilt
# resolver returns 0 attempts on every release and the installer
# falls all the way back to a source build. Upstream ggml-org ships
# CPU-only Linux ARM64 (Ampere Altra, Raspberry Pi 5, GitHub
# `ubuntu-24.04-arm`, CPU-only Jetson rescue mode, ...). The fork ships no
# arm64 CPU bundle, so without this branch the prebuilt resolver returns 0
# attempts and the installer falls back to a source build. ggml-org ships
# llama-bNNNN-bin-ubuntu-arm64.tar.gz from at least b9072 onward.
_HELPER_RELEASE_REPO="ggml-org/llama.cpp"
else
# GPU Linux (x64 CUDA/ROCm, arm64 CUDA), Windows (CUDA/ROCm), and macOS.
_HELPER_RELEASE_REPO="unslothai/llama.cpp"
fi
unset _GPU_TOOL
@ -1000,7 +1014,6 @@ else
--install-dir "$LLAMA_CPP_DIR"
--llama-tag "$_REQUESTED_LLAMA_TAG"
--published-repo "$_HELPER_RELEASE_REPO"
--simple-policy
)
if [ -n "${UNSLOTH_LLAMA_RELEASE_TAG:-}" ]; then
_PREBUILT_CMD+=(--published-release-tag "$UNSLOTH_LLAMA_RELEASE_TAG")
@ -1555,7 +1568,6 @@ if [ "$_LLAMA_CPP_DEGRADED" = true ] \
--install-dir "$LLAMA_CPP_DIR"
--llama-tag "$_REQUESTED_LLAMA_TAG"
--published-repo "ggml-org/llama.cpp"
--simple-policy
--cpu-fallback
)
# Trust the installer's exit code: it validates the server before exiting 0,

342
tests/studio/install/test_install_llama_prebuilt_logic.py
View file

@ -338,336 +338,6 @@ def test_validate_prebuilt_choice_creates_repo_shaped_linux_install(
assert (install_dir / "BUILD_INFO.txt").exists()
def test_simple_linux_direct_release_uses_published_source_checksums_for_branch(
monkeypatch: pytest.MonkeyPatch,
):
source_commit = "25b1bc9c2f9aa0a390b968ee1ffd9ff01340a3fe"
release = {
"tag_name": "llama-prebuilt-master-3a92bc9",
"assets": [
{
"name": "app-master-linux-x64-cuda13-newer.tar.gz",
"browser_download_url": "https://example.test/app-master-linux-x64-cuda13-newer.tar.gz",
},
{
"name": "llama-prebuilt-sha256.json",
"browser_download_url": "https://example.test/llama-prebuilt-sha256.json",
},
],
}
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "llama-prebuilt-master-3a92bc9",
upstream_tag = "b9174",
source_commit = source_commit,
source_repo = "ggml-org/llama.cpp",
source_repo_url = "https://github.com/ggml-org/llama.cpp",
source_ref_kind = "branch",
requested_source_ref = "master",
resolved_source_ref = "master",
artifacts = {
"app-master-linux-x64-cuda13-newer.tar.gz": ApprovedArtifactHash(
asset_name = "app-master-linux-x64-cuda13-newer.tar.gz",
sha256 = "a" * 64,
repo = "unslothai/llama.cpp",
kind = "linux-cuda-app",
),
INSTALL_LLAMA_PREBUILT.exact_source_archive_logical_name(
source_commit
): ApprovedArtifactHash(
asset_name = INSTALL_LLAMA_PREBUILT.exact_source_archive_logical_name(source_commit),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "exact-source",
),
},
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"load_approved_release_checksums",
lambda repo, release_tag: checksums,
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"detected_linux_runtime_lines",
lambda: (["cuda13"], {"cuda13": ["/usr/local/cuda/lib64"]}),
)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (13, 1),
compute_caps = ["100"],
visible_cuda_devices = None,
has_physical_nvidia = True,
has_usable_nvidia = True,
)
plan = INSTALL_LLAMA_PREBUILT.direct_linux_release_plan(
release,
host,
"unslothai/llama.cpp",
"latest",
)
assert plan is not None
assert plan.llama_tag == "master"
assert plan.approved_checksums.upstream_tag == "b9174"
assert plan.approved_checksums.source_commit == source_commit
assert plan.attempts[0].expected_sha256 == "a" * 64
source_repo, source_ref, _source_archive, exact_source = (
INSTALL_LLAMA_PREBUILT.preferred_source_archive(plan.approved_checksums, plan.llama_tag)
)
assert source_repo == "ggml-org/llama.cpp"
assert source_ref == source_commit
assert exact_source is True
def test_simple_linux_direct_release_honors_torch_cudart_preference(
monkeypatch: pytest.MonkeyPatch,
):
# Regression: a Blackwell host (sm_120, driver 13.0) with BOTH cudart majors
# visible -- a stray cuda13 wheel plus torch's cuda12 -- must install the
# cuda12 build that matches the runtime torch, not the newest-major cuda13
# build (which loads no GPU and silently falls back to CPU).
release = {
"tag_name": "b9334",
"assets": [
{
"name": f"app-b9334-linux-x64-{profile}.tar.gz",
"browser_download_url": f"https://example.test/app-b9334-linux-x64-{profile}.tar.gz",
}
for profile in (
"cuda12-newer",
"cuda12-portable",
"cuda13-newer",
"cuda13-portable",
)
],
}
# cuda13 detected first (newest-major order); both compatible with driver 13.0.
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"detected_linux_runtime_lines",
lambda: (
["cuda13", "cuda12"],
{
"cuda13": ["/usr/local/lib/python3.13/site-packages/nvidia/cu13/lib"],
"cuda12": ["/venv/lib/python3.13/site-packages/nvidia/cuda_runtime/lib"],
},
),
)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = "nvidia-smi",
driver_cuda_version = (13, 0),
compute_caps = ["120"],
visible_cuda_devices = None,
has_physical_nvidia = True,
has_usable_nvidia = True,
)
def first_asset_for_torch(line):
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"detect_torch_cuda_runtime_preference",
lambda h: INSTALL_LLAMA_PREBUILT.CudaRuntimePreference(
runtime_line = line, selection_log = []
),
)
plan = INSTALL_LLAMA_PREBUILT.direct_linux_release_plan(
release, host, "unslothai/llama.cpp", "latest"
)
return plan.attempts[0]
# torch reports cuda12 (the cu128 runtime) -> install the cuda12 build.
primary = first_asset_for_torch("cuda12")
assert primary.name == "app-b9334-linux-x64-cuda12-newer.tar.gz"
assert primary.runtime_line == "cuda12"
# torch unavailable -> unchanged newest-major fallback (documents the residual).
assert first_asset_for_torch(None).name == "app-b9334-linux-x64-cuda13-newer.tar.gz"
@pytest.mark.parametrize(
"mutate, expected_match",
[
# Missing source_commit.
(
lambda c: setattr(c, "source_commit", None) or setattr(c, "source_commit_short", None),
"exact source provenance",
),
# source_commit present, but no exact-source archive hash.
(
lambda c: c.artifacts.pop(
INSTALL_LLAMA_PREBUILT.exact_source_archive_logical_name(c.source_commit),
None,
),
"exact source provenance",
),
# source_commit + exact-source archive present, but no source_repo.
(
lambda c: setattr(c, "source_repo", None) or setattr(c, "source_repo_url", None),
"exact source provenance",
),
],
ids = [
"missing_source_commit",
"missing_exact_source_artifact",
"missing_source_repo",
],
)
def test_simple_linux_direct_release_rejects_branch_without_exact_source_metadata(
monkeypatch: pytest.MonkeyPatch, mutate, expected_match
):
source_commit = "25b1bc9c2f9aa0a390b968ee1ffd9ff01340a3fe"
release = {
"tag_name": "llama-prebuilt-master-3a92bc9",
"assets": [
{
"name": "app-master-linux-x64-cuda13-newer.tar.gz",
"browser_download_url": "https://example.test/app-master-linux-x64-cuda13-newer.tar.gz",
},
{
"name": "llama-prebuilt-sha256.json",
"browser_download_url": "https://example.test/llama-prebuilt-sha256.json",
},
],
}
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "llama-prebuilt-master-3a92bc9",
upstream_tag = "b9174",
source_commit = source_commit,
source_repo = "ggml-org/llama.cpp",
source_repo_url = "https://github.com/ggml-org/llama.cpp",
source_ref_kind = "branch",
requested_source_ref = "master",
resolved_source_ref = "master",
artifacts = {
"app-master-linux-x64-cuda13-newer.tar.gz": ApprovedArtifactHash(
asset_name = "app-master-linux-x64-cuda13-newer.tar.gz",
sha256 = "a" * 64,
repo = "unslothai/llama.cpp",
kind = "linux-cuda-app",
),
INSTALL_LLAMA_PREBUILT.exact_source_archive_logical_name(
source_commit
): ApprovedArtifactHash(
asset_name = INSTALL_LLAMA_PREBUILT.exact_source_archive_logical_name(source_commit),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "exact-source",
),
},
)
mutate(checksums)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"load_approved_release_checksums",
lambda repo, release_tag: checksums,
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"detected_linux_runtime_lines",
lambda: (["cuda13"], {"cuda13": ["/usr/local/cuda/lib64"]}),
)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (13, 1),
compute_caps = ["100"],
visible_cuda_devices = None,
has_physical_nvidia = True,
has_usable_nvidia = True,
)
with pytest.raises(PrebuiltFallback, match = expected_match):
INSTALL_LLAMA_PREBUILT.direct_linux_release_plan(
release,
host,
"unslothai/llama.cpp",
"latest",
)
def test_simple_linux_direct_release_keeps_legacy_b_tag_path_without_checksums(
monkeypatch: pytest.MonkeyPatch,
):
release = {
"tag_name": "b9999",
"assets": [
{
"name": "app-b9999-linux-x64-cuda13-newer.tar.gz",
"browser_download_url": "https://example.test/app-b9999-linux-x64-cuda13-newer.tar.gz",
},
{
"name": "llama-prebuilt-sha256.json",
"browser_download_url": "https://example.test/llama-prebuilt-sha256.json",
},
],
}
def unexpected_checksum_load(repo: str, release_tag: str):
raise AssertionError("legacy b-tag direct releases should not require checksum metadata")
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"load_approved_release_checksums",
unexpected_checksum_load,
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"detected_linux_runtime_lines",
lambda: (["cuda13"], {"cuda13": ["/usr/local/cuda/lib64"]}),
)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (13, 1),
compute_caps = ["100"],
visible_cuda_devices = None,
has_physical_nvidia = True,
has_usable_nvidia = True,
)
plan = INSTALL_LLAMA_PREBUILT.direct_linux_release_plan(
release,
host,
"unslothai/llama.cpp",
"latest",
)
assert plan is not None
assert plan.llama_tag == "b9999"
assert plan.release_tag == "b9999"
assert plan.approved_checksums.source_commit is None
assert plan.attempts[0].expected_sha256 is None
def test_validate_prebuilt_choice_creates_repo_shaped_windows_install(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
@ -993,7 +663,7 @@ def test_install_prebuilt_falls_back_to_older_release_plan(
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_install_release_plans",
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[first_plan, second_plan],
@ -1921,7 +1591,7 @@ def test_install_prebuilt_skips_download_when_existing_install_matches(
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_install_release_plans",
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[plan],
@ -2011,7 +1681,7 @@ def test_install_prebuilt_does_not_skip_unhealthy_existing_install(
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_install_release_plans",
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[plan],
@ -2139,7 +1809,7 @@ def test_install_prebuilt_skips_when_older_release_fallback_matches_existing_ins
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_install_release_plans",
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[latest_plan, fallback_plan],
@ -2286,7 +1956,7 @@ def test_install_prebuilt_skips_same_release_fallback_attempt_when_installed(
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_install_release_plans",
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[plan],
@ -2405,7 +2075,7 @@ def test_install_prebuilt_same_tag_upstream_failure_uses_older_unsloth_release_p
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_install_release_plans",
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[latest_plan, older_plan],

7
tests/studio/install/test_llama_pr_force_and_source.py
View file

@ -463,8 +463,13 @@ class TestSourcePatternsPs1:
assert "$LlamaSource = $DefaultLlamaSource" in self.content
def test_release_repo_override_removed(self):
# No env-based release-repo override; the repo is chosen by GPU detection
# (GPU -> fork, CPU -> ggml-org), mirroring setup.sh.
assert "$HelperReleaseRepo = if ($env:UNSLOTH_LLAMA_RELEASE_REPO)" not in self.content
assert '$HelperReleaseRepo = "ggml-org/llama.cpp"' in self.content
assert (
"$HelperReleaseRepo = if ($HasNvidiaSmi -or $HasROCm) "
'{ "unslothai/llama.cpp" } else { "ggml-org/llama.cpp" }' in self.content
)
def test_force_compile_skips_prebuilt_resolution_early(self):
assert 'if ($env:UNSLOTH_LLAMA_FORCE_COMPILE -eq "1") {' in self.content

24
tests/studio/install/test_pr4562_bugfixes.py
View file

@ -686,14 +686,26 @@ class TestSourceCodePatterns:
assert "_RESOLVED_SOURCE_REF_KIND" in content
assert "_RESOLVED_SOURCE_REF" in content
def test_setup_sh_prebuilt_install_uses_simple_policy_only(self):
"""Shell prebuilt path should use the simplified helper install entrypoint."""
def test_setup_sh_prebuilt_install_entrypoint(self):
"""Shell prebuilt path should call the helper install entrypoint, not the
old tag-resolution / releases-latest flow."""
content = SETUP_SH.read_text()
assert "--simple-policy" in content
assert "--resolve-install-tag" not in content
assert "_HELPER_RELEASE_REPO}/releases/latest" not in content
assert "ggml-org/llama.cpp/releases/latest" not in content
def test_setup_sh_routes_to_fork_only_on_usable_gpu(self):
"""Linux fork-vs-ggml routing must gate NVIDIA on actual GPU usability,
not mere nvidia-smi presence, so CPU-only / hidden-GPU hosts (e.g.
CUDA_VISIBLE_DEVICES=-1) get the ggml CPU prebuilt instead of a source
build. Guards against a silent revert to the old presence-only loop."""
content = SETUP_SH.read_text()
assert '[ "$_setup_nvidia_usable" = true ]' in content
assert "CUDA_VISIBLE_DEVICES" in content
# nvidia-smi must NOT be back in the bare presence loop.
assert "for _GPU_TOOL in nvidia-smi" not in content
assert "for _GPU_TOOL in rocminfo amd-smi hipconfig hipinfo" in content
def test_setup_sh_reports_installed_prebuilt_release(self):
"""Shell wrapper should report the installed prebuilt release from metadata."""
content = SETUP_SH.read_text()
@ -832,10 +844,10 @@ class TestSourceCodePatterns:
if "LlamaCppDir" in context:
pytest.fail(f"Found 'git pull' in llama.cpp build section at line {i+1}")
def test_setup_ps1_prebuilt_install_uses_simple_policy_only(self):
"""PS1 prebuilt path should use the simplified helper install entrypoint."""
def test_setup_ps1_prebuilt_install_entrypoint(self):
"""PS1 prebuilt path should call the helper install entrypoint, not the
old tag-resolution / releases-latest flow."""
content = SETUP_PS1.read_text()
assert '"--simple-policy"' in content
assert "--resolve-install-tag" not in content
assert "$HelperReleaseRepo/releases/latest" not in content
assert "ggml-org/llama.cpp/releases/latest" not in content

695
tests/studio/install/test_selection_logic.py
View file

@ -54,12 +54,11 @@ compatible_windows_runtime_lines = INSTALL_LLAMA_PREBUILT.compatible_windows_run
runtime_line_from_cuda_version = INSTALL_LLAMA_PREBUILT.runtime_line_from_cuda_version
apply_approved_hashes = INSTALL_LLAMA_PREBUILT.apply_approved_hashes
linux_cuda_choice_from_release = INSTALL_LLAMA_PREBUILT.linux_cuda_choice_from_release
parse_direct_linux_release_bundle = INSTALL_LLAMA_PREBUILT.parse_direct_linux_release_bundle
windows_cuda_attempts = INSTALL_LLAMA_PREBUILT.windows_cuda_attempts
resolve_upstream_asset_choice = INSTALL_LLAMA_PREBUILT.resolve_upstream_asset_choice
resolve_requested_install_tag = INSTALL_LLAMA_PREBUILT.resolve_requested_install_tag
resolve_install_attempts = INSTALL_LLAMA_PREBUILT.resolve_install_attempts
resolve_install_release_plans = INSTALL_LLAMA_PREBUILT.resolve_install_release_plans
_fork_manifest_release_plans = INSTALL_LLAMA_PREBUILT._fork_manifest_release_plans
resolve_published_release = INSTALL_LLAMA_PREBUILT.resolve_published_release
resolve_source_build_plan = INSTALL_LLAMA_PREBUILT.resolve_source_build_plan
validated_checksums_for_bundle = INSTALL_LLAMA_PREBUILT.validated_checksums_for_bundle
@ -114,7 +113,9 @@ def load_studio_run_module(monkeypatch):
return module
# ---------------------------------------------------------------------------
# Helper factories
# ---------------------------------------------------------------------------
def make_host(**overrides):
@ -272,7 +273,9 @@ def mock_windows_runtime(monkeypatch, lines):
)
# ===========================================================================
# Studio run.py localhost warning
# ===========================================================================
class TestStudioLocalhostIpv6Warning:
@ -453,7 +456,9 @@ class TestStudioLocalhostIpv6Warning:
assert calls["stop_hint"] == 1
# ===========================================================================
# A. normalize_compute_cap
# ===========================================================================
class TestNormalizeComputeCap:
@ -485,7 +490,9 @@ class TestNormalizeComputeCap:
assert normalize_compute_cap("9.0") == "90"
# ===========================================================================
# B. normalize_compute_caps
# ===========================================================================
class TestNormalizeComputeCaps:
@ -502,7 +509,9 @@ class TestNormalizeComputeCaps:
assert normalize_compute_caps([]) == []
# ===========================================================================
# C. parse_cuda_visible_devices
# ===========================================================================
class TestParseCudaVisibleDevices:
@ -525,7 +534,9 @@ class TestParseCudaVisibleDevices:
assert parse_cuda_visible_devices(" 0 , 1 ") == ["0", "1"]
# ===========================================================================
# D. supports_explicit_visible_device_matching
# ===========================================================================
class TestSupportsExplicitVisibleDeviceMatching:
@ -545,7 +556,9 @@ class TestSupportsExplicitVisibleDeviceMatching:
assert supports_explicit_visible_device_matching(["0", "MIG-device"]) is False
# ===========================================================================
# E. select_visible_gpu_rows
# ===========================================================================
class TestSelectVisibleGpuRows:
@ -578,7 +591,9 @@ class TestSelectVisibleGpuRows:
assert result == []
# ===========================================================================
# F. compatible_linux_runtime_lines
# ===========================================================================
class TestCompatibleLinuxRuntimeLines:
@ -604,38 +619,9 @@ class TestCompatibleLinuxRuntimeLines:
assert compatible_linux_runtime_lines(host) == ["cuda14", "cuda13", "cuda12"]
class TestParseDirectLinuxReleaseBundle:
def _release(self, *targets):
names = [f"app-bTEST-linux-x64-{t}.tar.gz" for t in targets]
return {
"tag_name": "bTEST",
"assets": [{"name": n, "browser_download_url": "https://x/" + n} for n in names],
}
def _cuda_artifact(self, bundle):
return [a for a in bundle.artifacts if a.install_kind == "linux-cuda"][0]
def test_parses_known_cuda13_bundle(self):
bundle = parse_direct_linux_release_bundle(
"unslothai/llama.cpp", self._release("cuda13-newer")
)
assert bundle is not None
assert self._cuda_artifact(bundle).runtime_line == "cuda13"
def test_parses_future_cuda_major_with_forward_profile(self):
# A future major name parses and inherits the newest known major's
# coverage for the same class as a forward default.
bundle = parse_direct_linux_release_bundle(
"unslothai/llama.cpp", self._release("cuda14-newer")
)
assert bundle is not None
art = self._cuda_artifact(bundle)
assert art.runtime_line == "cuda14"
assert art.coverage_class == "newer"
assert art.max_sm == 120 # inherited from cuda13-newer
# ===========================================================================
# G. pick_windows_cuda_runtime + compatible_windows_runtime_lines
# ===========================================================================
class TestPickWindowsCudaRuntime:
@ -669,6 +655,14 @@ class TestCompatibleWindowsRuntimeLines:
host = make_host(driver_cuda_version = (12, 4))
assert compatible_windows_runtime_lines(host) == ["cuda12"]
@pytest.mark.parametrize("minor", [0, 1, 2, 3])
def test_cuda12_runs_on_any_12_x_driver(self, minor):
# cuda12 app bundles are toolkit-12.8 builds with bundled runtime; CUDA
# minor-version compatibility runs them on any 12.x driver, same as Linux.
# Previously Windows wrongly gated cuda12 below a 12.4 driver.
host = make_host(driver_cuda_version = (12, minor))
assert compatible_windows_runtime_lines(host) == ["cuda12"]
def test_driver_13_1(self):
host = make_host(driver_cuda_version = (13, 1))
assert compatible_windows_runtime_lines(host) == ["cuda13", "cuda12"]
@ -682,7 +676,9 @@ class TestCompatibleWindowsRuntimeLines:
assert compatible_windows_runtime_lines(host) == ["cuda14", "cuda13", "cuda12"]
# ===========================================================================
# H. runtime_line_from_cuda_version
# ===========================================================================
class TestRuntimeLineFromCudaVersion:
@ -702,7 +698,9 @@ class TestRuntimeLineFromCudaVersion:
assert runtime_line_from_cuda_version("") is None
# ===========================================================================
# I. apply_approved_hashes
# ===========================================================================
class TestApplyApprovedHashes:
@ -816,7 +814,9 @@ class TestApplyApprovedHashes:
apply_approved_hashes([], checksums)
# ===========================================================================
# J. published release resolution
# ===========================================================================
class TestPublishedReleaseResolution:
@ -948,6 +948,7 @@ class TestPublishedReleaseResolution:
[],
release_tag = release_tag,
upstream_tag = "b9000",
source_repo = "example/custom-llama.cpp",
source_commit = commit,
),
)
@ -1149,8 +1150,55 @@ class TestValidatedChecksumsForBundle:
with pytest.raises(PrebuiltFallback, match = "manifest checksum"):
validated_checksums_for_bundle("unslothai/llama.cpp", bundle)
def test_rejects_exact_source_without_repo(self, monkeypatch):
# An exact source archive with no source repo to clone from would let
# preferred_source_archive silently fall back to upstream source at the
# tag, so validation must fail closed (clean source build instead).
bundle = make_release([], release_tag = "r1", upstream_tag = "b8508")
checksums = make_checksums_with_source(
[], release_tag = "r1", upstream_tag = "b8508", source_commit = "a" * 40
) # exact source archive, but no source_repo
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"load_approved_release_checksums",
lambda repo, release_tag: checksums,
)
with pytest.raises(PrebuiltFallback, match = "exact source archive"):
validated_checksums_for_bundle("unslothai/llama.cpp", bundle)
def test_accepts_exact_source_when_only_bundle_has_repo(self, monkeypatch):
# The source repo can live only in the manifest bundle, not the checksum
# payload. source_build_plan_for_release coalesces checksums-or-bundle, so
# validation must accept the bundle's repo rather than failing closed.
bundle = make_release(
[],
release_tag = "r1",
upstream_tag = "b8508",
source_repo = "ggml-org/llama.cpp",
source_repo_url = "https://github.com/ggml-org/llama.cpp",
)
checksums = make_checksums_with_source(
[], release_tag = "r1", upstream_tag = "b8508", source_commit = "a" * 40
) # exact source archive, repo only on the bundle
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"load_approved_release_checksums",
lambda repo, release_tag: checksums,
)
assert validated_checksums_for_bundle("unslothai/llama.cpp", bundle) is checksums
plan = INSTALL_LLAMA_PREBUILT.source_build_plan_for_release(
INSTALL_LLAMA_PREBUILT.ResolvedPublishedRelease(bundle = bundle, checksums = checksums)
)
assert plan.source_url == "https://github.com/ggml-org/llama.cpp"
assert plan.source_ref_kind == "commit"
assert plan.source_ref == "a" * 40
# ===========================================================================
# K. linux_cuda_choice_from_release -- core selection
# ===========================================================================
class TestLinuxCudaChoiceFromRelease:
@ -1200,6 +1248,36 @@ class TestLinuxCudaChoiceFromRelease:
log_entries = result.selection_log
assert any("unavailable_on_host" in entry for entry in log_entries)
def test_arm64_host_selects_linux_arm64_cuda_kind(self, monkeypatch):
# An arm64 CUDA host (DGX Spark / Grace Hopper) selects the
# linux-arm64-cuda bundle and ignores the x64 linux-cuda one.
mock_linux_runtime(monkeypatch, ["cuda13"])
host = make_host(
machine = "aarch64",
driver_cuda_version = (13, 0),
compute_caps = ["90"],
)
arm = make_artifact(
"app-b9457-linux-arm64-cuda13-portable.tar.gz",
install_kind = "linux-arm64-cuda",
runtime_line = "cuda13",
coverage_class = "portable",
supported_sms = ["90", "100", "120", "121"],
min_sm = 90,
max_sm = 121,
bundle_profile = "cuda13-portable",
)
x64 = make_artifact(
"app-b9457-linux-x64-cuda13-portable.tar.gz",
install_kind = "linux-cuda",
runtime_line = "cuda13",
)
release = make_release([arm, x64])
result = linux_cuda_choice_from_release(host, release)
assert result is not None
assert result.primary.install_kind == "linux-arm64-cuda"
assert result.primary.name == "app-b9457-linux-arm64-cuda13-portable.tar.gz"
# --- SM matching ---
def test_exact_sm_match(self, monkeypatch):
@ -1404,80 +1482,9 @@ class TestLinuxCudaChoiceFromRelease:
assert result is None
def make_profile_artifact(asset_name, profile_name, **overrides):
profile = INSTALL_LLAMA_PREBUILT.DIRECT_LINUX_BUNDLE_PROFILES[profile_name]
defaults = dict(
runtime_line = profile["runtime_line"],
coverage_class = profile["coverage_class"],
supported_sms = [str(value) for value in profile["supported_sms"]],
min_sm = int(profile["min_sm"]),
max_sm = int(profile["max_sm"]),
bundle_profile = profile_name,
rank = int(profile["rank"]),
)
defaults.update(overrides)
return make_artifact(asset_name, **defaults)
class TestBlackwellUltraSm103Coverage:
"""sm_103 (B300 / GB300) runs on the bundled base compute_100 PTX via JIT."""
def test_profiles_list_sm103_wherever_sm100_is_shipped(self):
for (
name,
profile,
) in INSTALL_LLAMA_PREBUILT.DIRECT_LINUX_BUNDLE_PROFILES.items():
sms = {str(value) for value in profile["supported_sms"]}
if "100" in sms:
assert "103" in sms, name
else:
assert "103" not in sms, name
def test_b300_selects_cuda13_newer_prebuilt(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda13"])
host = make_host(compute_caps = ["103"], driver_cuda_version = (13, 0))
art = make_profile_artifact("cuda13-newer.tar.gz", "cuda13-newer")
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is not None
assert result.primary.name == "cuda13-newer.tar.gz"
def test_b300_selects_cuda12_newer_prebuilt(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["103"], driver_cuda_version = (12, 8))
art = make_profile_artifact("cuda12-newer.tar.gz", "cuda12-newer")
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is not None
assert result.primary.name == "cuda12-newer.tar.gz"
def test_b300_reported_as_decimal_normalizes_and_matches(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda13"])
host = make_host(compute_caps = ["10.3"], driver_cuda_version = (13, 0))
art = make_profile_artifact("cuda13-portable.tar.gz", "cuda13-portable")
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is not None
def test_b300_falls_back_to_portable_when_only_portable_present(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda13"])
host = make_host(compute_caps = ["103"], driver_cuda_version = (13, 0))
art = make_profile_artifact("cuda13-portable.tar.gz", "cuda13-portable")
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is not None
assert result.primary.name == "cuda13-portable.tar.gz"
def test_older_bundle_still_rejects_b300(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda13"])
host = make_host(compute_caps = ["103"], driver_cuda_version = (13, 0))
art = make_profile_artifact("cuda13-older.tar.gz", "cuda13-older")
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is None
# ===========================================================================
# L. resolve_install_attempts
# ===========================================================================
class TestResolveInstallAttempts:
@ -1602,7 +1609,11 @@ class TestResolveInstallAttempts:
assert attempts[0].expected_sha256 == "a" * 64
assert approved.release_tag == "llama-prebuilt-latest"
def test_linux_cpu_uses_same_tag_upstream_asset(self, monkeypatch):
def test_linux_cpu_fork_without_bundle_raises_no_upstream_fallback(self, monkeypatch):
# A CPU-only Linux host on the fork no longer falls back to the ggml-org
# CPU asset: production routes CPU-only Linux to ggml-org, never the fork.
# With no fork CPU bundle in the manifest the resolver raises rather than
# quietly reaching for an upstream asset.
host = make_host(
has_usable_nvidia = False,
has_physical_nvidia = False,
@ -1610,7 +1621,7 @@ class TestResolveInstallAttempts:
)
release = make_release([], release_tag = "llama-prebuilt-latest", upstream_tag = "b9000")
checksums = make_checksums_with_source(
["llama-b9000-bin-ubuntu-x64.tar.gz"],
[],
release_tag = release.release_tag,
upstream_tag = "b9000",
)
@ -1630,22 +1641,13 @@ class TestResolveInstallAttempts:
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"github_release_assets",
lambda repo, tag: {
f"llama-{tag}-bin-ubuntu-x64.tar.gz": f"https://example.com/llama-{tag}-bin-ubuntu-x64.tar.gz"
},
lambda repo, tag: (_ for _ in ()).throw(
AssertionError("fork CPU host must not query upstream assets")
),
)
_requested_tag, resolved_tag, attempts, _approved = resolve_install_attempts(
"latest",
host,
"unslothai/llama.cpp",
"",
)
assert resolved_tag == "b9000"
assert attempts[0].name == "llama-b9000-bin-ubuntu-x64.tar.gz"
assert attempts[0].source_label == "upstream"
assert attempts[0].expected_sha256 == "a" * 64
with pytest.raises(PrebuiltFallback, match = "no compatible Linux prebuilt asset was found"):
resolve_install_attempts("latest", host, "unslothai/llama.cpp", "")
def test_linux_cuda_does_not_fall_back_to_upstream_cpu(self, monkeypatch):
host = make_host(system = "Linux", machine = "x86_64", compute_caps = ["86"])
@ -1670,7 +1672,7 @@ class TestResolveInstallAttempts:
)
mock_linux_runtime(monkeypatch, ["cuda12"])
with pytest.raises(PrebuiltFallback, match = "no compatible published Linux CUDA bundle"):
with pytest.raises(PrebuiltFallback, match = "no compatible Linux prebuilt asset was found"):
resolve_install_attempts("latest", host, "unslothai/llama.cpp", "")
def test_windows_cpu_prefers_published_asset(self, monkeypatch):
@ -1866,37 +1868,35 @@ class TestResolveInstallAttempts:
class TestResolveInstallReleasePlans:
def test_latest_collects_multiple_older_release_plans_up_to_limit(self, monkeypatch):
host = make_host(
has_usable_nvidia = False,
has_physical_nvidia = False,
nvidia_smi = None,
def _cuda_bundle(self, asset_name, release_tag, upstream_tag):
# A fork CUDA bundle that covers the default NVIDIA host (sm 86,
# cuda12 runtime), so each release yields a plan via
# linux_cuda_choice_from_release.
art = make_artifact(
asset_name,
install_kind = "linux-cuda",
runtime_line = "cuda12",
coverage_class = "portable",
supported_sms = ["75", "80", "86", "89", "90"],
min_sm = 75,
max_sm = 90,
)
return INSTALL_LLAMA_PREBUILT.ResolvedPublishedRelease(
bundle = make_release([art], release_tag = release_tag, upstream_tag = upstream_tag),
checksums = make_checksums_with_source(
[asset_name],
release_tag = release_tag,
upstream_tag = upstream_tag,
),
)
def test_latest_collects_multiple_older_release_plans_up_to_limit(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(system = "Linux", machine = "x86_64", compute_caps = ["86"])
releases = [
INSTALL_LLAMA_PREBUILT.ResolvedPublishedRelease(
bundle = make_release([], release_tag = "r3", upstream_tag = "b9003"),
checksums = make_checksums_with_source(
["llama-b9003-bin-ubuntu-x64.tar.gz"],
release_tag = "r3",
upstream_tag = "b9003",
),
),
INSTALL_LLAMA_PREBUILT.ResolvedPublishedRelease(
bundle = make_release([], release_tag = "r2", upstream_tag = "b9002"),
checksums = make_checksums_with_source(
["llama-b9002-bin-ubuntu-x64.tar.gz"],
release_tag = "r2",
upstream_tag = "b9002",
),
),
INSTALL_LLAMA_PREBUILT.ResolvedPublishedRelease(
bundle = make_release([], release_tag = "r1", upstream_tag = "b9001"),
checksums = make_checksums_with_source(
["llama-b9001-bin-ubuntu-x64.tar.gz"],
release_tag = "r1",
upstream_tag = "b9001",
),
),
self._cuda_bundle("app-b9003-linux-x64-cuda12.tar.gz", "r3", "b9003"),
self._cuda_bundle("app-b9002-linux-x64-cuda12.tar.gz", "r2", "b9002"),
self._cuda_bundle("app-b9001-linux-x64-cuda12.tar.gz", "r1", "b9001"),
]
monkeypatch.setattr(
@ -1904,15 +1904,8 @@ class TestResolveInstallReleasePlans:
"iter_resolved_published_releases",
lambda requested_tag, published_repo, published_release_tag = "": iter(releases),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"github_release_assets",
lambda repo, tag: {
f"llama-{tag}-bin-ubuntu-x64.tar.gz": f"https://example.com/llama-{tag}-bin-ubuntu-x64.tar.gz"
},
)
requested_tag, plans = resolve_install_release_plans(
requested_tag, plans = _fork_manifest_release_plans(
"latest",
host,
"unslothai/llama.cpp",
@ -1925,12 +1918,10 @@ class TestResolveInstallReleasePlans:
assert [plan.llama_tag for plan in plans] == ["b9003", "b9002"]
def test_latest_skips_non_installable_release_and_keeps_searching(self, monkeypatch):
host = make_host(
has_usable_nvidia = False,
has_physical_nvidia = False,
nvidia_smi = None,
)
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(system = "Linux", machine = "x86_64", compute_caps = ["86"])
releases = [
# r2 ships no fork bundle, so it yields no plan and is skipped.
INSTALL_LLAMA_PREBUILT.ResolvedPublishedRelease(
bundle = make_release([], release_tag = "r2", upstream_tag = "b9002"),
checksums = make_checksums_with_source(
@ -1939,14 +1930,7 @@ class TestResolveInstallReleasePlans:
upstream_tag = "b9002",
),
),
INSTALL_LLAMA_PREBUILT.ResolvedPublishedRelease(
bundle = make_release([], release_tag = "r1", upstream_tag = "b9001"),
checksums = make_checksums_with_source(
["llama-b9001-bin-ubuntu-x64.tar.gz"],
release_tag = "r1",
upstream_tag = "b9001",
),
),
self._cuda_bundle("app-b9001-linux-x64-cuda12.tar.gz", "r1", "b9001"),
]
monkeypatch.setattr(
@ -1954,19 +1938,8 @@ class TestResolveInstallReleasePlans:
"iter_resolved_published_releases",
lambda requested_tag, published_repo, published_release_tag = "": iter(releases),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"github_release_assets",
lambda repo, tag: (
{}
if tag == "b9002"
else {
f"llama-{tag}-bin-ubuntu-x64.tar.gz": f"https://example.com/llama-{tag}-bin-ubuntu-x64.tar.gz"
}
),
)
_requested_tag, plans = resolve_install_release_plans(
_requested_tag, plans = _fork_manifest_release_plans(
"latest",
host,
"unslothai/llama.cpp",
@ -1998,7 +1971,9 @@ class TestResolveInstallReleasePlans:
sys.modules.pop(spec.name, None)
# ===========================================================================
# N. windows_cuda_attempts
# ===========================================================================
class TestWindowsCudaAttempts:
@ -2216,7 +2191,9 @@ class TestWindowsCudaAttempts:
assert result[0].name == f"llama-{self.TAG}-bin-win-cuda-13.3-x64.zip"
# ===========================================================================
# N.1b. _pinned_windows_cuda_fallback -- pinned b9360 cuda-13.1 Blackwell fallback
# ===========================================================================
class TestPinnedBlackwellCudaFallback:
@ -2357,8 +2334,49 @@ class TestPinnedBlackwellCudaFallback:
)
assert _windows_cuda_attempt_covers_blackwell(cpu) is False
def _app_attempt(self, profile, runtime_line, max_sm):
# The fork's app-named windows-cuda bundle: no toolkit minor in the name,
# SM coverage declared directly (as published_windows_cuda_attempts sets it).
return AssetChoice(
repo = UPSTREAM_REPO,
tag = self.TAG,
name = f"app-{self.TAG}-windows-x64-{runtime_line}-{profile}.zip",
url = "https://example.com/x",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = runtime_line,
coverage_class = "newer" if profile == "newer" else profile,
max_sm = max_sm,
min_sm = 80,
supported_sms = ["120"] if max_sm >= 120 else ["86", "89"],
)
@pytest.mark.parametrize(
"profile, runtime_line, max_sm, covers",
[
("newer", "cuda13", 120, True), # native Blackwell build
("newer", "cuda12", 120, True), # 12.8 toolkit app bundle reaches sm120
("older", "cuda12", 89, False), # 12.4 toolkit app bundle stops at Ada
],
)
def test_attempt_covers_blackwell_app_bundle(self, profile, runtime_line, max_sm, covers):
# App-named bundles carry no toolkit minor; coverage is read from max_sm.
attempt = self._app_attempt(profile, runtime_line, max_sm)
assert _windows_cuda_attempt_covers_blackwell(attempt) is covers
def test_pin_dormant_when_app_bundle_covers_blackwell(self):
# Regression: the fork's app-named cuda13 bundle covers Blackwell, so the
# b9360 pin must retire instead of being prepended ahead of the native
# in-release build (previously the coverage check only matched legacy
# -bin-win-cuda-X.Y-x64.zip names, so the pin never went dormant).
host = self._win_host((13, 1), ["120"])
existing = [self._app_attempt("newer", "cuda13", 120)]
assert _pinned_windows_cuda_fallback(host, existing) is None
# ===========================================================================
# N.1c. direct_upstream_release_plan -- pinned Blackwell fallback ordering
# ===========================================================================
class TestDirectUpstreamBlackwellPin:
@ -2593,7 +2611,9 @@ class TestDirectLinuxNvidiaCpuGate:
assert [a.install_kind for a in plan.attempts] == ["linux-cpu"]
# ===========================================================================
# N.1d. published_windows_cuda_attempts -- version-dynamic ordering seed
# ===========================================================================
class TestPublishedWindowsCudaAttemptsDynamicMajor:
@ -2608,6 +2628,7 @@ class TestPublishedWindowsCudaAttemptsDynamicMajor:
f"llama-{self.TAG}-bin-win-cuda-{minor}-x64.zip",
install_kind = "windows-cuda",
runtime_line = runtime_line,
supported_sms = ["75", "80", "86", "89", "90", "100", "120"],
max_sm = 120,
)
@ -2659,12 +2680,14 @@ class TestPublishedWindowsCudaAttemptsDynamicMajor:
assert result[0].runtime_line == "cuda12"
# ===========================================================================
# N.1e. resolve_release_asset_choice -- pin on the published install path
# ===========================================================================
class TestResolveReleaseAssetChoicePin:
"""The published (non --simple-policy) install path reaches the same b9360
Blackwell pin as the simple path, with its verified hash threaded."""
"""The manifest install path reaches the same b9360 Blackwell pin as the
filename path, with its verified hash threaded."""
TAG = "b8508"
@ -2674,6 +2697,7 @@ class TestResolveReleaseAssetChoicePin:
f"llama-{self.TAG}-bin-win-cuda-{minor}-x64.zip",
install_kind = "windows-cuda",
runtime_line = line,
supported_sms = ["75", "80", "86", "89", "90", "100", "120"],
max_sm = 120,
)
for minor, line in minors_lines
@ -2754,7 +2778,243 @@ class TestResolveReleaseAssetChoicePin:
assert "b9360" not in [a.tag for a in result]
class TestPublishedWindowsCudaAppBundleSmSelection:
"""app-named windows-cuda bundles carry no minor in the filename, so the
driver-minor gate is skipped. Selection must instead filter by SM coverage,
or every host gets the lowest-rank "older" bundle regardless of its GPU."""
TAG = "b9457"
def _app(self, klass, supported, min_sm, max_sm, rank):
return make_artifact(
f"app-{self.TAG}-windows-x64-cuda12-{klass}.zip",
install_kind = "windows-cuda",
runtime_line = "cuda12",
coverage_class = klass,
supported_sms = supported,
min_sm = min_sm,
max_sm = max_sm,
bundle_profile = f"cuda12-{klass}",
rank = rank,
)
def test_blackwell_sm120_skips_older_bundle(self, monkeypatch):
mock_windows_runtime(monkeypatch, ["cuda12"])
older = self._app("older", ["70", "75", "80", "86", "89"], 70, 89, 10)
newer = self._app("newer", ["86", "89", "90", "100", "120"], 86, 120, 20)
portable = self._app(
"portable", ["70", "75", "80", "86", "89", "90", "100", "120"], 70, 120, 30
)
release = make_release([older, newer, portable], upstream_tag = self.TAG)
host = make_host(
system = "Windows",
machine = "AMD64",
driver_cuda_version = (12, 8),
compute_caps = ["120"],
)
result = published_windows_cuda_attempts(host, release, None)
assert result, "expected a windows-cuda attempt for an sm120 host"
# The lowest-rank "older" bundle (max_sm 89) must not be chosen, and the
# tightest covering bundle is cuda12-newer (range 86-120).
assert result[0].name == f"app-{self.TAG}-windows-x64-cuda12-newer.zip"
def _line(self, line, klass, rank):
return make_artifact(
f"app-{self.TAG}-windows-x64-{line}-{klass}.zip",
install_kind = "windows-cuda",
runtime_line = line,
coverage_class = klass,
supported_sms = ["86", "89", "90", "100", "120"],
min_sm = 86,
max_sm = 120,
bundle_profile = f"{line}-{klass}",
rank = rank,
)
def test_cuda13_reachable_on_driver_13_0(self, monkeypatch):
# app-named cuda13 bundles must be reachable on a 13.0 driver. The old
# synthetic '13.1' minor gate dropped the whole cuda13 line (13.1 > 13.0),
# so a cu13 host fell to cuda12. cuda13 is gated at the major level now.
mock_windows_runtime(monkeypatch, ["cuda13", "cuda12"])
release = make_release(
[self._line("cuda12", "newer", 20), self._line("cuda13", "newer", 50)],
upstream_tag = self.TAG,
)
host = make_host(
system = "Windows",
machine = "AMD64",
driver_cuda_version = (13, 0),
compute_caps = ["120"],
)
result = published_windows_cuda_attempts(host, release, "cuda13")
assert result
assert result[0].runtime_line == "cuda13"
assert result[0].name == f"app-{self.TAG}-windows-x64-cuda13-newer.zip"
def test_app_bundle_offered_when_no_runtime_dll_detected(self, monkeypatch):
# Windows torch bundles cudart in torch/lib, which runtime-DLL probing
# misses, so detected_windows_runtime_lines() returns nothing. The app
# bundle ships its own runtime, so selection must fall back to the
# driver-derived order instead of yielding no attempt (which would drop
# the host to the upstream build).
mock_windows_runtime(monkeypatch, [])
release = make_release(
[self._line("cuda12", "newer", 20), self._line("cuda13", "newer", 50)],
upstream_tag = self.TAG,
)
host = make_host(
system = "Windows",
machine = "AMD64",
driver_cuda_version = (13, 0),
compute_caps = ["120"],
)
result = published_windows_cuda_attempts(host, release, "cuda13")
assert result, "torch-only host must still get the fork app bundle"
assert result[0].name == f"app-{self.TAG}-windows-x64-cuda13-newer.zip"
class TestPublishedRocmGfxSelection:
"""Published ROCm bundles are matched by the host's detected gfx family, not
by rank -- rank ties would alphabetically hand every AMD GPU the gfx103X
bundle (e.g. a gfx1151 Strix Halo host)."""
GFX = ["gfx103X", "gfx110X", "gfx120X", "gfx1150", "gfx1151"]
MEMBERS = {
"gfx103X": ["gfx1030", "gfx1031", "gfx1032", "gfx1034"],
"gfx110X": ["gfx1100", "gfx1101", "gfx1102", "gfx1103"],
"gfx120X": ["gfx1200", "gfx1201"],
"gfx1150": ["gfx1150"],
"gfx1151": ["gfx1151"],
}
def _release(self, install_kind, prefix):
artifacts = [
make_artifact(
f"{prefix}-{gfx}.{'zip' if 'windows' in install_kind else 'tar.gz'}",
install_kind = install_kind,
runtime_line = None,
coverage_class = None,
supported_sms = [],
min_sm = None,
max_sm = None,
bundle_profile = None,
rank = 1000,
gfx_target = gfx,
mapped_targets = self.MEMBERS[gfx],
)
for gfx in self.GFX
]
return make_release(artifacts, upstream_tag = "b9457")
def _host(self, gfx):
return make_host(
machine = "x86_64",
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
has_physical_nvidia = False,
has_usable_nvidia = False,
has_rocm = True,
rocm_gfx_target = gfx,
)
def test_gfx1100_selects_gfx110X_family(self):
release = self._release("linux-rocm", "app-b9457-linux-x64-rocm")
choice = INSTALL_LLAMA_PREBUILT.published_rocm_choice_for_host(
release, self._host("gfx1100"), "linux-rocm"
)
assert choice is not None
assert choice.name == "app-b9457-linux-x64-rocm-gfx110X.tar.gz"
def test_gfx1151_strix_halo_not_handed_gfx103X(self):
release = self._release("linux-rocm", "app-b9457-linux-x64-rocm")
choice = INSTALL_LLAMA_PREBUILT.published_rocm_choice_for_host(
release, self._host("gfx1151"), "linux-rocm"
)
assert choice is not None
assert choice.name == "app-b9457-linux-x64-rocm-gfx1151.tar.gz"
def test_windows_rocm_gfx_match(self):
release = self._release("windows-rocm", "app-b9457-windows-x64-rocm")
choice = INSTALL_LLAMA_PREBUILT.published_rocm_choice_for_host(
release, self._host("gfx1201"), "windows-rocm"
)
assert choice is not None
assert choice.name == "app-b9457-windows-x64-rocm-gfx120X.zip"
def test_uncovered_gpu_returns_none(self):
release = self._release("linux-rocm", "app-b9457-linux-x64-rocm")
assert (
INSTALL_LLAMA_PREBUILT.published_rocm_choice_for_host(
release, self._host("gfx900"), "linux-rocm"
)
is None
)
def test_in_prefix_but_unbuilt_arch_returns_none(self):
# gfx1033 shares the gfx103 prefix but is not in any bundle's
# mapped_targets, so it must fall back to source, not be served gfx103X.
release = self._release("linux-rocm", "app-b9457-linux-x64-rocm")
for unbuilt in ("gfx1033", "gfx1035", "gfx1104", "gfx1202"):
assert (
INSTALL_LLAMA_PREBUILT.published_rocm_choice_for_host(
release, self._host(unbuilt), "linux-rocm"
)
is None
), unbuilt
class TestPublishedMacosForkSelection:
"""macOS now routes to the fork (setup.sh), which ships
llama-<tag>-bin-macos-<arch>.tar.gz with pinned deployment targets, selected
by install_kind."""
def _release(self):
arts = [
make_artifact(
"llama-b9457-bin-macos-arm64.tar.gz",
install_kind = "macos-arm64",
runtime_line = None,
coverage_class = None,
supported_sms = [],
min_sm = None,
max_sm = None,
bundle_profile = "macos-metal-arm64",
rank = 50,
),
make_artifact(
"llama-b9457-bin-macos-x64.tar.gz",
install_kind = "macos-x64",
runtime_line = None,
coverage_class = None,
supported_sms = [],
min_sm = None,
max_sm = None,
bundle_profile = "macos-cpu-x64",
rank = 50,
),
]
return make_release(arts, upstream_tag = "b9457")
def test_macos_arm64_selects_fork_bundle(self):
choice = INSTALL_LLAMA_PREBUILT.published_asset_choice_for_kind(
self._release(), "macos-arm64"
)
assert choice is not None
assert choice.name == "llama-b9457-bin-macos-arm64.tar.gz"
assert choice.install_kind == "macos-arm64"
def test_macos_x64_selects_fork_bundle(self):
choice = INSTALL_LLAMA_PREBUILT.published_asset_choice_for_kind(
self._release(), "macos-x64"
)
assert choice is not None
assert choice.name == "llama-b9457-bin-macos-x64.tar.gz"
# ===========================================================================
# N.1. apply_approved_hashes -- runtime archive checksum threading
# ===========================================================================
class TestApplyApprovedHashesRuntimePair:
@ -2826,7 +3086,9 @@ class TestApplyApprovedHashesRuntimePair:
assert result[0].runtime_sha256 is None
# ===========================================================================
# O. resolve_upstream_asset_choice -- platform routing
# ===========================================================================
class TestResolveUpstreamAssetChoice:
@ -2969,7 +3231,9 @@ class TestResolveUpstreamAssetChoice:
assert result.name == cuda_name
# ===========================================================================
# N.2. Deterministic macOS prebuilt pin (b9415)
# ===========================================================================
def _macos_host(machine = "arm64", version = (15, 5)):
@ -3092,37 +3356,47 @@ class TestResolveSimpleMacosPin:
assert calls[0][2] == "latest"
# ===========================================================================
# Linux arm64 + GPU must not install the x64-only fork bundle
# ===========================================================================
class TestLinuxArm64ForkFallsBackToSource:
"""The unslothai/llama.cpp fork ships only linux-x64 bundles. An arm64
Linux host with a GPU (GH200/GB200/DGX Spark) routes to the fork and must
fall back to a source build instead of selecting an x64 binary."""
"""The fork now ships linux-arm64-cuda bundles (GH200/GB200/DGX Spark). An
arm64 Linux host on the fork no longer hard-fails on the simple path; it
delegates to the manifest-aware resolver, which selects the arm64 CUDA
bundle (or falls back to source only if none matches)."""
def test_arm64_nvidia_fork_raises_before_fetching_releases(self, monkeypatch):
# Guard fires before any release is fetched: poison the iterator to prove
# it is never called.
def _boom(*_a, **_k):
raise AssertionError("iterator must not run for arm64 fork hosts")
def test_arm64_nvidia_fork_delegates_to_manifest_resolver(self, monkeypatch):
# arm64 fork hosts are no longer blocked up front; the simple resolver
# hands them to the manifest-aware resolver instead.
called = {}
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "iter_release_payloads_by_time", _boom)
def _full(llama_tag, host, repo, tag, **_kw):
called["args"] = (host.machine, repo)
return "b9457", ["plan"]
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "_fork_manifest_release_plans", _full)
host = make_host(system = "Linux", machine = "aarch64")
with pytest.raises(PrebuiltFallback, match = "linux-x64 prebuilts"):
resolve_simple_install_release_plans("latest", host, "unslothai/llama.cpp", "")
tag, plans = resolve_simple_install_release_plans("latest", host, "unslothai/llama.cpp", "")
assert called.get("args") == ("aarch64", "unslothai/llama.cpp")
assert plans == ["plan"]
def test_x86_64_fork_is_not_blocked_by_the_arch_guard(self, monkeypatch):
# x64 host must pass the guard and reach the iterator (here empty, so it
# raises the generic message, not the arch one).
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"iter_release_payloads_by_time",
lambda *_a, **_k: iter(()),
)
def test_x86_64_fork_delegates_to_manifest_resolver(self, monkeypatch):
# The old linux-x64 arch guard is gone: an x64 fork host is routed to the
# manifest resolver exactly like every other fork host, not down a
# separate filename-parsing path.
called = {}
def _full(llama_tag, host, repo, tag, **_kw):
called["args"] = (host.machine, repo)
return "b9457", ["plan"]
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "_fork_manifest_release_plans", _full)
host = make_host(system = "Linux", machine = "x86_64")
with pytest.raises(PrebuiltFallback) as exc:
resolve_simple_install_release_plans("latest", host, "unslothai/llama.cpp", "")
assert "linux-x64 prebuilts" not in str(exc.value)
tag, plans = resolve_simple_install_release_plans("latest", host, "unslothai/llama.cpp", "")
assert called.get("args") == ("x86_64", "unslothai/llama.cpp")
assert plans == ["plan"]
def test_arm64_cpu_on_ggml_org_is_not_blocked(self, monkeypatch):
# CPU-only arm64 routes to ggml-org (not the fork), so the guard must not
@ -3146,7 +3420,9 @@ class TestLinuxArm64ForkFallsBackToSource:
assert "linux-x64 prebuilts" not in str(exc.value)
# ===========================================================================
# arm64 Linux GPU: CPU prebuilt fallback after a failed source build (--cpu-fallback)
# ===========================================================================
class TestCpuFallback:
@ -3187,7 +3463,6 @@ class TestCpuFallback:
llama_tag = "latest",
published_repo = "ggml-org/llama.cpp",
published_release_tag = "",
simple_policy = True,
force_cpu = True,
)
host = captured["host"]