unsloth/tests/studio/install/test_selection_logic.py
DoubleMathew f4d8a246bf
Use prebuilt llama.cpp for unsloth studio setup (#4562)
* Use prebuilt llama.cpp for unsloth studio setup

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix 3 issues that cause unnecessary fallback to source build

1. Make filelock import optional -- environments without filelock
   (e.g. minimal installs) crashed at import time instead of
   gracefully skipping the lock.

2. Use already-verified converter script from the hydrated source
   tree instead of re-downloading from raw.githubusercontent.com
   with no checksum. Adds symlink with copy fallback for the
   legacy filename.

3. Initialize $SkipPrebuiltInstall in setup.ps1 before first use
   to prevent potential uninitialized variable errors.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep network fallback in ensure_converter_scripts

Prefer the local verified copy from the hydrated source tree, but
retain the original network download as a fallback if the file is
missing. Create the legacy hyphenated filename as a symlink with a
copy fallback instead of writing a second full copy.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix 4 bugs in source-build fallback and binary_env paths

- setup.ps1: Replace git pull + checkout FETCH_HEAD with fetch + checkout -B
  to avoid detached HEAD state that breaks re-runs. Use pinned tag in both
  fetch and clone paths.
- setup.sh: Move rm -rf after cmake/git prerequisite checks so a missing
  tool no longer deletes the existing install. Add --branch tag to clone.
- install_llama_prebuilt.py: Add binary_path.parent to Linux LD_LIBRARY_PATH
  in binary_env() so bundled .so files in build/bin are found even without
  RPATH, matching the existing Windows PATH logic.
- Add test for binary_env LD_LIBRARY_PATH on Linux.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Handle unresolved "latest" tag in source-build fallback clone

When tag resolution fails and the requested tag is "latest", both
setup scripts now omit --branch from git clone so the default branch
is cloned instead of failing on a nonexistent "latest" branch/tag.
Similarly, the PS1 fetch path fetches the default ref when the tag
is "latest".

* Resolve actual latest ggml-org tag instead of using literal "latest"

When both Python tag resolution attempts fail and the requested tag
is "latest", query the GitHub API for the actual latest release tag
from ggml-org/llama.cpp (e.g. b8508) instead of passing the literal
string "latest" to git clone --branch, which would fail since no
such branch/tag exists.

setup.sh uses curl + python json parsing; setup.ps1 uses
Invoke-RestMethod. Both fall back to the raw requested tag if the
API call also fails.

* Try Unsloth release repo before ggml-org when resolving latest tag

When falling back to the GitHub API to resolve "latest", query the
Unsloth release repo (unslothai/llama.cpp) first since it has the
prebuilt binaries pinned to tested tags. Only fall back to
ggml-org/llama.cpp if the Unsloth repo query fails.

* Add comprehensive sandbox tests for PR #4562 bug fixes

35 tests covering all fixes across platforms:
- binary_env cross-platform (Linux LD_LIBRARY_PATH, Windows PATH,
  macOS DYLD_LIBRARY_PATH) with edge cases (dedup, ordering, existing paths)
- resolve_requested_llama_tag (concrete, latest, None, empty)
- setup.sh logic via subprocess: prereq check ordering (cmake/git missing
  preserves install), pinned tag in clone, fetch+checkout -B pattern,
  fetch failure warns instead of aborting
- "latest" tag resolution fallback chain (Unsloth API -> ggml-org ->
  raw) with mock curl: success, failure, malformed JSON, empty body,
  empty tag_name, env overrides
- Source code pattern verification for both .sh and .ps1 files

All 138 tests pass in isolated uv venv.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add binary_path.parent to macOS DYLD_LIBRARY_PATH in binary_env

macOS prebuilt .dylib files are overlaid into build/bin (same as
Linux), but binary_env only added install_dir to DYLD_LIBRARY_PATH.
Add binary_path.parent so the loader can find sibling dylibs even
without embedded loader paths.

Mirrors the existing fix for Linux LD_LIBRARY_PATH and the Windows
PATH pattern.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Guard --branch when resolved tag is "latest"; fix broken test assertion

When all API fallbacks fail and the tag stays as literal "latest",
omit --branch from git clone (clones default branch instead of
failing). Both setup.sh and setup.ps1 now check for "latest" before
passing --branch to git clone/fetch.

Also fix test_setup_ps1_clone_uses_branch_tag which used Python
tuple syntax (assert "x", "y" in z) that always passes. Changed to
assert "x" in z and "y" in z.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix macOS DYLD trailing colon, install_lock no-op, and debug log

- binary_env macOS: use dedupe_existing_dirs instead of raw string
  concatenation. Eliminates trailing colon in DYLD_LIBRARY_PATH
  (which causes dyld to search CWD for libraries) and deduplicates
  when binary_path.parent == install_dir. Now consistent with the
  Linux and Windows branches.
- install_lock: when filelock is not installed, use os.O_CREAT|O_EXCL
  as a fallback exclusive file lock with timeout, instead of yielding
  with no locking. Prevents concurrent installs from corrupting each
  other's staging directories.
- setup.ps1: remove [DEBUG] log line that printed to every user on
  every Windows setup run.

* Add stale-lock detection and atomic clone-then-swap

install_lock fallback (no filelock): write PID to lock file and
check if the holder process is still alive on contention. Dead PIDs
(ProcessLookupError) and unreadable lock files trigger immediate
cleanup. Live processes owned by other users (PermissionError) are
correctly recognized as alive -- the lock is not removed.

setup.sh/setup.ps1 source-build: clone into a temporary directory
first, then swap into place only on success. If git clone fails,
the existing install is preserved instead of being deleted by the
premature rm -rf.

* Remove redundant upstream_tag != release_tag check

load_approved_release_checksums compared checksums.upstream_tag
against the Unsloth release_tag, which are different namespaces
(upstream ggml-org tag vs Unsloth published tag). This only worked
because both happened to be "b8508" by convention. Would break if
Unsloth ever uses a different release naming scheme.

The existing check at parse_approved_release_checksums (line 950)
already validates the release_tag field correctly.

* Fix lock TOCTOU race and build-in-temp-dir swap

install_lock fallback: add os.fsync(fd) after writing PID to ensure
the PID is visible to racing processes before they check. Treat
empty lock files (PID not yet written) as "wait and retry" instead
of stale, closing the window where two processes could both see an
empty file, both unlink it, and both acquire the lock.

setup.sh/setup.ps1 source-build: clone AND build in a temp directory
(LLAMA_CPP_DIR.build.$$). Only swap into the final LLAMA_CPP_DIR
after the build succeeds. If clone or cmake or build fails, the temp
dir is cleaned up and the existing working install is preserved.
Previously, rm -rf ran after clone but before build, destroying the
existing install even if the build later failed.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-03-25 05:42:43 -07:00

903 lines
34 KiB
Python

"""Tests for binary selection logic in install_llama_prebuilt.py.
Covers: normalize_compute_cap, normalize_compute_caps, parse_cuda_visible_devices,
supports_explicit_visible_device_matching, select_visible_gpu_rows,
compatible_linux_runtime_lines, pick_windows_cuda_runtime,
compatible_windows_runtime_lines, runtime_line_from_cuda_version,
apply_approved_hashes, linux_cuda_choice_from_release, windows_cuda_attempts,
resolve_upstream_asset_choice.
No GPU, no network, no torch required -- all I/O is monkeypatched.
"""
import importlib.util
import sys
from pathlib import Path
import pytest
PACKAGE_ROOT = Path(__file__).resolve().parents[3]
MODULE_PATH = PACKAGE_ROOT / "studio" / "install_llama_prebuilt.py"
SPEC = importlib.util.spec_from_file_location(
"studio_install_llama_prebuilt", MODULE_PATH
)
assert SPEC is not None and SPEC.loader is not None
INSTALL_LLAMA_PREBUILT = importlib.util.module_from_spec(SPEC)
sys.modules[SPEC.name] = INSTALL_LLAMA_PREBUILT
SPEC.loader.exec_module(INSTALL_LLAMA_PREBUILT)
HostInfo = INSTALL_LLAMA_PREBUILT.HostInfo
AssetChoice = INSTALL_LLAMA_PREBUILT.AssetChoice
PublishedLlamaArtifact = INSTALL_LLAMA_PREBUILT.PublishedLlamaArtifact
PublishedReleaseBundle = INSTALL_LLAMA_PREBUILT.PublishedReleaseBundle
ApprovedArtifactHash = INSTALL_LLAMA_PREBUILT.ApprovedArtifactHash
ApprovedReleaseChecksums = INSTALL_LLAMA_PREBUILT.ApprovedReleaseChecksums
PrebuiltFallback = INSTALL_LLAMA_PREBUILT.PrebuiltFallback
LinuxCudaSelection = INSTALL_LLAMA_PREBUILT.LinuxCudaSelection
UPSTREAM_REPO = INSTALL_LLAMA_PREBUILT.UPSTREAM_REPO
normalize_compute_cap = INSTALL_LLAMA_PREBUILT.normalize_compute_cap
normalize_compute_caps = INSTALL_LLAMA_PREBUILT.normalize_compute_caps
parse_cuda_visible_devices = INSTALL_LLAMA_PREBUILT.parse_cuda_visible_devices
supports_explicit_visible_device_matching = (
INSTALL_LLAMA_PREBUILT.supports_explicit_visible_device_matching
)
select_visible_gpu_rows = INSTALL_LLAMA_PREBUILT.select_visible_gpu_rows
compatible_linux_runtime_lines = INSTALL_LLAMA_PREBUILT.compatible_linux_runtime_lines
pick_windows_cuda_runtime = INSTALL_LLAMA_PREBUILT.pick_windows_cuda_runtime
compatible_windows_runtime_lines = (
INSTALL_LLAMA_PREBUILT.compatible_windows_runtime_lines
)
runtime_line_from_cuda_version = INSTALL_LLAMA_PREBUILT.runtime_line_from_cuda_version
apply_approved_hashes = INSTALL_LLAMA_PREBUILT.apply_approved_hashes
linux_cuda_choice_from_release = INSTALL_LLAMA_PREBUILT.linux_cuda_choice_from_release
windows_cuda_attempts = INSTALL_LLAMA_PREBUILT.windows_cuda_attempts
resolve_upstream_asset_choice = INSTALL_LLAMA_PREBUILT.resolve_upstream_asset_choice
# ---------------------------------------------------------------------------
# Helper factories
# ---------------------------------------------------------------------------
def make_host(**overrides):
system = overrides.pop("system", "Linux")
machine = overrides.pop("machine", "x86_64")
defaults = dict(
system = system,
machine = machine,
is_linux = system == "Linux",
is_windows = system == "Windows",
is_macos = system == "Darwin",
is_x86_64 = machine.lower() in {"x86_64", "amd64"},
is_arm64 = machine.lower() in {"arm64", "aarch64"},
nvidia_smi = "/usr/bin/nvidia-smi",
driver_cuda_version = (12, 8),
compute_caps = ["86"],
visible_cuda_devices = None,
has_physical_nvidia = True,
has_usable_nvidia = True,
)
defaults.update(overrides)
return HostInfo(**defaults)
def make_artifact(asset_name, **overrides):
defaults = dict(
asset_name = asset_name,
install_kind = "linux-cuda",
runtime_line = "cuda12",
coverage_class = "targeted",
supported_sms = ["75", "80", "86", "89", "90"],
min_sm = 75,
max_sm = 90,
bundle_profile = "cuda12-newer",
rank = 100,
)
defaults.update(overrides)
return PublishedLlamaArtifact(**defaults)
def make_release(artifacts, **overrides):
defaults = dict(
repo = "unslothai/llama.cpp",
release_tag = "v1.0",
upstream_tag = "b8508",
assets = {a.asset_name: f"https://example.com/{a.asset_name}" for a in artifacts},
manifest_asset_name = "llama-prebuilt-manifest.json",
artifacts = artifacts,
selection_log = [],
)
defaults.update(overrides)
return PublishedReleaseBundle(**defaults)
def make_checksums(asset_names):
return ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "v1.0",
upstream_tag = "b8508",
source_commit = None,
artifacts = {
name: ApprovedArtifactHash(
asset_name = name,
sha256 = "a" * 64,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
)
for name in asset_names
},
)
def mock_linux_runtime(monkeypatch, lines):
dirs = {line: ["/usr/lib/stub"] for line in lines}
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"detected_linux_runtime_lines",
lambda: (list(lines), dict(dirs)),
)
def mock_windows_runtime(monkeypatch, lines):
dirs = {line: ["C:\\Windows\\System32"] for line in lines}
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"detected_windows_runtime_lines",
lambda: (list(lines), dict(dirs)),
)
# ===========================================================================
# A. normalize_compute_cap
# ===========================================================================
class TestNormalizeComputeCap:
def test_dotted_86(self):
assert normalize_compute_cap("8.6") == "86"
def test_dotted_leading_zero(self):
assert normalize_compute_cap("07.05") == "75"
def test_already_normalized(self):
assert normalize_compute_cap("75") == "75"
def test_int_input(self):
assert normalize_compute_cap(86) == "86"
def test_empty_string(self):
assert normalize_compute_cap("") is None
def test_whitespace(self):
assert normalize_compute_cap(" ") is None
def test_non_numeric(self):
assert normalize_compute_cap("x.y") is None
def test_triple_part(self):
assert normalize_compute_cap("8.6.0") is None
def test_zero_minor(self):
assert normalize_compute_cap("9.0") == "90"
# ===========================================================================
# B. normalize_compute_caps
# ===========================================================================
class TestNormalizeComputeCaps:
def test_deduplication(self):
assert normalize_compute_caps(["8.6", "86", "8.6"]) == ["86"]
def test_numeric_sort(self):
assert normalize_compute_caps(["9.0", "7.5", "8.6"]) == ["75", "86", "90"]
def test_drops_invalid(self):
assert normalize_compute_caps(["8.6", "bad", "", "7.5"]) == ["75", "86"]
def test_empty_input(self):
assert normalize_compute_caps([]) == []
# ===========================================================================
# C. parse_cuda_visible_devices
# ===========================================================================
class TestParseCudaVisibleDevices:
def test_none(self):
assert parse_cuda_visible_devices(None) is None
def test_empty(self):
assert parse_cuda_visible_devices("") == []
def test_minus_one(self):
assert parse_cuda_visible_devices("-1") == []
def test_single(self):
assert parse_cuda_visible_devices("0") == ["0"]
def test_multi(self):
assert parse_cuda_visible_devices("0,1,2") == ["0", "1", "2"]
def test_whitespace_stripped(self):
assert parse_cuda_visible_devices(" 0 , 1 ") == ["0", "1"]
# ===========================================================================
# D. supports_explicit_visible_device_matching
# ===========================================================================
class TestSupportsExplicitVisibleDeviceMatching:
def test_all_digits(self):
assert supports_explicit_visible_device_matching(["0", "1", "2"]) is True
def test_gpu_prefix(self):
assert supports_explicit_visible_device_matching(["GPU-abc123"]) is True
def test_none(self):
assert supports_explicit_visible_device_matching(None) is False
def test_empty(self):
assert supports_explicit_visible_device_matching([]) is False
def test_mixed_invalid(self):
assert supports_explicit_visible_device_matching(["0", "MIG-device"]) is False
# ===========================================================================
# E. select_visible_gpu_rows
# ===========================================================================
class TestSelectVisibleGpuRows:
ROWS = [
("0", "GPU-aaa", "8.6"),
("1", "GPU-bbb", "7.5"),
("2", "GPU-ccc", "8.9"),
]
def test_none_returns_all(self):
assert select_visible_gpu_rows(self.ROWS, None) == list(self.ROWS)
def test_empty_returns_empty(self):
assert select_visible_gpu_rows(self.ROWS, []) == []
def test_filter_by_index(self):
result = select_visible_gpu_rows(self.ROWS, ["0", "2"])
assert result == [("0", "GPU-aaa", "8.6"), ("2", "GPU-ccc", "8.9")]
def test_filter_by_uuid_case_insensitive(self):
result = select_visible_gpu_rows(self.ROWS, ["gpu-bbb"])
assert result == [("1", "GPU-bbb", "7.5")]
def test_dedup_same_device(self):
result = select_visible_gpu_rows(self.ROWS, ["0", "0"])
assert result == [("0", "GPU-aaa", "8.6")]
def test_missing_token(self):
result = select_visible_gpu_rows(self.ROWS, ["99"])
assert result == []
# ===========================================================================
# F. compatible_linux_runtime_lines
# ===========================================================================
class TestCompatibleLinuxRuntimeLines:
def test_no_driver(self):
host = make_host(driver_cuda_version = None)
assert compatible_linux_runtime_lines(host) == []
def test_driver_11_8(self):
host = make_host(driver_cuda_version = (11, 8))
assert compatible_linux_runtime_lines(host) == []
def test_driver_12_4(self):
host = make_host(driver_cuda_version = (12, 4))
assert compatible_linux_runtime_lines(host) == ["cuda12"]
def test_driver_13_0(self):
host = make_host(driver_cuda_version = (13, 0))
assert compatible_linux_runtime_lines(host) == ["cuda13", "cuda12"]
# ===========================================================================
# G. pick_windows_cuda_runtime + compatible_windows_runtime_lines
# ===========================================================================
class TestPickWindowsCudaRuntime:
def test_no_driver(self):
host = make_host(driver_cuda_version = None)
assert pick_windows_cuda_runtime(host) is None
def test_below_threshold(self):
host = make_host(driver_cuda_version = (12, 3))
assert pick_windows_cuda_runtime(host) is None
def test_driver_12_4(self):
host = make_host(driver_cuda_version = (12, 4))
assert pick_windows_cuda_runtime(host) == "12.4"
def test_driver_13_1(self):
host = make_host(driver_cuda_version = (13, 1))
assert pick_windows_cuda_runtime(host) == "13.1"
class TestCompatibleWindowsRuntimeLines:
def test_no_driver(self):
host = make_host(driver_cuda_version = None)
assert compatible_windows_runtime_lines(host) == []
def test_driver_12_4(self):
host = make_host(driver_cuda_version = (12, 4))
assert compatible_windows_runtime_lines(host) == ["cuda12"]
def test_driver_13_1(self):
host = make_host(driver_cuda_version = (13, 1))
assert compatible_windows_runtime_lines(host) == ["cuda13", "cuda12"]
# ===========================================================================
# H. runtime_line_from_cuda_version
# ===========================================================================
class TestRuntimeLineFromCudaVersion:
def test_cuda_12(self):
assert runtime_line_from_cuda_version("12.6") == "cuda12"
def test_cuda_13(self):
assert runtime_line_from_cuda_version("13.0") == "cuda13"
def test_cuda_11(self):
assert runtime_line_from_cuda_version("11.8") is None
def test_none(self):
assert runtime_line_from_cuda_version(None) is None
def test_empty(self):
assert runtime_line_from_cuda_version("") is None
# ===========================================================================
# I. apply_approved_hashes
# ===========================================================================
class TestApplyApprovedHashes:
def _choice(self, name):
return AssetChoice(
repo = "test",
tag = "v1",
name = name,
url = f"https://x/{name}",
source_label = "test",
)
def test_both_approved(self):
c1, c2 = self._choice("a.tar.gz"), self._choice("b.tar.gz")
checksums = make_checksums(["a.tar.gz", "b.tar.gz"])
result = apply_approved_hashes([c1, c2], checksums)
assert len(result) == 2
assert all(c.expected_sha256 == "a" * 64 for c in result)
def test_one_approved(self):
c1, c2 = self._choice("a.tar.gz"), self._choice("missing.tar.gz")
checksums = make_checksums(["a.tar.gz"])
result = apply_approved_hashes([c1, c2], checksums)
assert len(result) == 1
assert result[0].name == "a.tar.gz"
def test_none_approved(self):
c1 = self._choice("missing.tar.gz")
checksums = make_checksums(["other.tar.gz"])
with pytest.raises(PrebuiltFallback, match = "approved checksum"):
apply_approved_hashes([c1], checksums)
def test_empty_input(self):
checksums = make_checksums(["a.tar.gz"])
with pytest.raises(PrebuiltFallback, match = "approved checksum"):
apply_approved_hashes([], checksums)
# ===========================================================================
# J. linux_cuda_choice_from_release -- core selection
# ===========================================================================
class TestLinuxCudaChoiceFromRelease:
# --- Runtime line resolution ---
def test_no_runtime_lines_detected(self, monkeypatch):
mock_linux_runtime(monkeypatch, [])
host = make_host(driver_cuda_version = (12, 8))
art = make_artifact("bundle-cuda12.tar.gz")
release = make_release([art])
assert linux_cuda_choice_from_release(host, release) is None
def test_detected_lines_incompatible_with_driver(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda13"])
host = make_host(driver_cuda_version = (12, 4))
art = make_artifact("bundle-cuda13.tar.gz", runtime_line = "cuda13")
release = make_release([art])
assert linux_cuda_choice_from_release(host, release) is None
def test_driver_13_only_cuda12_detected(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(driver_cuda_version = (13, 0))
art = make_artifact("bundle-cuda12.tar.gz", runtime_line = "cuda12")
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is not None
assert result.primary.runtime_line == "cuda12"
def test_preferred_runtime_line_reorders(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda13", "cuda12"])
host = make_host(driver_cuda_version = (13, 0))
art12 = make_artifact("bundle-cuda12.tar.gz", runtime_line = "cuda12")
art13 = make_artifact("bundle-cuda13.tar.gz", runtime_line = "cuda13")
release = make_release([art12, art13])
result = linux_cuda_choice_from_release(
host, release, preferred_runtime_line = "cuda12"
)
assert result is not None
assert result.primary.runtime_line == "cuda12"
def test_preferred_runtime_line_unavailable(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(driver_cuda_version = (12, 8))
art = make_artifact("bundle-cuda12.tar.gz", runtime_line = "cuda12")
release = make_release([art])
result = linux_cuda_choice_from_release(
host, release, preferred_runtime_line = "cuda13"
)
assert result is not None
assert result.primary.runtime_line == "cuda12"
log_entries = result.selection_log
assert any("unavailable_on_host" in entry for entry in log_entries)
# --- SM matching ---
def test_exact_sm_match(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["86"])
art = make_artifact(
"bundle.tar.gz", supported_sms = ["75", "86", "89"], min_sm = 75, max_sm = 89
)
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is not None
assert result.primary.name == "bundle.tar.gz"
def test_sm_not_in_supported_sms(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["86"])
art = make_artifact(
"bundle.tar.gz", supported_sms = ["75", "80", "89"], min_sm = 75, max_sm = 89
)
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is None
def test_sm_outside_min_range(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["50"])
art = make_artifact(
"bundle.tar.gz", supported_sms = ["50", "75", "86"], min_sm = 75, max_sm = 90
)
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is None
def test_sm_outside_max_range(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["100"])
art = make_artifact(
"bundle.tar.gz", supported_sms = ["100", "75", "86"], min_sm = 75, max_sm = 90
)
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is None
def test_very_old_sm(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["50"])
art = make_artifact("bundle.tar.gz", min_sm = 75, max_sm = 90)
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is None
def test_very_new_sm(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["100"])
art = make_artifact("bundle.tar.gz", min_sm = 75, max_sm = 90)
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is None
# --- Unknown compute caps (empty list) ---
def test_unknown_caps_only_portable(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = [])
targeted = make_artifact("targeted.tar.gz", coverage_class = "targeted")
portable = make_artifact("portable.tar.gz", coverage_class = "portable")
release = make_release([targeted, portable])
result = linux_cuda_choice_from_release(host, release)
assert result is not None
assert result.primary.name == "portable.tar.gz"
def test_unknown_caps_no_portable(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = [])
targeted = make_artifact("targeted.tar.gz", coverage_class = "targeted")
release = make_release([targeted])
result = linux_cuda_choice_from_release(host, release)
assert result is None
# --- Multi-GPU ---
def test_multi_gpu_all_covered(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["75", "89"])
art = make_artifact(
"bundle.tar.gz",
supported_sms = ["75", "80", "86", "89", "90"],
min_sm = 75,
max_sm = 90,
)
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is not None
def test_multi_gpu_not_all_covered(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["50", "89"])
art = make_artifact(
"bundle.tar.gz", supported_sms = ["75", "89"], min_sm = 75, max_sm = 89
)
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is None
# --- Artifact selection priority ---
def test_narrowest_sm_range_wins(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["86"])
wide = make_artifact(
"wide.tar.gz",
supported_sms = ["75", "86", "90"],
min_sm = 75,
max_sm = 90,
rank = 100,
)
narrow = make_artifact(
"narrow.tar.gz",
supported_sms = ["80", "86", "89"],
min_sm = 80,
max_sm = 89,
rank = 100,
)
release = make_release([wide, narrow])
result = linux_cuda_choice_from_release(host, release)
assert result is not None
assert result.primary.name == "narrow.tar.gz"
def test_range_tie_lower_rank_wins(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["86"])
high = make_artifact(
"high.tar.gz",
supported_sms = ["75", "86", "90"],
min_sm = 75,
max_sm = 90,
rank = 200,
)
low = make_artifact(
"low.tar.gz",
supported_sms = ["75", "86", "90"],
min_sm = 75,
max_sm = 90,
rank = 50,
)
release = make_release([high, low])
result = linux_cuda_choice_from_release(host, release)
assert result is not None
assert result.primary.name == "low.tar.gz"
def test_targeted_preferred_portable_fallback(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["86"])
targeted = make_artifact("targeted.tar.gz", coverage_class = "targeted", rank = 100)
portable = make_artifact("portable.tar.gz", coverage_class = "portable", rank = 100)
release = make_release([targeted, portable])
result = linux_cuda_choice_from_release(host, release)
assert result is not None
assert result.primary.name == "targeted.tar.gz"
assert len(result.attempts) == 2
assert result.attempts[1].name == "portable.tar.gz"
# --- Edge cases ---
def test_asset_missing_from_release_assets(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["86"])
art = make_artifact("bundle.tar.gz")
release = make_release([art], assets = {})
result = linux_cuda_choice_from_release(host, release)
assert result is None
def test_artifact_empty_supported_sms(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["86"])
art = make_artifact("bundle.tar.gz", supported_sms = [])
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is None
def test_artifact_missing_min_sm(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["86"])
art = make_artifact("bundle.tar.gz", min_sm = None, max_sm = 90)
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is None
def test_artifact_missing_max_sm(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["86"])
art = make_artifact("bundle.tar.gz", min_sm = 75, max_sm = None)
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is None
def test_no_linux_cuda_artifacts(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["86"])
art = make_artifact("bundle.tar.gz", install_kind = "windows-cuda")
release = make_release([art])
result = linux_cuda_choice_from_release(host, release)
assert result is None
def test_empty_artifacts_list(self, monkeypatch):
mock_linux_runtime(monkeypatch, ["cuda12"])
host = make_host(compute_caps = ["86"])
release = make_release([])
result = linux_cuda_choice_from_release(host, release)
assert result is None
# ===========================================================================
# K. windows_cuda_attempts
# ===========================================================================
class TestWindowsCudaAttempts:
TAG = "b8508"
def _upstream(self, *runtime_versions):
assets = {}
for rv in runtime_versions:
name = f"llama-{self.TAG}-bin-win-cuda-{rv}-x64.zip"
assets[name] = f"https://example.com/{name}"
return assets
def test_driver_12_4_no_dlls_fallback(self, monkeypatch):
mock_windows_runtime(monkeypatch, [])
host = make_host(system = "Windows", machine = "AMD64", driver_cuda_version = (12, 4))
assets = self._upstream("12.4")
result = windows_cuda_attempts(host, self.TAG, assets, None)
assert len(result) == 1
assert result[0].runtime_line == "cuda12"
def test_driver_13_1_both_dlls(self, monkeypatch):
mock_windows_runtime(monkeypatch, ["cuda13", "cuda12"])
host = make_host(system = "Windows", machine = "AMD64", driver_cuda_version = (13, 1))
assets = self._upstream("13.1", "12.4")
result = windows_cuda_attempts(host, self.TAG, assets, None)
assert len(result) == 2
assert result[0].runtime_line == "cuda13"
assert result[1].runtime_line == "cuda12"
def test_preferred_reorders(self, monkeypatch):
mock_windows_runtime(monkeypatch, ["cuda13", "cuda12"])
host = make_host(system = "Windows", machine = "AMD64", driver_cuda_version = (13, 1))
assets = self._upstream("13.1", "12.4")
result = windows_cuda_attempts(host, self.TAG, assets, "cuda12")
assert len(result) == 2
assert result[0].runtime_line == "cuda12"
def test_preferred_unavailable(self, monkeypatch):
mock_windows_runtime(monkeypatch, ["cuda12"])
host = make_host(system = "Windows", machine = "AMD64", driver_cuda_version = (12, 4))
assets = self._upstream("12.4")
result = windows_cuda_attempts(host, self.TAG, assets, "cuda13")
assert len(result) == 1
assert result[0].runtime_line == "cuda12"
def test_detected_incompatible_with_driver(self, monkeypatch):
mock_windows_runtime(monkeypatch, ["cuda13"])
host = make_host(system = "Windows", machine = "AMD64", driver_cuda_version = (12, 4))
assets = self._upstream("12.4")
result = windows_cuda_attempts(host, self.TAG, assets, None)
assert len(result) == 1
assert result[0].runtime_line == "cuda12"
def test_driver_too_old(self, monkeypatch):
mock_windows_runtime(monkeypatch, [])
host = make_host(system = "Windows", machine = "AMD64", driver_cuda_version = (11, 8))
assets = self._upstream("12.4")
result = windows_cuda_attempts(host, self.TAG, assets, None)
assert result == []
def test_asset_missing_from_upstream(self, monkeypatch):
mock_windows_runtime(monkeypatch, ["cuda12"])
host = make_host(system = "Windows", machine = "AMD64", driver_cuda_version = (12, 4))
result = windows_cuda_attempts(host, self.TAG, {}, None)
assert result == []
def test_both_assets_present(self, monkeypatch):
mock_windows_runtime(monkeypatch, ["cuda13", "cuda12"])
host = make_host(system = "Windows", machine = "AMD64", driver_cuda_version = (13, 1))
assets = self._upstream("13.1", "12.4")
result = windows_cuda_attempts(host, self.TAG, assets, None)
assert len(result) == 2
# ===========================================================================
# L. resolve_upstream_asset_choice -- platform routing
# ===========================================================================
class TestResolveUpstreamAssetChoice:
TAG = "b8508"
def _mock_github_assets(self, monkeypatch, assets):
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"github_release_assets",
lambda repo, tag: assets,
)
def test_linux_x86_64_cpu(self, monkeypatch):
name = f"llama-{self.TAG}-bin-ubuntu-x64.tar.gz"
self._mock_github_assets(monkeypatch, {name: f"https://x/{name}"})
host = make_host(
has_usable_nvidia = False, nvidia_smi = None, has_physical_nvidia = False
)
result = resolve_upstream_asset_choice(host, self.TAG)
assert result.install_kind == "linux-cpu"
assert result.name == name
def test_linux_cpu_missing(self, monkeypatch):
self._mock_github_assets(monkeypatch, {})
host = make_host(
has_usable_nvidia = False, nvidia_smi = None, has_physical_nvidia = False
)
with pytest.raises(PrebuiltFallback, match = "Linux CPU"):
resolve_upstream_asset_choice(host, self.TAG)
def test_windows_x86_64_cpu(self, monkeypatch):
name = f"llama-{self.TAG}-bin-win-cpu-x64.zip"
self._mock_github_assets(monkeypatch, {name: f"https://x/{name}"})
host = make_host(
system = "Windows",
machine = "AMD64",
has_usable_nvidia = False,
nvidia_smi = None,
has_physical_nvidia = False,
)
result = resolve_upstream_asset_choice(host, self.TAG)
assert result.install_kind == "windows-cpu"
assert result.name == name
def test_windows_cpu_missing(self, monkeypatch):
self._mock_github_assets(monkeypatch, {})
host = make_host(
system = "Windows",
machine = "AMD64",
has_usable_nvidia = False,
nvidia_smi = None,
has_physical_nvidia = False,
)
with pytest.raises(PrebuiltFallback, match = "Windows CPU"):
resolve_upstream_asset_choice(host, self.TAG)
def test_macos_arm64(self, monkeypatch):
name = f"llama-{self.TAG}-bin-macos-arm64.tar.gz"
self._mock_github_assets(monkeypatch, {name: f"https://x/{name}"})
host = make_host(
system = "Darwin",
machine = "arm64",
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
has_physical_nvidia = False,
has_usable_nvidia = False,
)
result = resolve_upstream_asset_choice(host, self.TAG)
assert result.install_kind == "macos-arm64"
assert result.name == name
def test_macos_arm64_missing(self, monkeypatch):
self._mock_github_assets(monkeypatch, {})
host = make_host(
system = "Darwin",
machine = "arm64",
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
has_physical_nvidia = False,
has_usable_nvidia = False,
)
with pytest.raises(PrebuiltFallback, match = "macOS arm64"):
resolve_upstream_asset_choice(host, self.TAG)
def test_macos_x86_64(self, monkeypatch):
name = f"llama-{self.TAG}-bin-macos-x64.tar.gz"
self._mock_github_assets(monkeypatch, {name: f"https://x/{name}"})
host = make_host(
system = "Darwin",
machine = "x86_64",
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
has_physical_nvidia = False,
has_usable_nvidia = False,
)
result = resolve_upstream_asset_choice(host, self.TAG)
assert result.install_kind == "macos-x64"
assert result.name == name
def test_linux_aarch64(self, monkeypatch):
self._mock_github_assets(monkeypatch, {})
host = make_host(
system = "Linux",
machine = "aarch64",
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
has_physical_nvidia = False,
has_usable_nvidia = False,
)
with pytest.raises(
PrebuiltFallback, match = "no prebuilt policy exists for Linux aarch64"
):
resolve_upstream_asset_choice(host, self.TAG)
def test_windows_usable_nvidia_delegates(self, monkeypatch):
cuda_name = f"llama-{self.TAG}-bin-win-cuda-12.4-x64.zip"
self._mock_github_assets(monkeypatch, {cuda_name: f"https://x/{cuda_name}"})
mock_windows_runtime(monkeypatch, ["cuda12"])
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_windows_cuda_choices",
lambda host, tag, assets: [
AssetChoice(
repo = UPSTREAM_REPO,
tag = tag,
name = cuda_name,
url = f"https://x/{cuda_name}",
source_label = "upstream",
install_kind = "windows-cuda",
runtime_line = "cuda12",
)
],
)
host = make_host(
system = "Windows",
machine = "AMD64",
driver_cuda_version = (12, 4),
has_usable_nvidia = True,
)
result = resolve_upstream_asset_choice(host, self.TAG)
assert result.install_kind == "windows-cuda"
assert result.name == cuda_name