unsloth/tests/studio/install/test_install_llama_prebuilt_logic.py
Souravrajvi0 7917c7828c
Installer: opt-in Vulkan llama.cpp backend (and fallback when no AMD card is HIP-supported) (#7373)
* feat(install): opt-in Vulkan llama.cpp backend and HIP gfx fallback (#7357)

Add UNSLOTH_LLAMA_BACKEND=vulkan and --llama-backend vulkan to force the
upstream Vulkan prebuilt on any host, persist llama_backend in the install
marker, and re-assert it during Studio updates.

On Windows AMD, auto-fallback to Vulkan when no detected gfx arch is in the
upstream win-hip-radeon GPU_TARGETS set (e.g. gfx803 / RX 480). Mixed setups
where at least one card is HIP-supported still default to HIP unless opted in.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(install): address Codex P2s on Vulkan gfx routing (#7357)

Honor ROCm family tokens (gfx110X), include fork-supported gfx1103, require
a known active gfx before auto-Vulkan, and base the HIP floor check on the
visible-device target instead of every physical GPU in hipinfo.

* Address Codex review: env namespace, physical-NVIDIA guard, test kwarg

- llama_backend_from_env: stop reading UNSLOTH_LLAMA_CPP_BACKEND. That is a
  separate pre-existing setup variable meaning auto/cpu; setup.sh/setup.ps1
  warn and ignore other values, so reading it here forced Vulkan behind that
  warning. Vulkan opt-in stays on UNSLOTH_LLAMA_BACKEND / UNSLOTH_FORCE_VULKAN.
- _should_auto_vulkan_for_amd_windows: gate on not has_physical_nvidia (not
  merely has_usable_nvidia). A CUDA-masked NVIDIA card keeps has_physical_nvidia
  while has_usable_nvidia goes False; Vulkan ignores CUDA_VISIBLE_DEVICES and
  could enumerate the reserved card. Mirrors the Intel auto path. Explicit
  opt-in still overrides.
- test fakes: validate_prebuilt_attempts/validate_prebuilt_choice gained a
  llama_backend kwarg; the four fake signatures in the fallback tests now
  accept it, clearing the TypeError that reddened Backend CI / Repo tests (CPU).

Tests: UNSLOTH_LLAMA_CPP_BACKEND=vulkan no longer triggers Vulkan; hidden
physical NVIDIA suppresses AMD auto-Vulkan while explicit opt-in overrides.

* Keep gfx1034 on the ROCm path (fork gfx103X bundle covers it)

The WINDOWS_HIP_PREBUILT_GFX_TARGETS allow-list omitted gfx1034, so
_route_to_vulkan_prebuilt downgraded RX 6500/6400-class hosts to the upstream
Vulkan prebuilt before published_rocm_choice_for_host could match the fork
windows-rocm gfx103X bundle (whose members include gfx1034). Add gfx1034 to the
allow-list and a regression test asserting it stays on the fork ROCm asset.

* Fix auto-Vulkan stealing fork windows-rocm gfx908/gfx90a hosts for PR #7373

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix Vulkan marker claiming a backend that was never installed for PR #7373

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the Vulkan backend routing comments for PR #7373

* Keep the visible-device-aware gfx when setup forwards --rocm-gfx

setup.ps1 resolves the gfx arch from its own probe, and that pick is not
fully visible-device aware: neither the hipinfo nor the amd-smi branch
reads CUDA_VISIBLE_DEVICES, and the amd-smi branch matches a bare integer
only, so a comma-separated HIP/ROCR mask such as 1,0 also falls back to
GPU 0. The resulting arch was then forwarded through --rocm-gfx and
replaced the arch detect_host() had already resolved for the
runtime-visible GPU.

On a mixed-AMD Windows host that flipped the auto-Vulkan decision: with
GPU 0 gfx1100 and a masked-in gfx1010, the forward reinstated gfx1100,
_should_auto_vulkan_for_amd_windows() saw a HIP-supported arch and the
HIP bundle was installed for a GPU that cannot run it.

Fold the forward in as a fill rather than a replacement: it still supplies
the arch on amd-smi-only, driver-only and name-inferred hosts where the
probe reports none, which is what --rocm-gfx exists for, but no longer
overwrites a successfully detected active arch. An explicit
UNSLOTH_ROCM_GFX_ARCH stays authoritative, since it is the documented
manual override for hosts whose arch the probes get wrong.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scope the Windows AMD Vulkan fallback per device and per repo

Three follow-ups on the auto-Vulkan routing for #7357.

Keep an explicit --rocm-gfx authoritative. The previous round stopped a
forwarded gfx from replacing an arch detect_host() had already resolved,
but --rocm-gfx is also the documented operator override for hosts whose
probe is wrong or stale, and both arrive as the same argv. Narrow the
advisory case to the two shapes setup can actually be describing: an arch
the probe saw on this host (setup picked a different physical GPU of the
same box), or a family label such as gfx110X, which is a bundle name the
update path derives from the marker asset rather than a real GPU arch.
Any other value is an override for an arch no probe reported and stays
authoritative. Keeping family labels advisory also preserves the rule that
an in-generation-but-unbuilt arch (gfx1033) is never upgraded into the
gfx103X bundle.

Do not auto-route to Vulkan from a HIP-only device mask. HIP_VISIBLE_DEVICES,
ROCR_VISIBLE_DEVICES and CUDA_VISIBLE_DEVICES select the active arch, but the
Vulkan runtime honours none of them: it enumerates through
GGML_VK_VISIBLE_DEVICES and Vulkan ordinals in
LlamaCppBackend._get_gpu_free_memory_vulkan. Masking down to a below-floor
card therefore used to install a backend that could still enumerate the
HIP-capable card the user deliberately hid, possibly one reserved for another
workload. Require every physical AMD gfx to be below the floor, matching the
has_physical_nvidia gate right above it. So the per-GPU list survives to that
check, a forward that agrees with the probe no longer collapses
rocm_gfx_targets to a single entry.

Make the HIP support predicate repository-specific. The floor constant is a
union of ggml-org's windows-hip gpu_targets and the fork's windows-rocm
bundles, so it only answers "is this arch served" for the fork. With
--published-repo ggml-org/llama.cpp, direct_upstream_release_plan() offers
win-hip-radeon then CPU and never Vulkan, so the four fork-only archs
(gfx908, gfx90a, gfx1034, gfx1103) were declared supported and fell through
to CPU instead of the Vulkan bundle that would actually run. Add
UPSTREAM_WINDOWS_HIP_GFX_TARGETS and select the set from the planned repo.

* Keep probe-confirmed AMD GPUs in the physical list when a gfx is forwarded

rocm_gfx_targets is the physical inventory _should_auto_vulkan_for_amd_windows()
reads, so a forwarded --rocm-gfx that the probe never reported was deleting cards
the probe had confirmed. On a mixed Windows AMD box whose active device is masked
down to a below-floor card, a stale UNSLOTH_ROCM_GFX_ARCH or a name-inferred arch
for the other GPU collapsed the list to that one arch, the floor check concluded no
AMD GPU on the host reaches the Windows HIP prebuilt, and the install auto-fell back
to Vulkan, which honours no HIP mask and would enumerate the reserved HIP-capable
card. Add the forwarded arch to the list instead of replacing it: it selects the HIP
target, it does not redefine what hardware is present.

An empty probe still yields a single-entry list, so the driver-only Windows AMD host
the forward exists for keeps its automatic Vulkan fallback, and an explicit
--llama-backend vulkan is unaffected.

* Do not auto-fall back to Vulkan when a HIP device mask filtered the probe

hipinfo is itself a HIP application, and AMD documents HIP_VISIBLE_DEVICES as
"only devices whose index is present in the sequence are visible to HIP", with
that spelling recommended on Windows. Under a mask the Windows probe therefore
enumerates the visible devices, so rocm_gfx_targets is what survived the mask
rather than the physical inventory the auto-Vulkan floor check assumes. A
masked-out gfx1100 next to a visible gfx803 made the check conclude that no AMD
GPU on the box reaches the Windows HIP prebuilt and route the install to Vulkan,
which honours none of these masks and would enumerate the reserved card.

Decline to guess when a mask is set: the physical inventory is unknowable from a
masked probe, so keep the HIP / fork / source path. This only ever turns the
automatic fallback off, never on. The driver-only single-GPU host the fallback
exists for sets no mask, an all-hiding "" / -1 mask is still handled as no active
target rather than a partial view, and an explicit --llama-backend vulkan or
UNSLOTH_LLAMA_BACKEND=vulkan is unaffected.

Reading the physical inventory through an unmasked re-probe would also correct
_pick_rocm_gfx_target, which indexes the token list by the mask value and so
already assumes an unmasked probe. That is pre-existing behaviour on main and is
left alone here.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Treat an all-hiding HIP device mask as suppressing the Vulkan fallback too

The mask guard exempted an empty or -1 value on the grounds that the probe reports
no active target under it, but that only holds for the probe: a forwarded
--rocm-gfx still reconstructs an active arch, and setup infers that arch from the
display-adapter name, which no HIP mask touches. A user who hid every AMD GPU from
HIP could therefore still be auto-routed to Vulkan, which honours none of these
masks and would then use all of them. That is the strongest form of the hazard the
guard exists for, not an exemption from it.

Presence of any of the three variables is now the whole test, which also removes
the value parsing. An explicit --llama-backend vulkan or UNSLOTH_LLAMA_BACKEND is
still unaffected.

* Grant the fork-only Windows HIP coverage to the fork, not to every mirror

The floor set is a union of the fork's windows-rocm bundles and only the fork is
planned from its manifest: resolve_simple_install_release_plans() compares
== DEFAULT_PUBLISHED_REPO and sends every other --published-repo through
direct_upstream_release_plan(), whose AMD branch offers win-hip-radeon then CPU
and never Vulkan. Exempting only the exact ggml-org spelling therefore told a
mirror carrying upstream-standard assets that fork-only archs such as gfx1034,
gfx1103 and gfx908 were HIP-served, landing them on HIP or CPU instead of the
Vulkan bundle that would actually run. Gate on the fork instead.

Matching the dispatch exactly, spelling included, also fixes a differently cased
repo: that really does take the upstream path, so it must be answered with
upstream coverage rather than the fork superset. An empty repo still defaults to
the fork, as the resolver does.

* Derive the Windows HIP gfx floor guard from the published manifest

The guard compared WINDOWS_HIP_PREBUILT_GFX_TARGETS against a second hardcoded
tuple in the same test file, so a windows-rocm arch newly published by the fork
passed both. Affected hosts would then be routed off the hash-approved fork ROCm
bundle onto an unhashed upstream Vulkan build with nothing failing.

Read the fork's llama-prebuilt-manifest.json through the installer's own
resolver instead, and assert the floor, the family labels, and the routing
tuple all still cover what it publishes. The manifest ships only as a release
asset, so an unreachable release skips with an explicit reason rather than
flaking. Both literals match the manifest as published today.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Compress the Vulkan backend routing comments and docstrings for PR #7373

* Correct the family-label rationale in the Windows HIP coverage check

The comment justified serving gfx103X / gfx110X against any repository by
claiming upstream's windows-hip targets build every member of those families.
The fork manifest maps gfx103X to gfx1030..1032 plus gfx1034 and gfx110X to
gfx1100..1102 plus gfx1103, and UPSTREAM_WINDOWS_HIP_GFX_TARGETS carries
neither gfx1034 nor gfx1103, so the stated reason is wrong even though the
answer is right.

State the real reason instead. A family label is a bundle name, not an arch,
so the concrete GPU is unknown at this point; answering unsupported to cover
the two uncovered members would move gfx1030..1032 and gfx1100..1102 off a
working HIP build onto Vulkan for a card the label cannot identify. Those two
archs still reach Vulkan through the concrete-arch branch below, which does
answer per repository.

Comment only. No behaviour change: the 5850-combination override sweep still
reports 0 rocm_gfx_target changes, 0 auto_vulkan False to True flips and 680
True to False flips all backed by a probe-confirmed HIP GPU, and both the
feature and override profile matrices are byte-identical.

* Pin that a deliberate CPU install outranks Vulkan for PR #7373

UNSLOTH_LLAMA_CPP_BACKEND (setup.sh / setup.ps1, "auto" or "cpu") and
UNSLOTH_LLAMA_BACKEND (this module, a backend name) are separate variables at
separate layers, and both accept "cpu". setup translates its own =cpu into
--force-cpu, which is what pins the CPU-only bundle on a GPU host and keeps
Intel iGPU Vulkan crashes away (#7213), so no trigger this PR adds may
outrank it.

_route_to_vulkan_prebuilt already gets this right, since force_cpu
short-circuits ahead of the forced, auto-Intel and auto-no-HIP triggers.
Cover it so it stays that way: the matrix runs [Linux, Windows, macOS] x
[NVIDIA, AMD, Intel, CPU only] x [unset, vulkan, hip, rocm, cpu] with the
legacy UNSLOTH_FORCE_VULKAN set as well, and asserts the published bundle
survives every one. WSL presents as Linux to this resolver, so it rides the
Linux row.

Also assert the guard is not vacuous: the same host still takes Vulkan once
the CPU pin is gone, so the matrix cannot pass on a resolver that had simply
stopped routing to Vulkan.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-27 06:57:19 -07:00

3472 lines
120 KiB
Python

import errno
import importlib.util
import io
import json
import os
import shutil
import subprocess
import sys
import tarfile
import zipfile
from pathlib import Path
import pytest
PACKAGE_ROOT = Path(__file__).resolve().parents[3]
MODULE_PATH = PACKAGE_ROOT / "studio" / "install_llama_prebuilt.py"
SPEC = importlib.util.spec_from_file_location("studio_install_llama_prebuilt", MODULE_PATH)
assert SPEC is not None and SPEC.loader is not None
INSTALL_LLAMA_PREBUILT = importlib.util.module_from_spec(SPEC)
sys.modules[SPEC.name] = INSTALL_LLAMA_PREBUILT
SPEC.loader.exec_module(INSTALL_LLAMA_PREBUILT)
PrebuiltFallback = INSTALL_LLAMA_PREBUILT.PrebuiltFallback
binary_env = INSTALL_LLAMA_PREBUILT.binary_env
is_secret_env_name = INSTALL_LLAMA_PREBUILT.is_secret_env_name
scrub_env = INSTALL_LLAMA_PREBUILT.scrub_env
isolated_runtime_home = INSTALL_LLAMA_PREBUILT.isolated_runtime_home
HostInfo = INSTALL_LLAMA_PREBUILT.HostInfo
AssetChoice = INSTALL_LLAMA_PREBUILT.AssetChoice
ApprovedArtifactHash = INSTALL_LLAMA_PREBUILT.ApprovedArtifactHash
ApprovedReleaseChecksums = INSTALL_LLAMA_PREBUILT.ApprovedReleaseChecksums
hydrate_source_tree = INSTALL_LLAMA_PREBUILT.hydrate_source_tree
remove_agent_instruction_files = INSTALL_LLAMA_PREBUILT.remove_agent_instruction_files
validate_prebuilt_choice = INSTALL_LLAMA_PREBUILT.validate_prebuilt_choice
activate_install_tree = INSTALL_LLAMA_PREBUILT.activate_install_tree
activate_staged_dir = INSTALL_LLAMA_PREBUILT.activate_staged_dir
create_install_staging_dir = INSTALL_LLAMA_PREBUILT.create_install_staging_dir
sha256_file = INSTALL_LLAMA_PREBUILT.sha256_file
source_archive_logical_name = INSTALL_LLAMA_PREBUILT.source_archive_logical_name
install_prebuilt = INSTALL_LLAMA_PREBUILT.install_prebuilt
write_prebuilt_metadata = INSTALL_LLAMA_PREBUILT.write_prebuilt_metadata
existing_install_matches_plan = INSTALL_LLAMA_PREBUILT.existing_install_matches_plan
existing_install_matches_choice = INSTALL_LLAMA_PREBUILT.existing_install_matches_choice
ensure_diffusion_visual_server = INSTALL_LLAMA_PREBUILT.ensure_diffusion_visual_server
def linux_host() -> HostInfo:
return HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
def approved_release_checksums_for_asset(asset_name: str, sha256: str) -> ApprovedReleaseChecksums:
return ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "b9334",
upstream_tag = "b9334",
artifacts = {
asset_name: ApprovedArtifactHash(
asset_name = asset_name,
sha256 = sha256,
repo = "unslothai/llama.cpp",
kind = "diffusion-visual-server",
)
},
)
def approved_checksums_for(
upstream_tag: str, *, source_archive: Path, bundle_archive: Path, bundle_name: str
) -> ApprovedReleaseChecksums:
return ApprovedReleaseChecksums(
repo = "local",
release_tag = upstream_tag,
upstream_tag = upstream_tag,
source_commit = None,
artifacts = {
source_archive_logical_name(upstream_tag): ApprovedArtifactHash(
asset_name = source_archive_logical_name(upstream_tag),
sha256 = sha256_file(source_archive),
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
bundle_name: ApprovedArtifactHash(
asset_name = bundle_name,
sha256 = sha256_file(bundle_archive),
repo = "local",
kind = "local-test-bundle",
),
},
)
# The extract_archive guard tests (safe symlink chain / hardlink, absolute or
# escaping or unresolved symlink targets, zip symlink entries) moved verbatim
# to tests/studio/install/test_prebuilt_core.py: extract_archive is the shared
# prebuilt_core implementation, re-exported by this installer.
def test_remove_agent_instruction_files_does_not_follow_links(tmp_path: Path):
managed = tmp_path / "managed"
nested = managed / "nested"
external = tmp_path / "external"
nested.mkdir(parents = True)
external.mkdir()
(managed / "AGENTS.md").write_text("managed root", encoding = "utf-8")
(nested / "AGENTS.md").write_text("managed nested", encoding = "utf-8")
(managed / "CLAUDE.md").write_text("managed Claude root", encoding = "utf-8")
(nested / "CLAUDE.md").write_text("managed Claude nested", encoding = "utf-8")
(external / "AGENTS.md").write_text("user owned", encoding = "utf-8")
(external / "CLAUDE.md").write_text("user-owned Claude", encoding = "utf-8")
try:
(managed / "external-link").symlink_to(external, target_is_directory = True)
linked_root = tmp_path / "linked-root"
linked_root.symlink_to(external, target_is_directory = True)
except OSError as exc:
pytest.skip(f"directory symlinks unavailable: {exc}")
assert remove_agent_instruction_files(managed) == 4
assert not list(managed.rglob("AGENTS.md"))
assert not list(managed.rglob("CLAUDE.md"))
assert (external / "AGENTS.md").read_text(encoding = "utf-8") == "user owned"
assert (external / "CLAUDE.md").read_text(encoding = "utf-8") == "user-owned Claude"
assert remove_agent_instruction_files(linked_root) == 0
assert (external / "AGENTS.md").exists()
assert (external / "CLAUDE.md").exists()
@pytest.mark.skipif(os.name != "nt", reason = "Windows junction behavior")
def test_remove_agent_instruction_files_does_not_follow_windows_junctions(tmp_path: Path):
managed = tmp_path / "managed"
external = tmp_path / "external"
managed.mkdir()
external.mkdir()
(external / "AGENTS.md").write_text("user owned", encoding = "utf-8")
(external / "CLAUDE.md").write_text("user-owned Claude", encoding = "utf-8")
nested_junction = managed / "external-junction"
root_junction = tmp_path / "linked-root"
for junction in (nested_junction, root_junction):
result = subprocess.run(
["cmd", "/d", "/c", "mklink", "/J", str(junction), str(external)],
capture_output = True,
text = True,
check = False,
)
if result.returncode != 0:
pytest.skip(f"directory junctions unavailable: {result.stderr or result.stdout}")
assert remove_agent_instruction_files(managed) == 0
assert remove_agent_instruction_files(root_junction) == 0
assert (external / "AGENTS.md").read_text(encoding = "utf-8") == "user owned"
assert (external / "CLAUDE.md").read_text(encoding = "utf-8") == "user-owned Claude"
def test_remove_agent_instruction_files_prunes_linklike_directories(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
managed = tmp_path / "managed"
simulated_junction = managed / "simulated-junction"
simulated_junction.mkdir(parents = True)
agents = simulated_junction / "AGENTS.md"
claude = simulated_junction / "CLAUDE.md"
agents.write_text("external instructions", encoding = "utf-8")
claude.write_text("external Claude instructions", encoding = "utf-8")
real_is_link_or_junction = INSTALL_LLAMA_PREBUILT._is_link_or_junction
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"_is_link_or_junction",
lambda path: path == simulated_junction or real_is_link_or_junction(path),
)
assert remove_agent_instruction_files(managed) == 0
assert agents.exists()
assert claude.exists()
def test_remove_agent_instruction_files_continues_after_unlink_error(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
):
managed = tmp_path / "managed"
managed.mkdir()
blocked = managed / "AGENTS.md"
removable = managed / "CLAUDE.md"
blocked.write_text("blocked", encoding = "utf-8")
removable.write_text("remove me", encoding = "utf-8")
real_unlink = Path.unlink
def selective_unlink(path: Path, *args, **kwargs):
if path == blocked:
raise PermissionError(errno.EACCES, "Access is denied", str(path))
return real_unlink(path, *args, **kwargs)
monkeypatch.setattr(Path, "unlink", selective_unlink)
assert remove_agent_instruction_files(managed) == 1
assert blocked.exists()
assert not removable.exists()
captured = capsys.readouterr()
assert "could not remove contributor-only instruction" in captured.out + captured.err
def test_main_resolves_linked_install_path_and_preserves_cleanup_root(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
target = tmp_path / "target"
linked_root = tmp_path / "linked-root"
target.mkdir()
try:
linked_root.symlink_to(target, target_is_directory = True)
except OSError as exc:
pytest.skip(f"directory symlinks unavailable: {exc}")
received = {}
monkeypatch.setattr(
sys,
"argv",
["install_llama_prebuilt.py", "--install-dir", str(linked_root)],
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"install_prebuilt",
lambda **kwargs: received.update(kwargs),
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "_LOG_TO_STDOUT", False)
assert INSTALL_LLAMA_PREBUILT.main() == 0
assert received["install_dir"] == target.resolve()
assert received["instruction_cleanup_root"] == linked_root.absolute()
assert received["instruction_cleanup_root"].is_symlink()
def test_install_prebuilt_uses_explicit_instruction_cleanup_root(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "target"
linked_root = tmp_path / "linked-root"
install_dir.mkdir()
(install_dir / "UNSLOTH_PREBUILT_INFO.json").write_text("{}", encoding = "utf-8")
try:
linked_root.symlink_to(install_dir, target_is_directory = True)
except OSError as exc:
pytest.skip(f"directory symlinks unavailable: {exc}")
cleanup_roots = []
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", linux_host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"remove_agent_instruction_files",
lambda root: cleanup_roots.append(root) or 0,
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_simple_install_release_plans",
lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("stop after cleanup")),
)
with pytest.raises(RuntimeError, match = "stop after cleanup"):
install_prebuilt(
install_dir.resolve(),
"latest",
"unslothai/llama.cpp",
"",
instruction_cleanup_root = linked_root.absolute(),
)
assert cleanup_roots == [linked_root.absolute()]
def test_hydrate_source_tree_extracts_upstream_archive_contents(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
upstream_tag = "b9999"
archive_path = tmp_path / "llama.cpp-source.tar.gz"
with tarfile.open(archive_path, "w:gz") as archive:
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/CMakeLists.txt",
b"cmake_minimum_required(VERSION 3.14)\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/convert_hf_to_gguf.py",
b"#!/usr/bin/env python3\nimport gguf\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/gguf-py/gguf/__init__.py",
b"__all__ = []\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/AGENTS.md",
b"upstream contributor instructions\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/examples/AGENTS.md",
b"nested contributor instructions\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/CLAUDE.md",
b"Claude contributor instructions\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/examples/CLAUDE.md",
b"nested Claude contributor instructions\n",
)
source_urls = set(INSTALL_LLAMA_PREBUILT.upstream_source_archive_urls(upstream_tag))
def fake_download_file(url: str, destination: Path) -> None:
assert url in source_urls
destination.write_bytes(archive_path.read_bytes())
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
install_dir = tmp_path / "install"
work_dir = tmp_path / "work"
work_dir.mkdir()
hydrate_source_tree(
upstream_tag, install_dir, work_dir, expected_sha256 = sha256_file(archive_path)
)
assert (install_dir / "CMakeLists.txt").exists()
assert (install_dir / "convert_hf_to_gguf.py").exists()
assert (install_dir / "gguf-py" / "gguf" / "__init__.py").exists()
assert not (install_dir / f"llama.cpp-{upstream_tag}").exists()
assert not list(install_dir.rglob("AGENTS.md"))
assert not list(install_dir.rglob("CLAUDE.md"))
def test_release_asset_download_url():
fn = INSTALL_LLAMA_PREBUILT.release_asset_download_url
assert fn(
"unslothai/llama.cpp", "b9000-mix-abc1234", "llama.cpp-source-commit-deadbeef.tar.gz"
) == (
"https://github.com/unslothai/llama.cpp/releases/download/"
"b9000-mix-abc1234/llama.cpp-source-commit-deadbeef.tar.gz"
)
# Any missing component -> None (no asset url, caller falls back to codeload).
assert fn(None, "b9000", "x.tar.gz") is None
assert fn("unslothai/llama.cpp", None, "x.tar.gz") is None
assert fn("unslothai/llama.cpp", "b9000", None) is None
def _mk_source_tarball(path: Path, tag: str) -> None:
with tarfile.open(path, "w:gz") as archive:
add_bytes_to_tar(
archive, f"llama.cpp-{tag}/CMakeLists.txt", b"cmake_minimum_required(VERSION 3.14)\n"
)
add_bytes_to_tar(
archive,
f"llama.cpp-{tag}/convert_hf_to_gguf.py",
b"#!/usr/bin/env python3\nimport gguf\n",
)
add_bytes_to_tar(archive, f"llama.cpp-{tag}/gguf-py/gguf/__init__.py", b"__all__ = []\n")
def test_hydrate_source_tree_prefers_release_asset_for_mix(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
# A mix build's merge commit 404s on codeload, so hydrate must fetch the release asset.
commit = "a" * 40
archive_path = tmp_path / "merged-source.tar.gz"
_mk_source_tarball(archive_path, f"b9000-mix-{commit[:7]}")
asset_url = INSTALL_LLAMA_PREBUILT.release_asset_download_url(
"unslothai/llama.cpp", "b9000-mix-abc1234", f"llama.cpp-source-commit-{commit}.tar.gz"
)
codeload_urls = set(
INSTALL_LLAMA_PREBUILT.commit_source_archive_urls("unslothai/llama.cpp", commit)
)
seen = []
def fake_download_file(url: str, destination: Path) -> None:
seen.append(url)
if url in codeload_urls:
raise AssertionError("codeload was hit even though the release asset was available")
assert url == asset_url
destination.write_bytes(archive_path.read_bytes())
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
install_dir = tmp_path / "install"
work_dir = tmp_path / "work"
work_dir.mkdir()
hydrate_source_tree(
commit,
install_dir,
work_dir,
source_repo = "unslothai/llama.cpp",
expected_sha256 = sha256_file(archive_path),
exact_source = True,
asset_url = asset_url,
)
assert seen == [asset_url]
assert (install_dir / "CMakeLists.txt").exists()
assert (install_dir / "convert_hf_to_gguf.py").exists()
def test_hydrate_source_tree_falls_back_to_codeload_when_asset_missing(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
# If the release asset 404s, fall back to codeload/archive (vanilla path).
commit = "b" * 40
archive_path = tmp_path / "vanilla-source.tar.gz"
_mk_source_tarball(archive_path, f"commit-{commit[:7]}")
asset_url = INSTALL_LLAMA_PREBUILT.release_asset_download_url(
"unslothai/llama.cpp", "b9000", f"llama.cpp-source-commit-{commit}.tar.gz"
)
codeload_urls = INSTALL_LLAMA_PREBUILT.commit_source_archive_urls("unslothai/llama.cpp", commit)
def fake_download_file(url: str, destination: Path) -> None:
if url == asset_url:
raise RuntimeError("404 Not Found")
assert url in codeload_urls
destination.write_bytes(archive_path.read_bytes())
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
install_dir = tmp_path / "install"
work_dir = tmp_path / "work"
work_dir.mkdir()
hydrate_source_tree(
commit,
install_dir,
work_dir,
source_repo = "unslothai/llama.cpp",
expected_sha256 = sha256_file(archive_path),
exact_source = True,
asset_url = asset_url,
)
assert (install_dir / "CMakeLists.txt").exists()
def test_validate_prebuilt_choice_creates_repo_shaped_linux_install(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
upstream_tag = "b9998"
bundle_name = "app-b9998-linux-x64-cuda13-newer.tar.gz"
source_archive = tmp_path / "source.tar.gz"
bundle_archive = tmp_path / "bundle.tar.gz"
with tarfile.open(source_archive, "w:gz") as archive:
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/CMakeLists.txt",
b"cmake_minimum_required(VERSION 3.14)\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/convert_hf_to_gguf.py",
b"#!/usr/bin/env python3\nimport gguf\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/gguf-py/gguf/__init__.py",
b"__all__ = []\n",
)
with tarfile.open(bundle_archive, "w:gz") as archive:
add_bytes_to_tar(archive, "llama-server", b"#!/bin/sh\nexit 0\n", mode = 0o755)
add_bytes_to_tar(archive, "llama-quantize", b"#!/bin/sh\nexit 0\n", mode = 0o755)
add_bytes_to_tar(archive, "libllama.so.0.0.1", b"libllama")
add_symlink_to_tar(archive, "libllama.so.0", "libllama.so.0.0.1")
add_symlink_to_tar(archive, "libllama.so", "libllama.so.0")
add_bytes_to_tar(archive, "libggml.so.0.9.8", b"libggml")
add_symlink_to_tar(archive, "libggml.so.0", "libggml.so.0.9.8")
add_symlink_to_tar(archive, "libggml.so", "libggml.so.0")
add_bytes_to_tar(archive, "libggml-base.so.0.9.8", b"libggml-base")
add_symlink_to_tar(archive, "libggml-base.so.0", "libggml-base.so.0.9.8")
add_symlink_to_tar(archive, "libggml-base.so", "libggml-base.so.0")
add_bytes_to_tar(archive, "libggml-cpu-x64.so.0.9.8", b"libggml-cpu")
add_symlink_to_tar(archive, "libggml-cpu-x64.so.0", "libggml-cpu-x64.so.0.9.8")
add_symlink_to_tar(archive, "libggml-cpu-x64.so", "libggml-cpu-x64.so.0")
add_bytes_to_tar(archive, "libmtmd.so.0.0.1", b"libmtmd")
add_symlink_to_tar(archive, "libmtmd.so.0", "libmtmd.so.0.0.1")
add_symlink_to_tar(archive, "libmtmd.so", "libmtmd.so.0")
add_bytes_to_tar(archive, "BUILD_INFO.txt", b"bundle metadata\n")
add_bytes_to_tar(archive, "THIRD_PARTY_LICENSES.txt", b"licenses\n")
source_urls = set(INSTALL_LLAMA_PREBUILT.upstream_source_archive_urls(upstream_tag))
def fake_download_file(url: str, destination: Path) -> None:
if url in source_urls:
destination.write_bytes(source_archive.read_bytes())
return
if url == "file://bundle":
destination.write_bytes(bundle_archive.read_bytes())
return
raise AssertionError(f"unexpected download url: {url}")
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_bytes",
lambda url, **_: b"#!/usr/bin/env python3\nimport gguf\n",
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"preflight_linux_installed_binaries",
lambda *args, **kwargs: None,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "validate_quantize", lambda *args, **kwargs: None)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "validate_server", lambda *args, **kwargs: None)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "local",
tag = upstream_tag,
name = bundle_name,
url = "file://bundle",
source_label = "local",
is_ready_bundle = True,
install_kind = "linux-cuda",
bundle_profile = "cuda13-newer",
runtime_line = "cuda13",
expected_sha256 = sha256_file(bundle_archive),
)
install_dir = tmp_path / "install"
work_dir = tmp_path / "work"
work_dir.mkdir()
probe_path = tmp_path / "stories260K.gguf"
quantized_path = tmp_path / "stories260K-q4.gguf"
validate_prebuilt_choice(
choice,
host,
install_dir,
work_dir,
probe_path,
requested_tag = upstream_tag,
llama_tag = upstream_tag,
release_tag = upstream_tag,
approved_checksums = approved_checksums_for(
upstream_tag,
source_archive = source_archive,
bundle_archive = bundle_archive,
bundle_name = bundle_name,
),
prebuilt_fallback_used = False,
quantized_path = quantized_path,
)
assert (install_dir / "gguf-py" / "gguf" / "__init__.py").exists()
assert (install_dir / "convert_hf_to_gguf.py").exists()
assert (install_dir / "build" / "bin" / "llama-server").exists()
assert (install_dir / "build" / "bin" / "llama-quantize").exists()
assert (install_dir / "build" / "bin" / "libllama.so").exists()
assert (install_dir / "llama-server").exists()
assert (install_dir / "llama-quantize").exists()
assert (install_dir / "UNSLOTH_PREBUILT_INFO.json").exists()
assert (install_dir / "BUILD_INFO.txt").exists()
def test_validate_prebuilt_choice_creates_repo_shaped_windows_install(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
upstream_tag = "b9997"
bundle_name = "app-b9997-windows-x64-cpu.zip"
source_archive = tmp_path / "source.tar.gz"
bundle_archive = tmp_path / "bundle.zip"
with tarfile.open(source_archive, "w:gz") as archive:
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/CMakeLists.txt",
b"cmake_minimum_required(VERSION 3.14)\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/convert_hf_to_gguf.py",
b"#!/usr/bin/env python3\nimport gguf\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/gguf-py/gguf/__init__.py",
b"__all__ = []\n",
)
with zipfile.ZipFile(bundle_archive, "w") as archive:
archive.writestr("llama-server.exe", b"MZ")
archive.writestr("llama-quantize.exe", b"MZ")
archive.writestr("llama.dll", b"DLL")
archive.writestr("BUILD_INFO.txt", b"bundle metadata\n")
source_urls = set(INSTALL_LLAMA_PREBUILT.upstream_source_archive_urls(upstream_tag))
def fake_download_file(url: str, destination: Path) -> None:
if url in source_urls:
destination.write_bytes(source_archive.read_bytes())
return
if url == "file://bundle.zip":
destination.write_bytes(bundle_archive.read_bytes())
return
raise AssertionError(f"unexpected download url: {url}")
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_bytes",
lambda url, **_: b"#!/usr/bin/env python3\nimport gguf\n",
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"preflight_linux_installed_binaries",
lambda *args, **kwargs: None,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "validate_quantize", lambda *args, **kwargs: None)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "validate_server", lambda *args, **kwargs: None)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "local",
tag = upstream_tag,
name = bundle_name,
url = "file://bundle.zip",
source_label = "local",
is_ready_bundle = True,
install_kind = "windows-cpu",
expected_sha256 = sha256_file(bundle_archive),
)
install_dir = tmp_path / "install"
work_dir = tmp_path / "work"
work_dir.mkdir()
probe_path = tmp_path / "stories260K.gguf"
quantized_path = tmp_path / "stories260K-q4.gguf"
validate_prebuilt_choice(
choice,
host,
install_dir,
work_dir,
probe_path,
requested_tag = upstream_tag,
llama_tag = upstream_tag,
release_tag = upstream_tag,
approved_checksums = approved_checksums_for(
upstream_tag,
source_archive = source_archive,
bundle_archive = bundle_archive,
bundle_name = bundle_name,
),
prebuilt_fallback_used = False,
quantized_path = quantized_path,
)
assert (install_dir / "gguf-py" / "gguf" / "__init__.py").exists()
assert (install_dir / "convert_hf_to_gguf.py").exists()
assert (install_dir / "build" / "bin" / "Release" / "llama-server.exe").exists()
assert (install_dir / "build" / "bin" / "Release" / "llama-quantize.exe").exists()
assert (install_dir / "build" / "bin" / "Release" / "llama.dll").exists()
assert not (install_dir / "llama-server.exe").exists()
assert (install_dir / "UNSLOTH_PREBUILT_INFO.json").exists()
assert (install_dir / "BUILD_INFO.txt").exists()
def test_activate_install_tree_restores_existing_install_after_activation_failure(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
(install_dir / "old.txt").write_text("old install\n")
staging_dir = create_install_staging_dir(install_dir)
(staging_dir / "new.txt").write_text("new install\n")
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"confirm_install_tree",
lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("activation confirm failed")),
)
with pytest.raises(
PrebuiltFallback,
match = "activation failed; restored previous install",
):
activate_install_tree(staging_dir, install_dir, host)
assert (install_dir / "old.txt").read_text() == "old install\n"
assert not (install_dir / "new.txt").exists()
assert not staging_dir.exists()
assert not (tmp_path / ".staging").exists()
captured = capsys.readouterr()
output = captured.out + captured.err
assert "moving existing install to rollback path" in output
assert "restored previous install from rollback path" in output
def test_activate_install_tree_preserves_symlink_to_resolved_target(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "target"
linked_root = tmp_path / "linked-root"
staging_dir = tmp_path / "staging"
install_dir.mkdir()
staging_dir.mkdir()
(install_dir / "old.txt").write_text("old", encoding = "utf-8")
(staging_dir / "new.txt").write_text("new", encoding = "utf-8")
try:
linked_root.symlink_to(install_dir, target_is_directory = True)
except OSError as exc:
pytest.skip(f"directory symlinks unavailable: {exc}")
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "confirm_install_tree", lambda *_args: None)
activate_install_tree(staging_dir, linked_root.resolve(), linux_host())
assert linked_root.is_symlink()
assert (linked_root / "new.txt").read_text(encoding = "utf-8") == "new"
assert not (linked_root / "old.txt").exists()
def test_activate_install_tree_cleans_all_paths_when_rollback_restore_fails(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
(install_dir / "old.txt").write_text("old install\n")
staging_dir = create_install_staging_dir(install_dir)
(staging_dir / "new.txt").write_text("new install\n")
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"confirm_install_tree",
lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("activation confirm failed")),
)
original_replace = INSTALL_LLAMA_PREBUILT.os.replace
def flaky_replace(src, dst):
src_path = Path(src)
dst_path = Path(dst)
if "rollback-" in src_path.name and dst_path == install_dir:
raise OSError("restore failed")
return original_replace(src, dst)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT.os, "replace", flaky_replace)
with pytest.raises(
PrebuiltFallback,
match = "activation and rollback failed; cleaned install state for fresh source build",
):
activate_install_tree(staging_dir, install_dir, host)
assert not install_dir.exists()
assert not staging_dir.exists()
assert not (tmp_path / ".staging").exists()
captured = capsys.readouterr()
output = captured.out + captured.err
assert "rollback after failed activation also failed: restore failed" in output
assert "cleaning staging, install, and rollback paths before source build fallback" in output
assert "removing failed install path" in output
assert "removing rollback path" in output
def test_activate_staged_dir_copies_when_replace_hits_busy_lock(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
):
staging_dir = tmp_path / "llama.cpp.staging-test"
(staging_dir / "bin").mkdir(parents = True)
(staging_dir / "bin" / "ggml-base.dll").write_bytes(b"fake dll")
dst = tmp_path / "llama.cpp"
def denied_replace(src, dst_arg):
raise PermissionError(errno.EACCES, "Access is denied", str(src))
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT.os, "replace", denied_replace)
activate_staged_dir(staging_dir, dst)
assert (dst / "bin" / "ggml-base.dll").read_bytes() == b"fake dll"
assert not staging_dir.exists()
captured = capsys.readouterr()
assert "falling back to file-by-file copy" in captured.out + captured.err
def test_activate_staged_dir_reraises_non_busy_errors(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
staging_dir = tmp_path / "llama.cpp.staging-test"
staging_dir.mkdir()
(staging_dir / "new.txt").write_text("new install\n")
dst = tmp_path / "llama.cpp"
def out_of_space_replace(src, dst_arg):
raise OSError(errno.ENOSPC, "No space left on device", str(src))
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT.os, "replace", out_of_space_replace)
with pytest.raises(OSError, match = "No space left on device"):
activate_staged_dir(staging_dir, dst)
assert not dst.exists()
assert (staging_dir / "new.txt").read_text() == "new install\n"
def test_binary_env_linux_includes_binary_parent_in_ld_library_path(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
bin_dir = install_dir / "build" / "bin"
bin_dir.mkdir(parents = True)
binary_path = bin_dir / "llama-server"
binary_path.write_bytes(b"fake")
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "linux_runtime_dirs", lambda _bp: [])
env = binary_env(binary_path, install_dir, host)
ld_dirs = env["LD_LIBRARY_PATH"].split(os.pathsep)
assert (
str(bin_dir) in ld_dirs
), f"binary_path.parent ({bin_dir}) must be in LD_LIBRARY_PATH, got: {ld_dirs}"
assert str(install_dir) in ld_dirs
def test_scrub_env_drops_secrets_and_keeps_runtime_vars():
raw = {
# secrets
"HF_TOKEN": "hf_x",
"HUGGING_FACE_HUB_TOKEN": "hf_y",
"GH_TOKEN": "gh_x",
"GITHUB_TOKEN": "gh_y",
"WANDB_API_KEY": "wandb_x",
"AWS_SECRET_ACCESS_KEY": "aws_x",
"ACTIONS_ID_TOKEN_REQUEST_TOKEN": "oidc_x",
"ACTIONS_ID_TOKEN_REQUEST_URL": "https://oidc",
"SOME_VENDOR_API_KEY": "vendor_x",
"DB_PASSWORD": "pw",
"MY_PRIVATE_KEY": "pk",
"KUBECONFIG": "/home/runner/.kube/config",
"SSH_AUTH_SOCK": "/tmp/ssh-agent.sock",
"SSH_PASSPHRASE": "ssh_pass",
# runtime vars to keep
"PATH": "/usr/bin",
"LD_LIBRARY_PATH": "/opt/lib",
"DYLD_LIBRARY_PATH": "/opt/dyld",
"HOME": "/home/runner",
"TMPDIR": "/tmp",
"CUDA_VISIBLE_DEVICES": "0",
"HSA_OVERRIDE_GFX_VERSION": "11.0.0",
}
cleaned = scrub_env(raw)
for secret in (
"HF_TOKEN",
"HUGGING_FACE_HUB_TOKEN",
"GH_TOKEN",
"GITHUB_TOKEN",
"WANDB_API_KEY",
"AWS_SECRET_ACCESS_KEY",
"ACTIONS_ID_TOKEN_REQUEST_TOKEN",
"ACTIONS_ID_TOKEN_REQUEST_URL",
"SOME_VENDOR_API_KEY",
"DB_PASSWORD",
"MY_PRIVATE_KEY",
"KUBECONFIG",
"SSH_AUTH_SOCK",
"SSH_PASSPHRASE",
):
assert secret not in cleaned, f"{secret} must be stripped from binary env"
for keep in (
"PATH",
"LD_LIBRARY_PATH",
"DYLD_LIBRARY_PATH",
"HOME",
"TMPDIR",
"CUDA_VISIBLE_DEVICES",
"HSA_OVERRIDE_GFX_VERSION",
):
assert cleaned[keep] == raw[keep], f"{keep} must be preserved for the binary"
# no bare "KEY" marker: benign KEY-containing names survive
assert is_secret_env_name("API_KEY") is True
assert is_secret_env_name("SSH_KEYFILE_PATH") is False
assert is_secret_env_name("PATH") is False
def test_scrub_env_drops_proxy_index_and_embedded_url_credentials():
raw = {
# proxy / package-index URLs whose values commonly embed credentials
"HTTPS_PROXY": "https://user:secret@proxy:8080",
"https_proxy": "https://user:secret@proxy:8080", # lower-case variant
"ALL_PROXY": "socks5://user:secret@proxy:1080",
"PIP_INDEX_URL": "https://u:p@pypi.internal/simple",
"UV_INDEX_URL": "https://u:p@index.internal/simple",
# credentials embedded in an otherwise benign-named variable's value
"MY_DB_DSN": "postgres://admin:secret@db:5432/app",
# benign vars the binary needs, including a URL with no userinfo
"PATH": "/usr/bin",
"CUDA_VISIBLE_DEVICES": "0",
"NO_PROXY": "localhost,127.0.0.1",
"SOME_ENDPOINT": "https://example.com:8080/v1",
}
cleaned = scrub_env(raw)
for secret in (
"HTTPS_PROXY",
"https_proxy",
"ALL_PROXY",
"PIP_INDEX_URL",
"UV_INDEX_URL",
"MY_DB_DSN",
):
assert secret not in cleaned, f"{secret} must be stripped from binary env"
for keep in ("PATH", "CUDA_VISIBLE_DEVICES", "NO_PROXY", "SOME_ENDPOINT"):
assert cleaned[keep] == raw[keep], f"{keep} must be preserved for the binary"
assert is_secret_env_name("HTTPS_PROXY") is True
assert is_secret_env_name("https_proxy") is True
assert is_secret_env_name("NO_PROXY") is False
def test_binary_env_strips_secrets_from_downloaded_binary_environment(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
bin_dir = install_dir / "build" / "bin"
bin_dir.mkdir(parents = True)
binary_path = bin_dir / "llama-server"
binary_path.write_bytes(b"fake")
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "linux_runtime_dirs", lambda _bp: [])
monkeypatch.setenv("HF_TOKEN", "hf_secret_from_ci")
monkeypatch.setenv("GITHUB_TOKEN", "gh_secret_from_ci")
monkeypatch.setenv("GH_TOKEN", "gh_secret_from_ci")
monkeypatch.setenv("WANDB_API_KEY", "wandb_secret_from_ci")
monkeypatch.setenv("CUDA_VISIBLE_DEVICES", "1")
env = binary_env(binary_path, install_dir, host)
assert "HF_TOKEN" not in env
assert "GITHUB_TOKEN" not in env
assert "GH_TOKEN" not in env
assert "WANDB_API_KEY" not in env
# library/runtime resolution unaffected
assert str(bin_dir) in env["LD_LIBRARY_PATH"].split(os.pathsep)
assert env["CUDA_VISIBLE_DEVICES"] == "1"
def test_binary_env_redirects_home_away_from_real_credential_stores(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
bin_dir = install_dir / "build" / "bin"
bin_dir.mkdir(parents = True)
binary_path = bin_dir / "llama-server"
binary_path.write_bytes(b"fake")
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "linux_runtime_dirs", lambda _bp: [])
real_home = str(tmp_path / "real_home")
monkeypatch.setenv("HOME", real_home)
monkeypatch.setenv("HF_HOME", real_home + "/.cache/huggingface")
env = binary_env(binary_path, install_dir, host)
# HOME and the cache pointers are redirected to a single empty, existing dir.
assert env["HOME"] != real_home
assert env["HF_HOME"] == env["HOME"]
assert env["HOME"] == isolated_runtime_home()
assert os.path.isdir(env["HOME"])
assert os.listdir(env["HOME"]) == []
# Windows reconstructs the profile from HOMEDRIVE + HOMEPATH.
assert env["HOMEDRIVE"] + env["HOMEPATH"] == env["HOME"]
def test_scrub_env_drops_token_only_url_userinfo():
raw = {
"GENERIC_REPO": "https://ghp_tokenonly@github.com/org/repo",
"GENERIC_OK": "https://example.com:8080/v1",
}
cleaned = scrub_env(raw)
assert "GENERIC_REPO" not in cleaned
assert cleaned["GENERIC_OK"] == raw["GENERIC_OK"]
def test_binary_env_drops_explicit_credential_file_pointers(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "linux_runtime_dirs", lambda _bp: [])
dropped = (
"NETRC",
"PIP_CONFIG_FILE",
"DOCKER_CONFIG",
"GIT_CONFIG_GLOBAL",
"GITHUB_ENV",
"GITHUB_PATH",
"GITHUB_OUTPUT",
"GITHUB_STEP_SUMMARY",
"BASH_ENV",
)
for var in dropped:
monkeypatch.setenv(var, "/home/realuser/secret")
env = binary_env(tmp_path / "llama-server", tmp_path, host)
for var in dropped:
assert var not in env
def test_linux_runtime_dirs_probes_with_secret_free_env(monkeypatch: pytest.MonkeyPatch):
captured: dict[str, object] = {}
def fake_missing(binary_path, *, env = None):
captured["env"] = env
return []
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "linux_missing_libraries", fake_missing)
monkeypatch.setenv("HF_TOKEN", "hf_secret")
monkeypatch.setenv("GITHUB_TOKEN", "gh_secret")
INSTALL_LLAMA_PREBUILT.linux_runtime_dirs(Path("/fake/llama-server"))
probe_env = captured["env"]
assert probe_env is not None
assert "HF_TOKEN" not in probe_env
assert "GITHUB_TOKEN" not in probe_env
def test_install_prebuilt_falls_back_to_older_release_plan(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
first_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "old-release",
name = "app-b9002-linux-x64.tar.gz",
url = "https://example.com/app-b9002-linux-x64.tar.gz",
source_label = "published",
install_kind = "linux-cpu",
)
second_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "older-release",
name = "app-b9001-linux-x64.tar.gz",
url = "https://example.com/app-b9001-linux-x64.tar.gz",
source_label = "published",
install_kind = "linux-cpu",
)
first_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9002",
release_tag = "release-2",
attempts = [first_choice],
approved_checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-2",
upstream_tag = "b9002",
source_commit = None,
artifacts = {},
),
)
second_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [second_choice],
approved_checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = None,
artifacts = {},
),
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[first_plan, second_plan],
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_validation_model",
lambda probe_path, cache_path: probe_path.write_bytes(b"probe"),
)
call_log: list[tuple[str, bool]] = []
def fake_validate(
attempts,
host,
install_dir,
work_dir,
probe_path,
*,
requested_tag,
llama_tag,
release_tag,
approved_checksums,
initial_fallback_used = False,
existing_install_dir = None,
force_cpu = False,
llama_backend = None,
):
call_log.append((llama_tag, initial_fallback_used))
if llama_tag == "b9002":
raise PrebuiltFallback("validation failed for latest release")
staging_dir = create_install_staging_dir(install_dir)
(staging_dir / "marker.txt").write_text("ready\n")
return attempts[0], staging_dir, initial_fallback_used
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"validate_prebuilt_attempts",
fake_validate,
)
activated = {}
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"activate_install_tree",
lambda staging_dir, install_dir, host: activated.update(
{"staging_dir": staging_dir, "install_dir": install_dir}
),
)
ensured_tags: list[str] = []
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"ensure_converter_scripts",
lambda install_dir, llama_tag: ensured_tags.append(llama_tag),
)
install_prebuilt(install_dir, "latest", "unslothai/llama.cpp", "")
assert call_log == [("b9002", False), ("b9001", True)]
assert activated["install_dir"] == install_dir
assert ensured_tags == ["b9001"]
def write_linux_install_shape(install_dir: Path) -> None:
runtime_dir = install_dir / "build" / "bin"
runtime_dir.mkdir(parents = True, exist_ok = True)
(install_dir / "llama-server").write_text("#!/bin/sh\n", encoding = "utf-8")
(install_dir / "llama-quantize").write_text("#!/bin/sh\n", encoding = "utf-8")
(runtime_dir / "llama-server").write_text("#!/bin/sh\n", encoding = "utf-8")
(runtime_dir / "llama-quantize").write_text("#!/bin/sh\n", encoding = "utf-8")
# libllama-common.so* (PR #5135) is a required runtime payload health group.
(runtime_dir / "libllama-common.so.0").write_bytes(b"DLL")
(runtime_dir / "libllama.so.0").write_bytes(b"DLL")
(runtime_dir / "libggml.so.0").write_bytes(b"DLL")
(runtime_dir / "libggml-base.so.0").write_bytes(b"DLL")
(runtime_dir / "libggml-cpu-x64.so.0").write_bytes(b"DLL")
(runtime_dir / "libmtmd.so.0").write_bytes(b"DLL")
(install_dir / "convert_hf_to_gguf.py").write_text("#!/usr/bin/env python3\n", encoding = "utf-8")
(install_dir / "gguf-py" / "gguf").mkdir(parents = True, exist_ok = True)
def write_windows_install_shape(
install_dir: Path,
*,
include_llama_dll: bool = True,
include_cuda_dll: bool = False,
include_cudart_dlls: bool = False,
) -> None:
runtime_dir = install_dir / "build" / "bin" / "Release"
runtime_dir.mkdir(parents = True, exist_ok = True)
(runtime_dir / "llama-server.exe").write_bytes(b"MZ")
(runtime_dir / "llama-quantize.exe").write_bytes(b"MZ")
if include_llama_dll:
(runtime_dir / "llama.dll").write_bytes(b"DLL")
if include_cuda_dll:
(runtime_dir / "ggml-cuda.dll").write_bytes(b"DLL")
if include_cudart_dlls:
# cudart bundle DLLs that ship in cudart-llama-bin-win-cuda-*-x64.zip
(runtime_dir / "cudart64_12.dll").write_bytes(b"DLL")
(runtime_dir / "cublas64_12.dll").write_bytes(b"DLL")
(runtime_dir / "cublasLt64_12.dll").write_bytes(b"DLL")
(install_dir / "convert_hf_to_gguf.py").write_text("#!/usr/bin/env python3\n", encoding = "utf-8")
(install_dir / "gguf-py" / "gguf").mkdir(parents = True, exist_ok = True)
def write_macos_install_shape(
install_dir: Path,
*,
include_libllama: bool = True,
include_libggml: bool = True,
include_libmtmd: bool = True,
) -> None:
runtime_dir = install_dir / "build" / "bin"
runtime_dir.mkdir(parents = True, exist_ok = True)
(install_dir / "llama-server").write_text("#!/bin/sh\n", encoding = "utf-8")
(install_dir / "llama-quantize").write_text("#!/bin/sh\n", encoding = "utf-8")
(runtime_dir / "llama-server").write_text("#!/bin/sh\n", encoding = "utf-8")
(runtime_dir / "llama-quantize").write_text("#!/bin/sh\n", encoding = "utf-8")
if include_libllama:
(runtime_dir / "libllama.0.dylib").write_bytes(b"DLL")
if include_libggml:
(runtime_dir / "libggml.0.dylib").write_bytes(b"DLL")
if include_libmtmd:
(runtime_dir / "libmtmd.0.dylib").write_bytes(b"DLL")
(install_dir / "convert_hf_to_gguf.py").write_text("#!/usr/bin/env python3\n", encoding = "utf-8")
(install_dir / "gguf-py" / "gguf").mkdir(parents = True, exist_ok = True)
def test_existing_install_matches_plan_with_fingerprint_linux(tmp_path: Path):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
assert existing_install_matches_plan(install_dir, host, plan) is True
def test_existing_install_matches_plan_false_without_fingerprint(tmp_path: Path):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
(install_dir / "UNSLOTH_PREBUILT_INFO.json").write_text(
json.dumps({"tag": "b9001", "asset": "llama-b9001-bin-ubuntu-x64.tar.gz"}) + "\n",
encoding = "utf-8",
)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/x.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
assert existing_install_matches_plan(install_dir, host, plan) is False
def test_existing_install_matches_plan_false_with_malformed_metadata(tmp_path: Path):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
(install_dir / "UNSLOTH_PREBUILT_INFO.json").write_text("{not-json\n", encoding = "utf-8")
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/x.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
assert existing_install_matches_plan(install_dir, host, plan) is False
def test_existing_install_matches_plan_windows_cpu_requires_llama_dll(tmp_path: Path):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_windows_install_shape(install_dir, include_llama_dll = True)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-win-cpu-x64.zip",
url = "https://example.com/x.zip",
source_label = "published",
install_kind = "windows-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
assert existing_install_matches_plan(install_dir, host, plan) is True
(install_dir / "build" / "bin" / "Release" / "llama.dll").unlink()
assert existing_install_matches_plan(install_dir, host, plan) is False
def test_existing_install_matches_plan_windows_cuda_requires_cuda_dll(tmp_path: Path):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_windows_install_shape(install_dir, include_llama_dll = True, include_cuda_dll = True)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (12, 4),
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = True,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-win-cuda-12.4-x64.zip",
url = "https://example.com/x.zip",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
assert existing_install_matches_plan(install_dir, host, plan) is True
(install_dir / "build" / "bin" / "Release" / "ggml-cuda.dll").unlink()
assert existing_install_matches_plan(install_dir, host, plan) is False
def test_existing_install_matches_plan_windows_cuda_paired_requires_cudart(tmp_path: Path):
"""A paired cudart bundle (#5106) marks the install stale unless cudart64_* and cublas64_* are on disk."""
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_windows_install_shape(
install_dir,
include_llama_dll = True,
include_cuda_dll = True,
include_cudart_dlls = True,
)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (12, 4),
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = True,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-win-cuda-12.4-x64.zip",
url = "https://example.com/x.zip",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = "a" * 64,
runtime_name = "cudart-llama-bin-win-cuda-12.4-x64.zip",
runtime_url = "https://example.com/cudart.zip",
runtime_sha256 = "c" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
choice.runtime_name: ApprovedArtifactHash(
asset_name = choice.runtime_name,
sha256 = choice.runtime_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
# Fully populated install (main archive + cudart DLLs) matches.
assert existing_install_matches_plan(install_dir, host, plan) is True
# cublas missing -- stale, must reinstall.
(install_dir / "build" / "bin" / "Release" / "cublas64_12.dll").unlink()
assert existing_install_matches_plan(install_dir, host, plan) is False
# cudart missing -- stale, must reinstall.
write_windows_install_shape(
install_dir,
include_llama_dll = True,
include_cuda_dll = True,
include_cudart_dlls = True,
)
(install_dir / "build" / "bin" / "Release" / "cudart64_12.dll").unlink()
assert existing_install_matches_plan(install_dir, host, plan) is False
# cublasLt missing -- stale, must reinstall (all three DLLs are required).
write_windows_install_shape(
install_dir,
include_llama_dll = True,
include_cuda_dll = True,
include_cudart_dlls = True,
)
(install_dir / "build" / "bin" / "Release" / "cublasLt64_12.dll").unlink()
assert existing_install_matches_plan(install_dir, host, plan) is False
def test_existing_install_matches_plan_windows_cuda_unpaired_skips_cudart_check(tmp_path: Path):
"""With no paired runtime archive, a legacy install lacking cudart must still pass (else reinstall loops)."""
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_windows_install_shape(
install_dir,
include_llama_dll = True,
include_cuda_dll = True,
include_cudart_dlls = False,
)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (12, 4),
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = True,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-win-cuda-12.4-x64.zip",
url = "https://example.com/x.zip",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
assert existing_install_matches_plan(install_dir, host, plan) is True
def test_existing_install_fingerprint_changes_when_cudart_pair_added(tmp_path: Path):
"""A pre-#5322 CUDA install must go stale once the choice gains a runtime archive (#5106 fingerprint half)."""
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_windows_install_shape(
install_dir,
include_llama_dll = True,
include_cuda_dll = True,
include_cudart_dlls = False,
)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (12, 4),
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = True,
)
legacy_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-win-cuda-12.4-x64.zip",
url = "https://example.com/x.zip",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = "a" * 64,
)
paired_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-win-cuda-12.4-x64.zip",
url = "https://example.com/x.zip",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = "a" * 64,
runtime_name = "cudart-llama-bin-win-cuda-12.4-x64.zip",
runtime_url = "https://example.com/cudart.zip",
runtime_sha256 = "c" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
legacy_choice.name: ApprovedArtifactHash(
asset_name = legacy_choice.name,
sha256 = legacy_choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
paired_choice.runtime_name: ApprovedArtifactHash(
asset_name = paired_choice.runtime_name,
sha256 = paired_choice.runtime_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
},
)
# Metadata written for the legacy (no-pair) choice.
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = legacy_choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
# The paired choice's fingerprint must differ from the legacy one so the install refreshes.
legacy_fingerprint = INSTALL_LLAMA_PREBUILT.expected_install_fingerprint(
llama_tag = "b9001",
release_tag = "release-1",
choice = legacy_choice,
approved_checksums = checksums,
)
paired_fingerprint = INSTALL_LLAMA_PREBUILT.expected_install_fingerprint(
llama_tag = "b9001",
release_tag = "release-1",
choice = paired_choice,
approved_checksums = checksums,
)
assert legacy_fingerprint != paired_fingerprint, (
"expected_install_fingerprint must hash runtime_name/runtime_sha256 "
"so pre-#5322 installs are not falsely considered up-to-date"
)
paired_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [paired_choice],
approved_checksums = checksums,
)
assert existing_install_matches_plan(install_dir, host, paired_plan) is False
def test_existing_install_matches_plan_macos_requires_dylibs(tmp_path: Path):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_macos_install_shape(install_dir)
host = HostInfo(
system = "Darwin",
machine = "arm64",
is_windows = False,
is_linux = False,
is_macos = True,
is_x86_64 = False,
is_arm64 = True,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-macos-arm64.tar.gz",
url = "https://example.com/x.tar.gz",
source_label = "published",
install_kind = "macos-arm64",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
assert existing_install_matches_plan(install_dir, host, plan) is True
(install_dir / "build" / "bin" / "libggml.0.dylib").unlink()
assert existing_install_matches_plan(install_dir, host, plan) is False
def test_install_prebuilt_skips_download_when_existing_install_matches(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
(install_dir / "AGENTS.md").write_text("old root instructions", encoding = "utf-8")
nested_agents = install_dir / "examples" / "AGENTS.md"
nested_agents.parent.mkdir()
nested_agents.write_text("old nested instructions", encoding = "utf-8")
(install_dir / "CLAUDE.md").write_text("old Claude instructions", encoding = "utf-8")
(nested_agents.parent / "CLAUDE.md").write_text(
"old nested Claude instructions", encoding = "utf-8"
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[plan],
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_validation_model",
lambda *args, **kwargs: (_ for _ in ()).throw(
AssertionError("matching install should skip before validation model download")
),
)
install_prebuilt(install_dir, "latest", "unslothai/llama.cpp", "")
assert not list(install_dir.rglob("AGENTS.md"))
assert not list(install_dir.rglob("CLAUDE.md"))
def test_setup_scripts_prune_agent_files_without_shipping_a_repo_copy():
setup_sh = (PACKAGE_ROOT / "studio" / "setup.sh").read_text(encoding = "utf-8")
setup_ps1 = (PACKAGE_ROOT / "studio" / "setup.ps1").read_text(encoding = "utf-8")
assert (
"_remove_agent_instruction_files \\\n"
' "$SCRIPT_DIR/frontend/node_modules" \\\n'
' "$_OXC_DIR/node_modules"'
) in setup_sh
assert '_remove_agent_instruction_files "$SCRIPT_DIR/frontend" "$_OXC_DIR"' not in setup_sh
assert '_remove_agent_instruction_files "$LLAMA_CPP_DIR"' in setup_sh
assert "-name 'CLAUDE.md'" in setup_sh
assert 'if [ ! -L "$LLAMA_CPP_DIR" ] && {' in setup_sh
assert '${_LOCAL_LLAMA_CPP_LINKED:-false}" != true' not in setup_sh
assert "$LLAMA_CPP_DIR/$_STUDIO_OWNED_MARKER" in setup_sh
assert '_studio_owned_adoptable "$LLAMA_CPP_DIR"' in setup_sh
assert (
"Remove-AgentInstructionFiles -Roots @(\n"
' (Join-Path $FrontendDir "node_modules"),\n'
' (Join-Path $OxcValidatorDir "node_modules")\n'
")"
) in setup_ps1
assert "Remove-AgentInstructionFiles -Roots @($FrontendDir, $OxcValidatorDir)" not in setup_ps1
assert '"CLAUDE.md"' in setup_ps1
assert '-Include "AGENTS.md", "CLAUDE.md"' not in setup_ps1
assert '$child.Name -in @("AGENTS.md", "CLAUDE.md")' in setup_ps1
assert "$llamaCppIsLink" in setup_ps1
assert "if (-not $LocalLlamaCppLinked)" not in setup_ps1
assert "Join-Path $LlamaCppDir $StudioOwnedMarker" in setup_ps1
assert "Test-StudioOwnedAdoptable $LlamaCppDir" in setup_ps1
assert (
"Copy-Item -Recurse -LiteralPath $ResolvedLocal -Destination $LlamaCppDir\n"
" Remove-AgentInstructionFiles -Roots @($LlamaCppDir)"
) in setup_ps1
assert not (PACKAGE_ROOT / "studio" / "frontend" / "src" / "i18n" / "AGENTS.md").exists()
assert (PACKAGE_ROOT / "studio" / "frontend" / "src" / "i18n" / "README.md").is_file()
def test_setup_sh_cleanup_unlinks_instruction_symlink_only(tmp_path: Path):
if shutil.which("bash") is None:
pytest.skip("bash is not available")
setup_sh = (PACKAGE_ROOT / "studio" / "setup.sh").read_text(encoding = "utf-8")
start = setup_sh.index("_remove_agent_instruction_files() {")
end = setup_sh.index("\n}\n", start) + 2
function = setup_sh[start:end]
managed = tmp_path / "managed"
external = tmp_path / "external.md"
managed.mkdir()
external.write_text("external", encoding = "utf-8")
instruction = managed / "AGENTS.md"
try:
instruction.symlink_to(external)
except OSError as exc:
pytest.skip(f"symlinks unavailable: {exc}")
subprocess.run(
["bash", "-c", function + '\n_remove_agent_instruction_files "$1"', "bash", str(managed)],
check = True,
)
assert not os.path.lexists(instruction)
assert external.read_text(encoding = "utf-8") == "external"
def test_install_prebuilt_does_not_skip_unhealthy_existing_install(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
(install_dir / "llama-quantize").unlink()
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[plan],
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_validation_model",
lambda *args, **kwargs: (_ for _ in ()).throw(
AssertionError("unhealthy install must continue into normal install flow")
),
)
with pytest.raises(
AssertionError, match = "unhealthy install must continue into normal install flow"
):
install_prebuilt(install_dir, "latest", "unslothai/llama.cpp", "")
def test_install_prebuilt_skips_when_older_release_fallback_matches_existing_install(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
latest_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-2",
name = "llama-b9002-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9002-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "c" * 64,
)
fallback_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
latest_checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-2",
upstream_tag = "b9002",
source_commit = "beadfeed",
artifacts = {
source_archive_logical_name("b9002"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9002"),
sha256 = "d" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
latest_choice.name: ApprovedArtifactHash(
asset_name = latest_choice.name,
sha256 = latest_choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
fallback_checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
fallback_choice.name: ApprovedArtifactHash(
asset_name = fallback_choice.name,
sha256 = fallback_choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
latest_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9002",
release_tag = "release-2",
attempts = [latest_choice],
approved_checksums = latest_checksums,
)
fallback_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [fallback_choice],
approved_checksums = fallback_checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = fallback_choice,
approved_checksums = fallback_checksums,
prebuilt_fallback_used = True,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[latest_plan, fallback_plan],
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_validation_model",
lambda probe_path, cache_path: probe_path.write_bytes(b"probe"),
)
call_log: list[str] = []
def fake_validate(
attempts,
host,
install_dir,
work_dir,
probe_path,
*,
requested_tag,
llama_tag,
release_tag,
approved_checksums,
initial_fallback_used = False,
existing_install_dir = None,
force_cpu = False,
llama_backend = None,
):
call_log.append(llama_tag)
raise PrebuiltFallback("validation failed for latest release")
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"validate_prebuilt_attempts",
fake_validate,
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"activate_install_tree",
lambda *args, **kwargs: (_ for _ in ()).throw(
AssertionError("matching fallback install should not reactivate")
),
)
install_prebuilt(install_dir, "latest", "unslothai/llama.cpp", "")
assert call_log == ["b9002"]
def test_install_prebuilt_skips_same_release_fallback_attempt_when_installed(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
first_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64-bad.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64-bad.tar.gz",
source_label = "published",
install_kind = "linux-cpu",
expected_sha256 = "c" * 64,
)
fallback_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64-good.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64-good.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
first_choice.name: ApprovedArtifactHash(
asset_name = first_choice.name,
sha256 = first_choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
fallback_choice.name: ApprovedArtifactHash(
asset_name = fallback_choice.name,
sha256 = fallback_choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [first_choice, fallback_choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = fallback_choice,
approved_checksums = checksums,
prebuilt_fallback_used = True,
)
assert (
existing_install_matches_choice(
install_dir,
host,
llama_tag = "b9001",
release_tag = "release-1",
choice = fallback_choice,
approved_checksums = checksums,
)
is True
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[plan],
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_validation_model",
lambda probe_path, cache_path: probe_path.write_bytes(b"probe"),
)
attempted_names: list[str] = []
def fake_validate_choice(
choice,
host,
staging_dir,
work_dir,
probe_path,
*,
requested_tag,
llama_tag,
release_tag,
approved_checksums,
prebuilt_fallback_used,
quantized_path,
force_cpu = False,
llama_backend = None,
):
attempted_names.append(choice.name)
if choice.name == first_choice.name:
raise PrebuiltFallback("newest candidate failed")
raise AssertionError("installed fallback candidate should have been skipped")
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"validate_prebuilt_choice",
fake_validate_choice,
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"activate_install_tree",
lambda *args, **kwargs: (_ for _ in ()).throw(
AssertionError("installed fallback candidate should not be activated")
),
)
install_prebuilt(install_dir, "latest", "unslothai/llama.cpp", "")
assert attempted_names == [first_choice.name]
def test_install_prebuilt_same_tag_upstream_failure_uses_older_unsloth_release_plan(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
same_tag_upstream_choice = AssetChoice(
repo = "ggml-org/llama.cpp",
tag = "b9002",
name = "llama-b9002-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9002-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
older_release_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "b" * 64,
)
latest_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9002",
release_tag = "release-2",
attempts = [same_tag_upstream_choice],
approved_checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-2",
upstream_tag = "b9002",
source_commit = None,
artifacts = {},
),
)
older_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [older_release_choice],
approved_checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = None,
artifacts = {},
),
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[latest_plan, older_plan],
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_validation_model",
lambda probe_path, cache_path: probe_path.write_bytes(b"probe"),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"latest_upstream_release_tag",
lambda: (_ for _ in ()).throw(
AssertionError("install fallback should not walk upstream releases")
),
)
attempted = []
def fake_validate(
attempts,
host,
install_dir,
work_dir,
probe_path,
*,
requested_tag,
llama_tag,
release_tag,
approved_checksums,
initial_fallback_used = False,
existing_install_dir = None,
force_cpu = False,
llama_backend = None,
):
attempted.append((llama_tag, release_tag, attempts[0].source_label))
if llama_tag == "b9002":
raise PrebuiltFallback("same-tag upstream asset failed validation")
staging_dir = create_install_staging_dir(install_dir)
(staging_dir / "marker.txt").write_text("ready\n")
return attempts[0], staging_dir, initial_fallback_used
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "validate_prebuilt_attempts", fake_validate)
activated = {}
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"activate_install_tree",
lambda staging_dir, install_dir, host: activated.update(
{"staging_dir": staging_dir, "install_dir": install_dir}
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"ensure_converter_scripts",
lambda install_dir, llama_tag: None,
)
install_prebuilt(install_dir, "latest", "unslothai/llama.cpp", "")
assert attempted == [("b9002", "release-2", "upstream"), ("b9001", "release-1", "upstream")]
assert activated["install_dir"] == install_dir
def io_bytes(data: bytes):
return io.BytesIO(data)
def add_bytes_to_tar(
archive: tarfile.TarFile,
name: str,
data: bytes,
*,
mode: int = 0o644,
) -> None:
info = tarfile.TarInfo(name)
info.size = len(data)
info.mode = mode
archive.addfile(info, io_bytes(data))
def add_symlink_to_tar(archive: tarfile.TarFile, name: str, target: str) -> None:
info = tarfile.TarInfo(name)
info.type = tarfile.SYMTYPE
info.linkname = target
archive.addfile(info)
def test_existing_install_matches_choice_fails_when_install_tree_incomplete(tmp_path: Path):
"""confirm_install_tree guard rejects installs missing critical files."""
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
# Full install should match
assert (
existing_install_matches_choice(
install_dir,
host,
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
)
is True
)
# Remove convert_hf_to_gguf.py (confirm_install_tree checks it; runtime health does not).
(install_dir / "convert_hf_to_gguf.py").unlink()
assert (
existing_install_matches_choice(
install_dir,
host,
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
)
is False
)
def test_existing_install_matches_choice_fails_when_install_tree_incomplete_macos(tmp_path: Path):
"""confirm_install_tree guard rejects macOS arm64 installs missing critical files."""
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_macos_install_shape(install_dir)
host = HostInfo(
system = "Darwin",
machine = "arm64",
is_windows = False,
is_linux = False,
is_macos = True,
is_x86_64 = False,
is_arm64 = True,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-macos-arm64.tar.gz",
url = "https://example.com/llama-b9001-bin-macos-arm64.tar.gz",
source_label = "upstream",
install_kind = "macos-arm64",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
# Full install should match
assert (
existing_install_matches_choice(
install_dir,
host,
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
)
is True
)
# Remove a macOS-specific runtime artifact and verify the guard catches it
(install_dir / "build" / "bin" / "libmtmd.0.dylib").unlink()
assert (
existing_install_matches_choice(
install_dir,
host,
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
)
is False
)
def test_paired_runtime_dll_patterns_excludes_executables() -> None:
"""The paired runtime archive must contribute only CUDA DLLs (no *.exe/*.dll) so it can't overwrite binaries."""
paired_runtime_dll_patterns = INSTALL_LLAMA_PREBUILT.paired_runtime_dll_patterns
paired_choice = AssetChoice(
repo = "x",
tag = "t",
name = "llama-b9001-bin-win-cuda-12.4-x64.zip",
url = "u",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = "a" * 64,
runtime_name = "cudart-llama-bin-win-cuda-12.4-x64.zip",
runtime_url = "https://example.com/cudart.zip",
runtime_sha256 = "c" * 64,
)
patterns = paired_runtime_dll_patterns(paired_choice)
assert "cudart64_*.dll" in patterns
assert "cublas64_*.dll" in patterns
assert "cublasLt64_*.dll" in patterns
assert "*.exe" not in patterns
assert "*.dll" not in patterns
for kind in (
"linux-cpu",
"linux-cuda",
"linux-rocm",
"macos-arm64",
"macos-x64",
"windows-cpu",
"windows-hip",
):
non_windows = AssetChoice(
repo = "x",
tag = "t",
name = "x",
url = "u",
source_label = "published",
install_kind = kind,
expected_sha256 = "a" * 64,
)
assert paired_runtime_dll_patterns(non_windows) == []
def test_runtime_overlay_cannot_overwrite_main_archive_payload(tmp_path: Path) -> None:
"""A malformed runtime archive with llama-server.exe must NOT replace the main archive's binary."""
install_from_archives = INSTALL_LLAMA_PREBUILT.install_from_archives
work = tmp_path / "work"
install = tmp_path / "install"
archives = tmp_path / "archives"
work.mkdir()
install.mkdir()
archives.mkdir()
main_zip = archives / "llama-b9001-bin-win-cuda-12.4-x64.zip"
runtime_zip = archives / "cudart-llama-bin-win-cuda-12.4-x64.zip"
with zipfile.ZipFile(main_zip, "w", zipfile.ZIP_DEFLATED) as zf:
zf.writestr("llama-server.exe", b"MAIN-SERVER")
zf.writestr("llama-quantize.exe", b"MAIN-Q")
zf.writestr("llama.dll", b"DLL-llama")
zf.writestr("ggml-cuda.dll", b"DLL-ggml")
import hashlib
main_sha = hashlib.sha256(main_zip.read_bytes()).hexdigest()
with zipfile.ZipFile(runtime_zip, "w", zipfile.ZIP_DEFLATED) as zf:
zf.writestr("cudart64_12.dll", b"DLL-cudart")
zf.writestr("cublas64_12.dll", b"DLL-cublas")
zf.writestr("cublasLt64_12.dll", b"DLL-cublasLt")
zf.writestr("llama-server.exe", b"RUNTIME-OVERWRITE")
runtime_sha = hashlib.sha256(runtime_zip.read_bytes()).hexdigest()
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = main_zip.name,
url = f"https://example.com/{main_zip.name}",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = main_sha,
runtime_name = runtime_zip.name,
runtime_url = f"https://example.com/{runtime_zip.name}",
runtime_sha256 = runtime_sha,
)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (12, 4),
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = True,
)
import shutil as _shutil
orig_download = INSTALL_LLAMA_PREBUILT.download_file_verified
def fake_download(
url,
target_path,
*,
expected_sha256 = None,
label = None,
**kw,
):
src = main_zip if "cudart" not in url else runtime_zip
_shutil.copy2(src, target_path)
if expected_sha256:
actual = hashlib.sha256(Path(target_path).read_bytes()).hexdigest()
if actual != expected_sha256:
raise INSTALL_LLAMA_PREBUILT.PrebuiltFallback(f"sha256 mismatch on {label}")
INSTALL_LLAMA_PREBUILT.download_file_verified = fake_download
try:
install_from_archives(choice, host, install, work)
finally:
INSTALL_LLAMA_PREBUILT.download_file_verified = orig_download
release_dir = install / "build" / "bin" / "Release"
server = release_dir / "llama-server.exe"
assert server.exists()
assert server.read_bytes() == b"MAIN-SERVER", (
"runtime archive overwrote main llama-server.exe; " f"got {server.read_bytes()!r}"
)
for name in ("cudart64_12.dll", "cublas64_12.dll", "cublasLt64_12.dll"):
assert (release_dir / name).exists(), f"missing {name}"
def test_linux_runtime_overlay_copies_llama_tool_impl_libraries(tmp_path: Path) -> None:
install_from_archives = INSTALL_LLAMA_PREBUILT.install_from_archives
work = tmp_path / "work"
install = tmp_path / "install"
archives = tmp_path / "archives"
work.mkdir()
install.mkdir()
archives.mkdir()
bundle = archives / "app-b9334-linux-x64-cuda13-newer.tar.gz"
with tarfile.open(bundle, "w:gz") as archive:
for name in (
"llama-cli",
"llama-server",
"llama-quantize",
"libllama-cli-impl.so",
"libllama-server-impl.so",
"libllama-quantize-impl.so",
"libllama-common.so",
"libllama.so",
"libggml.so",
"libggml-base.so",
"libmtmd.so",
"libggml-cpu-x64.so",
"libggml-cuda.so",
):
payload = f"{name}\n".encode()
member = tarfile.TarInfo(name)
member.size = len(payload)
archive.addfile(member, io.BytesIO(payload))
import hashlib
import shutil as _shutil
bundle_sha = hashlib.sha256(bundle.read_bytes()).hexdigest()
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "b9334",
name = bundle.name,
url = f"https://example.com/{bundle.name}",
source_label = "published",
install_kind = "linux-cuda",
runtime_line = "cuda13",
expected_sha256 = bundle_sha,
)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (13, 0),
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = True,
has_usable_nvidia = True,
)
orig_download = INSTALL_LLAMA_PREBUILT.download_file_verified
def fake_download(
url,
target_path,
*,
expected_sha256 = None,
label = None,
**kw,
):
_shutil.copy2(bundle, target_path)
if expected_sha256:
actual = hashlib.sha256(Path(target_path).read_bytes()).hexdigest()
if actual != expected_sha256:
raise INSTALL_LLAMA_PREBUILT.PrebuiltFallback(f"sha256 mismatch on {label}")
INSTALL_LLAMA_PREBUILT.download_file_verified = fake_download
try:
install_from_archives(choice, host, install, work)
finally:
INSTALL_LLAMA_PREBUILT.download_file_verified = orig_download
runtime_dir = install / "build" / "bin"
for name in (
"libllama-cli-impl.so",
"libllama-server-impl.so",
"libllama-quantize-impl.so",
):
assert (runtime_dir / name).exists(), f"missing {name}"
assert not (runtime_dir / "llama-cli").exists()
def test_python_runtime_dirs_covers_cu13_and_library_bin(monkeypatch, tmp_path: Path) -> None:
"""Installer DLL discovery must scan the same path set as the backend (cu12/cu13/conda layouts + torch/lib)."""
import site as _site
python_runtime_dirs = INSTALL_LLAMA_PREBUILT.python_runtime_dirs
site_dir = tmp_path / "Lib" / "site-packages"
# cu12-style modular wheel
cu12_bin = site_dir / "nvidia" / "cuda_runtime" / "bin"
cu12_bin.mkdir(parents = True)
# cu13-style unsuffixed wheel
cu13_arch = site_dir / "nvidia" / "cu13" / "bin" / "x86_64"
cu13_arch.mkdir(parents = True)
# conda-style repack
library_bin = site_dir / "nvidia" / "cublas" / "Library" / "bin"
library_bin.mkdir(parents = True)
# PyTorch bundled-CUDA wheel
torch_lib = site_dir / "torch" / "lib"
torch_lib.mkdir(parents = True)
monkeypatch.setattr(sys, "path", [str(site_dir)])
monkeypatch.setattr(_site, "getsitepackages", lambda: [str(site_dir)])
monkeypatch.setattr(_site, "getusersitepackages", lambda: "")
dirs = python_runtime_dirs()
assert str(cu12_bin) in dirs
assert str(cu13_arch) in dirs
assert str(library_bin) in dirs
assert str(torch_lib) in dirs
def _nvidia_linux_host():
return HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = ["10.0"],
visible_cuda_devices = None,
has_physical_nvidia = True,
has_usable_nvidia = True,
)
def _run_validate_prebuilt_choice(monkeypatch, tmp_path, *, expected_sha256):
"""Run validate_prebuilt_choice with heavy steps stubbed; return the quantize/server smoke-test call counts."""
calls = {"quantize": 0, "server": 0}
server_path = tmp_path / "install" / "build" / "bin" / "llama-server"
quantize_path = tmp_path / "install" / "build" / "bin" / "llama-quantize"
src = INSTALL_LLAMA_PREBUILT
monkeypatch.setattr(
src, "preferred_source_archive", lambda *a, **k: ("repo", "ref", None, False)
)
monkeypatch.setattr(src, "hydrate_source_tree", lambda *a, **k: None)
monkeypatch.setattr(src, "install_from_archives", lambda *a, **k: (server_path, quantize_path))
monkeypatch.setattr(src, "preflight_linux_installed_binaries", lambda *a, **k: None)
monkeypatch.setattr(src, "preflight_macos_installed_binaries", lambda *a, **k: None)
monkeypatch.setattr(src, "ensure_repo_shape", lambda *a, **k: None)
monkeypatch.setattr(src, "write_prebuilt_metadata", lambda *a, **k: None)
monkeypatch.setattr(
src,
"validate_quantize",
lambda *a, **k: calls.__setitem__("quantize", calls["quantize"] + 1),
)
monkeypatch.setattr(
src, "validate_server", lambda *a, **k: calls.__setitem__("server", calls["server"] + 1)
)
bundle_name = "app-b9998-linux-x64-cuda13-newer.tar.gz"
source_archive = tmp_path / "source.tar.gz"
bundle_archive = tmp_path / "bundle.tar.gz"
source_archive.write_bytes(b"source")
bundle_archive.write_bytes(b"bundle")
choice = AssetChoice(
repo = "local",
tag = "b9998",
name = bundle_name,
url = "file://bundle",
source_label = "local",
is_ready_bundle = True,
install_kind = "linux-cuda",
bundle_profile = "cuda13-newer",
runtime_line = "cuda13",
expected_sha256 = expected_sha256,
)
src.validate_prebuilt_choice(
choice,
_nvidia_linux_host(),
tmp_path / "install",
tmp_path / "work",
tmp_path / "stories260K.gguf",
requested_tag = "b9998",
llama_tag = "b9998",
release_tag = "b9998",
approved_checksums = approved_checksums_for(
"b9998",
source_archive = source_archive,
bundle_archive = bundle_archive,
bundle_name = bundle_name,
),
prebuilt_fallback_used = False,
quantized_path = tmp_path / "stories260K-q4.gguf",
)
return calls
def test_validate_prebuilt_choice_approved_validation_skipped_when_flag_off(tmp_path, monkeypatch):
# An approved (sha256-verified) bundle skips the smoke test while the flag is off.
calls = _run_validate_prebuilt_choice(monkeypatch, tmp_path, expected_sha256 = "ab" * 32)
assert calls == {"quantize": 0, "server": 0}
def test_validate_prebuilt_choice_hashless_build_always_validated(tmp_path, monkeypatch):
# A hashless build has no sha256 gate, so the smoke test must run even with the flag off.
calls = _run_validate_prebuilt_choice(monkeypatch, tmp_path, expected_sha256 = None)
assert calls == {"quantize": 1, "server": 1}
def test_validate_prebuilt_choice_approved_validation_runs_when_flag_enabled(tmp_path, monkeypatch):
# _RUN_STAGED_PREBUILT_VALIDATION back on restores the smoke test for approved bundles too.
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "_RUN_STAGED_PREBUILT_VALIDATION", True)
calls = _run_validate_prebuilt_choice(monkeypatch, tmp_path, expected_sha256 = "ab" * 32)
assert calls == {"quantize": 1, "server": 1}
def test_staged_validation_enabled_default_off(monkeypatch):
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "_RUN_STAGED_PREBUILT_VALIDATION", False)
monkeypatch.delenv("UNSLOTH_LLAMA_STAGED_VALIDATION", raising = False)
assert INSTALL_LLAMA_PREBUILT.staged_validation_enabled() is False
@pytest.mark.parametrize("value", ["1", "true", "YES", "on"])
def test_staged_validation_enabled_env_opt_in(monkeypatch, value):
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "_RUN_STAGED_PREBUILT_VALIDATION", False)
monkeypatch.setenv("UNSLOTH_LLAMA_STAGED_VALIDATION", value)
assert INSTALL_LLAMA_PREBUILT.staged_validation_enabled() is True
def test_validate_prebuilt_choice_approved_validation_runs_when_env_enabled(tmp_path, monkeypatch):
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "_RUN_STAGED_PREBUILT_VALIDATION", False)
monkeypatch.setenv("UNSLOTH_LLAMA_STAGED_VALIDATION", "1")
calls = _run_validate_prebuilt_choice(monkeypatch, tmp_path, expected_sha256 = "ab" * 32)
assert calls == {"quantize": 1, "server": 1}
def test_validate_existing_install_runs_server_smoke(tmp_path, monkeypatch):
# setup.sh --validate-install path: exercise smoke helpers without a real GPU.
install_dir = tmp_path / "llama.cpp"
bin_dir = install_dir / "build" / "bin"
bin_dir.mkdir(parents = True)
(bin_dir / "llama-server").write_text("#!/bin/sh\n", encoding = "utf-8")
(bin_dir / "llama-quantize").write_text("#!/bin/sh\n", encoding = "utf-8")
calls: dict[str, int] = {"quantize": 0, "server": 0, "download": 0}
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_validation_model",
lambda path, cache = None: calls.__setitem__("download", calls["download"] + 1),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"validate_quantize",
lambda *a, **k: calls.__setitem__("quantize", calls["quantize"] + 1),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"validate_server",
lambda *a, **k: calls.__setitem__("server", calls["server"] + 1),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"detect_host",
lambda: linux_host(),
)
INSTALL_LLAMA_PREBUILT.validate_existing_install(install_dir, install_kind = "linux-cuda")
assert calls == {"quantize": 1, "server": 1, "download": 1}
def test_validate_existing_install_missing_server_raises(tmp_path, monkeypatch):
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: linux_host())
with pytest.raises(INSTALL_LLAMA_PREBUILT.PrebuiltFallback, match = "llama-server not found"):
INSTALL_LLAMA_PREBUILT.validate_existing_install(tmp_path / "missing")
def test_diffusion_visual_server_uses_approved_checksum_download(monkeypatch, tmp_path: Path):
asset_name = "llama-diffusion-gemma-visual-server-linux-x64"
expected_sha = "a" * 64
asset_url = "https://github.com/unslothai/llama.cpp/releases/download/b9334/" + asset_name
calls: list[tuple[str, Path, str | None, str | None]] = []
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"github_release_assets",
lambda repo, tag: {asset_name: asset_url},
)
def fake_download_file(url, destination):
raise AssertionError("diffusion visual server must not use unverified download_file")
def fake_download_file_verified(url, destination, *, expected_sha256, label):
calls.append((url, Path(destination), expected_sha256, label))
Path(destination).write_bytes(b"verified visual server")
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT, "download_file_verified", fake_download_file_verified
)
ensure_diffusion_visual_server(
tmp_path / "install",
linux_host(),
"b9334",
approved_release_checksums_for_asset(asset_name, expected_sha),
)
target = tmp_path / "install" / "build" / "bin" / "llama-diffusion-gemma-visual-server"
assert calls == [
(
asset_url,
target,
expected_sha,
f"diffusion visual server {asset_name}",
)
]
assert target.read_bytes() == b"verified visual server"
assert target.stat().st_mode & 0o777 == 0o755
def test_diffusion_visual_server_refuses_unapproved_release_asset(monkeypatch, tmp_path: Path):
asset_name = "llama-diffusion-gemma-visual-server-attacker-linux"
verified_calls: list[str] = []
raw_calls: list[str] = []
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"github_release_assets",
lambda repo, tag: {asset_name: "https://example.test/" + asset_name},
)
def fake_download_file(url, destination):
raw_calls.append(url)
def fake_download_file_verified(url, destination, *, expected_sha256, label):
verified_calls.append(url)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT, "download_file_verified", fake_download_file_verified
)
ensure_diffusion_visual_server(
tmp_path / "install",
linux_host(),
"b9334",
ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "b9334",
upstream_tag = "b9334",
artifacts = {},
),
)
target = tmp_path / "install" / "build" / "bin" / "llama-diffusion-gemma-visual-server"
assert not target.exists()
assert raw_calls == []
assert verified_calls == []