ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
unsloth/tests/studio/install/test_install_llama_prebuilt_logic.py
Daniel Han 4c72e09480
 Studio: stop handing CI/user secrets to downloaded llama.cpp binaries (#6696) 
* Studio: stop handing CI/user secrets to downloaded llama.cpp binaries

The macOS prebuilt path installs llama.cpp from the unslothai/llama.cpp
fork's latest (unpinned, mutable) release and then executes the
downloaded llama-server / llama-quantize binaries during install-time
validation. binary_env() built that child environment from a full
os.environ.copy(), so a compromised or tampered prebuilt would inherit
every secret in the process: HF_TOKEN and the workflow GitHub tokens in
CI, and HF / cloud credentials for end users running install.sh /
setup.sh.

We publish prebuilts daily, so pinning a release tag is not workable.
Instead, neutralise the impact: these binaries have no reason to read any
token, so strip secret-bearing variables (exact names plus
TOKEN/SECRET/PASSWORD/CREDENTIAL/PRIVATE_KEY/API_KEY markers) before
handing the env to a downloaded binary. The installer's own GitHub and
Hugging Face API calls read os.environ directly, so authentication and
release-API rate limiting are unaffected; PATH, LD_LIBRARY_PATH,
DYLD_LIBRARY_PATH and CUDA/ROCm vars are preserved. One change covers the
install-time validation path for all six macOS workflows and end users.

Follow-up (separate, sequenced): publish build-provenance attestations
from the fork's prebuilt workflows and verify them in CI, so a forged
release is rejected rather than merely starved of secrets.

* Strip KUBECONFIG, SSH_AUTH_SOCK, and PASSPHRASE-marked vars from binary env

Extend the deny-list per PR review: KUBECONFIG and SSH_AUTH_SOCK are
credential pointers/capabilities a downloaded binary never needs, and a
PASSPHRASE marker catches SSH_PASSPHRASE / GPG_PASSPHRASE. Tests updated.

* Studio: also scrub proxy/index env vars and URL-embedded credentials before running prebuilt binaries

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scope mlx-ci secrets to the install + download commands for PR #6696

Drop the ambient step-level env block and pass GH/GITHUB/HF tokens only
on the installer and GGUF-download commands, so the directly invoked
llama-quantize / llama-server smoke runs see no secrets. The installer
still reads tokens from os.environ for the releases API and probe fetch.

* Trim verbose comments around the secret-env scrubber for PR #6696

Comment-only: condense the block comments added across this PR. Logic
unchanged (comment_tools.py check confirms code-only signature equal).

* Redirect HOME / cache pointers to an empty dir for prebuilt binaries (PR #6696)

Address Codex P2: stripping token env vars still let a tampered binary
read on-disk token stores (~/.cache/huggingface/token, ~/.aws/credentials,
~/.config/gh) through $HOME and the cache/config pointers. Point HOME plus
the HF / XDG / Windows home pointers at a single empty throwaway dir for
the downloaded-binary env. Defense in depth: a binary resolving the real
home via getpwuid is out of scope and needs OS sandboxing.

* Close residual credential-probe gaps for PR #6696

Address the latest Codex review:
- Strip token-only URL userinfo too (scheme://ghp_token@host), not just
  the user:pass form.
- Redirect HOMEDRIVE/HOMEPATH alongside USERPROFILE so a Windows binary
  cannot reconstruct the real profile from %HOMEDRIVE%%HOMEPATH%.
- Drop explicit credential-file pointers (NETRC, PIP_CONFIG_FILE,
  DOCKER_CONFIG, GIT_CONFIG_GLOBAL) that live outside HOME.
- Probe ldd with a secret-free env: linux_runtime_dirs ran ldd on the
  untrusted prebuilt with the inherited os.environ, and ldd may execute
  the binary, so it could observe HF_TOKEN/GITHUB_TOKEN during the probe.

Factored the shared scrub into secret_free_environ().

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Separate token-bearing install from binary smoke; drop CI command files (PR #6696)

Address the two P1s in the latest review:
- mlx-ci: GitHub bakes secrets into the run-script text, so inline token
  assignments in a step that later runs the prebuilt let a tampered binary
  read them from the script. Split into a token-bearing install + download
  step that never launches a binary, and a secret-free smoke step that runs
  llama-quantize / llama-server.
- secret_free_environ now drops the GitHub Actions command files
  (GITHUB_ENV, GITHUB_PATH, GITHUB_OUTPUT, GITHUB_STEP_SUMMARY, BASH_ENV) and
  the smoke step unsets them, so a tampered prebuilt cannot inject PATH/env
  into the later token-bearing MLX steps.

* Run the prebuilt smoke last, after all token-bearing steps (PR #6696)

Address the P1 workspace-poisoning vector: even with no secrets in its env,
a tampered prebuilt could edit the checkout or installed modules, and the
later HF_TOKEN MLX steps would then execute that poisoned code on push
builds. Move the prebuilt install + smoke to the end of the job so the
untrusted binary runs after every token-bearing step, leaving nothing for it
to corrupt. The MLX GGUF reload uses a source-built llama-cli, not this
prebuilt, so nothing depends on the earlier position.

* Trim comments around the secret-env scrubber and prebuilt CI steps (PR #6696)

Comment-only: condense the security-rationale block comments and merge the
duplicated prebuilt-step description in mlx-ci. Logic unchanged
(comment_tools.py check confirms the code-only signature is equal; install
suite still passes).

* Authenticate the GGUF export release-API lookup with the read-only GITHUB_TOKEN (PR #6696)

* Rename env scrubber off the secret-named identifier CodeQL flags as a clear-text sink (PR #6696)

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-27 05:21:05 -07:00

3205 lines
108 KiB
Python
Raw Permalink Blame History

import errno
import importlib.util
import io
import json
import os
import sys
import tarfile
import zipfile
from pathlib import Path
import pytest
PACKAGE_ROOT = Path(__file__).resolve().parents[3]
MODULE_PATH = PACKAGE_ROOT / "studio" / "install_llama_prebuilt.py"
SPEC = importlib.util.spec_from_file_location("studio_install_llama_prebuilt", MODULE_PATH)
assert SPEC is not None and SPEC.loader is not None
INSTALL_LLAMA_PREBUILT = importlib.util.module_from_spec(SPEC)
sys.modules[SPEC.name] = INSTALL_LLAMA_PREBUILT
SPEC.loader.exec_module(INSTALL_LLAMA_PREBUILT)
PrebuiltFallback = INSTALL_LLAMA_PREBUILT.PrebuiltFallback
extract_archive = INSTALL_LLAMA_PREBUILT.extract_archive
binary_env = INSTALL_LLAMA_PREBUILT.binary_env
is_secret_env_name = INSTALL_LLAMA_PREBUILT.is_secret_env_name
scrub_env = INSTALL_LLAMA_PREBUILT.scrub_env
isolated_runtime_home = INSTALL_LLAMA_PREBUILT.isolated_runtime_home
HostInfo = INSTALL_LLAMA_PREBUILT.HostInfo
AssetChoice = INSTALL_LLAMA_PREBUILT.AssetChoice
ApprovedArtifactHash = INSTALL_LLAMA_PREBUILT.ApprovedArtifactHash
ApprovedReleaseChecksums = INSTALL_LLAMA_PREBUILT.ApprovedReleaseChecksums
hydrate_source_tree = INSTALL_LLAMA_PREBUILT.hydrate_source_tree
validate_prebuilt_choice = INSTALL_LLAMA_PREBUILT.validate_prebuilt_choice
activate_install_tree = INSTALL_LLAMA_PREBUILT.activate_install_tree
activate_staged_dir = INSTALL_LLAMA_PREBUILT.activate_staged_dir
create_install_staging_dir = INSTALL_LLAMA_PREBUILT.create_install_staging_dir
sha256_file = INSTALL_LLAMA_PREBUILT.sha256_file
source_archive_logical_name = INSTALL_LLAMA_PREBUILT.source_archive_logical_name
install_prebuilt = INSTALL_LLAMA_PREBUILT.install_prebuilt
write_prebuilt_metadata = INSTALL_LLAMA_PREBUILT.write_prebuilt_metadata
existing_install_matches_plan = INSTALL_LLAMA_PREBUILT.existing_install_matches_plan
existing_install_matches_choice = INSTALL_LLAMA_PREBUILT.existing_install_matches_choice
ensure_diffusion_visual_server = INSTALL_LLAMA_PREBUILT.ensure_diffusion_visual_server
def linux_host() -> HostInfo:
return HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
def approved_release_checksums_for_asset(asset_name: str, sha256: str) -> ApprovedReleaseChecksums:
return ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "b9334",
upstream_tag = "b9334",
artifacts = {
asset_name: ApprovedArtifactHash(
asset_name = asset_name,
sha256 = sha256,
repo = "unslothai/llama.cpp",
kind = "diffusion-visual-server",
)
},
)
def approved_checksums_for(
upstream_tag: str, *, source_archive: Path, bundle_archive: Path, bundle_name: str
) -> ApprovedReleaseChecksums:
return ApprovedReleaseChecksums(
repo = "local",
release_tag = upstream_tag,
upstream_tag = upstream_tag,
source_commit = None,
artifacts = {
source_archive_logical_name(upstream_tag): ApprovedArtifactHash(
asset_name = source_archive_logical_name(upstream_tag),
sha256 = sha256_file(source_archive),
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
bundle_name: ApprovedArtifactHash(
asset_name = bundle_name,
sha256 = sha256_file(bundle_archive),
repo = "local",
kind = "local-test-bundle",
),
},
)
def test_extract_archive_allows_safe_tar_symlink_chain(tmp_path: Path):
archive_path = tmp_path / "bundle.tar.gz"
payload = b"shared-object"
with tarfile.open(archive_path, "w:gz") as archive:
versioned = tarfile.TarInfo("libllama.so.0.0.1")
versioned.size = len(payload)
archive.addfile(versioned, io_bytes(payload))
soname = tarfile.TarInfo("libllama.so.0")
soname.type = tarfile.SYMTYPE
soname.linkname = "libllama.so.0.0.1"
archive.addfile(soname)
linker_name = tarfile.TarInfo("libllama.so")
linker_name.type = tarfile.SYMTYPE
linker_name.linkname = "libllama.so.0"
archive.addfile(linker_name)
destination = tmp_path / "extract"
extract_archive(archive_path, destination)
assert (destination / "libllama.so.0.0.1").read_bytes() == payload
assert (destination / "libllama.so.0").is_symlink()
assert (destination / "libllama.so").is_symlink()
assert (destination / "libllama.so").resolve().read_bytes() == payload
def test_extract_archive_allows_safe_tar_hardlink(tmp_path: Path):
archive_path = tmp_path / "bundle.tar.gz"
payload = b"quantize"
with tarfile.open(archive_path, "w:gz") as archive:
target = tarfile.TarInfo("llama-quantize")
target.size = len(payload)
archive.addfile(target, io_bytes(payload))
hardlink = tarfile.TarInfo("llama-quantize-copy")
hardlink.type = tarfile.LNKTYPE
hardlink.linkname = "llama-quantize"
archive.addfile(hardlink)
destination = tmp_path / "extract"
extract_archive(archive_path, destination)
assert (destination / "llama-quantize-copy").read_bytes() == payload
assert not (destination / "llama-quantize-copy").is_symlink()
def test_extract_archive_rejects_absolute_tar_symlink_target(tmp_path: Path):
archive_path = tmp_path / "bundle.tar.gz"
with tarfile.open(archive_path, "w:gz") as archive:
entry = tarfile.TarInfo("libllama.so")
entry.type = tarfile.SYMTYPE
entry.linkname = "/tmp/libllama.so.0"
archive.addfile(entry)
with pytest.raises(PrebuiltFallback, match = "archive link used an absolute target"):
extract_archive(archive_path, tmp_path / "extract")
def test_extract_archive_rejects_escaping_tar_symlink_target(tmp_path: Path):
archive_path = tmp_path / "bundle.tar.gz"
with tarfile.open(archive_path, "w:gz") as archive:
entry = tarfile.TarInfo("libllama.so")
entry.type = tarfile.SYMTYPE
entry.linkname = "../outside/libllama.so.0"
archive.addfile(entry)
with pytest.raises(PrebuiltFallback, match = "archive link escaped destination"):
extract_archive(archive_path, tmp_path / "extract")
def test_extract_archive_rejects_unresolved_tar_symlink_target(tmp_path: Path):
archive_path = tmp_path / "bundle.tar.gz"
with tarfile.open(archive_path, "w:gz") as archive:
entry = tarfile.TarInfo("libllama.so")
entry.type = tarfile.SYMTYPE
entry.linkname = "libllama.so.0"
archive.addfile(entry)
with pytest.raises(PrebuiltFallback, match = "unresolved link entries"):
extract_archive(archive_path, tmp_path / "extract")
def test_extract_archive_rejects_zip_symlink_entry(tmp_path: Path):
archive_path = tmp_path / "bundle.zip"
with zipfile.ZipFile(archive_path, "w") as archive:
info = zipfile.ZipInfo("libllama.so")
info.create_system = 3
info.external_attr = 0o120777 << 16
archive.writestr(info, "libllama.so.0")
with pytest.raises(PrebuiltFallback, match = "zip archive contained a symlink entry"):
extract_archive(archive_path, tmp_path / "extract")
def test_hydrate_source_tree_extracts_upstream_archive_contents(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
upstream_tag = "b9999"
archive_path = tmp_path / "llama.cpp-source.tar.gz"
with tarfile.open(archive_path, "w:gz") as archive:
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/CMakeLists.txt",
b"cmake_minimum_required(VERSION 3.14)\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/convert_hf_to_gguf.py",
b"#!/usr/bin/env python3\nimport gguf\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/gguf-py/gguf/__init__.py",
b"__all__ = []\n",
)
source_urls = set(INSTALL_LLAMA_PREBUILT.upstream_source_archive_urls(upstream_tag))
def fake_download_file(url: str, destination: Path) -> None:
assert url in source_urls
destination.write_bytes(archive_path.read_bytes())
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
install_dir = tmp_path / "install"
work_dir = tmp_path / "work"
work_dir.mkdir()
hydrate_source_tree(
upstream_tag, install_dir, work_dir, expected_sha256 = sha256_file(archive_path)
)
assert (install_dir / "CMakeLists.txt").exists()
assert (install_dir / "convert_hf_to_gguf.py").exists()
assert (install_dir / "gguf-py" / "gguf" / "__init__.py").exists()
assert not (install_dir / f"llama.cpp-{upstream_tag}").exists()
def test_release_asset_download_url():
fn = INSTALL_LLAMA_PREBUILT.release_asset_download_url
assert fn(
"unslothai/llama.cpp", "b9000-mix-abc1234", "llama.cpp-source-commit-deadbeef.tar.gz"
) == (
"https://github.com/unslothai/llama.cpp/releases/download/"
"b9000-mix-abc1234/llama.cpp-source-commit-deadbeef.tar.gz"
)
# Any missing component -> None (no asset url, caller falls back to codeload).
assert fn(None, "b9000", "x.tar.gz") is None
assert fn("unslothai/llama.cpp", None, "x.tar.gz") is None
assert fn("unslothai/llama.cpp", "b9000", None) is None
def _mk_source_tarball(path: Path, tag: str) -> None:
with tarfile.open(path, "w:gz") as archive:
add_bytes_to_tar(
archive, f"llama.cpp-{tag}/CMakeLists.txt", b"cmake_minimum_required(VERSION 3.14)\n"
)
add_bytes_to_tar(
archive,
f"llama.cpp-{tag}/convert_hf_to_gguf.py",
b"#!/usr/bin/env python3\nimport gguf\n",
)
add_bytes_to_tar(archive, f"llama.cpp-{tag}/gguf-py/gguf/__init__.py", b"__all__ = []\n")
def test_hydrate_source_tree_prefers_release_asset_for_mix(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
# A mix build's merge commit 404s on codeload, so hydrate must fetch the release asset.
commit = "a" * 40
archive_path = tmp_path / "merged-source.tar.gz"
_mk_source_tarball(archive_path, f"b9000-mix-{commit[:7]}")
asset_url = INSTALL_LLAMA_PREBUILT.release_asset_download_url(
"unslothai/llama.cpp", "b9000-mix-abc1234", f"llama.cpp-source-commit-{commit}.tar.gz"
)
codeload_urls = set(
INSTALL_LLAMA_PREBUILT.commit_source_archive_urls("unslothai/llama.cpp", commit)
)
seen = []
def fake_download_file(url: str, destination: Path) -> None:
seen.append(url)
if url in codeload_urls:
raise AssertionError("codeload was hit even though the release asset was available")
assert url == asset_url
destination.write_bytes(archive_path.read_bytes())
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
install_dir = tmp_path / "install"
work_dir = tmp_path / "work"
work_dir.mkdir()
hydrate_source_tree(
commit,
install_dir,
work_dir,
source_repo = "unslothai/llama.cpp",
expected_sha256 = sha256_file(archive_path),
exact_source = True,
asset_url = asset_url,
)
assert seen == [asset_url]
assert (install_dir / "CMakeLists.txt").exists()
assert (install_dir / "convert_hf_to_gguf.py").exists()
def test_hydrate_source_tree_falls_back_to_codeload_when_asset_missing(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
# If the release asset 404s, fall back to codeload/archive (vanilla path).
commit = "b" * 40
archive_path = tmp_path / "vanilla-source.tar.gz"
_mk_source_tarball(archive_path, f"commit-{commit[:7]}")
asset_url = INSTALL_LLAMA_PREBUILT.release_asset_download_url(
"unslothai/llama.cpp", "b9000", f"llama.cpp-source-commit-{commit}.tar.gz"
)
codeload_urls = INSTALL_LLAMA_PREBUILT.commit_source_archive_urls("unslothai/llama.cpp", commit)
def fake_download_file(url: str, destination: Path) -> None:
if url == asset_url:
raise RuntimeError("404 Not Found")
assert url in codeload_urls
destination.write_bytes(archive_path.read_bytes())
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
install_dir = tmp_path / "install"
work_dir = tmp_path / "work"
work_dir.mkdir()
hydrate_source_tree(
commit,
install_dir,
work_dir,
source_repo = "unslothai/llama.cpp",
expected_sha256 = sha256_file(archive_path),
exact_source = True,
asset_url = asset_url,
)
assert (install_dir / "CMakeLists.txt").exists()
def test_validate_prebuilt_choice_creates_repo_shaped_linux_install(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
upstream_tag = "b9998"
bundle_name = "app-b9998-linux-x64-cuda13-newer.tar.gz"
source_archive = tmp_path / "source.tar.gz"
bundle_archive = tmp_path / "bundle.tar.gz"
with tarfile.open(source_archive, "w:gz") as archive:
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/CMakeLists.txt",
b"cmake_minimum_required(VERSION 3.14)\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/convert_hf_to_gguf.py",
b"#!/usr/bin/env python3\nimport gguf\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/gguf-py/gguf/__init__.py",
b"__all__ = []\n",
)
with tarfile.open(bundle_archive, "w:gz") as archive:
add_bytes_to_tar(archive, "llama-server", b"#!/bin/sh\nexit 0\n", mode = 0o755)
add_bytes_to_tar(archive, "llama-quantize", b"#!/bin/sh\nexit 0\n", mode = 0o755)
add_bytes_to_tar(archive, "libllama.so.0.0.1", b"libllama")
add_symlink_to_tar(archive, "libllama.so.0", "libllama.so.0.0.1")
add_symlink_to_tar(archive, "libllama.so", "libllama.so.0")
add_bytes_to_tar(archive, "libggml.so.0.9.8", b"libggml")
add_symlink_to_tar(archive, "libggml.so.0", "libggml.so.0.9.8")
add_symlink_to_tar(archive, "libggml.so", "libggml.so.0")
add_bytes_to_tar(archive, "libggml-base.so.0.9.8", b"libggml-base")
add_symlink_to_tar(archive, "libggml-base.so.0", "libggml-base.so.0.9.8")
add_symlink_to_tar(archive, "libggml-base.so", "libggml-base.so.0")
add_bytes_to_tar(archive, "libggml-cpu-x64.so.0.9.8", b"libggml-cpu")
add_symlink_to_tar(archive, "libggml-cpu-x64.so.0", "libggml-cpu-x64.so.0.9.8")
add_symlink_to_tar(archive, "libggml-cpu-x64.so", "libggml-cpu-x64.so.0")
add_bytes_to_tar(archive, "libmtmd.so.0.0.1", b"libmtmd")
add_symlink_to_tar(archive, "libmtmd.so.0", "libmtmd.so.0.0.1")
add_symlink_to_tar(archive, "libmtmd.so", "libmtmd.so.0")
add_bytes_to_tar(archive, "BUILD_INFO.txt", b"bundle metadata\n")
add_bytes_to_tar(archive, "THIRD_PARTY_LICENSES.txt", b"licenses\n")
source_urls = set(INSTALL_LLAMA_PREBUILT.upstream_source_archive_urls(upstream_tag))
def fake_download_file(url: str, destination: Path) -> None:
if url in source_urls:
destination.write_bytes(source_archive.read_bytes())
return
if url == "file://bundle":
destination.write_bytes(bundle_archive.read_bytes())
return
raise AssertionError(f"unexpected download url: {url}")
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_bytes",
lambda url, **_: b"#!/usr/bin/env python3\nimport gguf\n",
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"preflight_linux_installed_binaries",
lambda *args, **kwargs: None,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "validate_quantize", lambda *args, **kwargs: None)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "validate_server", lambda *args, **kwargs: None)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "local",
tag = upstream_tag,
name = bundle_name,
url = "file://bundle",
source_label = "local",
is_ready_bundle = True,
install_kind = "linux-cuda",
bundle_profile = "cuda13-newer",
runtime_line = "cuda13",
expected_sha256 = sha256_file(bundle_archive),
)
install_dir = tmp_path / "install"
work_dir = tmp_path / "work"
work_dir.mkdir()
probe_path = tmp_path / "stories260K.gguf"
quantized_path = tmp_path / "stories260K-q4.gguf"
validate_prebuilt_choice(
choice,
host,
install_dir,
work_dir,
probe_path,
requested_tag = upstream_tag,
llama_tag = upstream_tag,
release_tag = upstream_tag,
approved_checksums = approved_checksums_for(
upstream_tag,
source_archive = source_archive,
bundle_archive = bundle_archive,
bundle_name = bundle_name,
),
prebuilt_fallback_used = False,
quantized_path = quantized_path,
)
assert (install_dir / "gguf-py" / "gguf" / "__init__.py").exists()
assert (install_dir / "convert_hf_to_gguf.py").exists()
assert (install_dir / "build" / "bin" / "llama-server").exists()
assert (install_dir / "build" / "bin" / "llama-quantize").exists()
assert (install_dir / "build" / "bin" / "libllama.so").exists()
assert (install_dir / "llama-server").exists()
assert (install_dir / "llama-quantize").exists()
assert (install_dir / "UNSLOTH_PREBUILT_INFO.json").exists()
assert (install_dir / "BUILD_INFO.txt").exists()
def test_validate_prebuilt_choice_creates_repo_shaped_windows_install(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
upstream_tag = "b9997"
bundle_name = "app-b9997-windows-x64-cpu.zip"
source_archive = tmp_path / "source.tar.gz"
bundle_archive = tmp_path / "bundle.zip"
with tarfile.open(source_archive, "w:gz") as archive:
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/CMakeLists.txt",
b"cmake_minimum_required(VERSION 3.14)\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/convert_hf_to_gguf.py",
b"#!/usr/bin/env python3\nimport gguf\n",
)
add_bytes_to_tar(
archive,
f"llama.cpp-{upstream_tag}/gguf-py/gguf/__init__.py",
b"__all__ = []\n",
)
with zipfile.ZipFile(bundle_archive, "w") as archive:
archive.writestr("llama-server.exe", b"MZ")
archive.writestr("llama-quantize.exe", b"MZ")
archive.writestr("llama.dll", b"DLL")
archive.writestr("BUILD_INFO.txt", b"bundle metadata\n")
source_urls = set(INSTALL_LLAMA_PREBUILT.upstream_source_archive_urls(upstream_tag))
def fake_download_file(url: str, destination: Path) -> None:
if url in source_urls:
destination.write_bytes(source_archive.read_bytes())
return
if url == "file://bundle.zip":
destination.write_bytes(bundle_archive.read_bytes())
return
raise AssertionError(f"unexpected download url: {url}")
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_bytes",
lambda url, **_: b"#!/usr/bin/env python3\nimport gguf\n",
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"preflight_linux_installed_binaries",
lambda *args, **kwargs: None,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "validate_quantize", lambda *args, **kwargs: None)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "validate_server", lambda *args, **kwargs: None)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "local",
tag = upstream_tag,
name = bundle_name,
url = "file://bundle.zip",
source_label = "local",
is_ready_bundle = True,
install_kind = "windows-cpu",
expected_sha256 = sha256_file(bundle_archive),
)
install_dir = tmp_path / "install"
work_dir = tmp_path / "work"
work_dir.mkdir()
probe_path = tmp_path / "stories260K.gguf"
quantized_path = tmp_path / "stories260K-q4.gguf"
validate_prebuilt_choice(
choice,
host,
install_dir,
work_dir,
probe_path,
requested_tag = upstream_tag,
llama_tag = upstream_tag,
release_tag = upstream_tag,
approved_checksums = approved_checksums_for(
upstream_tag,
source_archive = source_archive,
bundle_archive = bundle_archive,
bundle_name = bundle_name,
),
prebuilt_fallback_used = False,
quantized_path = quantized_path,
)
assert (install_dir / "gguf-py" / "gguf" / "__init__.py").exists()
assert (install_dir / "convert_hf_to_gguf.py").exists()
assert (install_dir / "build" / "bin" / "Release" / "llama-server.exe").exists()
assert (install_dir / "build" / "bin" / "Release" / "llama-quantize.exe").exists()
assert (install_dir / "build" / "bin" / "Release" / "llama.dll").exists()
assert not (install_dir / "llama-server.exe").exists()
assert (install_dir / "UNSLOTH_PREBUILT_INFO.json").exists()
assert (install_dir / "BUILD_INFO.txt").exists()
def test_activate_install_tree_restores_existing_install_after_activation_failure(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
(install_dir / "old.txt").write_text("old install\n")
staging_dir = create_install_staging_dir(install_dir)
(staging_dir / "new.txt").write_text("new install\n")
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"confirm_install_tree",
lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("activation confirm failed")),
)
with pytest.raises(
PrebuiltFallback,
match = "activation failed; restored previous install",
):
activate_install_tree(staging_dir, install_dir, host)
assert (install_dir / "old.txt").read_text() == "old install\n"
assert not (install_dir / "new.txt").exists()
assert not staging_dir.exists()
assert not (tmp_path / ".staging").exists()
captured = capsys.readouterr()
output = captured.out + captured.err
assert "moving existing install to rollback path" in output
assert "restored previous install from rollback path" in output
def test_activate_install_tree_cleans_all_paths_when_rollback_restore_fails(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
(install_dir / "old.txt").write_text("old install\n")
staging_dir = create_install_staging_dir(install_dir)
(staging_dir / "new.txt").write_text("new install\n")
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"confirm_install_tree",
lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("activation confirm failed")),
)
original_replace = INSTALL_LLAMA_PREBUILT.os.replace
def flaky_replace(src, dst):
src_path = Path(src)
dst_path = Path(dst)
if "rollback-" in src_path.name and dst_path == install_dir:
raise OSError("restore failed")
return original_replace(src, dst)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT.os, "replace", flaky_replace)
with pytest.raises(
PrebuiltFallback,
match = "activation and rollback failed; cleaned install state for fresh source build",
):
activate_install_tree(staging_dir, install_dir, host)
assert not install_dir.exists()
assert not staging_dir.exists()
assert not (tmp_path / ".staging").exists()
captured = capsys.readouterr()
output = captured.out + captured.err
assert "rollback after failed activation also failed: restore failed" in output
assert "cleaning staging, install, and rollback paths before source build fallback" in output
assert "removing failed install path" in output
assert "removing rollback path" in output
def test_activate_staged_dir_copies_when_replace_hits_busy_lock(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
):
staging_dir = tmp_path / "llama.cpp.staging-test"
(staging_dir / "bin").mkdir(parents = True)
(staging_dir / "bin" / "ggml-base.dll").write_bytes(b"fake dll")
dst = tmp_path / "llama.cpp"
def denied_replace(src, dst_arg):
raise PermissionError(errno.EACCES, "Access is denied", str(src))
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT.os, "replace", denied_replace)
activate_staged_dir(staging_dir, dst)
assert (dst / "bin" / "ggml-base.dll").read_bytes() == b"fake dll"
assert not staging_dir.exists()
captured = capsys.readouterr()
assert "falling back to file-by-file copy" in captured.out + captured.err
def test_activate_staged_dir_reraises_non_busy_errors(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
staging_dir = tmp_path / "llama.cpp.staging-test"
staging_dir.mkdir()
(staging_dir / "new.txt").write_text("new install\n")
dst = tmp_path / "llama.cpp"
def out_of_space_replace(src, dst_arg):
raise OSError(errno.ENOSPC, "No space left on device", str(src))
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT.os, "replace", out_of_space_replace)
with pytest.raises(OSError, match = "No space left on device"):
activate_staged_dir(staging_dir, dst)
assert not dst.exists()
assert (staging_dir / "new.txt").read_text() == "new install\n"
def test_binary_env_linux_includes_binary_parent_in_ld_library_path(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
bin_dir = install_dir / "build" / "bin"
bin_dir.mkdir(parents = True)
binary_path = bin_dir / "llama-server"
binary_path.write_bytes(b"fake")
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "linux_runtime_dirs", lambda _bp: [])
env = binary_env(binary_path, install_dir, host)
ld_dirs = env["LD_LIBRARY_PATH"].split(os.pathsep)
assert (
str(bin_dir) in ld_dirs
), f"binary_path.parent ({bin_dir}) must be in LD_LIBRARY_PATH, got: {ld_dirs}"
assert str(install_dir) in ld_dirs
def test_scrub_env_drops_secrets_and_keeps_runtime_vars():
raw = {
# secrets
"HF_TOKEN": "hf_x",
"HUGGING_FACE_HUB_TOKEN": "hf_y",
"GH_TOKEN": "gh_x",
"GITHUB_TOKEN": "gh_y",
"WANDB_API_KEY": "wandb_x",
"AWS_SECRET_ACCESS_KEY": "aws_x",
"ACTIONS_ID_TOKEN_REQUEST_TOKEN": "oidc_x",
"ACTIONS_ID_TOKEN_REQUEST_URL": "https://oidc",
"SOME_VENDOR_API_KEY": "vendor_x",
"DB_PASSWORD": "pw",
"MY_PRIVATE_KEY": "pk",
"KUBECONFIG": "/home/runner/.kube/config",
"SSH_AUTH_SOCK": "/tmp/ssh-agent.sock",
"SSH_PASSPHRASE": "ssh_pass",
# runtime vars to keep
"PATH": "/usr/bin",
"LD_LIBRARY_PATH": "/opt/lib",
"DYLD_LIBRARY_PATH": "/opt/dyld",
"HOME": "/home/runner",
"TMPDIR": "/tmp",
"CUDA_VISIBLE_DEVICES": "0",
"HSA_OVERRIDE_GFX_VERSION": "11.0.0",
}
cleaned = scrub_env(raw)
for secret in (
"HF_TOKEN",
"HUGGING_FACE_HUB_TOKEN",
"GH_TOKEN",
"GITHUB_TOKEN",
"WANDB_API_KEY",
"AWS_SECRET_ACCESS_KEY",
"ACTIONS_ID_TOKEN_REQUEST_TOKEN",
"ACTIONS_ID_TOKEN_REQUEST_URL",
"SOME_VENDOR_API_KEY",
"DB_PASSWORD",
"MY_PRIVATE_KEY",
"KUBECONFIG",
"SSH_AUTH_SOCK",
"SSH_PASSPHRASE",
):
assert secret not in cleaned, f"{secret} must be stripped from binary env"
for keep in (
"PATH",
"LD_LIBRARY_PATH",
"DYLD_LIBRARY_PATH",
"HOME",
"TMPDIR",
"CUDA_VISIBLE_DEVICES",
"HSA_OVERRIDE_GFX_VERSION",
):
assert cleaned[keep] == raw[keep], f"{keep} must be preserved for the binary"
# no bare "KEY" marker: benign KEY-containing names survive
assert is_secret_env_name("API_KEY") is True
assert is_secret_env_name("SSH_KEYFILE_PATH") is False
assert is_secret_env_name("PATH") is False
def test_scrub_env_drops_proxy_index_and_embedded_url_credentials():
raw = {
# proxy / package-index URLs whose values commonly embed credentials
"HTTPS_PROXY": "https://user:secret@proxy:8080",
"https_proxy": "https://user:secret@proxy:8080", # lower-case variant
"ALL_PROXY": "socks5://user:secret@proxy:1080",
"PIP_INDEX_URL": "https://u:p@pypi.internal/simple",
"UV_INDEX_URL": "https://u:p@index.internal/simple",
# credentials embedded in an otherwise benign-named variable's value
"MY_DB_DSN": "postgres://admin:secret@db:5432/app",
# benign vars the binary needs, including a URL with no userinfo
"PATH": "/usr/bin",
"CUDA_VISIBLE_DEVICES": "0",
"NO_PROXY": "localhost,127.0.0.1",
"SOME_ENDPOINT": "https://example.com:8080/v1",
}
cleaned = scrub_env(raw)
for secret in (
"HTTPS_PROXY",
"https_proxy",
"ALL_PROXY",
"PIP_INDEX_URL",
"UV_INDEX_URL",
"MY_DB_DSN",
):
assert secret not in cleaned, f"{secret} must be stripped from binary env"
for keep in ("PATH", "CUDA_VISIBLE_DEVICES", "NO_PROXY", "SOME_ENDPOINT"):
assert cleaned[keep] == raw[keep], f"{keep} must be preserved for the binary"
assert is_secret_env_name("HTTPS_PROXY") is True
assert is_secret_env_name("https_proxy") is True
assert is_secret_env_name("NO_PROXY") is False
def test_binary_env_strips_secrets_from_downloaded_binary_environment(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
bin_dir = install_dir / "build" / "bin"
bin_dir.mkdir(parents = True)
binary_path = bin_dir / "llama-server"
binary_path.write_bytes(b"fake")
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "linux_runtime_dirs", lambda _bp: [])
monkeypatch.setenv("HF_TOKEN", "hf_secret_from_ci")
monkeypatch.setenv("GITHUB_TOKEN", "gh_secret_from_ci")
monkeypatch.setenv("GH_TOKEN", "gh_secret_from_ci")
monkeypatch.setenv("WANDB_API_KEY", "wandb_secret_from_ci")
monkeypatch.setenv("CUDA_VISIBLE_DEVICES", "1")
env = binary_env(binary_path, install_dir, host)
assert "HF_TOKEN" not in env
assert "GITHUB_TOKEN" not in env
assert "GH_TOKEN" not in env
assert "WANDB_API_KEY" not in env
# library/runtime resolution unaffected
assert str(bin_dir) in env["LD_LIBRARY_PATH"].split(os.pathsep)
assert env["CUDA_VISIBLE_DEVICES"] == "1"
def test_binary_env_redirects_home_away_from_real_credential_stores(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
bin_dir = install_dir / "build" / "bin"
bin_dir.mkdir(parents = True)
binary_path = bin_dir / "llama-server"
binary_path.write_bytes(b"fake")
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "linux_runtime_dirs", lambda _bp: [])
real_home = str(tmp_path / "real_home")
monkeypatch.setenv("HOME", real_home)
monkeypatch.setenv("HF_HOME", real_home + "/.cache/huggingface")
env = binary_env(binary_path, install_dir, host)
# HOME and the cache pointers are redirected to a single empty, existing dir.
assert env["HOME"] != real_home
assert env["HF_HOME"] == env["HOME"]
assert env["HOME"] == isolated_runtime_home()
assert os.path.isdir(env["HOME"])
assert os.listdir(env["HOME"]) == []
# Windows reconstructs the profile from HOMEDRIVE + HOMEPATH.
assert env["HOMEDRIVE"] + env["HOMEPATH"] == env["HOME"]
def test_scrub_env_drops_token_only_url_userinfo():
raw = {
"GENERIC_REPO": "https://ghp_tokenonly@github.com/org/repo",
"GENERIC_OK": "https://example.com:8080/v1",
}
cleaned = scrub_env(raw)
assert "GENERIC_REPO" not in cleaned
assert cleaned["GENERIC_OK"] == raw["GENERIC_OK"]
def test_binary_env_drops_explicit_credential_file_pointers(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "linux_runtime_dirs", lambda _bp: [])
dropped = (
"NETRC",
"PIP_CONFIG_FILE",
"DOCKER_CONFIG",
"GIT_CONFIG_GLOBAL",
"GITHUB_ENV",
"GITHUB_PATH",
"GITHUB_OUTPUT",
"GITHUB_STEP_SUMMARY",
"BASH_ENV",
)
for var in dropped:
monkeypatch.setenv(var, "/home/realuser/secret")
env = binary_env(tmp_path / "llama-server", tmp_path, host)
for var in dropped:
assert var not in env
def test_linux_runtime_dirs_probes_with_secret_free_env(monkeypatch: pytest.MonkeyPatch):
captured: dict[str, object] = {}
def fake_missing(binary_path, *, env = None):
captured["env"] = env
return []
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "linux_missing_libraries", fake_missing)
monkeypatch.setenv("HF_TOKEN", "hf_secret")
monkeypatch.setenv("GITHUB_TOKEN", "gh_secret")
INSTALL_LLAMA_PREBUILT.linux_runtime_dirs(Path("/fake/llama-server"))
probe_env = captured["env"]
assert probe_env is not None
assert "HF_TOKEN" not in probe_env
assert "GITHUB_TOKEN" not in probe_env
def test_install_prebuilt_falls_back_to_older_release_plan(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
first_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "old-release",
name = "app-b9002-linux-x64.tar.gz",
url = "https://example.com/app-b9002-linux-x64.tar.gz",
source_label = "published",
install_kind = "linux-cpu",
)
second_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "older-release",
name = "app-b9001-linux-x64.tar.gz",
url = "https://example.com/app-b9001-linux-x64.tar.gz",
source_label = "published",
install_kind = "linux-cpu",
)
first_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9002",
release_tag = "release-2",
attempts = [first_choice],
approved_checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-2",
upstream_tag = "b9002",
source_commit = None,
artifacts = {},
),
)
second_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [second_choice],
approved_checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = None,
artifacts = {},
),
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[first_plan, second_plan],
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_validation_model",
lambda probe_path, cache_path: probe_path.write_bytes(b"probe"),
)
call_log: list[tuple[str, bool]] = []
def fake_validate(
attempts,
host,
install_dir,
work_dir,
probe_path,
*,
requested_tag,
llama_tag,
release_tag,
approved_checksums,
initial_fallback_used = False,
existing_install_dir = None,
):
call_log.append((llama_tag, initial_fallback_used))
if llama_tag == "b9002":
raise PrebuiltFallback("validation failed for latest release")
staging_dir = create_install_staging_dir(install_dir)
(staging_dir / "marker.txt").write_text("ready\n")
return attempts[0], staging_dir, initial_fallback_used
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"validate_prebuilt_attempts",
fake_validate,
)
activated = {}
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"activate_install_tree",
lambda staging_dir, install_dir, host: activated.update(
{"staging_dir": staging_dir, "install_dir": install_dir}
),
)
ensured_tags: list[str] = []
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"ensure_converter_scripts",
lambda install_dir, llama_tag: ensured_tags.append(llama_tag),
)
install_prebuilt(install_dir, "latest", "unslothai/llama.cpp", "")
assert call_log == [("b9002", False), ("b9001", True)]
assert activated["install_dir"] == install_dir
assert ensured_tags == ["b9001"]
def write_linux_install_shape(install_dir: Path) -> None:
runtime_dir = install_dir / "build" / "bin"
runtime_dir.mkdir(parents = True, exist_ok = True)
(install_dir / "llama-server").write_text("#!/bin/sh\n", encoding = "utf-8")
(install_dir / "llama-quantize").write_text("#!/bin/sh\n", encoding = "utf-8")
(runtime_dir / "llama-server").write_text("#!/bin/sh\n", encoding = "utf-8")
(runtime_dir / "llama-quantize").write_text("#!/bin/sh\n", encoding = "utf-8")
# libllama-common.so* (PR #5135) is a required runtime payload health group.
(runtime_dir / "libllama-common.so.0").write_bytes(b"DLL")
(runtime_dir / "libllama.so.0").write_bytes(b"DLL")
(runtime_dir / "libggml.so.0").write_bytes(b"DLL")
(runtime_dir / "libggml-base.so.0").write_bytes(b"DLL")
(runtime_dir / "libggml-cpu-x64.so.0").write_bytes(b"DLL")
(runtime_dir / "libmtmd.so.0").write_bytes(b"DLL")
(install_dir / "convert_hf_to_gguf.py").write_text("#!/usr/bin/env python3\n", encoding = "utf-8")
(install_dir / "gguf-py" / "gguf").mkdir(parents = True, exist_ok = True)
def write_windows_install_shape(
install_dir: Path,
*,
include_llama_dll: bool = True,
include_cuda_dll: bool = False,
include_cudart_dlls: bool = False,
) -> None:
runtime_dir = install_dir / "build" / "bin" / "Release"
runtime_dir.mkdir(parents = True, exist_ok = True)
(runtime_dir / "llama-server.exe").write_bytes(b"MZ")
(runtime_dir / "llama-quantize.exe").write_bytes(b"MZ")
if include_llama_dll:
(runtime_dir / "llama.dll").write_bytes(b"DLL")
if include_cuda_dll:
(runtime_dir / "ggml-cuda.dll").write_bytes(b"DLL")
if include_cudart_dlls:
# cudart bundle DLLs that ship in cudart-llama-bin-win-cuda-*-x64.zip
(runtime_dir / "cudart64_12.dll").write_bytes(b"DLL")
(runtime_dir / "cublas64_12.dll").write_bytes(b"DLL")
(runtime_dir / "cublasLt64_12.dll").write_bytes(b"DLL")
(install_dir / "convert_hf_to_gguf.py").write_text("#!/usr/bin/env python3\n", encoding = "utf-8")
(install_dir / "gguf-py" / "gguf").mkdir(parents = True, exist_ok = True)
def write_macos_install_shape(
install_dir: Path,
*,
include_libllama: bool = True,
include_libggml: bool = True,
include_libmtmd: bool = True,
) -> None:
runtime_dir = install_dir / "build" / "bin"
runtime_dir.mkdir(parents = True, exist_ok = True)
(install_dir / "llama-server").write_text("#!/bin/sh\n", encoding = "utf-8")
(install_dir / "llama-quantize").write_text("#!/bin/sh\n", encoding = "utf-8")
(runtime_dir / "llama-server").write_text("#!/bin/sh\n", encoding = "utf-8")
(runtime_dir / "llama-quantize").write_text("#!/bin/sh\n", encoding = "utf-8")
if include_libllama:
(runtime_dir / "libllama.0.dylib").write_bytes(b"DLL")
if include_libggml:
(runtime_dir / "libggml.0.dylib").write_bytes(b"DLL")
if include_libmtmd:
(runtime_dir / "libmtmd.0.dylib").write_bytes(b"DLL")
(install_dir / "convert_hf_to_gguf.py").write_text("#!/usr/bin/env python3\n", encoding = "utf-8")
(install_dir / "gguf-py" / "gguf").mkdir(parents = True, exist_ok = True)
def test_existing_install_matches_plan_with_fingerprint_linux(tmp_path: Path):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
assert existing_install_matches_plan(install_dir, host, plan) is True
def test_existing_install_matches_plan_false_without_fingerprint(tmp_path: Path):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
(install_dir / "UNSLOTH_PREBUILT_INFO.json").write_text(
json.dumps({"tag": "b9001", "asset": "llama-b9001-bin-ubuntu-x64.tar.gz"}) + "\n",
encoding = "utf-8",
)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/x.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
assert existing_install_matches_plan(install_dir, host, plan) is False
def test_existing_install_matches_plan_false_with_malformed_metadata(tmp_path: Path):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
(install_dir / "UNSLOTH_PREBUILT_INFO.json").write_text("{not-json\n", encoding = "utf-8")
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/x.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
assert existing_install_matches_plan(install_dir, host, plan) is False
def test_existing_install_matches_plan_windows_cpu_requires_llama_dll(tmp_path: Path):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_windows_install_shape(install_dir, include_llama_dll = True)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-win-cpu-x64.zip",
url = "https://example.com/x.zip",
source_label = "published",
install_kind = "windows-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
assert existing_install_matches_plan(install_dir, host, plan) is True
(install_dir / "build" / "bin" / "Release" / "llama.dll").unlink()
assert existing_install_matches_plan(install_dir, host, plan) is False
def test_existing_install_matches_plan_windows_cuda_requires_cuda_dll(tmp_path: Path):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_windows_install_shape(install_dir, include_llama_dll = True, include_cuda_dll = True)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (12, 4),
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = True,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-win-cuda-12.4-x64.zip",
url = "https://example.com/x.zip",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
assert existing_install_matches_plan(install_dir, host, plan) is True
(install_dir / "build" / "bin" / "Release" / "ggml-cuda.dll").unlink()
assert existing_install_matches_plan(install_dir, host, plan) is False
def test_existing_install_matches_plan_windows_cuda_paired_requires_cudart(tmp_path: Path):
"""A paired cudart bundle (#5106) marks the install stale unless cudart64_* and cublas64_* are on disk."""
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_windows_install_shape(
install_dir,
include_llama_dll = True,
include_cuda_dll = True,
include_cudart_dlls = True,
)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (12, 4),
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = True,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-win-cuda-12.4-x64.zip",
url = "https://example.com/x.zip",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = "a" * 64,
runtime_name = "cudart-llama-bin-win-cuda-12.4-x64.zip",
runtime_url = "https://example.com/cudart.zip",
runtime_sha256 = "c" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
choice.runtime_name: ApprovedArtifactHash(
asset_name = choice.runtime_name,
sha256 = choice.runtime_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
# Fully populated install (main archive + cudart DLLs) matches.
assert existing_install_matches_plan(install_dir, host, plan) is True
# cublas missing -- stale, must reinstall.
(install_dir / "build" / "bin" / "Release" / "cublas64_12.dll").unlink()
assert existing_install_matches_plan(install_dir, host, plan) is False
# cudart missing -- stale, must reinstall.
write_windows_install_shape(
install_dir,
include_llama_dll = True,
include_cuda_dll = True,
include_cudart_dlls = True,
)
(install_dir / "build" / "bin" / "Release" / "cudart64_12.dll").unlink()
assert existing_install_matches_plan(install_dir, host, plan) is False
# cublasLt missing -- stale, must reinstall (all three DLLs are required).
write_windows_install_shape(
install_dir,
include_llama_dll = True,
include_cuda_dll = True,
include_cudart_dlls = True,
)
(install_dir / "build" / "bin" / "Release" / "cublasLt64_12.dll").unlink()
assert existing_install_matches_plan(install_dir, host, plan) is False
def test_existing_install_matches_plan_windows_cuda_unpaired_skips_cudart_check(tmp_path: Path):
"""With no paired runtime archive, a legacy install lacking cudart must still pass (else reinstall loops)."""
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_windows_install_shape(
install_dir,
include_llama_dll = True,
include_cuda_dll = True,
include_cudart_dlls = False,
)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (12, 4),
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = True,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-win-cuda-12.4-x64.zip",
url = "https://example.com/x.zip",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
assert existing_install_matches_plan(install_dir, host, plan) is True
def test_existing_install_fingerprint_changes_when_cudart_pair_added(tmp_path: Path):
"""A pre-#5322 CUDA install must go stale once the choice gains a runtime archive (#5106 fingerprint half)."""
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_windows_install_shape(
install_dir,
include_llama_dll = True,
include_cuda_dll = True,
include_cudart_dlls = False,
)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (12, 4),
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = True,
)
legacy_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-win-cuda-12.4-x64.zip",
url = "https://example.com/x.zip",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = "a" * 64,
)
paired_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-win-cuda-12.4-x64.zip",
url = "https://example.com/x.zip",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = "a" * 64,
runtime_name = "cudart-llama-bin-win-cuda-12.4-x64.zip",
runtime_url = "https://example.com/cudart.zip",
runtime_sha256 = "c" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
legacy_choice.name: ApprovedArtifactHash(
asset_name = legacy_choice.name,
sha256 = legacy_choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
paired_choice.runtime_name: ApprovedArtifactHash(
asset_name = paired_choice.runtime_name,
sha256 = paired_choice.runtime_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
},
)
# Metadata written for the legacy (no-pair) choice.
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = legacy_choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
# The paired choice's fingerprint must differ from the legacy one so the install refreshes.
legacy_fingerprint = INSTALL_LLAMA_PREBUILT.expected_install_fingerprint(
llama_tag = "b9001",
release_tag = "release-1",
choice = legacy_choice,
approved_checksums = checksums,
)
paired_fingerprint = INSTALL_LLAMA_PREBUILT.expected_install_fingerprint(
llama_tag = "b9001",
release_tag = "release-1",
choice = paired_choice,
approved_checksums = checksums,
)
assert legacy_fingerprint != paired_fingerprint, (
"expected_install_fingerprint must hash runtime_name/runtime_sha256 "
"so pre-#5322 installs are not falsely considered up-to-date"
)
paired_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [paired_choice],
approved_checksums = checksums,
)
assert existing_install_matches_plan(install_dir, host, paired_plan) is False
def test_existing_install_matches_plan_macos_requires_dylibs(tmp_path: Path):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_macos_install_shape(install_dir)
host = HostInfo(
system = "Darwin",
machine = "arm64",
is_windows = False,
is_linux = False,
is_macos = True,
is_x86_64 = False,
is_arm64 = True,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-macos-arm64.tar.gz",
url = "https://example.com/x.tar.gz",
source_label = "published",
install_kind = "macos-arm64",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
assert existing_install_matches_plan(install_dir, host, plan) is True
(install_dir / "build" / "bin" / "libggml.0.dylib").unlink()
assert existing_install_matches_plan(install_dir, host, plan) is False
def test_install_prebuilt_skips_download_when_existing_install_matches(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[plan],
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_validation_model",
lambda *args, **kwargs: (_ for _ in ()).throw(
AssertionError("matching install should skip before validation model download")
),
)
install_prebuilt(install_dir, "latest", "unslothai/llama.cpp", "")
def test_install_prebuilt_does_not_skip_unhealthy_existing_install(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
(install_dir / "llama-quantize").unlink()
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[plan],
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_validation_model",
lambda *args, **kwargs: (_ for _ in ()).throw(
AssertionError("unhealthy install must continue into normal install flow")
),
)
with pytest.raises(
AssertionError, match = "unhealthy install must continue into normal install flow"
):
install_prebuilt(install_dir, "latest", "unslothai/llama.cpp", "")
def test_install_prebuilt_skips_when_older_release_fallback_matches_existing_install(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
latest_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-2",
name = "llama-b9002-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9002-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "c" * 64,
)
fallback_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
latest_checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-2",
upstream_tag = "b9002",
source_commit = "beadfeed",
artifacts = {
source_archive_logical_name("b9002"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9002"),
sha256 = "d" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
latest_choice.name: ApprovedArtifactHash(
asset_name = latest_choice.name,
sha256 = latest_choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
fallback_checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
fallback_choice.name: ApprovedArtifactHash(
asset_name = fallback_choice.name,
sha256 = fallback_choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
latest_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9002",
release_tag = "release-2",
attempts = [latest_choice],
approved_checksums = latest_checksums,
)
fallback_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [fallback_choice],
approved_checksums = fallback_checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = fallback_choice,
approved_checksums = fallback_checksums,
prebuilt_fallback_used = True,
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[latest_plan, fallback_plan],
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_validation_model",
lambda probe_path, cache_path: probe_path.write_bytes(b"probe"),
)
call_log: list[str] = []
def fake_validate(
attempts,
host,
install_dir,
work_dir,
probe_path,
*,
requested_tag,
llama_tag,
release_tag,
approved_checksums,
initial_fallback_used = False,
existing_install_dir = None,
):
call_log.append(llama_tag)
raise PrebuiltFallback("validation failed for latest release")
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"validate_prebuilt_attempts",
fake_validate,
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"activate_install_tree",
lambda *args, **kwargs: (_ for _ in ()).throw(
AssertionError("matching fallback install should not reactivate")
),
)
install_prebuilt(install_dir, "latest", "unslothai/llama.cpp", "")
assert call_log == ["b9002"]
def test_install_prebuilt_skips_same_release_fallback_attempt_when_installed(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
first_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64-bad.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64-bad.tar.gz",
source_label = "published",
install_kind = "linux-cpu",
expected_sha256 = "c" * 64,
)
fallback_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64-good.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64-good.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
first_choice.name: ApprovedArtifactHash(
asset_name = first_choice.name,
sha256 = first_choice.expected_sha256,
repo = "unslothai/llama.cpp",
kind = "prebuilt",
),
fallback_choice.name: ApprovedArtifactHash(
asset_name = fallback_choice.name,
sha256 = fallback_choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [first_choice, fallback_choice],
approved_checksums = checksums,
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = fallback_choice,
approved_checksums = checksums,
prebuilt_fallback_used = True,
)
assert (
existing_install_matches_choice(
install_dir,
host,
llama_tag = "b9001",
release_tag = "release-1",
choice = fallback_choice,
approved_checksums = checksums,
)
is True
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[plan],
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_validation_model",
lambda probe_path, cache_path: probe_path.write_bytes(b"probe"),
)
attempted_names: list[str] = []
def fake_validate_choice(
choice,
host,
staging_dir,
work_dir,
probe_path,
*,
requested_tag,
llama_tag,
release_tag,
approved_checksums,
prebuilt_fallback_used,
quantized_path,
):
attempted_names.append(choice.name)
if choice.name == first_choice.name:
raise PrebuiltFallback("newest candidate failed")
raise AssertionError("installed fallback candidate should have been skipped")
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"validate_prebuilt_choice",
fake_validate_choice,
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"activate_install_tree",
lambda *args, **kwargs: (_ for _ in ()).throw(
AssertionError("installed fallback candidate should not be activated")
),
)
install_prebuilt(install_dir, "latest", "unslothai/llama.cpp", "")
assert attempted_names == [first_choice.name]
def test_install_prebuilt_same_tag_upstream_failure_uses_older_unsloth_release_plan(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
):
install_dir = tmp_path / "llama.cpp"
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
same_tag_upstream_choice = AssetChoice(
repo = "ggml-org/llama.cpp",
tag = "b9002",
name = "llama-b9002-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9002-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
older_release_choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "b" * 64,
)
latest_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9002",
release_tag = "release-2",
attempts = [same_tag_upstream_choice],
approved_checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-2",
upstream_tag = "b9002",
source_commit = None,
artifacts = {},
),
)
older_plan = INSTALL_LLAMA_PREBUILT.InstallReleasePlan(
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
attempts = [older_release_choice],
approved_checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = None,
artifacts = {},
),
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "detect_host", lambda: host)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"resolve_simple_install_release_plans",
lambda llama_tag, host, published_repo, published_release_tag: (
"latest",
[latest_plan, older_plan],
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"download_validation_model",
lambda probe_path, cache_path: probe_path.write_bytes(b"probe"),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"latest_upstream_release_tag",
lambda: (_ for _ in ()).throw(
AssertionError("install fallback should not walk upstream releases")
),
)
attempted = []
def fake_validate(
attempts,
host,
install_dir,
work_dir,
probe_path,
*,
requested_tag,
llama_tag,
release_tag,
approved_checksums,
initial_fallback_used = False,
existing_install_dir = None,
):
attempted.append((llama_tag, release_tag, attempts[0].source_label))
if llama_tag == "b9002":
raise PrebuiltFallback("same-tag upstream asset failed validation")
staging_dir = create_install_staging_dir(install_dir)
(staging_dir / "marker.txt").write_text("ready\n")
return attempts[0], staging_dir, initial_fallback_used
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "validate_prebuilt_attempts", fake_validate)
activated = {}
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"activate_install_tree",
lambda staging_dir, install_dir, host: activated.update(
{"staging_dir": staging_dir, "install_dir": install_dir}
),
)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"ensure_converter_scripts",
lambda install_dir, llama_tag: None,
)
install_prebuilt(install_dir, "latest", "unslothai/llama.cpp", "")
assert attempted == [("b9002", "release-2", "upstream"), ("b9001", "release-1", "upstream")]
assert activated["install_dir"] == install_dir
def io_bytes(data: bytes):
return io.BytesIO(data)
def add_bytes_to_tar(
archive: tarfile.TarFile,
name: str,
data: bytes,
*,
mode: int = 0o644,
) -> None:
info = tarfile.TarInfo(name)
info.size = len(data)
info.mode = mode
archive.addfile(info, io_bytes(data))
def add_symlink_to_tar(archive: tarfile.TarFile, name: str, target: str) -> None:
info = tarfile.TarInfo(name)
info.type = tarfile.SYMTYPE
info.linkname = target
archive.addfile(info)
def test_existing_install_matches_choice_fails_when_install_tree_incomplete(tmp_path: Path):
"""confirm_install_tree guard rejects installs missing critical files."""
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_linux_install_shape(install_dir)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-ubuntu-x64.tar.gz",
url = "https://example.com/llama-b9001-bin-ubuntu-x64.tar.gz",
source_label = "upstream",
install_kind = "linux-cpu",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
# Full install should match
assert (
existing_install_matches_choice(
install_dir,
host,
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
)
is True
)
# Remove convert_hf_to_gguf.py (confirm_install_tree checks it; runtime health does not).
(install_dir / "convert_hf_to_gguf.py").unlink()
assert (
existing_install_matches_choice(
install_dir,
host,
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
)
is False
)
def test_existing_install_matches_choice_fails_when_install_tree_incomplete_macos(tmp_path: Path):
"""confirm_install_tree guard rejects macOS arm64 installs missing critical files."""
install_dir = tmp_path / "llama.cpp"
install_dir.mkdir()
write_macos_install_shape(install_dir)
host = HostInfo(
system = "Darwin",
machine = "arm64",
is_windows = False,
is_linux = False,
is_macos = True,
is_x86_64 = False,
is_arm64 = True,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
)
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = "llama-b9001-bin-macos-arm64.tar.gz",
url = "https://example.com/llama-b9001-bin-macos-arm64.tar.gz",
source_label = "upstream",
install_kind = "macos-arm64",
expected_sha256 = "a" * 64,
)
checksums = ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "release-1",
upstream_tag = "b9001",
source_commit = "deadbeef",
artifacts = {
source_archive_logical_name("b9001"): ApprovedArtifactHash(
asset_name = source_archive_logical_name("b9001"),
sha256 = "b" * 64,
repo = "ggml-org/llama.cpp",
kind = "upstream-source",
),
choice.name: ApprovedArtifactHash(
asset_name = choice.name,
sha256 = choice.expected_sha256,
repo = "ggml-org/llama.cpp",
kind = "upstream-prebuilt",
),
},
)
write_prebuilt_metadata(
install_dir,
requested_tag = "latest",
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
prebuilt_fallback_used = False,
)
# Full install should match
assert (
existing_install_matches_choice(
install_dir,
host,
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
)
is True
)
# Remove a macOS-specific runtime artifact and verify the guard catches it
(install_dir / "build" / "bin" / "libmtmd.0.dylib").unlink()
assert (
existing_install_matches_choice(
install_dir,
host,
llama_tag = "b9001",
release_tag = "release-1",
choice = choice,
approved_checksums = checksums,
)
is False
)
def test_paired_runtime_dll_patterns_excludes_executables() -> None:
"""The paired runtime archive must contribute only CUDA DLLs (no *.exe/*.dll) so it can't overwrite binaries."""
paired_runtime_dll_patterns = INSTALL_LLAMA_PREBUILT.paired_runtime_dll_patterns
paired_choice = AssetChoice(
repo = "x",
tag = "t",
name = "llama-b9001-bin-win-cuda-12.4-x64.zip",
url = "u",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = "a" * 64,
runtime_name = "cudart-llama-bin-win-cuda-12.4-x64.zip",
runtime_url = "https://example.com/cudart.zip",
runtime_sha256 = "c" * 64,
)
patterns = paired_runtime_dll_patterns(paired_choice)
assert "cudart64_*.dll" in patterns
assert "cublas64_*.dll" in patterns
assert "cublasLt64_*.dll" in patterns
assert "*.exe" not in patterns
assert "*.dll" not in patterns
for kind in (
"linux-cpu",
"linux-cuda",
"linux-rocm",
"macos-arm64",
"macos-x64",
"windows-cpu",
"windows-hip",
):
non_windows = AssetChoice(
repo = "x",
tag = "t",
name = "x",
url = "u",
source_label = "published",
install_kind = kind,
expected_sha256 = "a" * 64,
)
assert paired_runtime_dll_patterns(non_windows) == []
def test_runtime_overlay_cannot_overwrite_main_archive_payload(tmp_path: Path) -> None:
"""A malformed runtime archive with llama-server.exe must NOT replace the main archive's binary."""
install_from_archives = INSTALL_LLAMA_PREBUILT.install_from_archives
work = tmp_path / "work"
install = tmp_path / "install"
archives = tmp_path / "archives"
work.mkdir()
install.mkdir()
archives.mkdir()
main_zip = archives / "llama-b9001-bin-win-cuda-12.4-x64.zip"
runtime_zip = archives / "cudart-llama-bin-win-cuda-12.4-x64.zip"
with zipfile.ZipFile(main_zip, "w", zipfile.ZIP_DEFLATED) as zf:
zf.writestr("llama-server.exe", b"MAIN-SERVER")
zf.writestr("llama-quantize.exe", b"MAIN-Q")
zf.writestr("llama.dll", b"DLL-llama")
zf.writestr("ggml-cuda.dll", b"DLL-ggml")
import hashlib
main_sha = hashlib.sha256(main_zip.read_bytes()).hexdigest()
with zipfile.ZipFile(runtime_zip, "w", zipfile.ZIP_DEFLATED) as zf:
zf.writestr("cudart64_12.dll", b"DLL-cudart")
zf.writestr("cublas64_12.dll", b"DLL-cublas")
zf.writestr("cublasLt64_12.dll", b"DLL-cublasLt")
zf.writestr("llama-server.exe", b"RUNTIME-OVERWRITE")
runtime_sha = hashlib.sha256(runtime_zip.read_bytes()).hexdigest()
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "release-1",
name = main_zip.name,
url = f"https://example.com/{main_zip.name}",
source_label = "published",
install_kind = "windows-cuda",
runtime_line = "cuda12",
expected_sha256 = main_sha,
runtime_name = runtime_zip.name,
runtime_url = f"https://example.com/{runtime_zip.name}",
runtime_sha256 = runtime_sha,
)
host = HostInfo(
system = "Windows",
machine = "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (12, 4),
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = True,
)
import shutil as _shutil
orig_download = INSTALL_LLAMA_PREBUILT.download_file_verified
def fake_download(
url,
target_path,
*,
expected_sha256 = None,
label = None,
**kw,
):
src = main_zip if "cudart" not in url else runtime_zip
_shutil.copy2(src, target_path)
if expected_sha256:
actual = hashlib.sha256(Path(target_path).read_bytes()).hexdigest()
if actual != expected_sha256:
raise INSTALL_LLAMA_PREBUILT.PrebuiltFallback(f"sha256 mismatch on {label}")
INSTALL_LLAMA_PREBUILT.download_file_verified = fake_download
try:
install_from_archives(choice, host, install, work)
finally:
INSTALL_LLAMA_PREBUILT.download_file_verified = orig_download
release_dir = install / "build" / "bin" / "Release"
server = release_dir / "llama-server.exe"
assert server.exists()
assert server.read_bytes() == b"MAIN-SERVER", (
"runtime archive overwrote main llama-server.exe; " f"got {server.read_bytes()!r}"
)
for name in ("cudart64_12.dll", "cublas64_12.dll", "cublasLt64_12.dll"):
assert (release_dir / name).exists(), f"missing {name}"
def test_linux_runtime_overlay_copies_llama_tool_impl_libraries(tmp_path: Path) -> None:
install_from_archives = INSTALL_LLAMA_PREBUILT.install_from_archives
work = tmp_path / "work"
install = tmp_path / "install"
archives = tmp_path / "archives"
work.mkdir()
install.mkdir()
archives.mkdir()
bundle = archives / "app-b9334-linux-x64-cuda13-newer.tar.gz"
with tarfile.open(bundle, "w:gz") as archive:
for name in (
"llama-cli",
"llama-server",
"llama-quantize",
"libllama-cli-impl.so",
"libllama-server-impl.so",
"libllama-quantize-impl.so",
"libllama-common.so",
"libllama.so",
"libggml.so",
"libggml-base.so",
"libmtmd.so",
"libggml-cpu-x64.so",
"libggml-cuda.so",
):
payload = f"{name}\n".encode()
member = tarfile.TarInfo(name)
member.size = len(payload)
archive.addfile(member, io.BytesIO(payload))
import hashlib
import shutil as _shutil
bundle_sha = hashlib.sha256(bundle.read_bytes()).hexdigest()
choice = AssetChoice(
repo = "unslothai/llama.cpp",
tag = "b9334",
name = bundle.name,
url = f"https://example.com/{bundle.name}",
source_label = "published",
install_kind = "linux-cuda",
runtime_line = "cuda13",
expected_sha256 = bundle_sha,
)
host = HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = (13, 0),
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = True,
has_usable_nvidia = True,
)
orig_download = INSTALL_LLAMA_PREBUILT.download_file_verified
def fake_download(
url,
target_path,
*,
expected_sha256 = None,
label = None,
**kw,
):
_shutil.copy2(bundle, target_path)
if expected_sha256:
actual = hashlib.sha256(Path(target_path).read_bytes()).hexdigest()
if actual != expected_sha256:
raise INSTALL_LLAMA_PREBUILT.PrebuiltFallback(f"sha256 mismatch on {label}")
INSTALL_LLAMA_PREBUILT.download_file_verified = fake_download
try:
install_from_archives(choice, host, install, work)
finally:
INSTALL_LLAMA_PREBUILT.download_file_verified = orig_download
runtime_dir = install / "build" / "bin"
for name in (
"libllama-cli-impl.so",
"libllama-server-impl.so",
"libllama-quantize-impl.so",
):
assert (runtime_dir / name).exists(), f"missing {name}"
assert not (runtime_dir / "llama-cli").exists()
def test_python_runtime_dirs_covers_cu13_and_library_bin(monkeypatch, tmp_path: Path) -> None:
"""Installer DLL discovery must scan the same path set as the backend (cu12/cu13/conda layouts + torch/lib)."""
import site as _site
python_runtime_dirs = INSTALL_LLAMA_PREBUILT.python_runtime_dirs
site_dir = tmp_path / "Lib" / "site-packages"
# cu12-style modular wheel
cu12_bin = site_dir / "nvidia" / "cuda_runtime" / "bin"
cu12_bin.mkdir(parents = True)
# cu13-style unsuffixed wheel
cu13_arch = site_dir / "nvidia" / "cu13" / "bin" / "x86_64"
cu13_arch.mkdir(parents = True)
# conda-style repack
library_bin = site_dir / "nvidia" / "cublas" / "Library" / "bin"
library_bin.mkdir(parents = True)
# PyTorch bundled-CUDA wheel
torch_lib = site_dir / "torch" / "lib"
torch_lib.mkdir(parents = True)
monkeypatch.setattr(sys, "path", [str(site_dir)])
monkeypatch.setattr(_site, "getsitepackages", lambda: [str(site_dir)])
monkeypatch.setattr(_site, "getusersitepackages", lambda: "")
dirs = python_runtime_dirs()
assert str(cu12_bin) in dirs
assert str(cu13_arch) in dirs
assert str(library_bin) in dirs
assert str(torch_lib) in dirs
def _nvidia_linux_host():
return HostInfo(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = ["10.0"],
visible_cuda_devices = None,
has_physical_nvidia = True,
has_usable_nvidia = True,
)
def _run_validate_prebuilt_choice(monkeypatch, tmp_path, *, expected_sha256):
"""Run validate_prebuilt_choice with heavy steps stubbed; return the quantize/server smoke-test call counts."""
calls = {"quantize": 0, "server": 0}
server_path = tmp_path / "install" / "build" / "bin" / "llama-server"
quantize_path = tmp_path / "install" / "build" / "bin" / "llama-quantize"
src = INSTALL_LLAMA_PREBUILT
monkeypatch.setattr(
src, "preferred_source_archive", lambda *a, **k: ("repo", "ref", None, False)
)
monkeypatch.setattr(src, "hydrate_source_tree", lambda *a, **k: None)
monkeypatch.setattr(src, "install_from_archives", lambda *a, **k: (server_path, quantize_path))
monkeypatch.setattr(src, "preflight_linux_installed_binaries", lambda *a, **k: None)
monkeypatch.setattr(src, "preflight_macos_installed_binaries", lambda *a, **k: None)
monkeypatch.setattr(src, "ensure_repo_shape", lambda *a, **k: None)
monkeypatch.setattr(src, "write_prebuilt_metadata", lambda *a, **k: None)
monkeypatch.setattr(
src,
"validate_quantize",
lambda *a, **k: calls.__setitem__("quantize", calls["quantize"] + 1),
)
monkeypatch.setattr(
src, "validate_server", lambda *a, **k: calls.__setitem__("server", calls["server"] + 1)
)
bundle_name = "app-b9998-linux-x64-cuda13-newer.tar.gz"
source_archive = tmp_path / "source.tar.gz"
bundle_archive = tmp_path / "bundle.tar.gz"
source_archive.write_bytes(b"source")
bundle_archive.write_bytes(b"bundle")
choice = AssetChoice(
repo = "local",
tag = "b9998",
name = bundle_name,
url = "file://bundle",
source_label = "local",
is_ready_bundle = True,
install_kind = "linux-cuda",
bundle_profile = "cuda13-newer",
runtime_line = "cuda13",
expected_sha256 = expected_sha256,
)
src.validate_prebuilt_choice(
choice,
_nvidia_linux_host(),
tmp_path / "install",
tmp_path / "work",
tmp_path / "stories260K.gguf",
requested_tag = "b9998",
llama_tag = "b9998",
release_tag = "b9998",
approved_checksums = approved_checksums_for(
"b9998",
source_archive = source_archive,
bundle_archive = bundle_archive,
bundle_name = bundle_name,
),
prebuilt_fallback_used = False,
quantized_path = tmp_path / "stories260K-q4.gguf",
)
return calls
def test_validate_prebuilt_choice_approved_validation_skipped_when_flag_off(tmp_path, monkeypatch):
# An approved (sha256-verified) bundle skips the smoke test while the flag is off.
calls = _run_validate_prebuilt_choice(monkeypatch, tmp_path, expected_sha256 = "ab" * 32)
assert calls == {"quantize": 0, "server": 0}
def test_validate_prebuilt_choice_hashless_build_always_validated(tmp_path, monkeypatch):
# A hashless build has no sha256 gate, so the smoke test must run even with the flag off.
calls = _run_validate_prebuilt_choice(monkeypatch, tmp_path, expected_sha256 = None)
assert calls == {"quantize": 1, "server": 1}
def test_validate_prebuilt_choice_approved_validation_runs_when_flag_enabled(tmp_path, monkeypatch):
# _RUN_STAGED_PREBUILT_VALIDATION back on restores the smoke test for approved bundles too.
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "_RUN_STAGED_PREBUILT_VALIDATION", True)
calls = _run_validate_prebuilt_choice(monkeypatch, tmp_path, expected_sha256 = "ab" * 32)
assert calls == {"quantize": 1, "server": 1}
def test_diffusion_visual_server_uses_approved_checksum_download(monkeypatch, tmp_path: Path):
asset_name = "llama-diffusion-gemma-visual-server-linux-x64"
expected_sha = "a" * 64
asset_url = "https://github.com/unslothai/llama.cpp/releases/download/b9334/" + asset_name
calls: list[tuple[str, Path, str | None, str | None]] = []
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"github_release_assets",
lambda repo, tag: {asset_name: asset_url},
)
def fake_download_file(url, destination):
raise AssertionError("diffusion visual server must not use unverified download_file")
def fake_download_file_verified(url, destination, *, expected_sha256, label):
calls.append((url, Path(destination), expected_sha256, label))
Path(destination).write_bytes(b"verified visual server")
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT, "download_file_verified", fake_download_file_verified
)
ensure_diffusion_visual_server(
tmp_path / "install",
linux_host(),
"b9334",
approved_release_checksums_for_asset(asset_name, expected_sha),
)
target = tmp_path / "install" / "build" / "bin" / "llama-diffusion-gemma-visual-server"
assert calls == [
(
asset_url,
target,
expected_sha,
f"diffusion visual server {asset_name}",
)
]
assert target.read_bytes() == b"verified visual server"
assert target.stat().st_mode & 0o777 == 0o755
def test_diffusion_visual_server_refuses_unapproved_release_asset(monkeypatch, tmp_path: Path):
asset_name = "llama-diffusion-gemma-visual-server-attacker-linux"
verified_calls: list[str] = []
raw_calls: list[str] = []
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"github_release_assets",
lambda repo, tag: {asset_name: "https://example.test/" + asset_name},
)
def fake_download_file(url, destination):
raw_calls.append(url)
def fake_download_file_verified(url, destination, *, expected_sha256, label):
verified_calls.append(url)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "download_file", fake_download_file)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT, "download_file_verified", fake_download_file_verified
)
ensure_diffusion_visual_server(
tmp_path / "install",
linux_host(),
"b9334",
ApprovedReleaseChecksums(
repo = "unslothai/llama.cpp",
release_tag = "b9334",
upstream_tag = "b9334",
artifacts = {},
),
)
target = tmp_path / "install" / "build" / "bin" / "llama-diffusion-gemma-visual-server"
assert not target.exists()
assert raw_calls == []
assert verified_calls == []
Reference in a new issue View git blame Copy permalink