* Studio: match the Linux llama.cpp prebuilt to the runtime cudart major
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Run the Linux llama.cpp prebuilt dependency preflight before reusing an existing install, so cached trees missing newly required per-tool shared libraries (libllama-server-impl.so / libllama-quantize-impl.so introduced upstream between b9279 and b9283) trigger a repair instead of silently skipping reinstall and failing at runtime. Adds a regression test for the new lib*-impl.so overlay layout.
* allow validation of custom releases to pass
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* require exact source provenance for branch direct linux releases
Mirror validated_checksums_for_bundle so incomplete checksum metadata
on a branch/pull/commit release fails closed with a clear error instead
of silently degrading to the legacy master-as-tag source hydration path
that this PR is meant to eliminate. Guard fires when source_commit,
the exact source archive hash, or a derivable source repo URL is
missing from the approved metadata.
Also set plan.llama_tag to approved_checksums.upstream_tag so the
ensure_converter_scripts fallback and the install fingerprint target
the concrete upstream tag (e.g. b9174) rather than the moving branch
label inferred from asset names (master). Legacy b#### releases are
unaffected: synthetic checksums already set upstream_tag to
bundle.upstream_tag, so the swap is a no-op on that path.
Add parametrized negative regression coverage for the three ways
exact provenance can be incomplete (missing source_commit, missing
exact source archive entry, missing source_repo) and update the
existing branch happy-path test to expect b9174.
* revert llama_tag swap to preserve install identity
Keep plan.llama_tag as bundle.upstream_tag (the branch label inferred
from asset names, e.g. master) rather than overriding it with the
approved metadata upstream tag (b9174). The override broke install
identity in two ways:
1. expected_install_fingerprint hashes upstream_tag = llama_tag, so
the same release would produce a different fingerprint depending on
which version of this code resolved it, causing spurious reinstalls
when users upgrade or roll back.
2. UNSLOTH_PREBUILT_INFO.json reports the value as the user-visible
record of which release was installed; tools and logs should see
the requested branch label, not the compatibility tag.
The approved metadata still records upstream_tag = b9174 internally
for source archive lookup; the new assertion makes that explicit.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: download paired cudart bundle on Windows CUDA installs
Upstream ggml-org/llama.cpp publishes Windows CUDA in two archives
that the release notes explicitly say are both required:
llama-<tag>-bin-win-cuda-X.Y-x64.zip (binaries + ggml DLLs)
cudart-llama-bin-win-cuda-X.Y-x64.zip (cudart64, cublas64, cublasLt64)
Studio's installer was downloading only the first one. The
``runtime_name`` / ``runtime_url`` fields on AssetChoice existed but
were never populated, and ``install_from_archives`` only handled
``choice.url``. With the cudart DLLs missing from
``install_dir/build/bin/Release``, the prebuilt binary's LoadLibrary
calls only resolved at runtime when the user happened to have a
version-matched system CUDA toolkit on PATH. That is the underlying
cause for the Windows reports in #5106 ("GPU detected but model
loaded entirely on RAM"): the prebuilt's CUDA backend silently fails
to load and llama-server falls back to CPU regardless of ``-ngl`` or
``--fit on``.
Wires the pairing through end to end:
* ``windows_cuda_attempts`` and ``published_windows_cuda_attempts``
look up the matching ``cudart-llama-bin-win-cuda-X.Y-x64.zip``
asset URL alongside the main archive and store it as
``runtime_url`` / ``runtime_name`` on the AssetChoice. We only
pair when the selected main archive is the binary archive
(``llama-...zip``) so the legacy cudart-only naming path is
unaffected.
* ``apply_approved_hashes`` resolves the runtime archive's hash from
the approved manifest. If the manifest does not list the runtime
archive, the pairing is dropped rather than installing without
checksum coverage. Preserves the supply-chain guarantee for
published bundles; upstream installs with no manifest are
unaffected (same risk surface as the existing main-archive
download).
* ``install_from_archives`` now downloads the runtime archive into a
separate temp dir and runs ``copy_globs`` against both source dirs.
Separate dirs avoid the "ambiguous archive layout" guard tripping
on shared filenames like LICENSE.txt, while the second
``copy_globs`` overlay drops the cudart DLLs into the same
``install_dir/build/bin/Release`` directory as the main binary.
Adds a ``runtime_sha256`` field on AssetChoice to carry the
verified hash through to the download step, alongside the existing
``runtime_name`` / ``runtime_url`` slots.
Tests: 5 new cases in tests/studio/install/test_selection_logic.py:
* upstream pairing populates runtime_url / runtime_name
* graceful degrade when cudart asset is absent in the release
* legacy cudart-only naming path does not self-pair
* apply_approved_hashes threads runtime_sha256 when the manifest
lists it
* apply_approved_hashes drops the pair when the runtime hash is
missing rather than installing without verification
130 install tests pass (125 baseline + 5 new). No regressions.
Refs #5106
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be more succinct
* Studio: refresh installs that pre-date the paired cudart bundle
expected_install_fingerprint did not hash the new runtime_name /
runtime_sha256 fields, and runtime_payload_health_groups for windows-
cuda only checked llama.dll / ggml-cuda.dll. The combination meant that
an install made before this PR -- the exact installs reporting #5106 --
would still match the post-PR choice: same main asset name + sha, same
llama.dll, same ggml-cuda.dll, missing cudart64_*.dll, but
existing_install_matches_choice returned True and the cudart download
path in install_from_archives never ran. Fresh installs got the fix;
existing affected installs did not.
This commit:
* Adds runtime_asset and runtime_sha256 to the fingerprint payload so
any change to (or first introduction of) the cudart pair invalidates
pre-existing installs.
* Refactors write_prebuilt_metadata to call expected_install_fingerprint
so the recorded fingerprint cannot drift from the expected one when
new keys are added.
* Extends runtime_payload_health_groups for windows-cuda to require
cudart64_*.dll and cublas64_*.dll *only when the choice carries a
paired runtime archive*. Gating on choice.runtime_name keeps the
no-pair fallback path (manifest missing cudart hash, upstream
without paired bundle) from looping on reinstall.
New tests:
* test_existing_install_matches_plan_windows_cuda_paired_requires_cudart
-- paired choice rejects installs missing cudart / cublas.
* test_existing_install_matches_plan_windows_cuda_unpaired_skips_cudart_check
-- unpaired choice still accepts legacy cudart-less installs.
* test_existing_install_fingerprint_changes_when_cudart_pair_added
-- direct fingerprint mismatch between the legacy and paired choice.
Refs #5106
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten paired Windows CUDA install gates
Three follow-ups from a 12-reviewer batch over 526894a4 (PR #5322):
1. (12/12) Health check required cudart64_*.dll and cublas64_*.dll but
not cublasLt64_*.dll. The upstream cudart-llama-bin-win-cuda-X.Y-x64
bundle ships all three (verified against b9103 cuda-12.4 and
cuda-13.1: 3 DLLs, no executables), and a Windows install missing
any one of them still fails CUDA initialisation. Adding
cublasLt64_*.dll to runtime_payload_health_groups so a partial
install or a deletion of the third DLL triggers reinstall instead
of silently staying broken.
2. The runtime overlay copy used the same broad runtime_patterns_for_choice
set as the main archive (windows-cuda returns *.exe and *.dll). A
malformed runtime zip that contained a llama-server.exe alongside
the real cudart DLLs would have overwritten the main archive's
server binary. Introduced paired_runtime_dll_patterns() that
returns the cudart bundle's three specific filename patterns and
nothing else, and use that for the second copy_globs pass.
New end-to-end regression test packs a fake runtime zip with an
extra llama-server.exe and asserts the main binary survives.
3. (7/12) python_runtime_dirs in install_llama_prebuilt.py and
_windows_pip_nvidia_dll_dirs in llama_cpp.py walked different path
sets. The installer side missed nvidia/<pkg>/Library/bin (conda
layout) and nvidia/<pkg>/bin/x86_64 (current CUDA 13 unsuffixed
wheel layout), so preflight CUDA detection could fail even when
usable DLLs were present. Mirrored the same six-path set the
backend resolver uses, including arch subdirs.
New tests:
- test_paired_runtime_dll_patterns_excludes_executables
- test_runtime_overlay_cannot_overwrite_main_archive_payload (end-to-end)
- test_python_runtime_dirs_covers_cu13_and_library_bin
- extended test_existing_install_matches_plan_windows_cuda_paired_requires_cudart
with a cublasLt-missing case
Upstream cudart bundle contents verified empirically by downloading
the b9103 release artifacts directly: each cuda-X.Y bundle contains
exactly cudart64_X.dll + cublas64_X.dll + cublasLt64_X.dll, no exes.
Refs #5106
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix 14 stale tests under tests/studio/install/ that drifted from code
All 14 failures audited locally and tracked back to test-side drift
(no production-code regressions). After these test updates the entire
tests/studio/install/ directory now passes: 346 passed, 1 skipped.
Per failure:
tests/studio/install/test_install_llama_prebuilt_logic.py (5 fails):
* test_existing_install_matches_plan_with_fingerprint_linux
* test_install_prebuilt_skips_download_when_existing_install_matches
* test_install_prebuilt_skips_when_older_release_fallback_matches_existing_install
* test_install_prebuilt_skips_same_release_fallback_attempt_when_installed
* test_existing_install_matches_choice_fails_when_install_tree_incomplete
All five build a fake Linux install tree via write_linux_install_shape
and call existing_install_matches_choice. The matcher returns False
because runtime_payload_is_healthy now requires a libllama-common.so*
library in build/bin/ (added by PR #5135), and the fixture never wrote
it. Add the missing library to write_linux_install_shape; matcher
passes for all five tests.
tests/studio/install/test_rocm_support.py (8 fails after the partial
audit, one collection-tier flake):
* TestEnsureRocmTorch::test_cpu_torch_gets_rocm_reinstall and
TestEnsureRocmTorch::test_probe_timeout_triggers_reinstall
_ensure_rocm_torch was refactored to call pip_install for the
torch reinstall and pip_install_try (not pip_install) for the
follow-up bitsandbytes install. The tests still asserted
mock_pip.call_count == 2. Add a second @patch.object on
pip_install_try and split the assertions across the two mocks.
* TestInstallShStructure::test_cuda_precedence
Asserted file-position-of-string ordering: looked for
`if [ -z "$_smi" ]` before the first `amd-smi` literal in
install.sh. The installer now defines top-level helpers
`_has_amd_rocm_gpu` (uses `amd-smi`) and `_has_usable_nvidia_gpu`
(uses `nvidia-smi`) before either is called from
`get_torch_index_url`, so file-position ordering carries no
semantic meaning. Rewrite the test to extract the
`get_torch_index_url` body via a small brace-matched helper and
assert the runtime ordering: NVIDIA call sits before the
`if [ -z "$_smi" ]` branch and the AMD call sits inside it.
* TestLiveRegression::test_get_torch_index_url_returns_cuda_on_nvidia
Sed-extracted only get_torch_index_url and eval'd it -- but the
function calls _has_amd_rocm_gpu and _has_usable_nvidia_gpu, so
the eval'd body crashed and fell through to the CPU URL on a
fully-loaded NVIDIA host. Extract the helpers alongside the
function. Also pre-skip when nvidia-smi is on PATH but does not
list a GPU (containers occasionally ship the binary without a
driver).
* TestWorkerRocmMambaSsm::test_probe_script_has_getattr_hip and
TestWorkerRocmMambaSsm::test_probe_returns_hip_version_field
The wheel-resolver probe subprocess (the only place where
`getattr(torch.version, 'hip', None)` is emitted) was hoisted out
of worker.py into studio/backend/utils/wheel_utils.py during the
wheel-resolver refactor. Point the file-content assertions at
wheel_utils.py and assert worker.py still consumes the
`hip_version` field.
* TestHardwareAmdBranching::test_hardware_branches_on_is_rocm_for_utilization
TestHardwareAmdBranching::test_hardware_branches_on_is_rocm_for_visible
TestHardwareAmdBranching::test_hardware_branches_on_is_rocm_for_physical_count
hardware.py refactored: the IS_ROCM branch and direct
`from . import amd` were hoisted out of get_gpu_utilization /
get_visible_gpu_utilization into the shared `_smi_query`
dispatcher. Update the first two tests to assert the dispatcher
call shape (`_smi_query("get_primary_gpu_utilization", ...)` etc.)
plus IS_ROCM + amd-import in `_smi_query` itself. Update the
physical-count test to assert IS_ROCM + the literal `from . import
amd` as that function still imports amd directly rather than going
through `_smi_query`.
No production-code changes; tests-only.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Expand test coverage for install_llama_prebuilt.py:
- Add tests for source build plan resolution with custom repos
- Add tests for branch/commit/PR ref matching and normalization
- Add tests for manifest checksum validation
- Add tests for Windows CUDA upstream asset name patterns
- Update capsys checks to capture stderr after log() redirect
* Fix script unbound variable error
* remove stale test script, add llama.cpp metal source builds, update tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Metal precedence, test sync, and add behavioral tests
- Move macOS arm64 Metal check before CUDA/ROCm in GPU backend
decision chain so Metal is not bypassed when nvcc is in PATH
- Remove RPATH flags from CPU fallback CMAKE_ARGS (only needed
for Metal library linking)
- Update test_llama_pr_force_and_source.py to match _CLONE_ARGS
rename from _CLONE_BRANCH_ARGS in setup.sh
- Add confirm_install_tree guard test for
existing_install_matches_choice
- Add TestMacOSMetalBuildLogic bash subprocess tests verifying
Metal flag selection, nvcc precedence, and CPU fallback behavior
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Metal CPU fallback to also cover cmake build failures and update tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* 1. _GPU_BACKEND_FRAGMENT synced -- removed dead CPU_FALLBACK_CMAKE_ARGS= init (6/8)
2. RPATH assertion replaced -- new test_macos_arm64_cpu_fallback_args_exclude_rpath checks the actual runtime CPU_FALLBACK_CMAKE_ARGS output for @loader_path and -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON (6/8)
3. _TRY_METAL_CPU_FALLBACK=false reset after both configure-failure and build-failure fallback branches in setup.sh (4/8)
4. macOS test now removes libmtmd.0.dylib instead of the platform-agnostic convert_hf_to_gguf.py (3/8)
5. Empty-string tag test added -- test_empty_tag_omits_branch_flag for resolved_tag= (2/8)
6. RPATH checks on cmake call logs -- both fallback tests now assert @loader_path and -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON are absent from CPU fallback cmake calls, plus baseline flag preservation (multiple)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests clean up
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Split out from #4741 to keep the main PR focused on installer logic.
- New test_install_llama_prebuilt_logic.py: tests for resolve logic,
fallback behavior, env_int, busy/lock handling
- New test_validate_llama_prebuilt.py: validator tests for staged
release_tag/upstream_tag handling
- New test_llama_pr_force_and_source.py: tests for PR_FORCE and
LLAMA_SOURCE maintainer defaults
- Updated test_selection_logic.py: expanded selection/fallback coverage
- Updated test_pr4562_bugfixes.py: updated bugfix tests for new logic
- Updated smoke_test_llama_prebuilt.py: minor update
* Use prebuilt llama.cpp for unsloth studio setup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix 3 issues that cause unnecessary fallback to source build
1. Make filelock import optional -- environments without filelock
(e.g. minimal installs) crashed at import time instead of
gracefully skipping the lock.
2. Use already-verified converter script from the hydrated source
tree instead of re-downloading from raw.githubusercontent.com
with no checksum. Adds symlink with copy fallback for the
legacy filename.
3. Initialize $SkipPrebuiltInstall in setup.ps1 before first use
to prevent potential uninitialized variable errors.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep network fallback in ensure_converter_scripts
Prefer the local verified copy from the hydrated source tree, but
retain the original network download as a fallback if the file is
missing. Create the legacy hyphenated filename as a symlink with a
copy fallback instead of writing a second full copy.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix 4 bugs in source-build fallback and binary_env paths
- setup.ps1: Replace git pull + checkout FETCH_HEAD with fetch + checkout -B
to avoid detached HEAD state that breaks re-runs. Use pinned tag in both
fetch and clone paths.
- setup.sh: Move rm -rf after cmake/git prerequisite checks so a missing
tool no longer deletes the existing install. Add --branch tag to clone.
- install_llama_prebuilt.py: Add binary_path.parent to Linux LD_LIBRARY_PATH
in binary_env() so bundled .so files in build/bin are found even without
RPATH, matching the existing Windows PATH logic.
- Add test for binary_env LD_LIBRARY_PATH on Linux.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle unresolved "latest" tag in source-build fallback clone
When tag resolution fails and the requested tag is "latest", both
setup scripts now omit --branch from git clone so the default branch
is cloned instead of failing on a nonexistent "latest" branch/tag.
Similarly, the PS1 fetch path fetches the default ref when the tag
is "latest".
* Resolve actual latest ggml-org tag instead of using literal "latest"
When both Python tag resolution attempts fail and the requested tag
is "latest", query the GitHub API for the actual latest release tag
from ggml-org/llama.cpp (e.g. b8508) instead of passing the literal
string "latest" to git clone --branch, which would fail since no
such branch/tag exists.
setup.sh uses curl + python json parsing; setup.ps1 uses
Invoke-RestMethod. Both fall back to the raw requested tag if the
API call also fails.
* Try Unsloth release repo before ggml-org when resolving latest tag
When falling back to the GitHub API to resolve "latest", query the
Unsloth release repo (unslothai/llama.cpp) first since it has the
prebuilt binaries pinned to tested tags. Only fall back to
ggml-org/llama.cpp if the Unsloth repo query fails.
* Add comprehensive sandbox tests for PR #4562 bug fixes
35 tests covering all fixes across platforms:
- binary_env cross-platform (Linux LD_LIBRARY_PATH, Windows PATH,
macOS DYLD_LIBRARY_PATH) with edge cases (dedup, ordering, existing paths)
- resolve_requested_llama_tag (concrete, latest, None, empty)
- setup.sh logic via subprocess: prereq check ordering (cmake/git missing
preserves install), pinned tag in clone, fetch+checkout -B pattern,
fetch failure warns instead of aborting
- "latest" tag resolution fallback chain (Unsloth API -> ggml-org ->
raw) with mock curl: success, failure, malformed JSON, empty body,
empty tag_name, env overrides
- Source code pattern verification for both .sh and .ps1 files
All 138 tests pass in isolated uv venv.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add binary_path.parent to macOS DYLD_LIBRARY_PATH in binary_env
macOS prebuilt .dylib files are overlaid into build/bin (same as
Linux), but binary_env only added install_dir to DYLD_LIBRARY_PATH.
Add binary_path.parent so the loader can find sibling dylibs even
without embedded loader paths.
Mirrors the existing fix for Linux LD_LIBRARY_PATH and the Windows
PATH pattern.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard --branch when resolved tag is "latest"; fix broken test assertion
When all API fallbacks fail and the tag stays as literal "latest",
omit --branch from git clone (clones default branch instead of
failing). Both setup.sh and setup.ps1 now check for "latest" before
passing --branch to git clone/fetch.
Also fix test_setup_ps1_clone_uses_branch_tag which used Python
tuple syntax (assert "x", "y" in z) that always passes. Changed to
assert "x" in z and "y" in z.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix macOS DYLD trailing colon, install_lock no-op, and debug log
- binary_env macOS: use dedupe_existing_dirs instead of raw string
concatenation. Eliminates trailing colon in DYLD_LIBRARY_PATH
(which causes dyld to search CWD for libraries) and deduplicates
when binary_path.parent == install_dir. Now consistent with the
Linux and Windows branches.
- install_lock: when filelock is not installed, use os.O_CREAT|O_EXCL
as a fallback exclusive file lock with timeout, instead of yielding
with no locking. Prevents concurrent installs from corrupting each
other's staging directories.
- setup.ps1: remove [DEBUG] log line that printed to every user on
every Windows setup run.
* Add stale-lock detection and atomic clone-then-swap
install_lock fallback (no filelock): write PID to lock file and
check if the holder process is still alive on contention. Dead PIDs
(ProcessLookupError) and unreadable lock files trigger immediate
cleanup. Live processes owned by other users (PermissionError) are
correctly recognized as alive -- the lock is not removed.
setup.sh/setup.ps1 source-build: clone into a temporary directory
first, then swap into place only on success. If git clone fails,
the existing install is preserved instead of being deleted by the
premature rm -rf.
* Remove redundant upstream_tag != release_tag check
load_approved_release_checksums compared checksums.upstream_tag
against the Unsloth release_tag, which are different namespaces
(upstream ggml-org tag vs Unsloth published tag). This only worked
because both happened to be "b8508" by convention. Would break if
Unsloth ever uses a different release naming scheme.
The existing check at parse_approved_release_checksums (line 950)
already validates the release_tag field correctly.
* Fix lock TOCTOU race and build-in-temp-dir swap
install_lock fallback: add os.fsync(fd) after writing PID to ensure
the PID is visible to racing processes before they check. Treat
empty lock files (PID not yet written) as "wait and retry" instead
of stale, closing the window where two processes could both see an
empty file, both unlink it, and both acquire the lock.
setup.sh/setup.ps1 source-build: clone AND build in a temp directory
(LLAMA_CPP_DIR.build.$$). Only swap into the final LLAMA_CPP_DIR
after the build succeeds. If clone or cmake or build fails, the temp
dir is cleaned up and the existing working install is preserved.
Previously, rm -rf ran after clone but before build, destroying the
existing install even if the build later failed.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Daniel Han