install.sh previously set MAC_INTEL=true unconditionally when uname
reported x86_64 on macOS, even after detecting Apple Silicon via
sysctl. This caused ARM Mac users running under Rosetta to get the
Intel llama.cpp prebuilt, skip PyTorch, and use Python 3.12.
Now when Rosetta is detected, _ARCH is overridden to arm64 so the
rest of the installer (Python version, torch, llama.cpp prebuilt)
picks the correct architecture.
Also adds matching Rosetta detection in install_llama_prebuilt.py's
detect_host() so the prebuilt resolver works correctly even when
invoked independently under Rosetta.
* update logic to incorporate custom prebuilt installs
* bug fixes
* update for review comments
* fix tags
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Separate test changes from main PR
Move test file changes out of this PR to keep the diff focused on
the install_llama_prebuilt.py and setup script changes. Test updates
will be submitted in a follow-up PR.
* Fix branch ref normalization and harden JSON parsing
- Add checkout_friendly_ref() to strip refs/heads/ prefix from branch
refs before emitting them in SourceBuildPlan. git clone --branch does
not accept fully qualified refs like refs/heads/main.
- Apply normalization in source_build_plan_for_release() and the
direct-ref fallback in resolve_source_build_plan().
- Allow validated_checksums_for_bundle() to accept releases that carry
only an exact-commit source archive without the legacy upstream-tag
source tarball.
- Add 2>/dev/null || true guards to all inline python -c JSON parsing
in setup.sh so a malformed payload does not abort the script under
set -e.
* Fix Windows CUDA asset ordering and tag ref normalization
- Reorder windows_cuda_upstream_asset_names to prefer the main binary
archive (llama-{tag}-bin-win-cuda-*) over the cudart sidecar archive
(cudart-llama-bin-win-cuda-*). The cudart ZIP only contains CUDA
runtime DLLs, not llama-server or llama-quantize binaries.
- Extend checkout_friendly_ref to also strip refs/tags/ prefix for tag
refs, matching the refs/heads/ handling for branch refs.
* Simplify JSON parsing consistency in setup.sh
Use json.load(sys.stdin) consistently for all inline JSON parsing
in setup.sh, instead of the more complex json.loads(raw) pattern
on the install-tag resolution path. The 2>/dev/null || true guard
already handles empty/malformed input gracefully.
* Fix source build plan fallback for commit ref kind in PR #4771
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <daniel@unsloth.ai>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Fix script unbound variable error
* remove stale test script, add llama.cpp metal source builds, update tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Metal precedence, test sync, and add behavioral tests
- Move macOS arm64 Metal check before CUDA/ROCm in GPU backend
decision chain so Metal is not bypassed when nvcc is in PATH
- Remove RPATH flags from CPU fallback CMAKE_ARGS (only needed
for Metal library linking)
- Update test_llama_pr_force_and_source.py to match _CLONE_ARGS
rename from _CLONE_BRANCH_ARGS in setup.sh
- Add confirm_install_tree guard test for
existing_install_matches_choice
- Add TestMacOSMetalBuildLogic bash subprocess tests verifying
Metal flag selection, nvcc precedence, and CPU fallback behavior
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Metal CPU fallback to also cover cmake build failures and update tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* 1. _GPU_BACKEND_FRAGMENT synced -- removed dead CPU_FALLBACK_CMAKE_ARGS= init (6/8)
2. RPATH assertion replaced -- new test_macos_arm64_cpu_fallback_args_exclude_rpath checks the actual runtime CPU_FALLBACK_CMAKE_ARGS output for @loader_path and -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON (6/8)
3. _TRY_METAL_CPU_FALLBACK=false reset after both configure-failure and build-failure fallback branches in setup.sh (4/8)
4. macOS test now removes libmtmd.0.dylib instead of the platform-agnostic convert_hf_to_gguf.py (3/8)
5. Empty-string tag test added -- test_empty_tag_omits_branch_flag for resolved_tag= (2/8)
6. RPATH checks on cmake call logs -- both fallback tests now assert @loader_path and -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON are absent from CPU fallback cmake calls, plus baseline flag preservation (multiple)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests clean up
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Replaces the fixed prebuilt llama.cpp tag with dynamic published-release
resolution, adds bounded fallback across older published releases, and
introduces maintainer-editable defaults for PR/source overrides.
Changes:
- Resolve latest from the latest usable published release in unslothai/llama.cpp
- Use the selected release upstream_tag as the authoritative llama.cpp version
- Prefer Unsloth-published platform assets when available
- Fall back to same-tag upstream ggml-org/llama.cpp assets where allowed
- Keep Linux CUDA anchored to Unsloth-published CUDA bundles only
- Add bounded fallback across older Unsloth published releases
- Add separate busy/in-use install handling (exit code 3)
- Skip reinstall when the installed bundle already matches the selected candidate
- Add maintainer-editable _DEFAULT_LLAMA_PR_FORCE and _DEFAULT_LLAMA_SOURCE
- Harden env parsing so malformed installer env vars do not crash import-time fallback logic
- Honor UNSLOTH_LLAMA_RELEASE_TAG in all resolve steps
- Always sync git remote URL in existing-checkout path
* fix(studio): source-build fallback prefers Unsloth's tested tag over upstream latest
When the prebuilt install fails and falls back to source build,
--resolve-llama-tag now queries the Unsloth release repo
(unslothai/llama.cpp) first to get the latest tested/approved tag
(e.g. b8508), instead of going straight to ggml-org/llama.cpp which
may return a newer untested tag (e.g. b8514).
This ensures the source-build fallback compiles the same version that
the prebuilt path would have installed, rather than a potentially
incompatible bleeding-edge release.
Resolution order for "latest":
1. Unsloth release repo (tested/approved)
2. ggml-org upstream (bleeding-edge)
3. Raw requested tag string (last resort)
Changes:
- resolve_requested_llama_tag() accepts optional published_repo param
with docstring explaining the resolution order
- CLI --resolve-llama-tag passes --published-repo through
- setup.sh and setup.ps1 pass --published-repo to --resolve-llama-tag
with inline comments explaining the preference
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Use prebuilt llama.cpp for unsloth studio setup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix 3 issues that cause unnecessary fallback to source build
1. Make filelock import optional -- environments without filelock
(e.g. minimal installs) crashed at import time instead of
gracefully skipping the lock.
2. Use already-verified converter script from the hydrated source
tree instead of re-downloading from raw.githubusercontent.com
with no checksum. Adds symlink with copy fallback for the
legacy filename.
3. Initialize $SkipPrebuiltInstall in setup.ps1 before first use
to prevent potential uninitialized variable errors.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep network fallback in ensure_converter_scripts
Prefer the local verified copy from the hydrated source tree, but
retain the original network download as a fallback if the file is
missing. Create the legacy hyphenated filename as a symlink with a
copy fallback instead of writing a second full copy.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix 4 bugs in source-build fallback and binary_env paths
- setup.ps1: Replace git pull + checkout FETCH_HEAD with fetch + checkout -B
to avoid detached HEAD state that breaks re-runs. Use pinned tag in both
fetch and clone paths.
- setup.sh: Move rm -rf after cmake/git prerequisite checks so a missing
tool no longer deletes the existing install. Add --branch tag to clone.
- install_llama_prebuilt.py: Add binary_path.parent to Linux LD_LIBRARY_PATH
in binary_env() so bundled .so files in build/bin are found even without
RPATH, matching the existing Windows PATH logic.
- Add test for binary_env LD_LIBRARY_PATH on Linux.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle unresolved "latest" tag in source-build fallback clone
When tag resolution fails and the requested tag is "latest", both
setup scripts now omit --branch from git clone so the default branch
is cloned instead of failing on a nonexistent "latest" branch/tag.
Similarly, the PS1 fetch path fetches the default ref when the tag
is "latest".
* Resolve actual latest ggml-org tag instead of using literal "latest"
When both Python tag resolution attempts fail and the requested tag
is "latest", query the GitHub API for the actual latest release tag
from ggml-org/llama.cpp (e.g. b8508) instead of passing the literal
string "latest" to git clone --branch, which would fail since no
such branch/tag exists.
setup.sh uses curl + python json parsing; setup.ps1 uses
Invoke-RestMethod. Both fall back to the raw requested tag if the
API call also fails.
* Try Unsloth release repo before ggml-org when resolving latest tag
When falling back to the GitHub API to resolve "latest", query the
Unsloth release repo (unslothai/llama.cpp) first since it has the
prebuilt binaries pinned to tested tags. Only fall back to
ggml-org/llama.cpp if the Unsloth repo query fails.
* Add comprehensive sandbox tests for PR #4562 bug fixes
35 tests covering all fixes across platforms:
- binary_env cross-platform (Linux LD_LIBRARY_PATH, Windows PATH,
macOS DYLD_LIBRARY_PATH) with edge cases (dedup, ordering, existing paths)
- resolve_requested_llama_tag (concrete, latest, None, empty)
- setup.sh logic via subprocess: prereq check ordering (cmake/git missing
preserves install), pinned tag in clone, fetch+checkout -B pattern,
fetch failure warns instead of aborting
- "latest" tag resolution fallback chain (Unsloth API -> ggml-org ->
raw) with mock curl: success, failure, malformed JSON, empty body,
empty tag_name, env overrides
- Source code pattern verification for both .sh and .ps1 files
All 138 tests pass in isolated uv venv.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add binary_path.parent to macOS DYLD_LIBRARY_PATH in binary_env
macOS prebuilt .dylib files are overlaid into build/bin (same as
Linux), but binary_env only added install_dir to DYLD_LIBRARY_PATH.
Add binary_path.parent so the loader can find sibling dylibs even
without embedded loader paths.
Mirrors the existing fix for Linux LD_LIBRARY_PATH and the Windows
PATH pattern.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard --branch when resolved tag is "latest"; fix broken test assertion
When all API fallbacks fail and the tag stays as literal "latest",
omit --branch from git clone (clones default branch instead of
failing). Both setup.sh and setup.ps1 now check for "latest" before
passing --branch to git clone/fetch.
Also fix test_setup_ps1_clone_uses_branch_tag which used Python
tuple syntax (assert "x", "y" in z) that always passes. Changed to
assert "x" in z and "y" in z.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix macOS DYLD trailing colon, install_lock no-op, and debug log
- binary_env macOS: use dedupe_existing_dirs instead of raw string
concatenation. Eliminates trailing colon in DYLD_LIBRARY_PATH
(which causes dyld to search CWD for libraries) and deduplicates
when binary_path.parent == install_dir. Now consistent with the
Linux and Windows branches.
- install_lock: when filelock is not installed, use os.O_CREAT|O_EXCL
as a fallback exclusive file lock with timeout, instead of yielding
with no locking. Prevents concurrent installs from corrupting each
other's staging directories.
- setup.ps1: remove [DEBUG] log line that printed to every user on
every Windows setup run.
* Add stale-lock detection and atomic clone-then-swap
install_lock fallback (no filelock): write PID to lock file and
check if the holder process is still alive on contention. Dead PIDs
(ProcessLookupError) and unreadable lock files trigger immediate
cleanup. Live processes owned by other users (PermissionError) are
correctly recognized as alive -- the lock is not removed.
setup.sh/setup.ps1 source-build: clone into a temporary directory
first, then swap into place only on success. If git clone fails,
the existing install is preserved instead of being deleted by the
premature rm -rf.
* Remove redundant upstream_tag != release_tag check
load_approved_release_checksums compared checksums.upstream_tag
against the Unsloth release_tag, which are different namespaces
(upstream ggml-org tag vs Unsloth published tag). This only worked
because both happened to be "b8508" by convention. Would break if
Unsloth ever uses a different release naming scheme.
The existing check at parse_approved_release_checksums (line 950)
already validates the release_tag field correctly.
* Fix lock TOCTOU race and build-in-temp-dir swap
install_lock fallback: add os.fsync(fd) after writing PID to ensure
the PID is visible to racing processes before they check. Treat
empty lock files (PID not yet written) as "wait and retry" instead
of stale, closing the window where two processes could both see an
empty file, both unlink it, and both acquire the lock.
setup.sh/setup.ps1 source-build: clone AND build in a temp directory
(LLAMA_CPP_DIR.build.$$). Only swap into the final LLAMA_CPP_DIR
after the build succeeds. If clone or cmake or build fails, the temp
dir is cleaned up and the existing working install is preserved.
Previously, rm -rf ran after clone but before build, destroying the
existing install even if the build later failed.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Roland Tannous