* Add --with-llama-cpp-dir flag to install.ps1 and install.sh Users can now pass --with-llama-cpp-dir /path/to/llama.cpp to the installer to skip downloading or building llama.cpp and use a local directory instead. A junction (Windows) or symlink (Linux/macOS) is created at the canonical install location, bypassing both the prebuilt download (Phase 3) and source build (Phase 4) steps in setup.ps1/setup.sh. The path is passed via UNSLOTH_LOCAL_LLAMA_CPP_DIR env var which setup.ps1 and setup.sh read directly. Ported from the idea in unslothai/unsloth#4384, reimplemented against current Studio architecture. * test: add static wiring test for --with-llama-cpp-dir flag Cross-checks install.sh, install.ps1, studio/setup.sh and studio/setup.ps1 so the flag's contract (parse -> UNSLOTH_LOCAL_LLAMA_CPP_DIR env var -> link local dir, skip prebuilt download and source build) can't silently regress. Wired into studio-backend-ci.yml alongside the other tests/sh installer tests. * Address review feedback on --with-llama-cpp-dir flag - setup.ps1: delete an existing junction/symlink via DirectoryInfo.Delete() instead of a recursive remove, which can traverse the link and wipe the user's real llama.cpp directory on PowerShell 5.1. - setup.ps1: short-circuit the build chain when a local dir is linked so CMake never runs inside the user's checkout when it lacks a Windows-layout binary. - install.sh / setup.sh: resolve paths with CDPATH= cd -P so a set CDPATH cannot corrupt the resolved path. - install.sh: seed _WITH_LLAMA_CPP_DIR from UNSLOTH_LOCAL_LLAMA_CPP_DIR so an exported env var (piped-install style) is honored instead of being clobbered. - setup.sh: create the root llama-quantize shim when linking a local source build so GGUF export's check_llama_cpp() still finds it. - setup.sh / setup.ps1: drop a stale link before the custom-home ownership assert so re-runs with the flag stay idempotent. - test: pin the new linked-dir build short-circuit. * Harden --with-llama-cpp-dir against Codex/Gemini review findings - install.sh: error when --with-llama-cpp-dir is the final arg with no path, matching the existing --package/--python post-loop guards (was a silent fallback to the normal prebuilt/source install). - studio/setup.sh: canonicalize LLAMA_CPP_DIR before the self-link no-op compare. _RESOLVED_LOCAL is fully resolved while LLAMA_CPP_DIR was textual, so a symlinked $HOME made the guard miss and the rm -rf could wipe the user's real llama.cpp tree. - studio/setup.sh: make the llama-quantize shim non-fatal; it writes through the link into the user's tree, which may be read-only (shared/CI cache), and under set -e a failed ln aborted an otherwise-good reuse. - studio/setup.ps1: detect a broken junction via Get-Item -Force instead of Test-Path so a dangling link from a prior run is removed and mklink can relink to a new valid directory. - studio/setup.ps1: use Copy-Item -LiteralPath so a source path containing [ ] isn't treated as a wildcard in the junction copy fallback. - tests: update the wiring assertions for the LiteralPath copy and the canonicalized compare. * Validate/reuse local llama.cpp tree and guard the in-use case Addresses the second Codex pass on the --with-llama-cpp-dir flag: - Validate the linked tree before disabling installs (setup.sh + setup.ps1): reusing a local dir skips BOTH the prebuilt download and the source build, so the dir must already contain a runnable llama-server (build/bin on Linux/macOS, build\bin\Release\llama-server.exe on Windows). Bail out with a clear message instead of linking an unbuilt/wrong-platform checkout and leaving Studio with no usable binary. - Treat a canonical-path target as already linked when it holds a build (setup.sh + setup.ps1): point the flag at ~/.unsloth/llama.cpp itself and an existing build is reused (skip prebuilt + source) rather than clobbered by the staged prebuilt installer (which uses os.replace()/replace). An empty canonical dir still falls through to the normal in-place install. - Abort when an in-use llama.cpp can't be removed on Windows (setup.ps1): Remove-Item -ErrorAction SilentlyContinue can silently leave a locked tree in place; detect that and stop with the same active-process message + exit 3 the prebuilt path uses, instead of junctioning over a half-present dir. Left as follow-up (already tracked by the PR author as a non-blocker): the in-app "Update llama.cpp" updater does not yet recognize a local-link install as externally managed; that fix belongs in studio/backend/utils/llama_cpp_update.py. * Accept all backend llama-server layouts in --with-llama-cpp-dir validation The linked-tree validation only accepted build/bin[/Release]/llama-server, but LlamaCppBackend._layout_candidates() resolves a root-level llama-server first, then build/bin, then build/bin/Release on Windows. A `make` build or a flat release extract (binary at the dir root) was therefore rejected with a hard installer failure even though Studio would have run it. Validate the same candidate set the backend uses in both setup scripts, and add wiring-test assertions so the check can't silently narrow again. * Treat --with-llama-cpp-dir local links as externally managed A --with-llama-cpp-dir install junctions/symlinks the canonical llama.cpp dir to the user's own checkout, but two backend paths still treated it as a Studio-owned tree: - The in-app updater (llama_cpp_update) offered and could apply an official prebuilt over the link, writing through it into the user's checkout (or failing) and silently dropping the link the flag created. - Orphan cleanup (LlamaCppBackend._kill_orphaned_servers) resolved the linked root into its kill allowlist, so a llama-server the user launched from the same checkout was classified as ours and killed on startup. Detect the canonical dir being a symlink/junction (reparse point) and treat the install as unmanaged: get_update_status reports unsupported, start_update refuses with reason "local_link", and the linked root is left out of the orphan allowlist. Adds behavioral tests (link vs plain dir, updater refusal, and the spared-vs-killed orphan control). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add behavioral shell test for --with-llama-cpp-dir linking The existing tests/sh/test_with_llama_cpp_dir_flag.sh is a static grep of the scripts. This adds a behavioral test that extracts the real link block from studio/setup.sh (by content anchors, with a self-validating extraction) and runs it against hermetic fake dirs, asserting the outcomes that matter: - an external CMake build links and arms neither the prebuilt download nor the source build - a flat / make tree (root-level llama-server, no build/bin) is accepted too - an unbuilt tree is rejected with a non-zero exit and no link left behind - relinking over a stale link preserves the target's contents (no data loss) - pointing at the canonical path is a no-op reuse, not a self-referential link Symlink-identity checks run only where real symlinks exist (skipped on Windows git-bash copy-mode); the link/skip/no-data-loss checks run everywhere. Wired into studio-backend-ci.yml next to the static test. * Install psutil in backend CI so orphan-cleanup tests run The new orphan-cleanup tests import psutil for the process scan, but the Backend CI deps step installed studio.txt plus a fixed extras list that omits it, so the two tests failed with ModuleNotFoundError. Add psutil to both backend pytest dep steps (kept in shared shape), and guard the import with pytest.importorskip so a minimal env without psutil skips these tests instead of erroring. --------- Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
172 lines
6.9 KiB
Bash
172 lines
6.9 KiB
Bash
#!/bin/bash
|
|
# Static analysis: the --with-llama-cpp-dir flag must be wired consistently
|
|
# across both installers (install.sh / install.ps1) and both setup scripts
|
|
# (studio/setup.sh / studio/setup.ps1).
|
|
#
|
|
# The flag lets a user point the installer at a local llama.cpp directory so it
|
|
# skips BOTH the prebuilt download (Phase 3) and the source build (Phase 4),
|
|
# linking the local dir into the canonical install location instead. The path
|
|
# crosses the installer->setup boundary via the UNSLOTH_LOCAL_LLAMA_CPP_DIR env
|
|
# var. These checks pin that contract so a future refactor of either side can't
|
|
# silently break it (e.g. installer parses the flag but setup never reads the
|
|
# env var, or setup links the dir but still runs the build).
|
|
#
|
|
# This is a shape/wiring test, not a behavioral one: it greps the committed
|
|
# scripts. It needs no Python, no GPU, no network.
|
|
set -e
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
INSTALL_SH="$SCRIPT_DIR/../../install.sh"
|
|
INSTALL_PS1="$SCRIPT_DIR/../../install.ps1"
|
|
SETUP_SH="$SCRIPT_DIR/../../studio/setup.sh"
|
|
SETUP_PS1="$SCRIPT_DIR/../../studio/setup.ps1"
|
|
ENV_VAR="UNSLOTH_LOCAL_LLAMA_CPP_DIR"
|
|
PASS=0
|
|
FAIL=0
|
|
|
|
assert_contains() {
|
|
_label="$1"; _file="$2"; _needle="$3"
|
|
if grep -qF -- "$_needle" "$_file"; then
|
|
echo " PASS: $_label"
|
|
PASS=$((PASS + 1))
|
|
else
|
|
echo " FAIL: $_label (expected to find '$_needle' in $(basename "$_file"))"
|
|
FAIL=$((FAIL + 1))
|
|
fi
|
|
}
|
|
|
|
# Count of distinct lines matching a regex, used to assert a guard appears
|
|
# in more than one place (e.g. env var forwarded on both setup invocations).
|
|
assert_min_count() {
|
|
_label="$1"; _file="$2"; _pattern="$3"; _min="$4"
|
|
_n=$(grep -cE -- "$_pattern" "$_file" || true)
|
|
if [ "$_n" -ge "$_min" ]; then
|
|
echo " PASS: $_label (found $_n, need >= $_min)"
|
|
PASS=$((PASS + 1))
|
|
else
|
|
echo " FAIL: $_label (found $_n in $(basename "$_file"), need >= $_min)"
|
|
FAIL=$((FAIL + 1))
|
|
fi
|
|
}
|
|
|
|
echo ""
|
|
echo "=== install.sh: parses --with-llama-cpp-dir and forwards the env var ==="
|
|
|
|
assert_contains \
|
|
"install.sh: accepts --with-llama-cpp-dir flag" \
|
|
"$INSTALL_SH" "--with-llama-cpp-dir"
|
|
assert_contains \
|
|
"install.sh: validates the path exists before forwarding" \
|
|
"$INSTALL_SH" 'if [ ! -d "$_WITH_LLAMA_CPP_DIR" ]; then'
|
|
# The path must be forwarded to setup.sh on BOTH the local and the
|
|
# non-local setup invocations, else --local users (the documented path)
|
|
# would silently lose the flag.
|
|
assert_min_count \
|
|
"install.sh: forwards $ENV_VAR on both setup invocations" \
|
|
"$INSTALL_SH" "$ENV_VAR=\"\\\$_WITH_LLAMA_CPP_DIR\"" 2
|
|
|
|
echo ""
|
|
echo "=== install.ps1: parses --with-llama-cpp-dir and forwards the env var ==="
|
|
|
|
assert_contains \
|
|
"install.ps1: accepts --with-llama-cpp-dir flag" \
|
|
"$INSTALL_PS1" '"--with-llama-cpp-dir"'
|
|
assert_contains \
|
|
"install.ps1: errors when flag is given with no path argument" \
|
|
"$INSTALL_PS1" "--with-llama-cpp-dir requires a path argument"
|
|
assert_contains \
|
|
"install.ps1: validates the path exists before forwarding" \
|
|
"$INSTALL_PS1" "--with-llama-cpp-dir path does not exist"
|
|
assert_contains \
|
|
"install.ps1: exports $ENV_VAR for setup.ps1" \
|
|
"$INSTALL_PS1" "\$env:$ENV_VAR ="
|
|
# The exported env var must be cleaned up so a later setup invocation in the
|
|
# same shell session doesn't inherit a stale local-dir link.
|
|
assert_contains \
|
|
"install.ps1: clears $ENV_VAR after the setup run" \
|
|
"$INSTALL_PS1" "Remove-Item Env:$ENV_VAR"
|
|
|
|
echo ""
|
|
echo "=== studio/setup.sh: reads the env var, links, and skips download+build ==="
|
|
|
|
assert_contains \
|
|
"setup.sh: reads $ENV_VAR" \
|
|
"$SETUP_SH" "$ENV_VAR"
|
|
assert_contains \
|
|
"setup.sh: symlinks the local dir into the canonical install location" \
|
|
"$SETUP_SH" 'ln -sfn "$_RESOLVED_LOCAL" "$LLAMA_CPP_DIR"'
|
|
assert_contains \
|
|
"setup.sh: disables the source build when the local dir is linked" \
|
|
"$SETUP_SH" "_NEED_LLAMA_SOURCE_BUILD=false"
|
|
assert_contains \
|
|
"setup.sh: skips the prebuilt download when the local dir is linked" \
|
|
"$SETUP_SH" "_SKIP_PREBUILT_INSTALL=true"
|
|
# The link branch must short-circuit the FORCE_COMPILE / prebuilt chain rather
|
|
# than fall through into it.
|
|
assert_contains \
|
|
"setup.sh: link branch gates the prebuilt/compile chain" \
|
|
"$SETUP_SH" 'if [ "$_LOCAL_LLAMA_CPP_LINKED" = true ]; then'
|
|
|
|
echo ""
|
|
echo "=== studio/setup.ps1: reads the env var, junctions, and skips download+build ==="
|
|
|
|
assert_contains \
|
|
"setup.ps1: reads $ENV_VAR" \
|
|
"$SETUP_PS1" "\$env:$ENV_VAR"
|
|
assert_contains \
|
|
"setup.ps1: creates a directory junction into the canonical location" \
|
|
"$SETUP_PS1" "mklink /J"
|
|
assert_contains \
|
|
"setup.ps1: falls back to a copy when the junction can't be created" \
|
|
"$SETUP_PS1" "Copy-Item -Recurse -LiteralPath \$ResolvedLocal -Destination \$LlamaCppDir"
|
|
assert_contains \
|
|
"setup.ps1: disables the source build when the local dir is linked" \
|
|
"$SETUP_PS1" '$NeedLlamaSourceBuild = $false'
|
|
# The link branch must gate the prebuilt-install chain (the elseif on
|
|
# FORCE_COMPILE), and the linked-dir case must short-circuit the build chain
|
|
# so neither a prebuilt download nor a source build runs against it.
|
|
assert_contains \
|
|
"setup.ps1: link branch gates the prebuilt/compile chain" \
|
|
"$SETUP_PS1" 'if ($LocalLlamaCppLinked) {'
|
|
assert_contains \
|
|
"setup.ps1: linked-dir case short-circuits the build chain" \
|
|
"$SETUP_PS1" 'step "llama.cpp" "linked (skipping build)"'
|
|
|
|
echo ""
|
|
echo "=== both setup scripts: validate against every layout the backend resolves ==="
|
|
|
|
# The linked tree is accepted only if it already holds a runnable llama-server,
|
|
# but the check must match LlamaCppBackend._layout_candidates() (root-level
|
|
# first, then build/bin, then build/bin/Release on Windows). A narrower check
|
|
# would reject a make/flat-release tree the backend could run.
|
|
assert_contains \
|
|
"setup.sh: accepts root-level or build/bin llama-server layouts" \
|
|
"$SETUP_SH" '[ -x "$1/llama-server" ] || [ -x "$1/build/bin/llama-server" ]'
|
|
assert_contains \
|
|
"setup.ps1: accepts the build\\bin (non-Release) llama-server.exe layout" \
|
|
"$SETUP_PS1" 'Join-Path $ResolvedLocal "build\bin\llama-server.exe"'
|
|
assert_contains \
|
|
"setup.ps1: accepts the root-level llama-server.exe layout" \
|
|
"$SETUP_PS1" 'Join-Path $ResolvedLocal "llama-server.exe"'
|
|
|
|
echo ""
|
|
echo "=== both setup scripts: a local dir pointing at the canonical path is a no-op ==="
|
|
|
|
# Guard against the self-link footgun: if the user passes the canonical install
|
|
# dir itself, neither script should delete-then-link it onto itself.
|
|
assert_contains \
|
|
"setup.sh: ignores a local dir equal to the canonical install location" \
|
|
"$SETUP_SH" 'if [ "$_RESOLVED_LOCAL" = "$_CANON_LLAMA_CPP_DIR" ]; then'
|
|
assert_contains \
|
|
"setup.ps1: ignores a local dir equal to the canonical install location" \
|
|
"$SETUP_PS1" 'if ($ResolvedLocal -eq $LlamaCppDir) {'
|
|
|
|
echo ""
|
|
echo "=== Results ==="
|
|
echo " PASS: $PASS"
|
|
echo " FAIL: $FAIL"
|
|
if [ "$FAIL" -gt 0 ]; then
|
|
echo "FAILED"
|
|
exit 1
|
|
fi
|
|
echo "ALL PASSED"
|