* studio: announce Cloudflare tunnel state and warn about public exposure on startup
The startup banner only printed a line when a tunnel URL was up, so a plain
`unsloth studio -H 0.0.0.0` launch silently created a public trycloudflare.com
URL with no indication that Studio had become reachable from the internet. The
only hint at the tunnel was the CLI help, shown when an invalid command was typed.
Make the banner always state the tunnel state for wildcard binds:
- ON: the public URL plus a warning that anyone with it can reach Studio from
outside the network, and that --no-cloudflare keeps it local-only.
- FAILED: requested but did not start (local network only).
- OFF: --no-cloudflare was passed (local network only).
Secure mode keeps its existing wording (the authenticated tunnel is intended and
--no-cloudflare is not valid there). Clarify the --cloudflare help text in both
the argparse and typer definitions. Default behavior is unchanged.
Also surface the state on the `unsloth studio run` banner, which runs the server
with silent=True and prints its own banner: it now calls _print_cloudflare_line
too, so the ON/OFF/FAILED notice and public-exposure warning are no longer
skipped on that path (previously it only echoed the URL when a tunnel was up).
For the OFF and FAILED notices, do not claim "local network only" when the
reachability probe just confirmed the raw port is reachable from the public
internet: --no-cloudflare and a failed tunnel disable only the Cloudflare link,
not the wildcard bind, so the message is reworded to flag the public raw port.
* Fix/adjust Cloudflare banner warnings for PR #6515
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust Cloudflare banner comments for PR #6515
* Fix/adjust IPv6 Cloudflare tunnel gate for PR #6515
* Fix/adjust Cloudflare review comments for PR #6515
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix silent run Cloudflare notice
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: defer llama.cpp update probes and self-heal MLX on macOS
Two macOS startup problems shared one root area in the FastAPI lifespan:
- The llama.cpp capability + freshness probes ran inline before the server
yielded, so a cold/slow/flaky network on the GitHub freshness check blocked
'Application startup complete' (~34s on CI, longer in the field). Move both
probes to a daemon thread; app.state stays None until ready (status routes
already re-probe at request time). Opt out with UNSLOTH_DISABLE_UPDATE_CHECK=1.
- Train and Export were greyed out because mlx/mlx-lm/mlx-vlm arrive only
transitively and a resolver backtrack silently drops them, so CHAT_ONLY stayed
true. Add utils/mlx_repair.py: when Apple Silicon is detected without MLX,
reinstall mlx/mlx-lm/mlx-vlm by name on a daemon thread and re-run hardware
detection (opt out UNSLOTH_DISABLE_MLX_AUTOREPAIR=1). Surface a chat_only_reason
in /api/health plus a sidebar tooltip so a greyed Train/Export explains itself
instead of failing silently.
* Studio: guard model defaults against a None model name
load_model_defaults(None) called model_name.lower() with no guard, raising
'Error loading model defaults for None' before any model is selected. Return
an empty dict for a falsy/non-str name.
* Studio: drop obsolete upstream macOS + Windows Blackwell prebuilt pins
Both pins worked around gaps in ggml-org upstream prebuilts, but Studio now
routes every GPU host and all of macOS to the unslothai/llama.cpp fork
(published_repo_for_host), which ships the needed bundles, so both pins are
dead code on the default install path:
- macOS b9415: macOS always routes to the fork (its own macOS bundles), and
host_supports_macos_minos() is the backstop. The pin only fired under an
explicit --published-repo ggml-org override.
- Windows Blackwell b9360: Windows-NVIDIA routes to the fork, whose
windows-x64-cuda13 bundle covers Blackwell (manifest max_sm 120, toolkit
13.3), so the pin's self-disable check makes it dormant on every default
install; it could only activate under the same upstream override on a
13.0-13.2 driver.
Remove the pin constants, functions, and call sites. Keep the Blackwell
capability detection (_drop_blackwell_incapable_windows_cuda, _host_is_blackwell,
_windows_cuda_attempt_covers_blackwell) that still drops a non-sm_120 cuda-12.4
build on a Blackwell host. After this, an explicit --published-repo ggml-org
override on a Blackwell 13.0-13.2 host loses its GPU fallback and lands on CPU;
the default fork path is unaffected. Update the install selection-logic and
macOS-compat unit tests for the new no-pin behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: walk back deeper on the macOS upstream prebuilt path
After removing the b9415 macOS pin, the explicit --published-repo ggml-org
upstream path still used the default 2-release fallback, so a pre-macOS-26 host
behind a run of macOS-26-only builds would exhaust two too-new plans (minos is
only checked post-download) and drop to a source build before reaching a
loadable older release. Walk back as deep as the fork macOS path
(DEFAULT_MAX_MACOS_RELEASE_FALLBACKS), turning the removed static pin into
dynamic discovery. Addresses review feedback on the macOS upstream fallback.
* Studio: pin transformers during MLX self-heal so it cannot break Studio
mlx-lm/mlx-vlm declare transformers>=5, but the single-env install pins
transformers==4.57.6. The self-heal used --upgrade with no constraint, so it
could upgrade transformers in the live venv and break the rest of Studio just to
make import mlx.core pass. Pin transformers to the installed version via a
constraint file: the resolver either finds an mlx build compatible with it or
fails (we stay chat-only), never upgrading transformers underneath Studio.
Addresses review feedback on the MLX repair install.
* Studio: harden MLX self-heal against an unsupported mlx-vlm
Pinning transformers alone made uv backtrack mlx-vlm to 0.3.9 (below unsloth-zoo's
mlx-vlm>=0.4.4), which imports but breaks VLM Train/Export -- so the self-heal
could clear chat-only onto a broken stack. Mirror the main installer: set
UV_OVERRIDE=overrides-darwin-arm64.txt so a current mlx-vlm coexists with the
transformers pin, require the same minimum versions unsloth-zoo declares, and
gate/validate on a full mlx_stack_available() check (not a bare import) so an
old or partial stack stays chat-only. Addresses PR review.
* Studio: filter Blackwell-incapable CUDA in resolve_upstream_asset_choice
resolve_upstream_asset_choice returned the first windows-cuda choice unfiltered,
so a Blackwell host could be handed an sm_120-incapable cuda-12.4 build while the
sibling planners drop it. Apply _drop_blackwell_incapable_windows_cuda here too
and fall through to the CPU bundle on a Blackwell host with no capable GPU asset.
Addresses PR review.
* Studio: re-poll health so MLX self-heal reaches an open UI
The sidebar cached the initial /api/health, so a successful background MLX
self-heal (chat_only flips false) did not re-enable Train/Export until a manual
reload. While chat-only for the recoverable mlx_unavailable reason, re-poll
/api/health and stop once Train/Export become available. Addresses PR review.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make the disabled Train/Export tooltip reachable
The greyed Train/Export items pass a tooltip explaining why (e.g. MLX missing),
but a disabled <button> fires no pointer events and SidebarMenuButton only showed
tooltips while collapsed, so the explanation never appeared. Wrap a disabled
button in a focusable span and show its tooltip while expanded too; enabled items
keep the collapsed-only behavior. Addresses PR review.
* Studio: gate Train/Export on the full MLX stack, not bare mlx.core
detect_hardware enabled MLX training whenever `import mlx.core` worked, but the
MLX self-heal (utils/mlx_repair) treats a stack without mlx-lm/mlx-vlm at the
versions unsloth-zoo requires as inadequate. That asymmetry let the UI enable
Train/Export on exactly the partial/backtracked stack the self-heal is trying to
repair (greyed-in-but-broken VLM export). Gate on the same mlx_stack_available()
criterion so a partial stack stays chat-only (reason mlx_unavailable) and the
background repair restores it. Addresses PR review.
* Fix MLX repair and health auth for PR #6494
* Fix macOS upstream prebuilt fallback for PR #6494
* Fix MLX stack validation for PR #6494
* Fix MLX self-heal validation for PR #6494
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Review fixes: isolate hardware-state test, robust transformers pin
- test_chat_only_reason.py: detect_hardware() assigns module globals directly,
which monkeypatch does not revert; the autouse fixture now saves and restores
DEVICE/CHAT_ONLY/CHAT_ONLY_REASON/IS_ROCM so a chat-only verdict here cannot
leak into other backend tests (e.g. test_utils.py) on a GPU host.
- mlx_repair.py: read the transformers version from importlib.metadata instead of
importing transformers, so the install pin is not silently dropped when
transformers has valid metadata but fails to import.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix CI: model full MLX stack in dispatch tests, keep selection test offline
dispatch (macOS) job:
- detect_hardware now gates MLX on the full stack (mlx_stack_available imports
mlx_lm/mlx_vlm and checks dist versions), so faking only mlx.core makes the
apple_silicon_mlx profile resolve to CPU. The dispatch tests assert the routing
decision when the stack IS usable, so model a complete stack:
test_hardware_dispatch_matrix patches utils.mlx_repair.mlx_stack_available and
test_is_mlx_dispatch_gate patches hardware._has_usable_mlx_stack. The stack
predicate's own internals stay covered by test_mlx_repair.py.
Repo tests (CPU) job:
- test_no_cuda_attempt_on_published_path_for_13_1 fell through to a live
github_release_assets() upstream fetch after the Blackwell filter dropped every
published attempt, which the offline security scanner blocks. Stub that fetch so
the walk-back deterministically finds no usable CUDA build and raises
PrebuiltFallback without network.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden MLX self-heal: prepare transformers constraint inside the try
attempt_mlx_repair runs on a daemon thread, but _transformers_constraint_args was
called before the try. A failure there (e.g. tempfile.mkstemp on a full disk or a
bad TMPDIR) would propagate unhandled and silently kill the self-heal thread.
Move the call inside the try and initialize constraint_path so any such failure
is caught and leaves Studio chat-only instead of crashing the thread.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: reach the published source asset when a mix build's commit 404s
A llama.cpp "mix" prebuilt records a merge commit that is never pushed to
the fork, so the codeload/archive URLs for that commit 404. The merged
source tree is instead published as a release asset alongside the prebuilt
(llama.cpp-source-commit-<sha>.tar.gz). The installer resolves that asset
URL from the approved-checksums manifest, but when the manifest omits the
top-level repo/release_tag the URL resolves empty, hydration falls through
to the 404-ing commit archive, and the prebuilt install drops to a slow
source build (or fails outright).
Extract exact_source_asset_url() and resolve the asset's host and tag
defensively: the artifact's own repo, then the manifest repo, then the
source repo; and the manifest release tag, then the tag we actually
installed the prebuilt from (the source asset is its sibling on the same
release). Normal installs build the identical URL as before, so this only
adds a working fallback for the degenerate manifest.
Add unit coverage for the resolver, including the empty repo/release_tag
regressions.
* Studio: cover exact_source_asset_url through the real parser chain
Add TestExactSourceAssetUrl.test_resolves_through_real_parser_chain, which runs
parse_approved_release_checksums -> preferred_source_archive -> exact_source_asset_url
so a regression in the parser or source-selection wiring cannot pass while only the
hand-built helper unit tests stay green.
* Keep server-side tools enabled under --secure and on every bind
--secure binds loopback and exposes Studio only through an authenticated
Cloudflare HTTPS tunnel, but it was grouped with a raw 0.0.0.0 bind and
force-disabled all server-side tools (web search, Python, terminal). The
process tool policy overrode the client's enable_tools request, so the
model was never told the tools existed and answered in plain text. The
plain 'unsloth studio' command had no way to re-enable and printed nothing.
Tools now default on for every bind. The bind host and --secure no longer
change the tool policy; only an explicit --enable-tools/--disable-tools
forces it on or off. Both 'unsloth studio' and 'unsloth studio run' accept
the flags and the startup banner states the resolved policy.
- run.py: replace _apply_default_tool_policy(host, secure) with
_apply_cli_tool_policy(enable_tools); add an enable_tools kwarg to
run_server and --enable-tools/--disable-tools to the argparse.
- _tool_policy.py: resolve_tool_policy defaults to on for every host and
no longer prompts on a network bind.
- studio.py: drop the secure-as-public tool gating, add the flags to the
plain command, and reword the startup banner.
- Update and extend the secure-flag and tool-policy tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add tool-policy notice to plain server banner and refresh run --help
Follow-up to PR review:
- run.py: the plain 'unsloth studio' / --secure / direct run.py path went
through _emit_startup_output without any tool-policy line, so a
network-reachable launch was silent about code execution now that tools
default on. Thread enable_tools through _emit_startup_output /
_emit_secure_startup_output and print a one-line policy notice, followed by
a single stop hint.
- studio.py: the 'unsloth studio run' --enable-tools/--disable-tools and --yes
help still described the removed loopback-on/network-off default and the
confirmation prompt; reword to match the new policy.
- Add tests for the banner notice and the refreshed help text.
* Update CI tool-policy resolver tests for default-on behavior
tests/python/test_unsloth_run_tool_policy_resolver.py still asserted the
removed network-bind policy (0.0.0.0 and LAN IP default off, explicit enable
prompts and aborts on a declined prompt), so it failed the Python CI jobs.
Rewrite the truth table: every bind defaults on, explicit on/off always wins,
and the resolver never prompts (yes/silent/prompt kept for compatibility).
* Trim comments for the tool-policy change
Shorten the verbose docstrings and block comments added for --secure tool
handling; keep the security-relevant intent. Verified comment-only via an AST
diff (code unchanged).
* Add deterministic test that server-side tools execute under --secure
Drive the GGUF agentic tool loop with a fake llama-server stream and let the
real execute_tool run: python counts 1..100, terminal returns a UTC datetime,
and web_search runs through real _web_search with only the ddgs network
boundary mocked. A policy assertion pins that the post-fix --secure path
(policy None + per-request enable_tools) is what keeps these executions
reachable. No model, GPU, or live network; runs in the existing backend CI.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align _emit_startup_output banner test with the moved stop hint
The tool-policy notice now prints between the access banner and the stop
hint, so the stop hint is emitted once at the end instead of inline in the
banner (include_stop_hint is False and print_studio_stop_hint runs once).
Update the plain-localhost case to match; the mismatch and wildcard cases
already asserted this wiring.
---------
Co-authored-by: Michael Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Reduce and tighten comments and docstrings in tests
Shorten verbose comments and docstrings across the test suite without
changing any test logic. Remove narration that restates the next line,
collapse long module and test docstrings to a single line, and drop banner
separators. Keep regression context (issue and PR references, run ids),
skip reasons, mocking and timing rationale, license headers, lint and type
directives, and commented-out code.
Comments and docstrings only: an AST signature check confirms no code,
assertions, or string literals changed, and the suite byte-compiles cleanly.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The Linux installer ordered its CUDA runtime-line attempts purely by torch's
reported CUDA major (preferred_runtime_line), so a Blackwell host running a
cu12x torch build hoisted cuda12 ahead of an available native cuda13 bundle.
This brings the Linux selector to parity with the existing Windows Blackwell
preference: on an sm_120 host, prefer the highest CUDA-major line that ships
a bundle covering every visible host SM, then fall back to the torch line.
Selection-time only. No external pin and no source build: in-release cuda13
bundles already cover sm_120, and the per-artifact SM filter still drops any
incapable bundle (cuda12-older / cuda13-older) and prevents fall-through to a
non-Blackwell build. The override is gated on _host_is_blackwell and only
reorders lines that are already detected and driver-compatible, so it never
forces cuda13 when its runtime libraries are absent or the driver is pre-13,
and non-Blackwell hosts keep the exact torch-preference behavior.
The runtime-line ranking only considers well-formed "cuda<major>" lines and
skips any malformed or future-format value (e.g. "cuda13.1") instead of
crashing the major sort, matching how the surrounding selector already
tolerates unknown lines.
Adds focused selection tests covering the override, the incapable-cuda13
skip, the cuda13-unavailable fallback, the non-Blackwell no-op, the
malformed-runtime_line skip, and cuda14 forward-compat.
* Studio: route arm64 Linux CUDA hosts to linux-arm64-cuda prebuilts
* Studio: SM-aware selection for windows-cuda app bundles
* Studio: select published ROCm bundles by gfx target (linux + windows)
* Studio: route macOS installs to the fork's prebuilt bundles
* Studio: fix windows cuda13 driver-13.0 gate and ROCm gfx prefix overreach
* Fix Blackwell Windows pin shadowing native app-bundle (b9360 over b9457)
* Match Windows cuda12 driver floor to Linux (12.x minor-version compat)
* Fix Windows app-bundle dropped when runtime DLLs come from torch/lib
* Fold the manifest resolver into the simple-path resolver (one entry, no dormant full path)
* Remove unused UNSLOTH_LLAMA_PUBLISHED_REPO override
* Route Windows GPU hosts to the fork prebuilts in setup.ps1
* Document sm_103 path divergence and mark --simple-policy as a no-op
* Note sm_103 coverage now comes from the producer manifest
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove the now-vestigial --simple-policy flag (one resolver handles all hosts)
* Unify the fork onto the manifest path; drop the linux-x64 filename path and hardcoded coverage tables
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strip whitespace from manifest gfx_target/mapped_targets when parsing
* Windows CUDA: sort coverage-unknown bundles last so they can't outrank targeted ones
* Share the SM-coverage sort key between the linux and windows selectors via _sm_range
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reject approved releases with an exact source archive but no source repo to clone from
* Studio: accept the fork's windows-rocm kind in the Windows reinstall check
* Studio: accept a manifest-bundle source repo in the exact-source release check
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: route Linux hosts to the fork only when a usable GPU is present
* Fix Windows AMD lemonade tag resolution for PR #5963
The fork release scan passes each scanned release's upstream tag
(b9518, ...) to the lemonade lookup, but lemonade publishes its own tag
series (b1292, ...) that never contains upstream tag numbers. On a
Windows AMD host every scanned release therefore 404s the lemonade
fetch twice, the upstream HIP zip is dropped by the approved-hash gate,
and the scan walks the whole release history until it dies on the
unauthenticated GitHub rate limit or falls to a HIP source build. The
Linux path already passes the requested tag ("latest") and works.
Thread the requested tag through resolve_release_asset_choice ->
resolve_asset_choice -> resolve_upstream_asset_choice as lemonade_tag,
used only by the lemonade lookups. Upstream asset names keep the
concrete per-release tag and all new parameters default to the old
behavior.
Verified on a gfx1151 box: before, the native Windows install scanned
b9518..b8811 and aborted on rate limit; after, it selects
llama-b1292-windows-rocm-gfx1151-x64.zip (lemonade) from fork release
b9518, passes staged validation, and the installed llama-server
enumerates ROCm0. WSL keeps selecting the matching ubuntu bundle.
Adds a regression test pinning that the Windows fork path resolves
lemonade via /releases/latest, never /releases/tags/<fork-tag>.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Plan lemonade for Linux ROCm hosts on the ggml-org direct path for PR #5963
Audit follow-up to 72f32364 across the other selection pathways. The
ggml-org direct planner kept its lemonade attempt for Windows ROCm
hosts but planned only the CPU tarball for Linux ROCm hosts, so an AMD
Linux box routed to ggml-org (for example a --published-repo override)
silently installed the CPU build. That lemonade planning used to live
in the --simple-policy dispatcher this PR removed.
Add the lemonade attempt ahead of the CPU tarball in the Linux x86_64
branch, mirroring the Windows branch, with the lookup keyed to the
requested tag. Adds a regression test asserting lemonade is the first
attempt for a Linux ROCm host on the direct path.
Also re-verified the other pathways on a gfx1151 box: the fork-routed
flows pass the requested tag everywhere, repeat runs over an existing
lemonade install correctly skip with "already matches selected release
b9518" on both native Windows and WSL, and macOS, CUDA and CPU
selection are untouched. Suites: 328 passed on Linux, Windows matches
the pre-existing baseline.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Two hardening fixes from the fleet-validation audit.
Blackwell Windows hosts drop windows-cuda attempts that cannot offload
sm_120 instead of leaving them ranked behind the b9360 pin. A cuda-12.4
upstream build loads and passes the functional validator but runs the
model on a slow non-native path (an RTX 5090 measured 7.1 tok/s vs
551.2 on cuda-13.3), so one failed pin download away from that is too
close. The coverage check now also reads manifest SM metadata first, so
published cuda12 app bundles (toolkit 12.8, sm_120 included) stay
selectable and make the pin go dormant correctly.
The fork-release Linux planner no longer appends the linux-cpu bundle
for NVIDIA hosts whose CUDA selection produced nothing; it raises so
the caller walks back to an older release with a usable CUDA line,
mirroring the deliberate ROCm policy. Today's walk-back only works
because partial releases ship no CPU bundle; this keeps it working if
a future partial release does.
* fix: clearer Studio setup error when GPU driver is too old for the installed CUDA toolkit
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor optional color arg in setup.sh substep so driver/toolkit warnings render in C_WARN
* Add regression test that setup.sh _cuda_version_gt compares numerically for PR #5993
* Update Resolve-CudaToolkit test for the new driver-too-old messaging
setup.ps1 now routes the too-new-toolkit case through
Write-CudaDriverToolkitMismatch instead of the old 'is installed but
INCOMPATIBLE' banner. Extract that helper alongside Resolve-CudaToolkit so the
child pwsh can run it, and assert the new driver-too-old guidance (and the
one-line source-build error) instead of the removed INCOMPATIBLE text.
* Address review nits: document Windows hard-exit asymmetry and add toolkit/driver edge tests
setup.ps1: note that only a forced source build reaches the hard-exit branch
(the prebuilt path returned above), unlike setup.sh which degrades to CPU.
test_selection_logic.py: cover the CUDA UMD Version variant, the empty nvcc
version guard, and the too_old (< 12.4) short-circuit.
* fix(studio): allow CUDA minor-version compat and try installed toolkits before CPU fallback
The driver check now compares CUDA major versions only, per NVIDIA
minor-version compatibility, and when the selected nvcc is still too
new the setup iterates other installed toolkits and uses the newest
driver-compatible one before falling back to a CPU llama.cpp build.
Same rule mirrored in setup.ps1.
* style: apply ruff kwarg-spacing format after rebase
* Accept a same-major CUDA toolkit found only on PATH in the Windows fallback
The major-only compatibility fix updated the side-by-side scan (Find-Nvcc
-MaxVersion) and the CUDA_PATH check, but the fallback that runs when
Find-Nvcc -MaxVersion returns null still recorded any plain Find-Nvcc result
as an incompatible toolkit without re-checking the major. A same-major
toolkit discoverable only via PATH, process CUDA_PATH, or a custom location
(e.g. toolkit 13.3 with a driver supporting CUDA 13.2) was therefore rejected
even though it is compatible.
Re-apply the same major-only rule in the fallback: use the toolkit when its
major is within the driver's, otherwise record it as too-new. Adds a
regression test covering the PATH-only same-major case.
* Add CUDA driver/toolkit selection edge-case tests for Studio setup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in Studio CUDA driver/toolkit setup
Collapse multi-line comments, drop obvious ones, keep the load-bearing intent
(the major-compat invariant, the Windows hard-exit vs setup.sh-CPU asymmetry,
the PATH-only fallback rationale). Comment-only; no code change.
* Clarify the Windows source-build hard-exit comment
The path is reached by any committed source build (forced, or after a
prebuilt-install failure), not only a forced one. Comment-only.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop two obvious comments in setup.ps1 CUDA detection
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison.
Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent.
* fix: warn when localhost resolves to ::1 but Studio is bound only to 127.0.0.1
* studio: fix localhost/::1 warning suppression and cover _run wiring
Addresses the Codex review on #5994 plus review-team findings:
- Remove the `_local_port_open("::1", port)` early-return. Studio binds
127.0.0.1 only, so a successful connect to ::1:<port> means a *different*
process is there -- exactly when http://localhost opens the wrong service
and the user most needs the warning. Dropping the probe also removes the
~0.25s startup latency and the probe/warn race.
- Extract the banner/warning block from `_run` into `_emit_startup_output`
so the wiring is unit-testable, and make the mismatch vs wildcard paths
an explicit if/elif (they are mutually exclusive by construction).
- Hoist the `_working_local_url` confirmation out of the try block and
reorder `_stdout_color_ok` before its only caller.
- Tests: add `_emit_startup_output` integration coverage (banner
include_stop_hint, warning emission, single stop hint), a regression test
that ::1 being occupied does NOT suppress the warning, dual-stack and
non-positive-port cases; drop the unreachable `None` getaddrinfo arm.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Etherll <mrmrmidessam@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: cover B300 (sm_103) with the Linux prebuilt bundles
sm_103 (B300 / GB300 Blackwell Ultra) was in no bundle's supported_sms,
so those hosts fell through to a slow source compile. The newer and
portable bundles already ship base compute_100 PTX, which the driver
JIT-compiles forward to sm_103, so list sm_103 alongside sm_100 in those
bundles and let B300 install the prebuilt.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: fall back to source build for arm64 Linux GPU hosts
setup.sh routes a Linux host with any GPU tool to the unslothai/llama.cpp fork,
which publishes only linux-x64 bundles. On an arm64 host with a GPU (GH200,
GB200, DGX Spark) the resolver then selected an x64 CUDA bundle, which cannot run
on aarch64. Routing those hosts to ggml-org instead would install a CPU-only
arm64 build, silently losing the GPU.
Guard resolve_simple_install_release_plans so an arm64 Linux host on the fork
raises PrebuiltFallback before any release is fetched, letting setup.sh do a
source build that actually targets the GPU. x86_64 hosts and arm64 CPU hosts
(which route to ggml-org) are unaffected.
Add tests covering the arm64 fork raise, the x86_64 pass-through, and the arm64
CPU ggml-org path.
* Studio: install ggml-org arm64 CPU prebuilt when the arm64 GPU source build fails
Per review of #5924: arm64 Linux GPU hosts have no CUDA prebuilt anywhere (the
unslothai fork is x64 only, ggml-org ships no Linux CUDA build), so they source
build for the GPU. If that build produces no binary, the host was left without
llama.cpp.
Add a --cpu-fallback flag to install_llama_prebuilt.py that drops the host GPU
attributes so the CPU prebuilt for the host arch is selected (a GPU host cannot
otherwise pick the CPU bundle). setup.sh calls it against ggml-org as a last
resort for arm64 Linux when the source build degraded, installing the
ubuntu-arm64 CPU build instead of leaving the host with no llama.cpp.
Add tests: force_cpu drops GPU attrs before planning, a CPU-forced arm64 host
selects the ggml-org ubuntu-arm64 bundle, and setup.sh wires the fallback.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The pinned b9360 cuda-13.1 Windows fallback for Blackwell (sm_120) only fired
when the driver advertised CUDA >= 13.1. The mainstream Blackwell branch ships
the r580 driver that reports CUDA 13.0, so those hosts missed the pin, were
gated off the in-release 13.3 build, and dropped to the CPU-only cuda-12.4
build.
b9360's binary is native sm_120a SASS (no PTX, so no JIT) and its bundled
runtime is cuda-13.1 cudart; both run on a 13.0 r580+ driver under CUDA
minor-version compatibility. Lower _PINNED_BLACKWELL_DRIVER_FLOOR to (13, 0) so
the pin covers the 13.0 branch too. The generic published-runtime gate stays
conservative (an unverified in-release 13.1 build is still gated off 13.0); only
the specific, hash-pinned, SASS-verified b9360 build is extended.
Refs #5887.
* Studio: pin the last pre-macOS-26 llama.cpp prebuilt instead of walking back
ggml-org moved their macOS build runner to macOS 26 (Tahoe) at b9428, so
b9428 and every newer upstream prebuilt is stamped minos 26 and fails to
dyld-load on macOS 14 / 15. #5883 handled this by walking back release by
release at install time.
Replace that with a deterministic pin: a host below macOS 26 selects b9415
directly (the last upstream build stamped below 26: arm64 minos 14, x64
minos 13.3), so it loads on macOS 13.3 / 14 / 15 / 26. Hosts on macOS 26+
and unknown-version hosts keep latest selection unchanged.
Only the ggml-org upstream path is pinned; the unslothai/llama.cpp fork
ships its own minos-13.3 prebuilts (#5893), so the pin is a no-op there and
goes dormant once macOS routes to the fork. The Mach-O minos preflight from
#5883 stays as a post-download backstop.
Refs #5883, #5893.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* docs: fork ships arm64 minos 14 / x64 minos 13.3, not uniform 13.3
The per-slice fork producer pins arm64 to 14.0 and x64 to 13.3; update the
pinned_macos_release_tag docstring to match. No behavior change.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Make the NVIDIA prebuilt selection version-dynamic and driver-gated on both the
Windows (ggml-org) and Linux (unslothai/llama.cpp) paths, so CUDA version bumps
are handled with no code change while staying safe across driver versions.
- Derive candidate CUDA runtime lines from the driver major; pick the highest
upstream-published minor the driver can actually run, so a sub-13.3 driver is
never handed an unguaranteed 13.3 build.
- Pin b9360's cuda-13.1 build (immutable, hash-verified) as a GPU fallback for a
Blackwell host on a 13.1/13.2 driver that the in-release 13.3 build gates off,
on both the simple and published install paths. Dormant for every other host
and self-disabling once upstream ships a driver-runnable build again.
- Seed the published-path ordering from the release's real published minors so a
future CUDA major is selectable with no code change.
Refs #5861, #5817, #5807, #5858, #5854, #5826, #5887.
* Studio: download paired cudart bundle on Windows CUDA installs
Upstream ggml-org/llama.cpp publishes Windows CUDA in two archives
that the release notes explicitly say are both required:
llama-<tag>-bin-win-cuda-X.Y-x64.zip (binaries + ggml DLLs)
cudart-llama-bin-win-cuda-X.Y-x64.zip (cudart64, cublas64, cublasLt64)
Studio's installer was downloading only the first one. The
``runtime_name`` / ``runtime_url`` fields on AssetChoice existed but
were never populated, and ``install_from_archives`` only handled
``choice.url``. With the cudart DLLs missing from
``install_dir/build/bin/Release``, the prebuilt binary's LoadLibrary
calls only resolved at runtime when the user happened to have a
version-matched system CUDA toolkit on PATH. That is the underlying
cause for the Windows reports in #5106 ("GPU detected but model
loaded entirely on RAM"): the prebuilt's CUDA backend silently fails
to load and llama-server falls back to CPU regardless of ``-ngl`` or
``--fit on``.
Wires the pairing through end to end:
* ``windows_cuda_attempts`` and ``published_windows_cuda_attempts``
look up the matching ``cudart-llama-bin-win-cuda-X.Y-x64.zip``
asset URL alongside the main archive and store it as
``runtime_url`` / ``runtime_name`` on the AssetChoice. We only
pair when the selected main archive is the binary archive
(``llama-...zip``) so the legacy cudart-only naming path is
unaffected.
* ``apply_approved_hashes`` resolves the runtime archive's hash from
the approved manifest. If the manifest does not list the runtime
archive, the pairing is dropped rather than installing without
checksum coverage. Preserves the supply-chain guarantee for
published bundles; upstream installs with no manifest are
unaffected (same risk surface as the existing main-archive
download).
* ``install_from_archives`` now downloads the runtime archive into a
separate temp dir and runs ``copy_globs`` against both source dirs.
Separate dirs avoid the "ambiguous archive layout" guard tripping
on shared filenames like LICENSE.txt, while the second
``copy_globs`` overlay drops the cudart DLLs into the same
``install_dir/build/bin/Release`` directory as the main binary.
Adds a ``runtime_sha256`` field on AssetChoice to carry the
verified hash through to the download step, alongside the existing
``runtime_name`` / ``runtime_url`` slots.
Tests: 5 new cases in tests/studio/install/test_selection_logic.py:
* upstream pairing populates runtime_url / runtime_name
* graceful degrade when cudart asset is absent in the release
* legacy cudart-only naming path does not self-pair
* apply_approved_hashes threads runtime_sha256 when the manifest
lists it
* apply_approved_hashes drops the pair when the runtime hash is
missing rather than installing without verification
130 install tests pass (125 baseline + 5 new). No regressions.
Refs #5106
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be more succinct
* Studio: refresh installs that pre-date the paired cudart bundle
expected_install_fingerprint did not hash the new runtime_name /
runtime_sha256 fields, and runtime_payload_health_groups for windows-
cuda only checked llama.dll / ggml-cuda.dll. The combination meant that
an install made before this PR -- the exact installs reporting #5106 --
would still match the post-PR choice: same main asset name + sha, same
llama.dll, same ggml-cuda.dll, missing cudart64_*.dll, but
existing_install_matches_choice returned True and the cudart download
path in install_from_archives never ran. Fresh installs got the fix;
existing affected installs did not.
This commit:
* Adds runtime_asset and runtime_sha256 to the fingerprint payload so
any change to (or first introduction of) the cudart pair invalidates
pre-existing installs.
* Refactors write_prebuilt_metadata to call expected_install_fingerprint
so the recorded fingerprint cannot drift from the expected one when
new keys are added.
* Extends runtime_payload_health_groups for windows-cuda to require
cudart64_*.dll and cublas64_*.dll *only when the choice carries a
paired runtime archive*. Gating on choice.runtime_name keeps the
no-pair fallback path (manifest missing cudart hash, upstream
without paired bundle) from looping on reinstall.
New tests:
* test_existing_install_matches_plan_windows_cuda_paired_requires_cudart
-- paired choice rejects installs missing cudart / cublas.
* test_existing_install_matches_plan_windows_cuda_unpaired_skips_cudart_check
-- unpaired choice still accepts legacy cudart-less installs.
* test_existing_install_fingerprint_changes_when_cudart_pair_added
-- direct fingerprint mismatch between the legacy and paired choice.
Refs #5106
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten paired Windows CUDA install gates
Three follow-ups from a 12-reviewer batch over 526894a4 (PR #5322):
1. (12/12) Health check required cudart64_*.dll and cublas64_*.dll but
not cublasLt64_*.dll. The upstream cudart-llama-bin-win-cuda-X.Y-x64
bundle ships all three (verified against b9103 cuda-12.4 and
cuda-13.1: 3 DLLs, no executables), and a Windows install missing
any one of them still fails CUDA initialisation. Adding
cublasLt64_*.dll to runtime_payload_health_groups so a partial
install or a deletion of the third DLL triggers reinstall instead
of silently staying broken.
2. The runtime overlay copy used the same broad runtime_patterns_for_choice
set as the main archive (windows-cuda returns *.exe and *.dll). A
malformed runtime zip that contained a llama-server.exe alongside
the real cudart DLLs would have overwritten the main archive's
server binary. Introduced paired_runtime_dll_patterns() that
returns the cudart bundle's three specific filename patterns and
nothing else, and use that for the second copy_globs pass.
New end-to-end regression test packs a fake runtime zip with an
extra llama-server.exe and asserts the main binary survives.
3. (7/12) python_runtime_dirs in install_llama_prebuilt.py and
_windows_pip_nvidia_dll_dirs in llama_cpp.py walked different path
sets. The installer side missed nvidia/<pkg>/Library/bin (conda
layout) and nvidia/<pkg>/bin/x86_64 (current CUDA 13 unsuffixed
wheel layout), so preflight CUDA detection could fail even when
usable DLLs were present. Mirrored the same six-path set the
backend resolver uses, including arch subdirs.
New tests:
- test_paired_runtime_dll_patterns_excludes_executables
- test_runtime_overlay_cannot_overwrite_main_archive_payload (end-to-end)
- test_python_runtime_dirs_covers_cu13_and_library_bin
- extended test_existing_install_matches_plan_windows_cuda_paired_requires_cudart
with a cublasLt-missing case
Upstream cudart bundle contents verified empirically by downloading
the b9103 release artifacts directly: each cuda-X.Y bundle contains
exactly cudart64_X.dll + cublas64_X.dll + cublasLt64_X.dll, no exes.
Refs #5106
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix windows llama.cpp compile from source issue
* undo local repo usage
* fix llama.cpp install
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix windows
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: route resolve-source-build call through Invoke-LlamaHelper
The --resolve-source-build call at the source-build resolution path
was still calling install_llama_prebuilt.py directly instead of going
through Invoke-LlamaHelper. On PS7+ with ErrorActionPreference=Stop,
stderr from the 422 response (when tag is "master") would trigger a
terminating NativeCommandError and crash setup.
* fix: suppress stderr error records from Invoke-LlamaHelper
ErrorActionPreference=Continue prevents termination but PowerShell
still displays stderr lines as visible ErrorRecord objects. Capture
all output via 2>&1 and split stdout from stderr manually so that
stderr lines never appear on the console. When StderrPath is given
the stderr content is written to that file for diagnostics.
* fix: always rebuild llama.cpp on Windows when tag is master
When the requested llama.cpp tag is "master" (a moving target), skip
the "already built" early exit so the build path runs and syncs to
the latest commit. Without this, existing llama-server binaries from
an older build (e.g. b8635 which lacks Gemma 4 support) are reused
and model loading fails.
Pinned tags (e.g. b8635) still skip the rebuild when the binary
already exists, since the tag is immutable.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Expand test coverage for install_llama_prebuilt.py:
- Add tests for source build plan resolution with custom repos
- Add tests for branch/commit/PR ref matching and normalization
- Add tests for manifest checksum validation
- Add tests for Windows CUDA upstream asset name patterns
- Update capsys checks to capture stderr after log() redirect
Split out from #4741 to keep the main PR focused on installer logic.
- New test_install_llama_prebuilt_logic.py: tests for resolve logic,
fallback behavior, env_int, busy/lock handling
- New test_validate_llama_prebuilt.py: validator tests for staged
release_tag/upstream_tag handling
- New test_llama_pr_force_and_source.py: tests for PR_FORCE and
LLAMA_SOURCE maintainer defaults
- Updated test_selection_logic.py: expanded selection/fallback coverage
- Updated test_pr4562_bugfixes.py: updated bugfix tests for new logic
- Updated smoke_test_llama_prebuilt.py: minor update
* Use prebuilt llama.cpp for unsloth studio setup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix 3 issues that cause unnecessary fallback to source build
1. Make filelock import optional -- environments without filelock
(e.g. minimal installs) crashed at import time instead of
gracefully skipping the lock.
2. Use already-verified converter script from the hydrated source
tree instead of re-downloading from raw.githubusercontent.com
with no checksum. Adds symlink with copy fallback for the
legacy filename.
3. Initialize $SkipPrebuiltInstall in setup.ps1 before first use
to prevent potential uninitialized variable errors.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep network fallback in ensure_converter_scripts
Prefer the local verified copy from the hydrated source tree, but
retain the original network download as a fallback if the file is
missing. Create the legacy hyphenated filename as a symlink with a
copy fallback instead of writing a second full copy.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix 4 bugs in source-build fallback and binary_env paths
- setup.ps1: Replace git pull + checkout FETCH_HEAD with fetch + checkout -B
to avoid detached HEAD state that breaks re-runs. Use pinned tag in both
fetch and clone paths.
- setup.sh: Move rm -rf after cmake/git prerequisite checks so a missing
tool no longer deletes the existing install. Add --branch tag to clone.
- install_llama_prebuilt.py: Add binary_path.parent to Linux LD_LIBRARY_PATH
in binary_env() so bundled .so files in build/bin are found even without
RPATH, matching the existing Windows PATH logic.
- Add test for binary_env LD_LIBRARY_PATH on Linux.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle unresolved "latest" tag in source-build fallback clone
When tag resolution fails and the requested tag is "latest", both
setup scripts now omit --branch from git clone so the default branch
is cloned instead of failing on a nonexistent "latest" branch/tag.
Similarly, the PS1 fetch path fetches the default ref when the tag
is "latest".
* Resolve actual latest ggml-org tag instead of using literal "latest"
When both Python tag resolution attempts fail and the requested tag
is "latest", query the GitHub API for the actual latest release tag
from ggml-org/llama.cpp (e.g. b8508) instead of passing the literal
string "latest" to git clone --branch, which would fail since no
such branch/tag exists.
setup.sh uses curl + python json parsing; setup.ps1 uses
Invoke-RestMethod. Both fall back to the raw requested tag if the
API call also fails.
* Try Unsloth release repo before ggml-org when resolving latest tag
When falling back to the GitHub API to resolve "latest", query the
Unsloth release repo (unslothai/llama.cpp) first since it has the
prebuilt binaries pinned to tested tags. Only fall back to
ggml-org/llama.cpp if the Unsloth repo query fails.
* Add comprehensive sandbox tests for PR #4562 bug fixes
35 tests covering all fixes across platforms:
- binary_env cross-platform (Linux LD_LIBRARY_PATH, Windows PATH,
macOS DYLD_LIBRARY_PATH) with edge cases (dedup, ordering, existing paths)
- resolve_requested_llama_tag (concrete, latest, None, empty)
- setup.sh logic via subprocess: prereq check ordering (cmake/git missing
preserves install), pinned tag in clone, fetch+checkout -B pattern,
fetch failure warns instead of aborting
- "latest" tag resolution fallback chain (Unsloth API -> ggml-org ->
raw) with mock curl: success, failure, malformed JSON, empty body,
empty tag_name, env overrides
- Source code pattern verification for both .sh and .ps1 files
All 138 tests pass in isolated uv venv.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add binary_path.parent to macOS DYLD_LIBRARY_PATH in binary_env
macOS prebuilt .dylib files are overlaid into build/bin (same as
Linux), but binary_env only added install_dir to DYLD_LIBRARY_PATH.
Add binary_path.parent so the loader can find sibling dylibs even
without embedded loader paths.
Mirrors the existing fix for Linux LD_LIBRARY_PATH and the Windows
PATH pattern.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard --branch when resolved tag is "latest"; fix broken test assertion
When all API fallbacks fail and the tag stays as literal "latest",
omit --branch from git clone (clones default branch instead of
failing). Both setup.sh and setup.ps1 now check for "latest" before
passing --branch to git clone/fetch.
Also fix test_setup_ps1_clone_uses_branch_tag which used Python
tuple syntax (assert "x", "y" in z) that always passes. Changed to
assert "x" in z and "y" in z.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix macOS DYLD trailing colon, install_lock no-op, and debug log
- binary_env macOS: use dedupe_existing_dirs instead of raw string
concatenation. Eliminates trailing colon in DYLD_LIBRARY_PATH
(which causes dyld to search CWD for libraries) and deduplicates
when binary_path.parent == install_dir. Now consistent with the
Linux and Windows branches.
- install_lock: when filelock is not installed, use os.O_CREAT|O_EXCL
as a fallback exclusive file lock with timeout, instead of yielding
with no locking. Prevents concurrent installs from corrupting each
other's staging directories.
- setup.ps1: remove [DEBUG] log line that printed to every user on
every Windows setup run.
* Add stale-lock detection and atomic clone-then-swap
install_lock fallback (no filelock): write PID to lock file and
check if the holder process is still alive on contention. Dead PIDs
(ProcessLookupError) and unreadable lock files trigger immediate
cleanup. Live processes owned by other users (PermissionError) are
correctly recognized as alive -- the lock is not removed.
setup.sh/setup.ps1 source-build: clone into a temporary directory
first, then swap into place only on success. If git clone fails,
the existing install is preserved instead of being deleted by the
premature rm -rf.
* Remove redundant upstream_tag != release_tag check
load_approved_release_checksums compared checksums.upstream_tag
against the Unsloth release_tag, which are different namespaces
(upstream ggml-org tag vs Unsloth published tag). This only worked
because both happened to be "b8508" by convention. Would break if
Unsloth ever uses a different release naming scheme.
The existing check at parse_approved_release_checksums (line 950)
already validates the release_tag field correctly.
* Fix lock TOCTOU race and build-in-temp-dir swap
install_lock fallback: add os.fsync(fd) after writing PID to ensure
the PID is visible to racing processes before they check. Treat
empty lock files (PID not yet written) as "wait and retry" instead
of stale, closing the window where two processes could both see an
empty file, both unlink it, and both acquire the lock.
setup.sh/setup.ps1 source-build: clone AND build in a temp directory
(LLAMA_CPP_DIR.build.$$). Only swap into the final LLAMA_CPP_DIR
after the build succeeds. If clone or cmake or build fails, the temp
dir is cleaned up and the existing working install is preserved.
Previously, rm -rf ran after clone but before build, destroying the
existing install even if the build later failed.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Daniel Han