* MLX CI: find llama-cli where save_pretrained_gguf actually installs it
The GGUF reload step hardcoded the CWD-relative paths llama.cpp/llama-cli and
llama.cpp/build/bin/llama-cli, but save_pretrained_gguf builds and installs llama.cpp
under unsloth_zoo's LLAMA_CPP_DEFAULT_DIR ($UNSLOTH_LLAMA_CPP_PATH, else
~/.unsloth/llama.cpp), so the reload could not find the binary and failed the Mac M1
job with "llama-cli not found". _find_llama_cli now searches that install directory
(and honors the env override) before falling back to the old CWD layout, with a
recursive glob as a last resort. The search is a strict superset of the previous
paths, so it cannot regress a layout that already worked.
* MLX CI: return an absolute llama-cli path from the locator
Resolve the located binary to an absolute path. If UNSLOTH_LLAMA_CPP_PATH is a
relative directory (e.g. "."), Path(".") / "llama-cli" normalizes to the bare name
"llama-cli", and subprocess.run treats a separator-less argument as a PATH lookup
rather than a file to execute, raising FileNotFoundError. resolve() makes the returned
path absolute so it always runs the intended binary.
* MLX CI: give llama-cli EOF on stdin so GGUF reload cannot hang
With the binary now found, the GGUF reload actually invokes llama-cli and it timed
out after 300s generating 24 tokens on a 270m model, which is a stdin block rather
than slow generation: subprocess.run captured stdout/stderr but left stdin inherited,
so -no-cnv still left llama-cli waiting for interactive input. Pass
stdin=subprocess.DEVNULL so it receives an immediate EOF and runs the single prompt to
completion.
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* studio: announce Cloudflare tunnel state and warn about public exposure on startup
The startup banner only printed a line when a tunnel URL was up, so a plain
`unsloth studio -H 0.0.0.0` launch silently created a public trycloudflare.com
URL with no indication that Studio had become reachable from the internet. The
only hint at the tunnel was the CLI help, shown when an invalid command was typed.
Make the banner always state the tunnel state for wildcard binds:
- ON: the public URL plus a warning that anyone with it can reach Studio from
outside the network, and that --no-cloudflare keeps it local-only.
- FAILED: requested but did not start (local network only).
- OFF: --no-cloudflare was passed (local network only).
Secure mode keeps its existing wording (the authenticated tunnel is intended and
--no-cloudflare is not valid there). Clarify the --cloudflare help text in both
the argparse and typer definitions. Default behavior is unchanged.
Also surface the state on the `unsloth studio run` banner, which runs the server
with silent=True and prints its own banner: it now calls _print_cloudflare_line
too, so the ON/OFF/FAILED notice and public-exposure warning are no longer
skipped on that path (previously it only echoed the URL when a tunnel was up).
For the OFF and FAILED notices, do not claim "local network only" when the
reachability probe just confirmed the raw port is reachable from the public
internet: --no-cloudflare and a failed tunnel disable only the Cloudflare link,
not the wildcard bind, so the message is reworded to flag the public raw port.
* Fix/adjust Cloudflare banner warnings for PR #6515
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust Cloudflare banner comments for PR #6515
* Fix/adjust IPv6 Cloudflare tunnel gate for PR #6515
* Fix/adjust Cloudflare review comments for PR #6515
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix silent run Cloudflare notice
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: stop handing CI/user secrets to downloaded llama.cpp binaries
The macOS prebuilt path installs llama.cpp from the unslothai/llama.cpp
fork's latest (unpinned, mutable) release and then executes the
downloaded llama-server / llama-quantize binaries during install-time
validation. binary_env() built that child environment from a full
os.environ.copy(), so a compromised or tampered prebuilt would inherit
every secret in the process: HF_TOKEN and the workflow GitHub tokens in
CI, and HF / cloud credentials for end users running install.sh /
setup.sh.
We publish prebuilts daily, so pinning a release tag is not workable.
Instead, neutralise the impact: these binaries have no reason to read any
token, so strip secret-bearing variables (exact names plus
TOKEN/SECRET/PASSWORD/CREDENTIAL/PRIVATE_KEY/API_KEY markers) before
handing the env to a downloaded binary. The installer's own GitHub and
Hugging Face API calls read os.environ directly, so authentication and
release-API rate limiting are unaffected; PATH, LD_LIBRARY_PATH,
DYLD_LIBRARY_PATH and CUDA/ROCm vars are preserved. One change covers the
install-time validation path for all six macOS workflows and end users.
Follow-up (separate, sequenced): publish build-provenance attestations
from the fork's prebuilt workflows and verify them in CI, so a forged
release is rejected rather than merely starved of secrets.
* Strip KUBECONFIG, SSH_AUTH_SOCK, and PASSPHRASE-marked vars from binary env
Extend the deny-list per PR review: KUBECONFIG and SSH_AUTH_SOCK are
credential pointers/capabilities a downloaded binary never needs, and a
PASSPHRASE marker catches SSH_PASSPHRASE / GPG_PASSPHRASE. Tests updated.
* Studio: also scrub proxy/index env vars and URL-embedded credentials before running prebuilt binaries
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope mlx-ci secrets to the install + download commands for PR #6696
Drop the ambient step-level env block and pass GH/GITHUB/HF tokens only
on the installer and GGUF-download commands, so the directly invoked
llama-quantize / llama-server smoke runs see no secrets. The installer
still reads tokens from os.environ for the releases API and probe fetch.
* Trim verbose comments around the secret-env scrubber for PR #6696
Comment-only: condense the block comments added across this PR. Logic
unchanged (comment_tools.py check confirms code-only signature equal).
* Redirect HOME / cache pointers to an empty dir for prebuilt binaries (PR #6696)
Address Codex P2: stripping token env vars still let a tampered binary
read on-disk token stores (~/.cache/huggingface/token, ~/.aws/credentials,
~/.config/gh) through $HOME and the cache/config pointers. Point HOME plus
the HF / XDG / Windows home pointers at a single empty throwaway dir for
the downloaded-binary env. Defense in depth: a binary resolving the real
home via getpwuid is out of scope and needs OS sandboxing.
* Close residual credential-probe gaps for PR #6696
Address the latest Codex review:
- Strip token-only URL userinfo too (scheme://ghp_token@host), not just
the user:pass form.
- Redirect HOMEDRIVE/HOMEPATH alongside USERPROFILE so a Windows binary
cannot reconstruct the real profile from %HOMEDRIVE%%HOMEPATH%.
- Drop explicit credential-file pointers (NETRC, PIP_CONFIG_FILE,
DOCKER_CONFIG, GIT_CONFIG_GLOBAL) that live outside HOME.
- Probe ldd with a secret-free env: linux_runtime_dirs ran ldd on the
untrusted prebuilt with the inherited os.environ, and ldd may execute
the binary, so it could observe HF_TOKEN/GITHUB_TOKEN during the probe.
Factored the shared scrub into secret_free_environ().
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Separate token-bearing install from binary smoke; drop CI command files (PR #6696)
Address the two P1s in the latest review:
- mlx-ci: GitHub bakes secrets into the run-script text, so inline token
assignments in a step that later runs the prebuilt let a tampered binary
read them from the script. Split into a token-bearing install + download
step that never launches a binary, and a secret-free smoke step that runs
llama-quantize / llama-server.
- secret_free_environ now drops the GitHub Actions command files
(GITHUB_ENV, GITHUB_PATH, GITHUB_OUTPUT, GITHUB_STEP_SUMMARY, BASH_ENV) and
the smoke step unsets them, so a tampered prebuilt cannot inject PATH/env
into the later token-bearing MLX steps.
* Run the prebuilt smoke last, after all token-bearing steps (PR #6696)
Address the P1 workspace-poisoning vector: even with no secrets in its env,
a tampered prebuilt could edit the checkout or installed modules, and the
later HF_TOKEN MLX steps would then execute that poisoned code on push
builds. Move the prebuilt install + smoke to the end of the job so the
untrusted binary runs after every token-bearing step, leaving nothing for it
to corrupt. The MLX GGUF reload uses a source-built llama-cli, not this
prebuilt, so nothing depends on the earlier position.
* Trim comments around the secret-env scrubber and prebuilt CI steps (PR #6696)
Comment-only: condense the security-rationale block comments and merge the
duplicated prebuilt-step description in mlx-ci. Logic unchanged
(comment_tools.py check confirms the code-only signature is equal; install
suite still passes).
* Authenticate the GGUF export release-API lookup with the read-only GITHUB_TOKEN (PR #6696)
* Rename env scrubber off the secret-named identifier CodeQL flags as a clear-text sink (PR #6696)
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Two intermittent Studio CI failures, both runner-environment flakes unrelated
to test logic:
Windows 'Studio install + inference without Visual Studio': the 'Hide Visual
Studio + CMake' step renames C:\Program Files\Microsoft Visual Studio to
simulate a host with no build tools. A background handle on a Program Files
directory (Defender scan or an MSBuild node) makes Rename-Item intermittently
fail with 'Access is denied', and $ErrorActionPreference = Stop turns that into
a hard job failure. Wrap the VS and cmake renames in both Hide steps in a short
Rename-WithRetry (6 tries, 3s apart) to ride out the transient lock.
macOS 'Chat UI Tests': the re-login goto to /login can be interrupted by the
SPA auth guard redirecting to the same /login URL, which Playwright reports as
'Navigation to .../login is interrupted by another navigation to .../login'.
The goto already tolerated ERR_ABORTED; broaden it to also tolerate the same-URL
interrupt (the password-field wait right after confirms we landed on /login),
and add the same signature to the two Playwright flake-retry harnesses as a
safety net for any other navigation.
Validated: playwright_chat_ui.py parses + byte-compiles, both workflow YAMLs
parse, bash -n on the retry harnesses, PowerShell AST parse on all pwsh steps,
and a functional check of Rename-WithRetry (succeeds, and rethrows after
exhausting retries).
* fix: wrap unprotected evaluate() calls with robust_evaluate() to handle navigation context loss
Fixes PR #5911 - Playwright UI test error: 'Execution context was destroyed'
The test had several direct page.evaluate() and locator.evaluate() calls that
weren't wrapped with robust_evaluate(), which retries when navigation destroys
the execution context mid-operation.
Changes:
- Wrap picker_visible_text() evaluate in robust_evaluate()
- Wrap _bubble_count() evaluate in robust_evaluate()
- Wrap assistant text query in robust_evaluate()
- Wrap theme_item click evaluation in robust_evaluate()
- Wrap background color/theme query in robust_evaluate()
This ensures all execution context losses from concurrent navigation are
properly caught and retried with exponential backoff, preventing transient
failures in the UI test suite.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: revert robust_evaluate on theme_item.evaluate per Codex review
The theme_item.evaluate('el => el.click()') is side-effecting — retrying
after a context loss could double-toggle the theme. It's already inside
a 3-attempt try/except loop that handles click failures gracefully.
The other 4 changes (all read-only queries) remain wrapped in
robust_evaluate() since retrying them is safe.
* fix: wrap remaining chat UI evaluate
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* feat: improve Unsloth Studio chat title generation quality
* fix: address self-review (guard echoed role labels before punctuation stripping)
* Address title generation review feedback
Consolidate the echo guard into a single leading-label check (now also
covering base and lora) and drop the post-punctuation duplicate that
could never match a colon once punctuation is stripped. Swap the
slice-based first-assistant lookup for an indexed find to avoid copying
the messages array, and note the brace counter's assumptions in the
test helper.
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Daniel Han <unslothai@gmail.com>
ensure_diffusion_visual_server() downloaded the visual-server release
asset with the unverified download_file() and marked it executable,
bypassing the approved-checksum manifest that gates every other prebuilt
llama.cpp artifact. The backend later auto-discovers that binary and
launches it through DG_VISUAL_BIN, so a compromised or substituted
release asset could place attacker-controlled native code in the install
tree and have it executed under the Studio user.
Require the matched asset to be present in the approved checksum manifest
and download it through download_file_verified() with the published
sha256. A name-matching asset that is absent from the manifest is refused
rather than executed.
Add regression tests covering the verified-download path and the refusal
of an unapproved asset.
* Pin isolated Node.js installer to committed sha256 digests
The isolated Node installer verified each downloaded archive only against
SHASUMS256.txt fetched from the same nodejs.org origin as the archive, so a
compromised CDN or TLS path could serve a malicious archive plus a matching
checksum and gain code execution when the extracted node is run during the
npm floor check and version probe.
Anchor trust in studio/node_prebuilt_pins.json, a committed manifest of
per-arch sha256 digests, and verify archives against it. The default channel
installs the pinned version and never fetches the remote SHASUMS. Unpinned
lts, latest, or explicit versions fail closed via UnpinnedNodeRefused unless
UNSLOTH_NODE_ALLOW_UNVERIFIED=1, and the refusal is not swallowed by the
keep-existing-on-transient-failure path. Ship the manifest in package-data.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review nits on the pinned Node installer
- Drop the unused npm_min_major field from node_prebuilt_pins.json; the floor is
the NPM_MIN_MAJOR module constant and the dead field could silently drift.
- Reword the unpinned-refusal message so it does not tell a user already on the
default to install it, and point the "add a pin" hint at the exact asset.
- Decode the opt-in SHASUMS body with errors="replace" so a non-UTF8 response
yields a clean PrebuiltFallback instead of an uncaught UnicodeDecodeError.
- Tests: assert the refusal message (guards the main() catch order, not just the
exit code), cover malformed-manifest parsing, and drive the opt-in remote-SHASUMS
path end to end through install_prebuilt.
* Tighten comments in the pinned Node installer
Collapse multi-line rationale comments to single lines, drop docstrings on the
obvious internal helpers (load_pins, pinned_sha256), and shorten the manifest
note. Comments/docstrings only; verified code-unchanged via AST comparison.
* Address Codex review: verify pins on existing installs; tomllib fallback
- existing_install_matches now takes an expected_sha and the short-circuit passes
the committed pin, so a version-matching but non-pinned or tampered install (e.g.
from the old remote-SHASUMS path) is re-verified instead of kept. An unpinned
target without opt-in no longer short-circuits on an existing install; it falls
through to the UnpinnedNodeRefused fail-closed path.
- The package-data test uses pytest.importorskip(tomllib/tomli) so it does not
ModuleNotFoundError on the supported 3.9/3.10 interpreters.
* Make the transient-failure keep-existing path pin-aware
The previous commit added the pinned-digest check to the existing-install
short-circuit but not to the post-download-failure fallback, which still kept any
runnable same-version install via existing_install_usable(). A same-version
install whose recorded sha256 is not the pin could therefore be kept on a
transient download failure, the exact artifact the short-circuit rejects. Refuse
to keep a same-version pin-mismatched install there too; a different usable
version is still kept for offline resilience.
* Bump pinned default Node to the current 24 LTS (24.18.0)
Node 24 LTS moved to 24.18.0; since the default channel now resolves straight to
the manifest, a frozen 24.17.0 would downgrade fresh installs and make
UNSLOTH_NODE_VERSION=lts refuse the current LTS as unpinned. Update default_version
and all six per-arch digests (verified against the official SHASUMS256.txt), and
point the test INDEX/short-circuit fixtures at the new LTS.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix _SameTaskStreamingResponse disconnect test bypassing __init__
test_same_task_response_closes_body_iterator_on_send_disconnect builds the
response via __new__ to skip Starlette's __init__, then wires body_iterator,
background, and stream_response by hand. It never set _unstarted_cleanup, so the
disconnect-before-first-chunk branch of __call__ raised AttributeError instead of
ClientDisconnect, failing the Backend CI "Repo tests (CPU)" job on main.
Set response._unstarted_cleanup = None in the manual construction, matching the
default __init__ assigns.
* Shorten the _unstarted_cleanup comment to one line
* Fix Gemma 4 GGUF OpenAI API streams
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Avoid duplicate Responses stream disconnect watcher
* Keep reasoning-only Responses output hidden
* Address Gemma stream review comments
* Avoid Responses stream task-group cleanup
* Harden OpenAI chat completion streams
* Address OpenAI stream review issues
* Clean up Studio OpenAI stream helpers
* Fix Studio passthrough cold stream timeout
* Fix tool parser compatibility exports lint
* Preserve audio stream disconnect cancellation
* Avoid synthetic finish after passthrough errors
* Address stream cleanup and Gemma parser reviews
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gemma 4: parse bare-string tool args and keep safetensors tools for native <|tool_call>
- Quote bare unquoted string values in Gemma native tool-call args (e.g.
{location:Tokyo,unit:celsius}) so they parse; JSON scalars stay typed.
- Stop _detect_safetensors_features from suppressing supports_tools for
templates that emit Gemma native <|tool_call>, which the shared parser
now reads.
- Add tests for both.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden Gemma tool-call parsing and stream-error detection
Address three issues in the Gemma-native tool-call path:
- _quote_gemma_object_keys stopped a bare (unquoted) string value at the
first comma, so an argument like `location:New York, NY` was split
mid-value and the synthesized JSON failed to parse, dropping the whole
tool call. A bare value now ends only at `}` or a comma that begins the
next `key:` pair.
- parse_tool_calls_from_text scanned the entire response for Gemma markers
even inside a tool call already parsed from a `<tool_call>{...}` JSON
block, so a marker-like string inside an argument (data) was promoted to
a second, unintended tool call. Matches inside an already-consumed call
span are now skipped.
- _openai_passthrough_stream relied on _monitor_openai_sse_line to flag a
stream error, which returns early when monitor_id is None
(skip_api_monitor), so an upstream error chunk left saw_stream_error
unset and the synthetic-finish guard emitted a successful finish_reason
after a failed stream. Error chunks are now detected independently of API
monitoring.
Adds tests/test_gemma_tool_parse_edge_cases.py covering the comma and
marker-injection cases.
* Emit the terminal finish_reason chunk in GGUF streams
The OpenAI chat-completions GGUF tool stream and plain stream both built a
final ChatCompletionChunk carrying finish_reason but never yielded it, so
clients received the optional usage chunk and [DONE] with no chunk carrying
finish_reason. OpenAI-compatible consumers rely on that terminal choice to
distinguish stop/length/tool_calls. Yield it before the usage chunk and
[DONE], matching the other streaming paths.
* Parse tool calls in document order and skip nested markers both ways
Unify the JSON- and Gemma-format tool-call passes into a single
position-ordered scan:
- Calls are now emitted in byte order across both formats, so a mixed
output like `<|tool_call>call:create{...}<tool_call|> ... <tool_call>
{"name":"read",...}</tool_call>` executes create before read, matching
the order they appear in (tools run in returned order).
- A candidate that starts inside an already-accepted call's span is
skipped, in both directions: a JSON marker inside a Gemma argument and a
Gemma marker inside a JSON argument are treated as data, not promoted to
a second executable tool call.
Extends tests/test_gemma_tool_parse_edge_cases.py with the ordering and
JSON-in-Gemma nesting cases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Quote bare Gemma array elements; order finish before trailing usage
- _quote_gemma_object_keys skipped array values, so a Gemma call with a
bare-string array argument like labels:[bug,ui] produced invalid JSON and
the whole tool call was dropped. Array values are now scanned and bare
string elements quoted, while numbers, quoted strings, and JSON literals
are preserved.
- In the OpenAI passthrough stream, a trailing usage-only chunk
(stream_options.include_usage) that arrived before any finish chunk was
relayed before the synthetic finish, producing usage -> finish -> [DONE].
Emit the synthetic finish before that usage chunk so the order matches the
other streams (finish -> usage -> [DONE]).
Extends tests/test_gemma_tool_parse_edge_cases.py with the bare-array cases.
* Harden Gemma array parsing, XML-parameter guard, and stream teardown
Address five review findings on the Gemma tool-call and OpenAI passthrough
streaming paths:
- parse_tool_calls_from_text collected JSON and Gemma markers without the
_inside_open_parameter guard, so a marker embedded in an existing
<function=...><parameter=...> value was promoted to a separate tool call.
Candidates that start inside an open XML parameter are now skipped, matching
the guard the XML-style parser already applies.
- _quote_gemma_array_elements preserved array elements starting with { or [
verbatim, so an array of objects (items:[{path:a}]) or a nested array failed
json.loads and the whole call was dropped. Object and nested-array elements
are now normalised recursively.
- _openai_passthrough_stream synthesized a finish chunk before a trailing
usage-only chunk and set saw_finish_reason, which made the EOF guard skip the
[DONE] sentinel. The EOF path now emits [DONE] whenever the upstream omitted
it, even after a finish chunk was already synthesized.
- /generate/stream drove generation through asyncio.to_thread with no
disconnect watcher, so a client disconnect during a long generation went
unnoticed until the next send. It now runs _await_disconnect_then_cancel
against the request, matching the other local streaming endpoints.
- _SameTaskStreamingResponse closed the body iterator with aclose() on a
send-side disconnect, raising GeneratorExit so the generators' cancellation
handlers (which finish the api_monitor entry) never ran. It now throws
CancelledError, falling back to aclose() when athrow is unavailable.
Extends tests/test_gemma_tool_parse_edge_cases.py with array-of-objects,
nested-array, and marker-inside-XML-parameter cases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Watch disconnects on Anthropic streams; keep timestamps in Gemma values
Two follow-ups on the streaming and tool-parse paths:
- _anthropic_tool_stream and _anthropic_plain_stream drove generation through
asyncio.to_thread(next, gen, ...) and only polled is_disconnected() between
events, so a client disconnect during prefill or a long generation/tool step
held the decode slot until the next event or a failed send. Both now run the
_await_disconnect_then_cancel watcher used by the other local streams, stop it
in finally, and break promptly when cancel_event is set.
- _GEMMA_NEXT_KEY_RE treated any comma followed by word-chars-then-colon as the
next key, so a bare value such as "meet at 10:00, 11:00 tomorrow" was split
into bogus keys. The next-key token must now be identifier-shaped (start with
a letter or underscore), so a comma before a timestamp, ratio, or other
numeric-then-colon text stays part of the value.
Adds a timestamp-in-bare-value regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard nested markers, reset on disconnect, clean unstarted streams
Three follow-ups on the tool-parse and streaming paths:
- parse_tool_calls_from_text only skipped markers that fell inside a span it
had already parsed successfully, so when an unquoted Gemma argument contained
a literal marker (code:<|tool_call>call:terminal{...}<tool_call|>) the outer
object failed to normalize, its span was never recorded, and the inner marker
was promoted to a standalone terminal call. Candidates nested inside any other
candidate's brace span are now skipped regardless of whether the enclosing
candidate parsed, so a marker in malformed outer data is never executed.
- /generate/stream skipped backend.reset_generation_state() when the disconnect
watcher set cancel_event between chunks: the loop broke and the finally's reset
is guarded on cancel_event being unset. A subprocess backend kept decoding
after the client left. The cancel-break path now resets the backend.
- _SameTaskStreamingResponse threw CancelledError / called aclose() on the body
iterator on a send-side disconnect, but neither runs the try/finally of a
generator that never started (early disconnect on http.response.start), so the
passthrough's eagerly-opened upstream httpx stream and cancel-registry entry
leaked. It now tracks whether the body started and, when it did not, runs an
optional unstarted_cleanup hook; the OpenAI passthrough wires it to close the
upstream resp/client and exit the cancel tracker.
Adds a nested-unquoted-marker regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
torchao has no working Windows ROCm build. transformers.quantizers imports it,
and it loads torch's c10d distributed backend at module level, which the AMD
Windows wheels omit (no RCCL). The import aborts, transformers can no longer
expose PreTrainedModel, and the sentence-transformers embedder silently falls
back to the llama-server GGUF embedder. Linux ROCm and NVIDIA are unaffected
(the c10d ops are present / torchao is real there).
The training and export workers already install the shared torchao stub before
importing transformers, but the RAG embedder runs in the main backend process,
which never did. Two fixes, both no-ops off Windows ROCm:
- embeddings.py: install_torchao_windows_rocm_stub() before the first
sentence-transformers import, so an already-installed torchao is neutralized
(fixes existing venvs).
- install_python_stack.py: stop installing torchao on Windows ROCm; it can only
crash on import there, so new venvs never ship it.
Add tests covering the embedder stub call and the install skip.
* studio/setup.sh: guard empty CUDA arch detection in the source build
PR #5826 hardened setup.sh for fresh CUDA toolkits, but the source build
still set -DCMAKE_CUDA_ARCHITECTURES only when nvidia-smi reported a
compute capability. When that query returns nothing the build proceeded
with no explicit arch list, so llama.cpp built PTX only. On a driver older
than the toolkit that binary fails at runtime with "the provided PTX was
compiled with an unsupported toolchain" - the build succeeds, so neither
the build-time check nor the CPU fallback caught it (issue #5854).
Resolve the arch list before committing to a CUDA build. A new pure helper
_resolve_cuda_archs parses and de-duplicates the nvidia-smi compute_cap
output and honors an explicit UNSLOTH_LLAMA_CUDA_ARCHS override. When the
result is empty, build CPU llama.cpp instead of a PTX-only binary, with a
clear message pointing at the override - so the user still ends up with a
working llama-server. The override also lets advanced users force a native
build on hosts where nvidia-smi cannot report compute_cap.
No behavior change when an arch is detected: -DGGML_CUDA=ON plus the arch,
CUDA flags and NVCC_PREPEND_FLAGS are assembled exactly as before.
Adds tests/sh/test_resolve_cuda_archs.sh (single/multi/dedup/empty/garbage/
whitespace/override cases), wired into tests/run_all.sh and the
studio-backend-ci.yml shell-test loop.
* studio/setup.sh: resolve nvidia-smi via /usr/bin fallback for arch detection
Addresses review feedback on the empty-CUDA-arch guard: _setup_has_usable_nvidia_gpu
classifies a host as NVIDIA-usable using nvidia-smi on PATH OR /usr/bin/nvidia-smi,
but the new arch detection probed only `command -v nvidia-smi`. On a GPU host where
nvidia-smi is off PATH (reachable only at /usr/bin), arch detection returned empty
and the new empty-arch branch dropped the build to CPU, losing CUDA. Mirror the same
PATH-then-/usr/bin resolution so those hosts still get a native CUDA build.
Also scope _resolve_cuda_archs locals with `local` (no behavior change; it already
runs under command substitution).
* tests: update compute_cap-probe assertion for $_smi_bin resolution
The nvidia-smi /usr/bin fallback parameterized the binary in the compute_cap
probe (_setup_run_smi "$_smi_bin" ...), so the literal-string assertion in
test_compute_cap_probe_timeout_wrapped no longer matched. Assert the probe is
preceded by _setup_run_smi (timeout-wrapped) instead, scanning all occurrences
so the comment mention is ignored. Same intent, binary-agnostic.
* tests: ruff-format the compute_cap probe assertion (pre-commit)
Collapse the backslash-continued assert onto one line and normalize slice
spacing so the ruff-format pre-commit hook (0.6.9) is satisfied. Formatting
only; no behavior change.
* Tighten code comments (no logic change)
* studio(windows): build CPU when CUDA arch is undetectable (#5854)
The Windows source build added -DGGML_CUDA=ON unconditionally but only set
-DCMAKE_CUDA_ARCHITECTURES when $CudaArch was detected. With no detectable
compute capability that produced a PTX-only binary, the same hole the Linux
fix closed. Build CPU llama.cpp in that case, and honor UNSLOTH_LLAMA_CUDA_ARCHS
to force a CUDA build, matching setup.sh. Detected-arch builds are unchanged.
* test: anchor NVCC_PREPEND_FLAGS scope check on the final CPU branch
The undetectable-arch CPU fallback adds an earlier -DGGML_CUDA=OFF, so the
ordering check now anchors on -DGGML_CUDA=ON and the last -DGGML_CUDA=OFF
instead of the first.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
* Studio Playwright: snooze update banner before sending
The llama.cpp update banner is a fixed bottom-right toast (z-9998). When an
update is available it overlaps the composer's Send button and its subtree
intercepts the click, so send_and_wait times out (flaky; surfaces on the
Windows studio UI smoke, passes otherwise). Snooze the banner if it is
showing before each send, then wait for it to detach.
* Also snooze the web update banner before sending
The web update banner (web-update-banner, z-9999) is a fixed bottom-right
toast like the llama.cpp one and can overlap the Send button too. Loop over
both banners and snooze whichever is showing.
* Windows installer: fix DiskPart UAC, drive-root cache, spurious rename warning, CPU-base messaging
amd-smi gate (DiskPart UAC mid-install): the AMD torch wheel ships hipInfo.exe
inside the venv, and the bitsandbytes fix prepends that venv Scripts dir to PATH.
shutil.which("hipinfo") then found it and flipped _amd_smi_allowed() to True, so
the post-install AMD probe fell through to `amd-smi list` (the venv hipInfo failed
to report gcnArchName, which is why the arch came from the GPU-name table) and
amd-smi elevated, popping the DiskPart UAC. Fix: a hipinfo resolved inside the
active venv (sys.prefix) is the torch-wheel binary, not a HIP SDK, and must not
open the gate. Mirrored in install_python_stack.py, install_llama_prebuilt.py, and
backend utils/hardware/amd.py (the runtime VRAM poller had the same latent prompt).
TORCHINDUCTOR_CACHE_DIR: move from C:\tc to <StudioHome>\TORCHINDUCTOR_CACHE_DIR so
the inductor/Triton cache lives under the user's Studio home, not the system drive
root. Long paths are already enabled above so deep inductor paths still fit.
unsloth.exe rename: skip the rename (and its "pip may fail with WinError 32"
warning) when SKIP_STUDIO_BASE=1. In the install.ps1 flow base packages are not
reinstalled, so unsloth.exe is never rewritten; the self-rename only failed because
setup runs via unsloth.exe (the running launcher holds its own file). The
'studio update' flow still attempts it.
CPU PyTorch messaging: clarify that the CPU base is temporary and setup replaces it
with GPU ROCm wheels, and print an explicit "GPU ROCm PyTorch installed" line after
the AMD wheels land, so the log makes clear the final install is GPU-accelerated.
Adds two regression tests covering the venv-internal vs external hipInfo gate.
Verified end-to-end on a Strix Halo box (Radeon 8060S / gfx1151): install.ps1
--local from this branch completed exit 0 with no DiskPart prompt, no rename
warning, the cache under the Studio home, and "GPU ROCm PyTorch installed
(gfx1151)"; Studio then booted and detected "ROCm (HIP 7.13.99004) -- AMD Radeon
8060S Graphics".
* Windows installer: drop the unreliable unsloth.exe rename and its WinError 32 warning
setup.ps1 used to rename the running unsloth.exe out of the way before the
base-package upgrade so pip could replace it. That rename never actually
worked: setup runs *via* unsloth.exe, so renaming our own running
uv-trampoline launcher failed with a sharing violation (WinError 32) and only
printed a scary 'could not rename unsloth.exe; pip may fail with WinError 32'
warning on every Windows install and update.
It also was not needed. pip tolerates a running/locked console-script .exe: it
moves the old one aside and writes the new one. The base upgrade routes through
pip on Windows, so the upgrade succeeds (or, in the install.ps1 flow with
SKIP_STUDIO_BASE=1, the base is not touched at all) and unsloth.exe is left
intact either way.
Removing the rename block and its failed-install restore block removes the
false warning for all Windows devices in both the install and update flows.
* Windows installer: gate venv-internal hipInfo.exe in PowerShell amd-smi probe; harden venv path checks
Follow-up to PR #6296.
- install.ps1 and setup.ps1: ignore the AMD torch wheel hipInfo.exe that lives
inside the Studio venv when probing for a HIP SDK, so amd-smi no longer reopens
the DiskPart UAC during install/update. Mirrors _path_inside_venv in the Python
installers, which already do this.
- amd.py, install_llama_prebuilt.py, install_python_stack.py: normcase the venv
containment check (Windows paths are case-insensitive) and run the
HIP_PATH/ROCM_PATH candidate through it too.
- setup.ps1: fall back to a short TORCHINDUCTOR cache dir when long paths are
unavailable, and create the dir wildcard-safely.
- tests: isolate sys.prefix in the gate helper, add HIP_PATH/ROCM_PATH cases, and
assert the PowerShell venv exclusion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Windows installer: install ROCm PyTorch directly for a known AMD arch
When the GPU arch is known (name-inferred from the GPU-name table) but ROCm
could not be probe-verified (no HIP SDK, no amd-smi), the bootstrap installed
a CPU PyTorch base that setup.ps1 then force-reinstalled as ROCm. The
repo.amd.com wheels bundle their own runtime (no HIP SDK required), which
setup.ps1 already relies on, so the CPU base was a pure wasted download/install.
- Gate the ROCm index on a known arch, not only on probe-verified ROCm, so a
mapped arch installs ROCm torch directly. Unmapped arches and no-GPU hosts
still get CPU (unchanged).
- Fall back to a CPU base if the ROCm-index install fails, so a transient
repo.amd.com outage does not abort the install (setup.ps1 retries ROCm).
- Correct the stale comment that claimed ROCm wheels need a confirmed HIP SDK.
- Add a regression test for the arch-based gate.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Windows installer: correct the unsloth.exe rename-removal comment
The comment claimed the base upgrade 'routes through pip on Windows' and that
pip 'moves the old unsloth.exe aside, then writes the new one'. That is not what
the code does. install_python_stack tries uv first; on a locked launcher uv
aborts and falls back to pip, but the pip fallback strips --upgrade-package and
base.txt lists only bare unsloth/unsloth-zoo, so pip finds them already
satisfied and no-ops. The running unsloth.exe is left intact at its current
version either way. Reword the comment to describe the real uv-first /
pip-fallback-no-op behavior. No functional change.
* Windows installer: close two gaps in the venv-internal hipinfo exclusion
Review follow-up. The amd-smi/DiskPart gate could still reopen in two cases:
- setup.ps1 ran the HIP probe long before $VenvDir is assigned, so without
VIRTUAL_ENV (the `unsloth studio update` path) $venvRoots was empty and the
venv-internal hipInfo.exe was not recognized. Seed the venv root from
UNSLOTH_SETUP_PYTHON and the default Studio home too (both installers).
- The HIP_PATH/ROCM_PATH candidate was accepted without the venv filter, so an
env var pointing into the venv (AMD wheel) still set $HipSdkInstalled. Run
Test-HipinfoIsVenvInternal on the candidate as well (both installers).
Extend the PS gate test to assert both. Both .ps1 parse clean; install tests
pass (the venv-internal / HIP probe coverage at 359 passed).
* Windows installer: correct the CPU-base message for arches with no ROCm wheels
After gating the ROCm index on a known arch, a mapped arch sets $ROCmIndexUrl
and installs ROCm directly, so it no longer reaches the "temporary CPU base"
branch. That branch is now reached only by a name-inferred arch with no ROCm
wheels (e.g. RDNA2 gfx103X), where setup.ps1 does NOT install ROCm. The old
text ("setup replaces it with GPU ROCm wheels ... the final install IS
GPU-accelerated") was therefore always wrong there. Say plainly that PyTorch
stays on CPU for this GPU.
* Windows installer: seed the venv-internal hipInfo check from a custom Studio home
Test-HipinfoIsVenvInternal seeded the venv root from VIRTUAL_ENV, VenvDir, the
setup python, and the default %USERPROFILE% path only. A standalone
`unsloth studio update` with a custom UNSLOTH_STUDIO_HOME (or STUDIO_HOME alias)
and none of those set would not recognize the venv hipInfo on PATH, reopening the
amd-smi/DiskPart gate. Seed the custom home too, in both installers, and assert
it in the gate test.
* Studio installer: resolve venv aliases and expand ~ in the hipInfo venv filter
Two review points on the amd-smi/DiskPart UAC gate:
1. _path_inside_venv compared os.path.abspath of sys.prefix and the hipInfo
path, which does not resolve symlinks, junctions, or 8.3 short names. A venv
reached through an aliased path then fails the check, so its bundled
hipInfo.exe is mistaken for an external HIP SDK and amd-smi runs (the
DiskPart prompt this fix exists to suppress). Switch to os.path.realpath in
all three copies (amd.py, install_llama_prebuilt.py, install_python_stack.py).
2. setup.ps1's early venv-internal hipInfo probe seeded the venv root from a
custom Studio home (UNSLOTH_STUDIO_HOME / STUDIO_HOME) without expanding a
leading ~, while the canonical resolver does. With a tilde form,
[IO.Path]::GetFullPath kept the literal ~ relative to cwd, so the custom-home
hipInfo escaped the filter and reopened the gate. Expand ~ in the probe the
same way as the resolver.
tests/studio/install/test_pr5940_followups.py: 30 passed (adds a symlink
realpath case and a setup.ps1 tilde-expansion guard).
* Studio installer: mirror the hipInfo venv filter and ROCm wheel pins into install.ps1
Follow-up review on the same install.ps1 paths:
1. install.ps1's venv-internal hipInfo probe (Test-HipinfoIsVenvInternal)
seeded the venv root from a custom Studio home without expanding a leading
~, unlike the canonical resolver and setup.ps1. A tilde form left
[IO.Path]::GetFullPath with the literal ~ (relative to cwd), so the
custom-home hipInfo escaped the filter and reopened the amd-smi/DiskPart
gate. Expand ~ in the probe, matching the setup.ps1 fix.
2. The AMD ROCm path installed torchvision/torchaudio bare while pinning torch
to below 2.12. AMD's per-arch index publishes the companions independently
and may ship torchvision 0.27 (for torch 2.12) before removing 0.26, so a
bare resolve can pick an ABI-incompatible set and fall back to CPU. Add
torchvision/torchaudio floor maps and pass the pinned specs, mirroring
setup.ps1 and install_python_stack.py.
3. The ROCm-to-CPU fallback torch install used Invoke-InstallCommand (no
retry), the only torch step in the file without it. Switch to
Invoke-InstallCommandRetry so the recovery path survives a transient index
failure.
tests/studio/install/test_pr5940_followups.py: 33 passed (parametrized tilde
check over both installers, a torch/companion floor-map parity test, and a
CPU-fallback retry guard).
* Studio installer: scan all PATH hipinfo so the venv copy can't shadow a real HIP SDK
The amd-smi HIP-SDK probe used shutil.which("hipinfo") / Get-Command hipinfo,
which return only the first hit on PATH. The AMD torch wheel ships hipInfo.exe
inside the venv and the bnb fix (plus the Studio backend) prepend the venv
Scripts dir to PATH, so that venv-internal copy lands first. When a real HIP SDK
hipinfo sits later on PATH with HIP_PATH/ROCM_PATH unset, the first-hit probe
stopped at the venv copy, treated it as "not a HIP SDK", and closed the amd-smi
gate -- AMD users in that PATH-only SDK setup lost amd-smi telemetry and could
fall back to CPU. Scan every PATH entry and keep the first hipinfo that is not
venv-internal; only the venv copy is ignored, so the UAC/DiskPart suppression is
unchanged.
Applied to all three Python copies (install_llama_prebuilt.py,
install_python_stack.py, backend/utils/hardware/amd.py) via a new
_external_hipinfo_on_path helper, and both PowerShell callers (install.ps1,
setup.ps1) now use Get-Command hipinfo -All filtered by Test-HipinfoIsVenvInternal.
tests/studio/install/test_pr5940_followups.py: 36 passed (real-PATH scan tests, a
shadow-regression test for the exact venv-first ordering, and a parity check that
every Python copy uses the scanning helper).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio uninstallers: fix leftovers (false "removed", shared icon, llama lock)
Auditing a dual native+WSL uninstall on a real device surfaced three leftovers:
1. uninstall.ps1 removed the data dir (which holds unsloth.ico) before the
shortcuts that reference that icon, so Explorer's icon cache briefly held it
open. Remove-Item -Recurse reported success yet left the locked file, and the
dir was never re-attempted, so it orphaned with a false "removed" log.
_RemovePath now verifies the path is actually gone (retrying transient locks)
and reports honestly, and the data dir is re-swept after the shortcuts go.
2. install.sh writes a shared unsloth.ico to %LOCALAPPDATA%\Unsloth Studio for
the WSL shortcut, but uninstall.sh never removed it, orphaning the icon (and
dir) after a WSL uninstall. uninstall.sh now drops that icon and the dir when
empty, in both the powershell.exe and drvfs-fallback paths.
3. ~/.unsloth/.llama.cpp.install.lock was never removed, so the rmdir of
~/.unsloth failed and the dir lingered. Both uninstallers now remove the lock.
Verified by running both uninstallers on a real dual install: device fully clean
(no install dirs, shortcuts, PATH/registry entries, shared icon, or lock left).
* install.sh: auto-route Strix Halo WSL to an existing Ubuntu 24.04
ROCm-on-WSL is the GPU runtime for Strix Halo and only targets Ubuntu
24.04. When the installer runs in a newer default distro (e.g. 26.04) it
cannot enable the GPU and silently falls back to CPU. If a 24.04 distro
already exists, re-run the install there and stop in the current one so the
GPU path is taken without the user having to know about the distro
requirement.
Runs before venv creation so the wrong distro is left untouched, guards
against re-route loops via UNSLOTH_WSL_REROUTED, leaves a working ROCm
distro alone (librocdxg present), and skips the GGUF-only / opt-out /
non-Strix cases. When no 24.04 distro exists we keep today's behaviour:
continue to CPU and print the `wsl --install Ubuntu-24.04` guidance, never
auto-downloading a distro.
Adds tests/sh/test_strixhalo_wsl_reroute.sh (hermetic: extracts the
function, rewrites its paths to fixtures, mocks wsl.exe) covering the full
decision matrix, wired into tests/run_all.sh.
* uninstall.ps1: keep shared unsloth.ico for a surviving WSL shortcut
A dual native+WSL install shares %LOCALAPPDATA%\Unsloth Studio\unsloth.ico:
install.sh points the WSL shortcut's icon there while the native install owns the
dir. The native uninstaller removed the whole dir unconditionally, so uninstalling
native while keeping WSL left the WSL shortcut with a blank icon. The old code only
avoided this when Explorer happened to hold the icon open, which is unreliable; on a
real dual install the dir was deleted and the WSL shortcut went blank.
_RemoveDataDirKeepingWslIcon now scans the Start Menu + Desktop for a surviving
"Unsloth Studio (WSL ...).lnk" and, if found, removes everything in the data dir
except unsloth.ico (keeping the dir) instead of deleting it; with no WSL shortcut it
removes the dir as before. uninstall.sh still drops the icon and the empty dir when
WSL itself is uninstalled, so every uninstall order ends clean.
Adds tests/studio/test_uninstall_dual_install_icon.ps1 (AST-extracts the helper and
runs it against a temp dir with controlled shortcut dirs) covering the dual,
native-only, empty, and missing-dir cases, wired into the windows-inference smoke
workflow. Verified on a real dual install: native uninstall now keeps unsloth.ico
and the WSL shortcut's icon stays intact.
* installer: condense AMD/ROCm code comments (no behavior change)
Tighten the comments added for the Strix Halo native+WSL installer work so
they are shorter and clearer without losing intent: the venv-internal hipInfo
amd-smi gate, the ROCm torch/companion floor maps, the WSL 24.04 reroute, and
the dual-install uninstall icon handling. Comment-only; code paths unchanged.
107 insertions, 166 deletions across 11 files.
* install.sh: run the Strix Halo WSL reroute before any STUDIO_HOME write
The reroute fired after mkdir -p "$STUDIO_HOME" and the legacy-venv migration,
so rerouting 26.04 -> 24.04 left an empty ~/.unsloth/studio stub in the origin
distro (and ran venv migration in the distro about to be abandoned). Move the
reroute ahead of the venv section so the origin distro is left untouched, matching
the function's own comment. Behavior is identical on every non-reroute path.
* installer: fix ROCm CPU-fallback, hipinfo gate edge cases, uninstall icon, WSL 22.04
- install.ps1: clear $ROCmIndexUrl/$ROCmTorchFloor after the CPU fallback so the
flavor-repair block does not retry the failed ROCm index and abort the install;
pin the ROCm companion specs ($visionSpec/$audioSpec) in the repair path too.
- install.ps1 + setup.ps1: skip a bare drive root in Test-HipinfoIsVenvInternal so a
non-venv UNSLOTH_SETUP_PYTHON does not match the whole drive; iterate
HIP_PATH/HIP_PATH_57/ROCM_PATH and take the first non-venv hipinfo.
- amd.py, install_llama_prebuilt.py, install_python_stack.py: strip surrounding
quotes from PATH entries before probing for hipinfo.
- install.sh: pipefail the WSL reroute curl|sh; do not reroute supported Ubuntu 22.04.
- uninstall.sh: keep the shared unsloth.ico while any Unsloth shortcut (native or
another WSL distro) still references it, in both the powershell and drvfs paths.
- tests: regression coverage for all of the above.
* installer: forward reroute options, guard ROCm bootstrap, harden hipinfo gate
- install.sh: forward the caller's --package/--python/--verbose/--tauri and a custom
UNSLOTH_STUDIO_HOME into the WSL reroute (was a bare default install); bail on
--local; run the reroute BEFORE dependency/uv install so the origin distro is left
untouched; set UNSLOTH_SKIP_ROCM_WSL_SETUP after a failed reroute so the later
ROCm-on-WSL bootstrap does not install into the unsupported origin distro.
- install.ps1 + setup.ps1: Get-Command hipinfo -CommandType Application so only real
executables match (not an alias/function named hipinfo).
- uninstall.ps1: guard $env:APPDATA when building the default shortcut search dirs.
- tests: cover option forwarding, --local bail, the bootstrap guard, and the gate change.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer: guard origin ROCm bootstrap on every CPU-only fallback; harden ~ expansion
WSL reroute: the no-wsl.exe, no-24.04-target and --local fallbacks all tell the
user the install continues CPU-only, but only the failed-reroute branch set
UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The later _maybe_bootstrap_rocm_wsl gate keys off
that flag, so the other three branches could still install ROCm into the
unsupported origin distro (e.g. 26.04). Set the skip guard on all of them.
Forward UNSLOTH_ROCM_WSL_AUTO into the reroute so a Tauri/consented GPU bootstrap
carries through to the rerouted 24.04 child instead of dropping to the prompt path.
install.ps1/setup.ps1: guard the venv-probe ~ expansion on a non-empty
$env:USERPROFILE so Join-Path does not throw on a profile-less service account.
Tests: add no-wsl.exe and UNSLOTH_ROCM_WSL_AUTO reroute cases, the USERPROFILE
guard assertion, and route shell-test fixtures through a single trap-cleaned root.
* installer: pin + soften Windows ROCm Python repair, reroute to 22.04, harden gates
install_python_stack.py: the Windows AMD ROCm repair in _ensure_rocm_torch()
installed bare torch/torchvision/torchaudio via the fatal pip_install -- the same
asymmetry already fixed on the PowerShell side. A transient repo.amd.com failure
could abort the whole install even after install.ps1/setup.ps1 fell back to CPU.
Pin companions per-arch (gfx120X/Strix -> the rocm7.2 trio, mirroring the PS floor
maps) and make the retry nonfatal: keep the existing build and let the user re-run
update to retry ROCm, so the chain install.ps1 -> setup.ps1 -> stack stays CPU-safe.
install.sh: reroute now targets an installed Ubuntu 24.04 OR 22.04 (24.04 preferred);
both are AMD-supported for ROCm-on-WSL, matching the leave-alone set, so a box with
only 22.04 reaches the GPU instead of staying CPU-only.
install.ps1/setup.ps1: a bare ~ for UNSLOTH_STUDIO_HOME left an empty Join-Path child
(PS 5.1 throws); fall back to USERPROFILE directly and only join a real remainder.
_path_inside_venv (amd.py + both installers): guard a root-dir sys.prefix so commonpath
can't classify every path on the drive as venv-internal (defensive; venv never at root).
uninstall.sh: guard an empty LOCALAPPDATA in the PS-interop icon cleanup (mirror APPDATA).
Tests: add 22.04-target reroute cases, Windows ROCm pin+nonfatal coverage (text +
behavioral), root-dir guard coverage, and bare-~/LOCALAPPDATA guard assertions.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: match WSL reroute target by exact distro name, not substring
The 24.04/22.04 reroute target was chosen with grep -F (substring), so a custom
distro such as 'Ubuntu-24.04-test' (with no exact Ubuntu-24.04) was picked as the
target; the later 'wsl -d Ubuntu-24.04' then fails and the Strix Halo install stays
CPU-only. Match whole lines (grep -ixF) and reuse the matched name so only a real
Ubuntu-24.04/22.04 is targeted. Adds substring-rejection + exact-vs-custom tests.
* install.sh: keep the WSL reroute target to Ubuntu 24.04 (helper-supported only)
The ROCm-on-WSL bootstrap (scripts/install_rocm_wsl_strixhalo.sh) dies on any
VERSION_ID other than 24.04 and pins the noble repo, so treating 22.04 as
GPU-supported let the parent report a successful reroute while the child fell
back to CPU. Drop 22.04 from the supported set and the reroute target list;
24.04 stays the sole target (keeping the exact whole-line distro match). An
already-working ROCm on any other version is still left alone by the librocdxg
check above.
tests: reroute 22.04 cases updated to the 24.04-only behavior; make the
"no wsl.exe" case hermetic so a real host wsl.exe can't leak in on dev boxes;
stop the tauri exit-order check from mis-flagging the reroute helper's
[ "$TAURI_MODE" = true ] && ... --tauri one-liner.
* installer: tighten comment wording across the Strix Halo install/uninstall paths
Condense the verbose multi-line comment blocks (amd-smi hipinfo gate, ROCm
torch install + CPU fallback, WSL reroute, uninstall icon-keep) into fewer,
clearer lines. Comments and a few docstrings only; no code, logic, or
behavior change. Verified with bash -n, the PowerShell parser, and ast.parse,
and the installer test suite still passes.
* add AGPL-3.0 SPDX headers to the .sh/.ps1 scripts missing them
Every shell and PowerShell script under the Studio/installer surface now
carries the standard SPDX-License-Identifier: AGPL-3.0-only + copyright
header (after the shebang where present): the installer (install.sh,
install.ps1), build.sh, the .github and src-tauri scripts, the installer
test suite, and the moe kernel test. Header-only, line endings preserved;
bash -n, the PowerShell parser, and the installer tests all pass.
* installer: drop the duplicate AGPL header from install.sh and install.ps1
Both already carry an SPDX-License-Identifier: AGPL-3.0-only header below
their usage comment block; the prior header pass added a second one at the
top because it only scanned the first few lines. Remove the duplicate so each
file keeps a single original header.
* installer: force-reinstall CPU fallback torch; propagate Tauri NEED_SUDO from reroute
install.ps1/setup.ps1: when the AMD ROCm wheel install fails and we fall back to a
CPU base, force-reinstall the torch/vision/audio triplet. A failed ROCm install can
leave an unpinned ROCm torch (e.g. 2.10.0+rocm on gfx110X/gfx90a) that still
satisfies the CPU torch>=2.4,<2.11.0 range, so without --force-reinstall uv keeps the
ROCm build and only swaps the companions -- a mismatched venv the flavor-repair block
won't fix. setup.ps1 scopes the forced reinstall to the ROCm-fallback path
() so the genuine CPU-only install stays fast.
install.sh: the Strix Halo WSL reroute treated every nonzero child exit as a reroute
failure and fell back to CPU. In --tauri mode the child uses exit 2 ([TAURI:NEED_SUDO])
to ask the desktop app to elevate for the target distro; capture the child's exit code
and propagate exit 2 in Tauri mode (the child already printed the NEED_SUDO line)
instead of masking it. CLI mode still falls back to CPU on a generic failure.
Tests: reroute Tauri exit-2 propagation (and non-Tauri CPU-fallback) cases;
run_func now preserves the child exit code; force-reinstall assertions for both
PowerShell installers.
Note: codex's _rr_q apostrophe finding is a false positive -- the helper already
emits POSIX-correct 'O'\''Brien' and round-trips under both sh and bash.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.ps1: fix $cpuForce array collapse in the ROCm->CPU torch fallback
An if-expression assignment ($cpuForce = if ($ROCmCpuFallback) { @("--force-reinstall") })
collapses the single-element array to a scalar string, so @cpuForce splatting enumerated
it character-by-character into broken single-letter args (- - f o r c e ...), which made
uv/pip reject the install and aborted the whole Studio setup on the AMD ROCm->CPU fallback
path. Build $cpuForce as a real array assigned outside the if-expression so the splat passes
a single --force-reinstall arg. Genuine CPU-only installs stay fast (empty array, no flag).
Test now asserts the array-build form and rejects the if-expression form.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* uninstall: remove the isolated Node.js runtime (~/.unsloth/node)
The isolated Node.js runtime (install_node_prebuilt.py, added with the managed-Node
change) installs to ~/.unsloth/node in default mode -- a sibling of studio, so deleting
<studio> leaves it behind (~200MB orphaned after uninstall). Both uninstallers already
remove the other default-mode siblings (llama.cpp/.cache/.staging); add node alongside
them. uninstall.ps1 also adds it to the handle-lock sweep so a held node.exe can't block
the delete. Env/custom mode nests node under the custom root, removed with that root.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: defer llama.cpp update probes and self-heal MLX on macOS
Two macOS startup problems shared one root area in the FastAPI lifespan:
- The llama.cpp capability + freshness probes ran inline before the server
yielded, so a cold/slow/flaky network on the GitHub freshness check blocked
'Application startup complete' (~34s on CI, longer in the field). Move both
probes to a daemon thread; app.state stays None until ready (status routes
already re-probe at request time). Opt out with UNSLOTH_DISABLE_UPDATE_CHECK=1.
- Train and Export were greyed out because mlx/mlx-lm/mlx-vlm arrive only
transitively and a resolver backtrack silently drops them, so CHAT_ONLY stayed
true. Add utils/mlx_repair.py: when Apple Silicon is detected without MLX,
reinstall mlx/mlx-lm/mlx-vlm by name on a daemon thread and re-run hardware
detection (opt out UNSLOTH_DISABLE_MLX_AUTOREPAIR=1). Surface a chat_only_reason
in /api/health plus a sidebar tooltip so a greyed Train/Export explains itself
instead of failing silently.
* Studio: guard model defaults against a None model name
load_model_defaults(None) called model_name.lower() with no guard, raising
'Error loading model defaults for None' before any model is selected. Return
an empty dict for a falsy/non-str name.
* Studio: drop obsolete upstream macOS + Windows Blackwell prebuilt pins
Both pins worked around gaps in ggml-org upstream prebuilts, but Studio now
routes every GPU host and all of macOS to the unslothai/llama.cpp fork
(published_repo_for_host), which ships the needed bundles, so both pins are
dead code on the default install path:
- macOS b9415: macOS always routes to the fork (its own macOS bundles), and
host_supports_macos_minos() is the backstop. The pin only fired under an
explicit --published-repo ggml-org override.
- Windows Blackwell b9360: Windows-NVIDIA routes to the fork, whose
windows-x64-cuda13 bundle covers Blackwell (manifest max_sm 120, toolkit
13.3), so the pin's self-disable check makes it dormant on every default
install; it could only activate under the same upstream override on a
13.0-13.2 driver.
Remove the pin constants, functions, and call sites. Keep the Blackwell
capability detection (_drop_blackwell_incapable_windows_cuda, _host_is_blackwell,
_windows_cuda_attempt_covers_blackwell) that still drops a non-sm_120 cuda-12.4
build on a Blackwell host. After this, an explicit --published-repo ggml-org
override on a Blackwell 13.0-13.2 host loses its GPU fallback and lands on CPU;
the default fork path is unaffected. Update the install selection-logic and
macOS-compat unit tests for the new no-pin behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: walk back deeper on the macOS upstream prebuilt path
After removing the b9415 macOS pin, the explicit --published-repo ggml-org
upstream path still used the default 2-release fallback, so a pre-macOS-26 host
behind a run of macOS-26-only builds would exhaust two too-new plans (minos is
only checked post-download) and drop to a source build before reaching a
loadable older release. Walk back as deep as the fork macOS path
(DEFAULT_MAX_MACOS_RELEASE_FALLBACKS), turning the removed static pin into
dynamic discovery. Addresses review feedback on the macOS upstream fallback.
* Studio: pin transformers during MLX self-heal so it cannot break Studio
mlx-lm/mlx-vlm declare transformers>=5, but the single-env install pins
transformers==4.57.6. The self-heal used --upgrade with no constraint, so it
could upgrade transformers in the live venv and break the rest of Studio just to
make import mlx.core pass. Pin transformers to the installed version via a
constraint file: the resolver either finds an mlx build compatible with it or
fails (we stay chat-only), never upgrading transformers underneath Studio.
Addresses review feedback on the MLX repair install.
* Studio: harden MLX self-heal against an unsupported mlx-vlm
Pinning transformers alone made uv backtrack mlx-vlm to 0.3.9 (below unsloth-zoo's
mlx-vlm>=0.4.4), which imports but breaks VLM Train/Export -- so the self-heal
could clear chat-only onto a broken stack. Mirror the main installer: set
UV_OVERRIDE=overrides-darwin-arm64.txt so a current mlx-vlm coexists with the
transformers pin, require the same minimum versions unsloth-zoo declares, and
gate/validate on a full mlx_stack_available() check (not a bare import) so an
old or partial stack stays chat-only. Addresses PR review.
* Studio: filter Blackwell-incapable CUDA in resolve_upstream_asset_choice
resolve_upstream_asset_choice returned the first windows-cuda choice unfiltered,
so a Blackwell host could be handed an sm_120-incapable cuda-12.4 build while the
sibling planners drop it. Apply _drop_blackwell_incapable_windows_cuda here too
and fall through to the CPU bundle on a Blackwell host with no capable GPU asset.
Addresses PR review.
* Studio: re-poll health so MLX self-heal reaches an open UI
The sidebar cached the initial /api/health, so a successful background MLX
self-heal (chat_only flips false) did not re-enable Train/Export until a manual
reload. While chat-only for the recoverable mlx_unavailable reason, re-poll
/api/health and stop once Train/Export become available. Addresses PR review.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make the disabled Train/Export tooltip reachable
The greyed Train/Export items pass a tooltip explaining why (e.g. MLX missing),
but a disabled <button> fires no pointer events and SidebarMenuButton only showed
tooltips while collapsed, so the explanation never appeared. Wrap a disabled
button in a focusable span and show its tooltip while expanded too; enabled items
keep the collapsed-only behavior. Addresses PR review.
* Studio: gate Train/Export on the full MLX stack, not bare mlx.core
detect_hardware enabled MLX training whenever `import mlx.core` worked, but the
MLX self-heal (utils/mlx_repair) treats a stack without mlx-lm/mlx-vlm at the
versions unsloth-zoo requires as inadequate. That asymmetry let the UI enable
Train/Export on exactly the partial/backtracked stack the self-heal is trying to
repair (greyed-in-but-broken VLM export). Gate on the same mlx_stack_available()
criterion so a partial stack stays chat-only (reason mlx_unavailable) and the
background repair restores it. Addresses PR review.
* Fix MLX repair and health auth for PR #6494
* Fix macOS upstream prebuilt fallback for PR #6494
* Fix MLX stack validation for PR #6494
* Fix MLX self-heal validation for PR #6494
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Review fixes: isolate hardware-state test, robust transformers pin
- test_chat_only_reason.py: detect_hardware() assigns module globals directly,
which monkeypatch does not revert; the autouse fixture now saves and restores
DEVICE/CHAT_ONLY/CHAT_ONLY_REASON/IS_ROCM so a chat-only verdict here cannot
leak into other backend tests (e.g. test_utils.py) on a GPU host.
- mlx_repair.py: read the transformers version from importlib.metadata instead of
importing transformers, so the install pin is not silently dropped when
transformers has valid metadata but fails to import.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix CI: model full MLX stack in dispatch tests, keep selection test offline
dispatch (macOS) job:
- detect_hardware now gates MLX on the full stack (mlx_stack_available imports
mlx_lm/mlx_vlm and checks dist versions), so faking only mlx.core makes the
apple_silicon_mlx profile resolve to CPU. The dispatch tests assert the routing
decision when the stack IS usable, so model a complete stack:
test_hardware_dispatch_matrix patches utils.mlx_repair.mlx_stack_available and
test_is_mlx_dispatch_gate patches hardware._has_usable_mlx_stack. The stack
predicate's own internals stay covered by test_mlx_repair.py.
Repo tests (CPU) job:
- test_no_cuda_attempt_on_published_path_for_13_1 fell through to a live
github_release_assets() upstream fetch after the Blackwell filter dropped every
published attempt, which the offline security scanner blocks. Stub that fetch so
the walk-back deterministically finds no usable CUDA build and raises
PrebuiltFallback without network.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden MLX self-heal: prepare transformers constraint inside the try
attempt_mlx_repair runs on a daemon thread, but _transformers_constraint_args was
called before the try. A failure there (e.g. tempfile.mkstemp on a full disk or a
bad TMPDIR) would propagate unhandled and silently kill the self-heal thread.
Move the call inside the try and initialize constraint_path so any such failure
is caught and leaves Studio chat-only instead of crashing the thread.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Update studio root-resilience tests for the inference-backend refactor
#6490 moved the studio_root() probe and its (ImportError, OSError, ValueError)
handler out of _find_llama_server_binary / _kill_orphaned_servers into the shared
_resolved_studio_root_and_is_legacy() classifier, and switched the WSL ROCm lib-dir
ordering to lib_dirs.extend(_wsl_system_rocm_lib_dirs()). These source-introspection
tests still asserted the old inline structure, so they fail on main (surfaced by any
PR that trips the Repo tests path filter, e.g. the Windows installer PRs). Point them
at the new structure and assert the defense in its new home; no runtime change.
* Address review: qualify the classifier call and harden helper-body extraction
Assert the callers invoke LlamaCppBackend._resolved_studio_root_and_is_legacy()
through the class namespace (more precise than the bare name), and end the
helper-source slice at the next sibling def/decorator at the same indent instead
of the literal @staticmethod string, so a future docstring that mentions a
decorator can't truncate the helper mid-body and break exec().
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Harden evaluate_fetch against execution-context-destroyed during navigation
The Mac Studio chat-UI Playwright test intermittently failed with 'Page.evaluate: Execution context was destroyed, most likely because of a navigation' (e.g. run 27904470348). evaluate_fetch already retried transport failures (the JS result status==0), but the page.evaluate call itself can throw at the Python level when a navigation or auth refresh destroys the execution context mid-call, which was uncaught and crashed the script.
Wrap the evaluate in a try/except that retries this transient class of error (execution context destroyed, frame detached, target closed) within the existing attempt budget, letting the page settle via wait_for_load_state before retrying. Real or persistent errors still propagate (re-raised on the final attempt or for non-transient messages).
* Add reusable robust_evaluate and route auth-token reads through it
Promote the execution-context-destroyed retry from evaluate_fetch into a reusable robust_evaluate(page_or_locator, expression, arg) helper, and have evaluate_fetch use it (no behaviour change to the transport retry). Route the post-login localStorage auth-token reads in playwright_chat_ui.py and playwright_extra_ui.py through it too, since those direct evaluates run right after auth redirects and are the same navigation-race class. The stable composer/IME evaluates that never overlap a navigation are left as-is. Helper retry semantics covered by unit checks (transient retries then succeeds, non-transient re-raises, persistent re-raises after the budget, locator settles via .page).
* Apply repo ruff-format kwarg-spacing to the hardened Playwright evaluates
* Don't replay single-use POSTs (auth/refresh) when robust_evaluate retries a context loss
* Match context-loss markers case-insensitively and only replay idempotent fetches
Playwright varies the casing of the detached-frame/destroyed-context error
across versions, so the substring check now lowercases both sides. evaluate_fetch
no longer replays mutating methods on a mid-call context loss: the default now
retries only GET/HEAD/OPTIONS, so a duplicate POST /api/inference/load (rejected
while the first is still loading) or a spent POST /api/auth/refresh is never
re-sent. Callers can still override per call.
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio: use an isolated Node.js for the frontend build instead of replacing the system Node/npm
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address Node isolation review (no-Node probe crash, PATH refresh, OXC provisioning, venv python, runtime node resolver)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust Node isolation for PR #6533
* Studio Node: don't cache a negative node resolution; accept Node metadata in setup.sh ownership guard
- node_runtime: memoize only a version-adequate executable so a Node installed
by a separate-process 'studio update' is picked up without a backend restart.
- setup.sh: _studio_owned_adoptable also accepts UNSLOTH_NODE_PREBUILT_INFO.json,
matching the setup.ps1 Node ownership guard (custom-home parity).
* Studio setup.ps1: skip OXC npm install gracefully when npm is absent
Mirror setup.sh's `command -v npm` guard so a pip-installed Studio with no
system Node skips the OXC runtime install (validator degrades at runtime) instead
of exit 1 aborting the whole setup. Tighten test_node_probe_guard.ps1's probe
regex so it only matches the two system-version probes, not this new npm guard.
* Wire test_node_probe_guard.ps1 into Windows CI for PR #6533
* Harden isolated Node install and probes for PR #6533
- install_node_prebuilt.py: keep an existing, still-usable isolated Node
when nodejs.org's dist index is unreachable instead of aborting the
update on a transient outage (existing_install_usable + tolerant fetch).
- install_node_prebuilt.py: pin NPM_CONFIG_PREFIX/npm_config_prefix and
drop NODE_PATH in _run_node so any npm -g stays inside the isolated
prefix; Windows npm otherwise writes to %APPDATA%\npm.
- install_node_prebuilt.py: resolve tar hard-link targets against the
archive root (symlink targets stay link-parent relative).
- setup.ps1: wrap the system node/npm probes in try/catch so a present
but broken shim degrades to the bundled Node instead of aborting setup.
- setup.ps1: run the isolated Node install with the handed-off/venv Python
(ReusedSetupPython); the main resolver runs later and bare python may be
a Store stub this early.
- setup.sh: log when the OXC validator runtime is skipped for missing npm,
matching setup.ps1.
- node_runtime.py: move the version-floor comment onto _version_meets_floor.
- Tests for the offline-reuse and broken-shim paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim verbose comments across the Studio Node installer for PR #6533
Comments-only pass: collapse the multi-line section banners to single lines,
drop comments that restate obvious code, and tighten the remaining docstrings
and "why" notes without losing intent. No code changes (verified with an AST
comment-only check on the Python files and a non-comment-diff scan on setup.sh
and setup.ps1). Net 109 fewer lines; the install, decision, and probe-guard
suites stay green.
* Harden Node install from review: validated Python, version floor, legacy home, lock race
For PR #6533, addressing the latest review pass:
- setup.ps1: run the isolated Node install with the validated reused/venv Python.
An incompatible reused interpreter (old venv, conda, stale UNSLOTH_SETUP_PYTHON)
is no longer used; fall back to the resolved python instead.
- setup.ps1: a STUDIO_HOME/UNSLOTH_STUDIO_HOME override equal to the legacy default
now uses the legacy sibling node dir (~/.unsloth/node), matching the runtime
resolver and setup.sh, so OXC can find the Node it installed.
- install_node_prebuilt.py: reject an explicit --node-version below the floor
(^20.19 || >=22.12 || >=23) instead of installing a Node the build cannot use.
- install_node_prebuilt.py: atomically rename a stale install lock before unlinking
so two concurrent runs without filelock cannot both acquire it.
Tests added for the version floor (parametrized + explicit-below-floor rejection).
Full install suite: 937 passed, 1 skipped; setup.ps1 parses; decision tests green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address latest review: armv7l + later-fetch offline reuse for PR #6533
- install_node_prebuilt.py: reject 32-bit ARM (armv7l) up front. Node 24 LTS
ships no linux-armv7l build, so the old path failed late with a confusing
"no sha256"; it now fails fast with a clear unsupported-architecture error.
- install_node_prebuilt.py: extend the offline-reuse fallback to the SHASUMS and
archive fetches. If index.json resolves a newer Node but a later download fails
and a usable isolated Node is already on disk, keep it instead of aborting a
non-force update.
Tests added: armv7l/armhf are unsupported; a SHASUMS failure keeps an existing
usable Node and re-raises when none is present. Full install suite: 941 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add UNSLOTH_STUDIO_HOME node-dir tests (install side + resolver) for PR #6533
* Add regression tests pinning the reuse path read-only and isolating installer writes
Lock in the two invariants behind the isolated-Node design: reusing a good
system Node never mutates the user's Node/npm, and the installer's own npm
calls only ever write inside its install_dir.
- tests/studio/install/test_install_node_prebuilt_logic.py: assert _run_node
redirects NPM_CONFIG_PREFIX/npm_config_prefix into install_dir and drops an
inherited NODE_PATH; assert _ensure_npm_floor scopes the npm self-upgrade to
install_dir (never -g against the system) and is a no-op once npm meets the floor.
- tests/sh/test_system_node_readonly.sh (new, wired into studio-backend-ci.yml):
the setup.sh NODE_SOURCE=system arm runs no global install and sets no
NPM_CONFIG_PREFIX, with a positive control that the bundled arm does.
- tests/studio/test_node_decision.ps1: symmetric structural guard that the prefix
pin and the only global install (bun) live in the bundled branch, not the system arm.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* studio: run /generate/stream's sync generator off the event loop to avoid blocking it
* fix: close generator in finally on client disconnect in generate_stream
* Fix/adjust generate stream test for PR #6466
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust generate stream cancellation for PR #6466
* Fix/adjust generate stream cleanup for PR #6466
* fix: cancel incomplete generate stream cleanup
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* Windows installer: repair a stale CPU PyTorch instead of looping forever
A Windows machine with an NVIDIA CUDA 13 driver (e.g. RTX 6000 Pro on enterprise
drivers) could get permanently stuck at:
Stale venv detected (torch cpu != required cu130).
[ERROR] The existing Studio environment needs repair.
Re-run install.ps1 so it can replace the environment safely with rollback.
Re-running install.ps1 did not help. install.ps1 installs torch with
"torch>=2.4,<2.11.0" --index-url .../cu130 but no --force-reinstall, so when a
torch==X+cpu is already present uv treats it as satisfying the range (PEP 440
ignores the +cpu/+cuXXX local label) and makes no change -- the CPU wheel is
never replaced. setup.ps1 then rejects the venv as cpu != cu130 and exits, but it
cannot create a venv or install torch, so the loop never resolves. The migrated-
venv branch also preserves existing torch and never reinstalls it.
After the install step, detect the installed torch flavor (cuXXX/cpu/rocm) and,
when it does not match the tag implied by the selected index, force-reinstall the
torch/torchvision/torchaudio triplet from the correct index via three
--reinstall-package flags. No-op on a healthy matching venv; skipped for
--no-torch, ROCm (already --force-reinstalls), and CPU-only machines.
Adds two pure helpers (ConvertTo-TorchFlavorTag, Get-ExpectedTorchFlavorTag), a
PowerShell unit test (tests/studio/test_torch_flavor.ps1), and a CI parse gate for
install.ps1 (previously unparsed).
* install.sh: repair a stale CPU PyTorch on Linux too (parity with install.ps1)
install.sh has the same latent bug as the Windows installer: the CUDA torch
install uses "torch>=2.4,<2.11.0" --index-url .../cuXXX with no
--force-reinstall, so an already-present torch==X+cpu satisfies the version
range (PEP 440 ignores the +cpu/+cuXXX local label) and uv leaves it in place.
The migrated-venv branch also preserves existing torch. Unlike Windows there is
no stale-venv check in setup.sh, so on Linux the symptom is silent CPU training
rather than a hard loop -- same root cause.
Mirror the install.ps1 fix: after the install block, detect the installed torch
flavor (_torch_flavor_tag) and, when it does not match the index tag
(_expected_torch_flavor_tag), force-reinstall the torch/torchvision/torchaudio
triplet from the selected index via --reinstall-package. No-op on a healthy
matching venv; skipped for --no-torch, ROCm (its own repair force-reinstalls),
and CPU-only / macOS hosts. Adds tests/sh/test_torch_flavor.sh (run in
studio-backend-ci and run_all.sh).
* Installer: catch CPU-fallback on AMD/WSL too (repair ROCm, warn when unfixable)
Extend the torch-flavor safety net beyond NVIDIA:
- install.sh now auto-repairs a stale CPU torch on standard pytorch.org ROCm
indexes too (the rocm-index install path lacked --force-reinstall, unlike the
Windows ROCm install). Reuses the rocm-adjusted $TORCH_CONSTRAINT + rocm index,
so it pulls the correct ROCm wheels.
- Both installers gain a universal post-install warning: when a GPU build was
expected (cuXXX / rocm, including the repo.amd.com gfx* arch indexes) but torch
is still CPU-only, warn loudly instead of silently training on CPU. This catches
the cases auto-repair cannot safely fix (AMD gfx arch indexes that need
--find-links, a migrated AMD venv on Windows where the ROCm install was skipped).
- Mac / Intel / CPU-only hosts resolve to the cpu index -> expected == installed
-> no-op, no false warning. WSL uses install.sh, so the NVIDIA repair + warning
apply there.
Adds Get-InstalledTorchTag (ps1) and _torch_index_repairable (sh) helpers and
extends both unit tests. gfx*/AMD indexes now map to the 'rocm' expected flavor.
* Installer: tighten torch-flavor comments (no logic change)
Condense the rationale comments added for the stale/CPU PyTorch repair in
install.ps1, install.sh and the two helper unit tests; same intent, fewer
lines. Comment-only: AST parse of install.ps1/setup.ps1 clean, helper unit
tests (15 ps1, 24 sh under bash and dash) and the integration sims
(24 ps1, 28 sh) still pass, banner markers the sims slice on are unchanged.
* Installer: bound torch probe, auto-repair gfx, fix ROCm gate parity
install.ps1: in Get-InstalledTorchTag, call WaitForExit(30000) and drain stdout
and stderr asynchronously instead of reading stdout synchronously first, so a
hung or noisy "import torch" (a wedged CUDA/driver, the exact failure this PR
targets) can no longer block the probe past the timeout.
install.sh and install.ps1: treat the repo.amd.com gfx* indexes as plain
--index-url reinstallable. They are PEP 503 simple indexes uv resolves in full
(torch plus every transitive dep) via --index-url, the same URLs the fresh
ROCm install paths already use, so a stale CPU torch on AMD Strix now auto-repairs
to the correct ROCm build instead of only warning.
install.sh: include */gfx* alongside */rocm* in the bitsandbytes install and
ROCm torch repair gates, so a custom UNSLOTH_AMD_ROCM_MIRROR whose path lacks
/rocm/ still installs the AMD bitsandbytes build and repairs ROCm torch.
tests/sh/test_torch_flavor.sh: gfx indexes now assert repairable, plus a
gfx1151 case and an unknown-mirror not-repairable case.
* install.ps1: guard Get-InstalledTorchTag against an empty python path
Make the early return explicit for an empty $PythonExe instead of relying on
Test-Path -LiteralPath '' returning false, so the probe stays safe under
Set-StrictMode or a future refactor that drops the [string] annotation.
* Studio: reach the published source asset when a mix build's commit 404s
A llama.cpp "mix" prebuilt records a merge commit that is never pushed to
the fork, so the codeload/archive URLs for that commit 404. The merged
source tree is instead published as a release asset alongside the prebuilt
(llama.cpp-source-commit-<sha>.tar.gz). The installer resolves that asset
URL from the approved-checksums manifest, but when the manifest omits the
top-level repo/release_tag the URL resolves empty, hydration falls through
to the 404-ing commit archive, and the prebuilt install drops to a slow
source build (or fails outright).
Extract exact_source_asset_url() and resolve the asset's host and tag
defensively: the artifact's own repo, then the manifest repo, then the
source repo; and the manifest release tag, then the tag we actually
installed the prebuilt from (the source asset is its sibling on the same
release). Normal installs build the identical URL as before, so this only
adds a working fallback for the degenerate manifest.
Add unit coverage for the resolver, including the empty repo/release_tag
regressions.
* Studio: cover exact_source_asset_url through the real parser chain
Add TestExactSourceAssetUrl.test_resolves_through_real_parser_chain, which runs
parse_approved_release_checksums -> preferred_source_archive -> exact_source_asset_url
so a regression in the parser or source-selection wiring cannot pass while only the
hand-built helper unit tests stay green.
* Keep server-side tools enabled under --secure and on every bind
--secure binds loopback and exposes Studio only through an authenticated
Cloudflare HTTPS tunnel, but it was grouped with a raw 0.0.0.0 bind and
force-disabled all server-side tools (web search, Python, terminal). The
process tool policy overrode the client's enable_tools request, so the
model was never told the tools existed and answered in plain text. The
plain 'unsloth studio' command had no way to re-enable and printed nothing.
Tools now default on for every bind. The bind host and --secure no longer
change the tool policy; only an explicit --enable-tools/--disable-tools
forces it on or off. Both 'unsloth studio' and 'unsloth studio run' accept
the flags and the startup banner states the resolved policy.
- run.py: replace _apply_default_tool_policy(host, secure) with
_apply_cli_tool_policy(enable_tools); add an enable_tools kwarg to
run_server and --enable-tools/--disable-tools to the argparse.
- _tool_policy.py: resolve_tool_policy defaults to on for every host and
no longer prompts on a network bind.
- studio.py: drop the secure-as-public tool gating, add the flags to the
plain command, and reword the startup banner.
- Update and extend the secure-flag and tool-policy tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add tool-policy notice to plain server banner and refresh run --help
Follow-up to PR review:
- run.py: the plain 'unsloth studio' / --secure / direct run.py path went
through _emit_startup_output without any tool-policy line, so a
network-reachable launch was silent about code execution now that tools
default on. Thread enable_tools through _emit_startup_output /
_emit_secure_startup_output and print a one-line policy notice, followed by
a single stop hint.
- studio.py: the 'unsloth studio run' --enable-tools/--disable-tools and --yes
help still described the removed loopback-on/network-off default and the
confirmation prompt; reword to match the new policy.
- Add tests for the banner notice and the refreshed help text.
* Update CI tool-policy resolver tests for default-on behavior
tests/python/test_unsloth_run_tool_policy_resolver.py still asserted the
removed network-bind policy (0.0.0.0 and LAN IP default off, explicit enable
prompts and aborts on a declined prompt), so it failed the Python CI jobs.
Rewrite the truth table: every bind defaults on, explicit on/off always wins,
and the resolver never prompts (yes/silent/prompt kept for compatibility).
* Trim comments for the tool-policy change
Shorten the verbose docstrings and block comments added for --secure tool
handling; keep the security-relevant intent. Verified comment-only via an AST
diff (code unchanged).
* Add deterministic test that server-side tools execute under --secure
Drive the GGUF agentic tool loop with a fake llama-server stream and let the
real execute_tool run: python counts 1..100, terminal returns a UTC datetime,
and web_search runs through real _web_search with only the ddgs network
boundary mocked. A policy assertion pins that the post-fix --secure path
(policy None + per-request enable_tools) is what keeps these executions
reachable. No model, GPU, or live network; runs in the existing backend CI.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align _emit_startup_output banner test with the moved stop hint
The tool-policy notice now prints between the access banner and the stop
hint, so the stop hint is emitted once at the end instead of inline in the
banner (include_stop_hint is False and print_studio_stop_hint runs once).
Update the plain-localhost case to match; the mismatch and wildcard cases
already asserted this wiring.
---------
Co-authored-by: Michael Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Reduce and tighten comments and docstrings in tests
Shorten verbose comments and docstrings across the test suite without
changing any test logic. Remove narration that restates the next line,
collapse long module and test docstrings to a single line, and drop banner
separators. Keep regression context (issue and PR references, run ids),
skip reasons, mocking and timing rationale, license headers, lint and type
directives, and commented-out code.
Comments and docstrings only: an AST signature check confirms no code,
assertions, or string literals changed, and the suite byte-compiles cleanly.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The Linux installer ordered its CUDA runtime-line attempts purely by torch's
reported CUDA major (preferred_runtime_line), so a Blackwell host running a
cu12x torch build hoisted cuda12 ahead of an available native cuda13 bundle.
This brings the Linux selector to parity with the existing Windows Blackwell
preference: on an sm_120 host, prefer the highest CUDA-major line that ships
a bundle covering every visible host SM, then fall back to the torch line.
Selection-time only. No external pin and no source build: in-release cuda13
bundles already cover sm_120, and the per-artifact SM filter still drops any
incapable bundle (cuda12-older / cuda13-older) and prevents fall-through to a
non-Blackwell build. The override is gated on _host_is_blackwell and only
reorders lines that are already detected and driver-compatible, so it never
forces cuda13 when its runtime libraries are absent or the driver is pre-13,
and non-Blackwell hosts keep the exact torch-preference behavior.
The runtime-line ranking only considers well-formed "cuda<major>" lines and
skips any malformed or future-format value (e.g. "cuda13.1") instead of
crashing the major sort, matching how the surrounding selector already
tolerates unknown lines.
Adds focused selection tests covering the override, the incapable-cuda13
skip, the cuda13-unavailable fallback, the non-Blackwell no-op, the
malformed-runtime_line skip, and cuda14 forward-compat.
* studio: set _stats_logger in kill-process test backend
#6377 added a self._stats_logger cleanup step to _kill_process's finally block.
test_kill_process_records_timestamp_on_actual_kill (added in #6400) builds the
backend via __new__, which bypasses __init__ where _stats_logger is set, so once
both landed on main the test raised AttributeError: 'LlamaCppBackend' object has
no attribute '_stats_logger'. Set _stats_logger on the hand-built backend,
mirroring __init__, so the kill path's finally has the attribute it expects.
* test: assert torchao override step on normal Linux, not overrides.txt
#6400 moved the torchao dependency override from a fixed pin in overrides.txt to
a torch-matched spec installed via --force-reinstall (_select_torchao_spec), and
turned overrides.txt into a comment-only pointer. It updated the Windows variant
(test_windows_only_includes_overrides) to check for --reinstall, but left
test_normal_linux_includes_overrides asserting overrides.txt is installed, which
no longer happens. Check for the override step (--reinstall) instead, matching
the Windows test.
* test(ui): tolerate ERR_ABORTED on /login re-login in shutdown step
The Shutdown step re-logs in after a CLI password rotation that revoked the prior
token. The SPA auth guard can client-side-redirect mid-navigation against the
stale token, aborting page.goto("/login") with net::ERR_ABORTED. It is a race
(passes on main most of the time). Resolve on domcontentloaded and tolerate the
abort, relying on the password-field wait that follows to confirm we reached
/login, matching the wait_until used by the other navigations in this file.
* Studio: drop the VBS launcher to clear the Kaspersky false positive
The Windows shortcut launched Unsloth Studio through wscript.exe ->
launch-studio.vbs, and that VBS used CreateObject("WScript.Shell").Run to
start a hidden -ExecutionPolicy Bypass PowerShell. That wscript + .vbs +
bypass-powershell shape is the canonical trigger for generic VBS-dropper
heuristics (Kaspersky HEUR:Trojan.VBS.Agent.gen). The launcher is benign;
only its shape is the problem.
- install.ps1: stop generating launch-studio.vbs and point the Desktop /
Start Menu .lnk straight at powershell.exe -WindowStyle Hidden running
launch-studio.ps1. The shortcut is saved WindowStyle 7 (minimized) so the
brief console flash is muted. launch-studio.ps1 (health poll, port,
mutex, browser) is byte-for-byte unchanged.
- install.ps1: delete a pre-existing launch-studio.vbs on upgrade, so the
flagged file does not linger on machines that already installed it.
- install.ps1 / install.sh: run the heavier ie4uinit -ClearIconCache plus
StartMenuExperienceHost tile-cache rebuild only on a first install or a
real icon change, instead of on every no-op reinstall. That repeated
clear-cache plus kill cluster is itself a dropper-like behavioral pattern.
- tests: forbid VBS generation and require the legacy-VBS cleanup.
Linux, macOS and WSL install paths are unchanged. WSL already targets
wsl.exe from its .lnk and never used a VBS; its only change is the same
icon-cache gating.
* Studio: add launcher-chain smoke coverage to the Windows UI CI
The shortcut launch path was previously untested: studio-windows-ui-smoke
installed then booted `unsloth studio` directly, so a broken .lnk could ship
silently. After install the job now seeds a legacy launch-studio.vbs, asserts
the upgrade removed it, asserts the .lnk targets hidden powershell.exe (never
wscript.exe), and launches via the shortcut's stored command, waiting for
/api/health to report healthy.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: use partial hipinfo output on crash to avoid CPU fallback (#6043)
`hipinfo.exe` on some RDNA 4 hosts (e.g. RX 9060 XT / gfx1200) exits
with STATUS_ACCESS_VIOLATION (0xC0000005) after printing the
gcnArchName line. The previous guard `$LASTEXITCODE -eq 0` in
studio/setup.ps1 and `if result.returncode == 0` in
install_python_stack.py discarded this partial-but-valid output,
causing the installer to fall through to WMI name inference which sets
HasROCm=false and installs CPU PyTorch instead of the ROCm wheel.
Fix: check for gcnArchName in stdout first; accept the arch regardless
of exit code. Only fall through to the amd-smi / WMI path when no
gcnArchName is present at all (crash before any output, or a genuine
"no device" error). A cyan INFO substep is emitted when the arch is
recovered from a crashed hipinfo run so users can see what happened.
Adds a regression test covering the crash-with-valid-output path.
Fixes#6043
* Fix/adjust hipinfo crash fallback for PR #6292
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Studio: fix Mac IME input-method switch leaving composer Send disabled
On macOS, switching input method (Ctrl+Space / menu-bar language icon)
fires compositionstart but never compositionend — leaving composingRef
pinned at true and the Send button permanently disabled even after
switching back to English.
Two immediate recovery paths added to useImeComposerInputHandlers
(thread.tsx) and SharedComposer (shared-composer.tsx):
* onKeyDown else-if: clears composingRef on the first non-IME keystroke
after a stuck composition, unblocking Send on that very keydown rather
than waiting for the 2500ms watchdog.
* onBlur handler: clears composingRef unconditionally on textarea focus
loss — safe because the OS always commits or cancels any active
composition before surrendering focus to another element.
Two new Playwright regression steps (6e, 6f) added to
playwright_chat_ime_i18n.py assert recovery within 1500ms (well below
the 2500ms watchdog), covering both recovery paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix IME regression test idle handoff for PR #5762
* Fix IME cleanup console guard for PR #5762
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix IME Enter guard for PR #5762
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* fix(studio): Windows GGUF cancel hang + CPU spinlock overhead (#5692)
Two fixes for Windows-native GGUF inference via llama-server:
**Issue 1 — GPU/CUDA Hang on Stream Cancellation:**
- Add `Connection: close` header to all httpx requests proxying to
llama-server, preventing Keep-Alive from masking downstream socket
closure.
- Introduce `_await_disconnect_then_close` background watcher that
polls `request.is_disconnected()` every 100ms and calls
`resp.aclose()` immediately when the client disconnects. This runs
alongside the existing cancel-POST watcher and covers client aborts
that never reach the /cancel endpoint (tab close, proxy aborts,
Colab, mobile navigation, etc.).
- Change all StreamingResponse `Connection: keep-alive` headers to
`Connection: close`.
**Issue 2 — High CPU Spinlock & KV Cache Backup Overhead:**
- Set OMP_WAIT_POLICY=PASSIVE and OMP_NUM_THREADS=2 in the
llama-server subprocess environment on Windows to prevent OpenMP
from spin-waiting on all logical cores while the GPU decodes.
- Limit `--threads` to 2 on Windows when the model is fully
GPU-offloaded (`-ngl -1`). Auto-detect otherwise.
- Pass `--cache-ram 0 --ctx-checkpoints 0 --no-cache-prompt
--checkpoint-every-n-tokens -1` on Windows to disable prompt-cache
snapshots that copy KV cache to system RAM over the WDDM/PCI-E bus.
Closes#5692.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: use local import to avoid ruff F823 (sys used before assignment)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* review: address gemini review feedback
- Simplify _fully_gpu_offloaded init: default to False, only set True
in the gpu_indices branch, drop redundant else.
- Log exceptions in _await_disconnect_then_close at debug level instead
of silent pass, per review suggestion.
* Adjust review feedback for PR #5749
- _await_disconnect_then_close: set cancel_event before resp.aclose() so
the streamer's RemoteProtocolError handler treats the watcher-driven
close as cancellation, not an upstream error. Both call sites pass
cancel_event through.
- Windows --cache-ram / --no-cache-prompt / --ctx-checkpoints block: gate
on _fully_gpu_offloaded so CPU and partial-offload Windows runs keep
prompt-cache reuse across turns.
- Windows OMP_WAIT_POLICY / OMP_NUM_THREADS env: same gate so CPU and
partial-offload Windows runs keep default OpenMP parallelism.
* Shorten code comments touched by PR #5749
* Clean up local imports and rename underscore locals in PR #5749
- Drop the function-local `import sys as _sys` introduced as an F823
workaround; remove the redundant in-function `import os`/`import sys`
block so module-level imports resolve sys/os instead. F823 no longer
triggers because no shadowing import remains inside load_model.
- Rename `_fully_gpu_offloaded` and `_t` to `fully_gpu_offloaded` and
`threads_arg`. Underscore-prefixed names usually mean private/module-
level; plain locals match Python style for in-function temporaries.
No behavior change. ruff clean, py_compile clean, 35 studio cancel-
infra tests + 13 launch-gating AST locks + 6 disconnect-watcher locks
+ 4 spoof live-import tests all pass.
* Fix Windows GGUF follow-ups for PR #5749
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix cache flag gating for PR #5749
* Fix Python 3.9 annotations for PR #5749
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Anmol Mishra <anmolx.work@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Expose MLX grad value clipping in Studio
* update test
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* dataset ordering + wd
* fix mlx smoke step expectations
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* cast norm activation output back to original input dtype
* address mlx studio review feedback
* Fix present-but-None seed override for PR #5656
studio/backend/core/training/worker.py
`config.get("model_random_state", random_seed)` only fills the
default when the key is absent. When a caller passes
`config["model_random_state"] = None` explicitly (which happens
any time a JSON payload sends an explicit `null`), the old code
forwarded `None` to FastMLXModel and disabled deterministic init
silently. Same for `lora_random_state`. Treat absent and explicit
None the same way: fall back to random_seed.
studio/backend/tests/test_training_raw_support.py
Update the source-string assertions to match the new lines.
* Guard optional MLXTrainingConfig fields and normalize random_seed for PR #5656
The MLX worker now passes `cast_norm_output_to_input_dtype` and
`dataset_order` only when the linked unsloth-zoo dataclass actually
declares them. Released zoo trees that predate the paired PR can still
construct `MLXTrainingConfig` without raising
`TypeError: unexpected keyword argument`. Once the dependency floor is
bumped to a release that contains both fields, the feature-detect
guards become no-ops.
`random_seed = config.get("random_seed", 3407)` was unguarded against
explicit `None` from raw / backend callers. The same value seeded the
trainer and was the fallback target for `model_random_state` /
`lora_random_state`. Normalize once at the top of the function and use
the normalized value everywhere so an explicit `None` cannot reach
FastMLXModel / get_peft_model / MLXTrainingConfig.
Existing seed source-pattern test updated to match the new normalize
helper. New test asserts the feature-detection guards exist and that
the unconditional kwargs do not include the gated fields.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Normalize seed / cast / max_grad_value at TrainingBackend for PR #5656
Round-3 review consensus: the per-field guards that landed in the MLX
worker only protect the MLX path. The same `TrainingBackend.start_training`
config still reaches the CUDA/text trainer at `worker.py:2267`, the
embedding LoRA init at `worker.py:2450`, and embedding TrainingArguments
at `worker.py:2624` with raw `None` values, so an explicit
`random_seed=None` from a raw / backend caller still breaks non-MLX
training even after the previous fix.
Move the normalization into `TrainingBackend.start_training` itself,
where it runs once for every training mode:
- `_coerce_seed(value)`: explicit `None`, non-int, or absent all become
3407. Every downstream worker now sees an int.
- `_coerce_optional_bool(value, default)`: explicit `None` falls back
to `default` instead of `bool(None) == False`. Also normalizes the
common raw-config / YAML string aliases ("true" / "false" / "0" /
"1"). Used for `cast_norm_output_to_input_dtype`.
- `_coerce_optional_nonneg_float(name, value)`: rejects negative
numerics from raw / backend callers, matching the Pydantic
`ge=0` constraint the HTTP route already enforces. Used for
`max_grad_value`.
worker.py MLX path: the existing `bool(config.get(key, True))` for
`cast_norm_output_to_input_dtype` was changed to also fall back on
explicit `None`, so direct worker callers (bypassing
`TrainingBackend.start_training`) are equally safe. `max_grad_value`
also raises on negative values inside the worker for the same reason.
TrainingStartRequest.random_seed default bumped from 42 to 3407 so
direct REST callers that omit the field receive the same default as
the Studio frontend and the MLX worker.
New regression test exercises the three new helpers across explicit
None, valid values, string aliases, and negative-value rejection.
* Tighten feature-detect test paren tracking for PR #5656
The block-extraction used , which stops at the
first inner closing paren (e.g. )
and would silently miss a future unconditional
/ added later in the same dict literal. Switched to
proper paren-depth tracking so the unconditional block is checked end-to-end.
* Shorten verbose comments in MLX Studio backend
* Handle MLX Studio EOS appending by mode
* Wire MLX leaf norm clipping through Studio
* Respect VLM layer filters for explicit LoRA targets
Rationale / guardrails for the local Studio/vision push:
When callers provide explicit VLM LoRA target_modules together with layer filters, FastVisionModel still needs to route the explicit targets through get_peft_regex. Otherwise the layer filters are ignored and adapters can be attached outside the requested language/vision scope.
Do not revert this to plain list(target_modules) for explicit module lists. The CUDA/Studio-facing contract is that explicit targets and layer filters compose: target_modules selects module names, while finetune_language_layers / finetune_vision_layers / finetune_attention_modules / finetune_mlp_modules constrain where those targets are allowed.
The regression test covers the language-only explicit q_proj case and source-checks that explicit targets are wrapped through get_peft_regex when filters are active.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refresh MLX smoke clip-config note for leaf_norm default
Trim the 11-line comment block to 5 lines and correct the stale claim
that MLXTrainingConfig defaults to max_grad_value=1.0. The new default
is max_grad_leaf_norm=1.0 (same memory profile as elementwise but
direction-preserving). The smoke still pins max_grad_value=1.0
explicitly to keep the 13-seed pass-rate fixture stable.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Forward max_grad_leaf_norm through the training route and warn when layer filters constrain explicit target_modules for PR #5656
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han-Chen <info@unsloth.ai>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Installer: drop redundant -WindowStyle Hidden from the Windows launcher VBS
The desktop / Start Menu shortcut launches Studio through a generated
launch-studio.vbs that runs:
shell.Run "powershell ... -WindowStyle Hidden -File launch-studio.ps1", 0, False
The second argument to shell.Run is intWindowStyle 0 (hidden), so WScript
already launches the child windowless. The child -WindowStyle Hidden is
therefore redundant: dropping it keeps the launcher hidden and behaviour
identical, while removing the WScript-spawns-hidden-ExecutionPolicy-Bypass
PowerShell token combination that antivirus heuristics weight. That shape was
reported as a Kaspersky HEUR:Trojan.VBS.Agent false positive during install.
Adds tests/studio/install/test_launch_studio_launcher.py to stop the flag from
being reintroduced and to assert the launcher stays windowless via
shell.Run(cmd, 0, False).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: extend llama.cpp first-token timeout
* fix: timeout label pluralization
* studio: distinguish llama stream timeout phases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust timeout handling for PR #5841
* Fix lint failure for PR #5841
* Fix/adjust stream timeout handling for PR #5841
* Fix/adjust first token timeout for PR #5841
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust passthrough timeouts for PR #5841
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust preheader stream cancellation for PR #5841
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust timeout PR diff for PR #5841
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust Python 3.9 stream iteration for PR #5841
* Fix first body timeout for PR #5841
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix first token timeout deadlines for PR #5841
---------
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
The prebuilt bundles ship llama-diffusion-gemma-visual-server, but
runtime_patterns_for_choice pruned it, so a fresh install never placed
it next to llama-server. ensure_diffusion_visual_server then found no
standalone release asset and skipped it, leaving Studio unable to serve
DiffusionGemma GGUFs natively (it required DG_VISUAL_BIN or a source
build). Keep the binary in the runtime allowlist on Linux, macOS and
Windows so it lands in build/bin and is activated automatically.
* install.sh: persist ROCm-on-WSL drop-in even when rocminfo already works
_maybe_bootstrap_rocm_wsl calls _ensure_rocm_probe_env (which exports a
transient HSA_ENABLE_DXG_DETECTION + adds /opt/rocm/bin to PATH on the
installer process) right before the "rocminfo enumerates gfx1151 -> already
set up, return early" gate. On any reinstall over an existing /opt/rocm --
the common case, since the uninstaller keeps shared ROCm userspace but
removes /etc/profile.d/unsloth-rocm-wsl.sh -- that probe env makes rocminfo
succeed, so the gate returns 0 WITHOUT ever persisting the drop-in. The
transient env dies with the installer, so the next login shell (Studio,
llama-server) sees no GPU: torch cuda_avail=False, rocminfo finds nothing,
the llama.cpp ROCm prebuilt segfaults on a GPU it can't reach.
Factor the drop-in writer into _persist_rocm_wsl_dropin() and call it before
the early return so the persistent env is restored whenever librocdxg is
present. Idempotent (only writes when the drop-in is missing), gated on
librocdxg so it never fires on non-WSL/non-ROCDXG hosts, root-writes or
sudo-tees like before. The fast-path branch now reuses the same helper.
Reproduced on gfx1151 (Radeon 8060S) under dash (the curl|sh shell):
before the fix a reinstall left the drop-in absent and torch cuda_avail
False; after, the drop-in is persisted and a fresh login shell reports
cuda_avail True. Verified under both dash and bash, and idempotent on
re-run.
* Studio WSL: load system HIP before a prebuilt's bundled runtime (gfx1151)
The lemonade / published llama.cpp ROCm prebuilts bundle their own HIP
runtime (libamdhip64) built for bare-metal Linux. In WSL the GPU is reached
through the system ROCm's librocdxg bridge over /dev/dxg, which the bundled
runtime cannot drive -- it segfaults on the first GPU call. So:
- install_llama_prebuilt.py: the prebuilt's llama-quantize/llama-server
validation runs with the bundle dir first on LD_LIBRARY_PATH, segfaults
(empty stderr), and the install silently falls back to a CPU source build
(which on this host can't even build for GPU -- hipcc absent). The Strix
Halo WSL user ends up on CPU despite a working GPU.
- llama_cpp.py: even if a GPU prebuilt were kept, the serve-time launcher
put the bundle dir first too, so it would crash at load.
Fix: on a ROCDXG WSL host (gated on /dev/dxg + "microsoft" /proc/version +
a librocdxg-providing /opt/rocm), prepend the system ROCm lib dir to
LD_LIBRARY_PATH so the WSL-capable libamdhip64 + librocdxg load first, while
the bundle still supplies libggml-hip / librocblas with the gfx1151 kernels.
Set HSA_ENABLE_DXG_DETECTION=1 alongside. Added _wsl_system_rocm_lib_dirs()
to both modules (kept identical so a prebuilt that passed install validation
runs the same way at serve time). Strict no-op on bare-metal Linux, NVIDIA,
macOS, and Windows.
Verified on gfx1151 (Radeon 8060S) in WSL (ROCm 7.2.1 + librocdxg, Adrenalin
ROCDXG): before, the lemonade gfx1151 prebuilt segfaulted and the install
fell back to a broken CPU build; after, install_llama_prebuilt validates and
keeps the GPU prebuilt (source=published, prebuilt_fallback_used=False), and
Studio serves Qwen3-1.7B-GGUF at 53 tok/s with the model resident in GPU
memory (llama-server device_info: ROCm0 = AMD Radeon 8060S).
* tests: cover the WSL ROCDXG drop-in + system-HIP-ordering fixes
- _wsl_system_rocm_lib_dirs: no-op without /dev/dxg, on bare-metal Linux,
and on WSL without librocdxg; returns the system lib dir on a ROCDXG WSL
host.
- binary_env: prepends the system ROCm lib dir ahead of the bundle and sets
HSA_ENABLE_DXG_DETECTION on WSL; unchanged on bare-metal Linux.
- install.sh: _persist_rocm_wsl_dropin exists, is gated on librocdxg, and the
rocminfo-already-works early return calls it before returning.
- llama_cpp.py: the serve-time launcher prepends the WSL rocm dirs before the
bundle dir (mirrors binary_env).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten WSL ROCDXG fix comments (no logic change)
Condense the drop-in / system-HIP-ordering comments and docstrings added in
this PR. Verified comment-only via AST parse + py_compile + sh/bash -n, the
308-test rocm_support suite, and a dash functional re-run of the bootstrap
(drop-in still persisted, env still set).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The post-download llama-quantize / llama-server smoke test JIT-compiles CUDA kernels on the first GPU forward pass and stalls every install and update by minutes on Blackwell (sm_100). Gate it behind _RUN_STAGED_PREBUILT_VALIDATION, disabled for now, keeping the smoke test and the source-build fallback it triggers fully intact so it can be restored by flipping the flag to True.
Hashless external prebuilts (e.g. lemonade) are not in the approved-sha256 manifest and rely on the functional smoke test as their only integrity gate, so they are always validated regardless of the flag; only approved bundles, already proven by the sha256 manifest, skip it.
The sha256 archive verification and the static Linux/macOS preflights are unchanged and still run for every install.
test_sidebar_account_block_uses_leading_tight hardcoded gap-0.5 in its selector, but the sidebar account-block div moved to gap-px during UI polish (#6196), so the regex stopped matching and the test failed across every studio PR's Repo tests (CPU). Match the gap utility loosely (gap-\S+) since this guard is about the leading-* class for descender clipping, not the spacing.
* fix(rocm): stop overwriting ROCR_VISIBLE_DEVICES in apply_gpu_ids
ROCR_VISIBLE_DEVICES uses HSA agent-level indexing, not physical GPU
indices. Setting it to a bare integer breaks multi-GPU ROCm systems
where the parent already set ROCR_VISIBLE_DEVICES=0,1: narrowing to
1 causes torch.cuda.is_available() to return False in the training
worker, producing a misleading 'no HIP accelerator' error even on a
correctly configured ROCm host.
HIP_VISIBLE_DEVICES is sufficient for GPU selection on ROCm.
Leave ROCR_VISIBLE_DEVICES inherited from the parent environment.
* test(rocm): update apply_gpu_ids test to assert ROCR_VISIBLE_DEVICES is not overwritten
* Stop HF 429 rate limits from sinking the llama.cpp prebuilt path in Studio CI
The Windows Studio API smoke job failed when anonymous huggingface.co
fetches of the tiny GGUF validation model (stories260K.gguf) hit HTTP 429
on the shared runner IP. The installer correctly refused the unvalidated
prebuilt and fell back to a source build, which the prebuilt assert then
flags. Three layers fix this:
1. Installer: auth_headers sends HF_TOKEN (or HUGGING_FACE_HUB_TOKEN) to
huggingface.co hosts, mirroring the existing GH_TOKEN handling for the
GitHub API rate limit. A redirect handler strips Authorization when a
download is redirected off-host (CDN signed URLs reject foreign auth;
urllib forwards headers on redirect, unlike requests/huggingface_hub).
2. Workflows: the HF_HOME prime steps also prefetch the validation model
so the install's hf_hub_download resolves from the local cache even
when the Hub is rate limiting; cache keys bumped v1 to v2 to repopulate.
This also covers fork PRs, which cannot see secrets.
3. Workflows: every Install Studio / update step that already passes
GH_TOKEN now also passes HF_TOKEN, so both the huggingface_hub path and
the direct URL fallback are authenticated.
Tests: tests/studio/install/test_hf_auth.py covers token-to-host routing,
the cross-host redirect strip, and the download_bytes wiring (offline).
Verified live: authenticated download of the validation model through the
new opener (CDN redirect exercised, pinned sha matches) and an offline
hf_hub_download cache hit against an HF_HOME primed by the new step.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Require a found ROCm DLL before forcing BNB_ROCM_VERSION in Studio paths
main.py previously set BNB_ROCM_VERSION=72 whenever HIP_PATH or ROCM_PATH
was set, and the training worker fell back to a blind 72 when DLL
detection found nothing. On a Windows machine with the AMD HIP SDK
installed but CUDA or CPU torch, that forces a ROCm backend onto a
non-ROCm bitsandbytes wheel, which raises at import. Both paths now only
write the override when a libbitsandbytes_rocm DLL actually exists (or a
seeded value is already present), matching the strict gates in
unsloth/import_fixes.py.
Also removes four redundant local import shutil statements in
unsloth/save.py that shadow the module-level import, the same pattern
that caused the UnboundLocalError fixed in #6149.
* Worker: gate the BNB override on a found ROCm DLL, preserving seeded marker
Review follow-ups: track _found_rocm_bnb in the worker like main.py so a
ROCm DLL with an unparsable name still gets the seeded or 72 fallback,
and skip the env write entirely when no DLL exists so a seeded value
keeps its sitecustomize marker and stays redetectable by later import
fixes.
* fix(studio): fall back to copy when os.replace is blocked during install activation
On Windows ARM64 the antivirus scanner can transiently hold a freshly
extracted DLL open while MoveFileEx runs, so activating the staged
llama.cpp prebuilt fails with [WinError 5] Access is denied. Attempt
os.replace first, then fall back to a file-by-file copytree which
bypasses the rename.
* address review: keep os.replace for rollback, scope copy-fallback to staging
The copy + rmtree fallback could silently corrupt a live install if the
existing directory is busy. Restrict it to freshly extracted staging
trees (renamed activate_staged_dir) and keep strict os.replace for the
rollback move so a busy active install raises immediately.
* fix(studio): scope copy-fallback to busy-lock errors, log it, and add tests
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* Studio: fetch the release source asset for exact (mix) source builds
The source-build fallback rebuilt the codeload/archive URL from the
source repo and commit. A mix build's merged tree is never pushed to any
repo (it ships only as the release's llama.cpp-source-commit-<sha>.tar.gz
asset), so codeload 404s on the merge commit and an uncovered host could
not build from source. When an exact-source asset exists, fetch it
directly from the release and keep codeload as the fallback for vanilla
builds whose commit is real.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Installer: harden GPU detection follow-ups after #6174
Ports the NVIDIA-priority and /proc/driver/nvidia/gpus hardening from #6174
to the remaining pathways and adds recovery for already-poisoned venvs:
- install_python_stack.py: add _ensure_cuda_torch so 'unsloth studio update'
force-reinstalls CUDA torch when the venv carries a ROCm build on an NVIDIA
Linux host (the pre-#6174 poisoning signature). Honors UNSLOTH_TORCH_BACKEND,
UNSLOTH_ROCM_TORCH_INSTALLED, and CUDA_VISIBLE_DEVICES=-1/'' opt-outs; never
touches healthy CUDA, deliberate CPU wheels, macOS, or Windows.
- install_llama_prebuilt.py: detect_host gains the /proc NVIDIA fallback and
skips ROCm probes when NVIDIA is usable; forwarded --rocm-gfx/--has-rocm
overrides still win.
- setup.sh: GPU summary classifies NVIDIA first through a timeout-bounded
probe with the /proc fallback; AMD probes are bounded and gain a KFD
vendor_id 4098 fallback; the llama.cpp source build only selects
GGML_CUDA/GGML_HIP when the matching GPU is actually detected.
- install.sh: bound both nvidia-smi calls with a 10s timeout (no behavior
change when healthy or when the timeout binary is absent); classify the
exported UNSLOTH_TORCH_BACKEND on the final index path segment so custom
mirrors containing 'rocm'/'gfx' in their base path are not mislabeled.
- install.ps1 + setup.ps1: NVIDIA probes now require a real 'GPU N:' row from
nvidia-smi -L under a 10s bound instead of bare exit code 0; later CUDA
version and compute_cap queries are bounded too.
Tests: 3 new test files (50+ tests), suite at 788 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Resolve-CudaToolkit driver probe for extracted-function unit test
tests/studio/test_resolve_cuda_toolkit.ps1 extracts Resolve-CudaToolkit alone
into a child pwsh and stubs nvidia-smi with a .ps1 script. The bounded runner
is not in scope there (and ProcessStartInfo cannot dispatch .ps1 stubs), so
the DriverMaxCuda parse silently returned nothing and the major-mismatch
scenarios failed. Fall back to direct invocation when Invoke-NvidiaSmiBounded
is unavailable; production setup.ps1 always has it defined and keeps the
10s bound.
* Treat CUDA_VISIBLE_DEVICES empty or -1 as hidden in NVIDIA-first guards
The NVIDIA-first guards added in this branch only special-cased
CUDA_VISIBLE_DEVICES=-1 at two setup.sh gates and ignored the empty-string
form entirely, while the Python detector (install_llama_prebuilt.py)
already treats both as hidden. On a mixed AMD+NVIDIA host steered to the
AMD card via CUDA_VISIBLE_DEVICES, the guards suppressed the AMD probes,
so setup.sh fell to a CPU llama.cpp build and install.sh picked CUDA
wheels instead of ROCm.
Move the policy into the helpers so every consumer agrees:
- install.sh: new _cvd_hides_nvidia checked first in _has_usable_nvidia_gpu
- studio/setup.sh: same via _setup_cvd_hides_nvidia; the two ad-hoc
CUDA_VISIBLE_DEVICES=-1 gate conditions are now redundant and removed
- studio/install_python_stack.py: _has_usable_nvidia_gpu returns False
when CUDA_VISIBLE_DEVICES is set to or -1 (whitespace tolerated)
Tests: 5 new sh scenarios (hidden via , -1, padded -1, visible device,
and mixed host with hidden NVIDIA restoring the ROCm route) plus a pytest
class covering all three implementations behaviourally.
Addresses the review comment on the NVIDIA-first setup.sh block.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Retrigger CI after PyPI 503 outage during the previous run
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
PR #6153 taught the chat UI driver to find Compare chat inside the
decluttered plus menu (#6140) but the extra UI driver kept looking for
the old top-level item, so all three Studio UI CI workflows have been
red on main since (Compare nav not found). Mirror the same
hover-then-click More fallback here.
* fix: prevent ROCm torch from installing on NVIDIA Linux hosts
NVIDIA's open kernel module (driver 560+) registers GPU topology nodes in
the KFD sysfs hierarchy with non-zero gpu_id values. The _has_amd_rocm_gpu
(install.sh) and _has_rocm_gpu (install_python_stack.py) sysfs fallbacks
previously treated any non-zero gpu_id as proof of an AMD GPU, so an
NVIDIA-only host with the open kernel driver was misrouted to the ROCm
install path, replacing the correctly-installed CUDA torch with ROCm wheels.
Fixes:
1. install.sh _has_amd_rocm_gpu sysfs fallback: require vendor_id 4098
(AMD 0x1002) in the KFD node properties file before declaring an AMD
GPU present. NVIDIA KFD nodes carry vendor_id 4318 (0x10DE) and are
now skipped.
2. install_python_stack.py _has_rocm_gpu sysfs fallback: same vendor_id
guard. Also preserves the existing fallback for older kernels that
don't ship a properties file (trusts gpu_id alone there).
3. install.sh now exports UNSLOTH_TORCH_BACKEND ("cuda"/"rocm"/"cpu")
immediately after get_torch_index_url() resolves the wheel family.
install_python_stack.py reads this as _TORCH_BACKEND and short-circuits
_ensure_rocm_torch() entirely on cuda/cpu hosts, providing a second
layer of defense that is independent of subprocess GPU detection.
Tests: 9 new cases in TestHasRocmGpuKfdVendorGuard,
TestEnsureRocmTorch, and TestInstallShStructure cover all three changes.
Full test_rocm_support.py suite: 289 passed, 2 skipped, 0 failed.
Closes#6172
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: show actual torch backend in progress step labels
The 'ROCm torch check' and 'ROCm torch (final)' step labels were
hardcoded regardless of whether the installer was targeting CUDA, ROCm,
or CPU. On NVIDIA hosts they showed 'ROCm' even though no ROCm wheels
were being installed, which was misleading.
Add _torch_step_label(suffix) which reads UNSLOTH_TORCH_BACKEND (set by
install.sh) and formats the label as e.g. 'torch check (cuda)' or
'torch final (rocm)'. Falls back to live GPU detection for standalone
studio update runs that bypass install.sh.
* fix: make KFD sysfs vendor check conservative -- skip if no properties file
The previous implementation fell through to `return True` when the KFD
node's properties file was missing (OSError), intending to support older
kernels. But NVIDIA open driver KFD nodes can also lack a properties file
on some kernel versions, so the fallback still produced a false positive.
Change the `except OSError: pass` to `continue` so any node without a
readable properties file is skipped rather than trusted. KFD properties
files exist on every kernel version that actually exposes /sys/class/kfd,
so this does not regress real AMD GPU detection -- if the directory exists
at all, properties files will be present for genuine GPU nodes.
* fix: bulletproof NVIDIA vs AMD GPU detection
Four changes that together ensure ROCm torch can never be installed on an
NVIDIA host regardless of which detection path fires:
1. _has_rocm_gpu() (Python): NVIDIA guard at the top -- returns False
immediately when _has_usable_nvidia_gpu() is True, blocking rocminfo,
amd-smi, and KFD sysfs from producing a false positive even when ROCm
tools are co-installed alongside the NVIDIA driver.
2. _has_amd_rocm_gpu() (install.sh): same NVIDIA guard -- calls
_has_usable_nvidia_gpu first and returns 1 if it succeeds.
3. _has_usable_nvidia_gpu() (Python): adds /proc/driver/nvidia/gpus/
sysfs fallback. The NVIDIA driver populates this directory on Linux
regardless of nvidia-smi state, so a subprocess PATH gap, timeout, or
driver initialisation race can no longer silence NVIDIA detection.
4. _has_usable_nvidia_gpu() (install.sh): same /proc/driver/nvidia/gpus
fallback, tried after nvidia-smi -L rather than instead of it.
Together: NVIDIA wins at every decision point. If nvidia-smi works, it
confirms NVIDIA. If it fails, /proc/driver/nvidia confirms NVIDIA. If
somehow both fail, _has_rocm_gpu still checks NVIDIA first before any AMD
path runs.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: two KFD/proc-only corner cases from Codex review
1. KFD awk state not reset per node file (Ryzen+NVIDIA false positive):
The awk glob processes all topology node properties files in one pass.
Without FNR==1 reset, a Ryzen+NVIDIA host where an AMD CPU-agent node
sets amd=1 (vendor_id 4098, gpu_id 0) can combine with a later NVIDIA
node setting gpu=1 (gpu_id > 0), triggering found=1 before vendor_id
4318 is seen. Added FNR==1{ gpu=0; amd=0 } to reset per file.
2. proc-only NVIDIA not reaching CUDA wheel selection:
_has_usable_nvidia_gpu returning true via /proc/driver/nvidia fallback
left _smi empty, so get_torch_index_url entered the AMD/CPU branch and
selected CPU wheels despite NVIDIA being confirmed. Introduced
_nvidia_detected flag (separate from _smi) so the AMD branch is skipped
whenever NVIDIA is confirmed by any path, while _cuda_ver reads from
_smi when available (with the existing cu126 fallback when _smi is absent).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: route arm64 Linux CUDA hosts to linux-arm64-cuda prebuilts
* Studio: SM-aware selection for windows-cuda app bundles
* Studio: select published ROCm bundles by gfx target (linux + windows)
* Studio: route macOS installs to the fork's prebuilt bundles
* Studio: fix windows cuda13 driver-13.0 gate and ROCm gfx prefix overreach
* Fix Blackwell Windows pin shadowing native app-bundle (b9360 over b9457)
* Match Windows cuda12 driver floor to Linux (12.x minor-version compat)
* Fix Windows app-bundle dropped when runtime DLLs come from torch/lib
* Fold the manifest resolver into the simple-path resolver (one entry, no dormant full path)
* Remove unused UNSLOTH_LLAMA_PUBLISHED_REPO override
* Route Windows GPU hosts to the fork prebuilts in setup.ps1
* Document sm_103 path divergence and mark --simple-policy as a no-op
* Note sm_103 coverage now comes from the producer manifest
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove the now-vestigial --simple-policy flag (one resolver handles all hosts)
* Unify the fork onto the manifest path; drop the linux-x64 filename path and hardcoded coverage tables
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strip whitespace from manifest gfx_target/mapped_targets when parsing
* Windows CUDA: sort coverage-unknown bundles last so they can't outrank targeted ones
* Share the SM-coverage sort key between the linux and windows selectors via _sm_range
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reject approved releases with an exact source archive but no source repo to clone from
* Studio: accept the fork's windows-rocm kind in the Windows reinstall check
* Studio: accept a manifest-bundle source repo in the exact-source release check
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: route Linux hosts to the fork only when a usable GPU is present
* Fix Windows AMD lemonade tag resolution for PR #5963
The fork release scan passes each scanned release's upstream tag
(b9518, ...) to the lemonade lookup, but lemonade publishes its own tag
series (b1292, ...) that never contains upstream tag numbers. On a
Windows AMD host every scanned release therefore 404s the lemonade
fetch twice, the upstream HIP zip is dropped by the approved-hash gate,
and the scan walks the whole release history until it dies on the
unauthenticated GitHub rate limit or falls to a HIP source build. The
Linux path already passes the requested tag ("latest") and works.
Thread the requested tag through resolve_release_asset_choice ->
resolve_asset_choice -> resolve_upstream_asset_choice as lemonade_tag,
used only by the lemonade lookups. Upstream asset names keep the
concrete per-release tag and all new parameters default to the old
behavior.
Verified on a gfx1151 box: before, the native Windows install scanned
b9518..b8811 and aborted on rate limit; after, it selects
llama-b1292-windows-rocm-gfx1151-x64.zip (lemonade) from fork release
b9518, passes staged validation, and the installed llama-server
enumerates ROCm0. WSL keeps selecting the matching ubuntu bundle.
Adds a regression test pinning that the Windows fork path resolves
lemonade via /releases/latest, never /releases/tags/<fork-tag>.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Plan lemonade for Linux ROCm hosts on the ggml-org direct path for PR #5963
Audit follow-up to 72f32364 across the other selection pathways. The
ggml-org direct planner kept its lemonade attempt for Windows ROCm
hosts but planned only the CPU tarball for Linux ROCm hosts, so an AMD
Linux box routed to ggml-org (for example a --published-repo override)
silently installed the CPU build. That lemonade planning used to live
in the --simple-policy dispatcher this PR removed.
Add the lemonade attempt ahead of the CPU tarball in the Linux x86_64
branch, mirroring the Windows branch, with the lookup keyed to the
requested tag. Adds a regression test asserting lemonade is the first
attempt for a Linux ROCm host on the direct path.
Also re-verified the other pathways on a gfx1151 box: the fork-routed
flows pass the requested tag everywhere, repeat runs over an existing
lemonade install correctly skip with "already matches selected release
b9518" on both native Windows and WSL, and macOS, CUDA and CPU
selection are untouched. Suites: 328 passed on Linux, Windows matches
the pre-existing baseline.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Daniel Han