* fix(studio): support hostname-based enterprise proxies
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): strip userinfo from proxy fetch targets
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* feat(studio): add DoRA support to studio
* fix: added use_dora fast encoder LoraConfig and gated use_dora on AdapterMethod
* fix(studio) serverside normalization for use_dora=true - add note documenting use_dora is silently dropped on diffusion
* fix: dora button disabled on mac, add preflight guard on GGUF lora export, mismatch now correctly falls through to existing error instead of silently no-opping
* Studio: add dora to the WizardState LoRA variant union for consistency
* Reject --use_dora on the MLX (Apple Silicon) CLI path
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Seed each request with the model's recommended sampling (matching the Chat UI), add per-field override flags, ignore oversized overrides, warn when sampling pins cannot apply to a reused server, and apply pins to the completions endpoint.
Add session-scoped MCP bridges so Codex, Claude plan mode, and Pi subagents run on the loaded local model, with cloud credentials and Codex state isolated per session and a process-wide Pi agent cap.
Keep the child process environment consistent with the cwd used to launch native POSIX coding agents. Some Node-based agents use PWD during project-root discovery, so inheriting a stale PWD can make them edit files in a parent or unrelated directory even when the wrapper process cwd is correct.
Only apply this normalization for native POSIX launches. WSL-launched Windows shims stay on the existing WSLENV bridge path so path translation behavior is unchanged.
Add regression coverage that launches an agent with a deliberately stale inherited PWD and asserts the child environment is normalized to os.getcwd().
Co-authored-by: Leo Borcherding <borchborchmail@gmail.com>
* unsloth start/run: tool-call flags, positional model, grouped help
Expose the existing tool-call controls as first-class CLI flags on both
unsloth run and unsloth start, add positional model detection with a GGUF
quant default, and group --help into rich panels.
Flags (unsloth run): --enable-tool-call-healing/--disable-tool-call-healing
(default on), --enable-tool-call-nudging/--disable-tool-call-nudging
(default on). Resolved before any re-exec and written to the existing env
controls (UNSLOTH_DISABLE_TOOL_CALL_HEALING, UNSLOTH_TOOL_CALL_NUDGE) so the
in-venv server reads them at import; an omitted flag respects a value the
parent already set.
Flags (unsloth start): --enable-tools/--disable-tools (default off, passthrough),
plus the same healing/nudging flags (default on). start conveys them to the
auto-started run via the child env and the tools flag, so it stays correct even
if run re-execs into an older Studio venv.
Positional model: a leading org/name(:variant) token routes to --model when
--model is absent, without stealing an option value or an agent passthrough arg.
A bare GGUF repo with no variant defaults to UD-Q4_K_XL for the unsloth namespace
and Q4_K_M elsewhere, applied only on the fresh auto-serve path so attaching to a
loaded model never reloads.
Help is grouped into rich panels (Model / Server / Session for start; Model /
Server and network / Tool calls / Advanced for run) so --help reads cleanly.
Adds unit coverage for the helpers, the start command-and-env forwarding, the
positional/quant defaulting, and the run env resolution.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Positional model: reuse _is_hub_model_id so local dirs and paths are not stolen
Route a bare org/name positional to --model only when it resolves as a hub id
(via the existing _is_hub_model_id, which rejects local paths and existing
dirs), so an OpenCode project dir like owner/repo is left for the agent. Apply
the same guard to the auto-serve GGUF quant default so a local -GGUF path is
not forced to a quant it may not contain.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* unsloth start: typer floor, drop redundant GGUF quant default, respect inherited tool-call env
- Require typer>=0.12.0. The rich_help_panel options added here crash at import
on typer<0.6, and the dependency was previously unbounded.
- Stop forcing a default GGUF quant for a bare org/name-GGUF on auto-serve. The
server's own quant preference already picks UD-Q4_K_XL for Unsloth uploads and
Q4_K_M otherwise, and falls back when that exact quant is missing, so forcing a
fixed variant broke external repos that only publish Q5_K_M/Q8_0.
- Make the healing/nudging start flags tri-state so an omitted flag keeps an
operator's inherited UNSLOTH_DISABLE_TOOL_CALL_HEALING / UNSLOTH_TOOL_CALL_NUDGE
instead of overwriting it with the start defaults.
* Fix start passthrough and inherited tool settings
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
* Studio: reject whitespace-only passwords
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reject any whitespace in passwords
* Studio: surface whitespace error in setup form, isolate auth test import
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Bring the local-subagent support onto main. The original change (#7316) merged
into the stacked pr/daniel-unsloth-start-audit branch rather than main, and #7313
reached main via squash, so these files never landed on main.
Adds --as-subagent for claude, codex, opencode and pi: the parent agent keeps its
own cloud model while a locally served GGUF is registered as a delegated subagent,
using ephemeral per-session config that never touches the user's real agent config.
* Improve unsloth start runtime lifecycle
* Remove speculative Gemma prompt override
* Polish model download progress output
* Refine unsloth start status output
* Clarify unsloth readiness banner
* Clarify model reuse and switching output
* Queue model switches behind active inference
* Tighten unsloth start model switching
* Reduce model switch bookkeeping
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Studio re-exec compatibility
* Recheck sidecar reservation after inference drain
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Pass start marker through child environment
* Fix key redaction, switch-waiter ordering, and stop/messaging gaps for PR #7313
- Redact minted sk-unsloth keys from the startup-failure log tail: the early
key marker lands in the server log before the model load finishes, so a
load-phase crash printed a live key to the terminal
- Deregister a finished switch waiter before releasing the swap gate so a
swap on another event loop cannot count it as still queued and unload the
model the finished request is about to generate against
- Warn on same-repo quant switches: an explicit variant replaces the resident
weights for every attached session, but the repo ids match so no switch
warning was printed
- Note the agent exit code when it is nonzero so the server keep-alive
message does not read as a successful session
- Use taskkill /T in unsloth studio stop so llama-server children stop too
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in start, studio, and inference changes
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* unsloth start: fix Windows agent install/launch and local model selection
- claude: pin availableModels to the served model in the session --settings
overlay so a user's ~/.claude/settings.json allowlist no longer substitutes
the org default for the local Unsloth model. The allowlist covers --model,
ANTHROPIC_MODEL and the model setting, and an empty [] is ignored, so the pin
lists the model explicitly.
- installs: run the Windows installer under -ExecutionPolicy Bypass
(process-scoped, nothing persistent) so npm's npm.ps1 and irm|iex scripts run
under the default Restricted policy; on failure, hint at Set-ExecutionPolicy
-Scope CurrentUser -ExecutionPolicy RemoteSigned for a hand-run retry.
- PATH: resolve agents installed to ~/.local/bin (claude) and %APPDATA%\npm (npm
agents) in-process, so a fresh install launches without opening a new shell and
an already-installed agent is not re-prompted for install.
- load message: "Loading <model> - please wait" while a model loads.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* unsloth start: resolve agent version against the launch PATH
The claude/codex/opencode version probes ran shutil.which while building the
command, before _launch augments PATH with the known install dirs. An agent
present only in ~/.local/bin or %APPDATA%\npm was therefore missed, assumed to
be a current build, and launched with flags an older build rejects (claude
aborts on the unknown flags). Route the three probes through a new
_which_with_install_dirs() so each resolves the same binary _launch will,
restoring PATH afterward so only _launch persists the augmentation.
Add regression tests for the three probes (POSIX and the Windows npm dir) and
make the Windows-branch tests run on POSIX hosts (pinning Path to the native
flavour so a simulated os.name does not make pathlib build WindowsPath).
* unsloth start: keep os.defpath when augmenting an unset PATH
_augment_path_with_install_dirs collapsed an unset PATH to just the install
dirs, dropping the os.defpath fallback (/bin:/usr/bin) that shutil.which and
exec*p* use when PATH is absent. A system-installed agent then looked missing
and the launched child lost its normal PATH. Seed os.defpath when PATH is
unset; an explicitly empty PATH is left as-is (search nothing), matching
shutil.which. Add regression tests for the augment helper and the version-probe
wrapper.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Bridge PWD through WSLENV /p when launching a Windows npm shim from WSL so project-root discovery uses the live cwd. The no-launch recipe adds PWD/p without freezing PWD; the concrete cwd override applies only on direct launch.
Pin the fetched Hermes install.sh/install.ps1 and the checkout they perform to an immutable upstream commit, and distinguish pinned from unpinned sources in the consent warning.
Route --yolo to OpenCode native --auto for the default TUI and run; keep the config permission fallback for no-auto subcommands (including hidden console/generate) and for --mini, which ignores --auto.
* Replace standalone Studio wording with Unsloth
Replace the single word Studio with Unsloth wherever it is used as
shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n
locales, workflow display names, comments and docstrings.
Kept unchanged: the full name Unsloth Studio, third party product
names (LM Studio, Visual Studio, Mac Studio), feature names
(Recipe Studio, Fine-tuning Studio and its translations), and all
identifiers such as env vars, commands, paths and filenames.
* Address review feedback on the Studio wording rename
Use "an" before Unsloth where the rename left the article as "a".
Restore the split brand where Unsloth and Studio render as two halves
of the full product name: the onboarding sidebar subtitle and the
IPv6 localhost warning. Scope two messages to the full name Unsloth
Studio where plain Unsloth was misleading: the AMD README bullet and
the CLI studio setup error.
* Studio: make the Cloudflare tunnel opt-in (off by default)
A wildcard bind (`-H 0.0.0.0`) auto-started a public trycloudflare.com
tunnel, so exposing Studio on the LAN also published it to the public
internet. Flip the default so the tunnel is opt-in.
- `--cloudflare` is now tri-state (Optional[bool], default None = off),
mirroring the existing --enable-tools/--disable-tools handling. Pass
--cloudflare to expose a public HTTPS link for a wildcard bind; --secure
still implies the tunnel.
- --secure + --no-cloudflare is still rejected as a contradiction.
- Update the parent-command guard, re-exec forwarding, startup-banner
wording, the colab comment, README, and tests.
* Studio: update installer/setup launch hints for opt-in Cloudflare
The post-install launch hints only mentioned --secure for a public link.
Now that the tunnel is opt-in, clarify that -H 0.0.0.0 exposes the raw
port on the LAN (not a public URL), and surface --cloudflare as the
explicit opt-in for a public HTTPS link (--secure keeps the raw port
private). Applied to install.ps1, install.sh, and studio/setup.sh.
* Studio: address review - keep cloudflare tri-state + harden run re-exec
Two review points from the bots:
- Gemini: keep `cloudflare` as Optional[bool] in run_server instead of
casting None -> False, so the startup banner can distinguish "OFF (default)"
(unset) from "OFF (--no-cloudflare)" (explicit). `_cloudflare_flag` and the
banner branch now carry the tri-state.
- Codex (P1): `unsloth studio run` re-execs the studio venv's console script,
which can be an older build whose --cloudflare defaulted on; omitting the
flag let it re-enable the tunnel. That path now forwards the default polarity
explicitly (--no-cloudflare, or nothing under --secure since --secure implies
the tunnel). The plain `unsloth studio` path runs the same-version in-tree
run.py (resolved via _find_run_py), so it keeps forwarding only an explicit
polarity and still shows the accurate "(default)" banner.
Tests updated for the tri-state banner labels, the None gate cases, and the
new re-exec forwarding.
* Studio: forward --no-cloudflare on plain re-exec too (mixed install)
Codex follow-up: _find_run_py falls back to STUDIO_HOME/.../studio/backend/
run.py when the package copy is absent, so the plain `unsloth studio` re-exec
can land on an older run.py whose --cloudflare defaults on. Forward the default
polarity explicitly there too (--no-cloudflare, or nothing under --secure),
matching the run subcommand. The common in-venv launch skips the re-exec and
still shows the tri-state "(default)" banner.
* Studio: fix launch hint - --cloudflare needs the wildcard bind
Codex P3: the launch hint listed --cloudflare next to the loopback
`unsloth studio -p 8888` command, but the tunnel only starts for wildcard
binds, so `--cloudflare` alone on 127.0.0.1 does nothing. Show
`-H 0.0.0.0 --cloudflare` in the hints (install.ps1, install.sh,
studio/setup.sh) and clarify the same in the README.
* Studio: cross-platform masked terminal password prompt helper
Per-keystroke '*' echo (POSIX termios cbreak / Windows msvcrt.getwch),
backspace editing, Ctrl-C abort, EOF handling, confirmation loop with
re-prompt on mismatch or policy failure. Pure should_prompt gate for the
--secure/--cloudflare exposure paths.
* Studio CLI: force a terminal password change before public tunnel exposure
When a launch will start the Cloudflare tunnel (--secure, or --cloudflare on
a non-api-only wildcard bind) and the admin account still has its seeded
bootstrap password, prompt for a new password in the terminal (masked with
'*', confirmed, re-prompting until valid) before any re-exec or server
exists. The change is committed in the parent so it never crosses argv or
the environment and older studio-venv children see it immediately. Without
a terminal, warn and fall back to the backend bootstrap shutdown timer.
Mirrors backend update_password semantics in one transaction: rehash,
rotate the JWT secret, clear must_change_password, revoke refresh tokens,
drop the desktop secret, then remove the stale credential files.
* Studio: terminal password gate before the public tunnel (backend backstop)
Never publish a trycloudflare URL while the seeded admin password is
active: run_server now runs a terminal password-change gate after the
tunnel decision and strictly before start_studio_tunnel. Interactive
refusal fails closed (shutdown + exit 1, mirroring the secure gate);
without a tty it warns and keeps the bootstrap deadline. Success applies
the same effects as the change-password route (update_password +
revoke_user_refresh_tokens) and drops the stale
app.state.bootstrap_password. MIN_PASSWORD_LENGTH centralised in
auth/storage.py and referenced by the HTTP schema. terminal_prompt.py
carries the pure gate helper (interactive loop stubbed; supplied by the
masked-input module). Also migrates the studio/setup.ps1 launch footer
that still showed the bare wildcard hint.
* README: reconcile remote-access section with opt-in Cloudflare tunnel
* Studio: harden the terminal password gate after review
- run.py: run the gate BEFORE the uvicorn socket binds. On a wildcard
--cloudflare launch the served HTML injects the bootstrap credential
for first login, so a pre-gate listener would hand the default
password to anyone who reaches the raw port while the operator is
still typing. The gate now also seeds the admin row itself (it can
run before lifespan startup).
- Headless launches that nothing would protect now fail closed: the
bootstrap deadline never arms for api-only serving and
UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT=0 disables it, so warn-and-proceed
would have promised a shutdown that never comes. Both the CLI and the
backend refuse to publish in that case; the ordinary headless path
still warns and relies on the 1h deadline, and no longer auto-fills
the default credential into HTML served on a public URL.
- storage.update_password gains revoke_refresh_tokens to delete the
user's refresh tokens in the SAME transaction as the password commit;
the change-password route and the backend gate use it (a separable
follow-up delete could fail after the commit and leave a stale
refresh token able to mint access tokens under the rotated secret).
- clear_bootstrap_password is best-effort: a locked/undeletable file
must not surface as a failed password change.
- CLI masked reader: disable ISIG like the backend so Ctrl-Z cannot
suspend the process with the shared terminal stuck in no-echo mode;
handle Ctrl-C/Ctrl-Z as characters; treat stream EOF mid-line as an
abort instead of submitting a partial password. Both readers restore
terminal attrs from a SIGTERM/SIGHUP handler since a finally block
cannot run when a default-disposition signal terminates the process.
- Backend reader: decode byte-at-a-time through an incremental UTF-8
decoder so multi-byte characters split across read boundaries are no
longer dropped; isatty checks tolerate closed/None streams.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: persist bootstrap suppression through lifespan startup
The pre-bind password gate nulled app.state.bootstrap_password, but the
FastAPI lifespan runs after it and re-reads the bootstrap password into
app.state on both admin paths, so a headless public launch could still
serve the injected credential in HTML. Carry a persistent
suppress_bootstrap_injection flag that the lifespan honors instead.
Also drop the quoted Tuple annotation on _terminal_password_gate that
tripped the import-hoist lint (the typing import looked unused).
* Studio CLI: keep the pre-exec auth DB private (0700 dir, 0600 db)
On a fresh install the pre-exposure password gate creates auth/ and
auth.db through the CLI before the backend ever runs, and
sqlite3.connect leaves the DB 0644 under a 022 umask. Mirror backend
storage.get_connection's chmod so the committed password hash and JWT
secret are never world-readable, even if the launch aborts before the
backend applies its own modes.
* Tighten pre-exposure password gate comments
* Studio: delete seeded bootstrap password before headless public re-exec
The headless warn-and-proceed path returns with the default admin
password still active, then re-execs a child Studio process. An old
studio-venv child (mixed-version install) predates the pre-bind gate and
its injection-suppress flag, so its lifespan reads .bootstrap_password
and injects the seeded credential into the public HTML for up to the
bootstrap deadline. A CLI-flag handshake cannot fix this uniformly: the
studio run path uses ignore_unknown_options and an old in-venv child
runs in-process, so it would never reject the flag.
Delete the seeded .bootstrap_password file in the parent before re-exec
so a fresh child of any version reads None and never serves it. This
covers both re-exec paths and both child versions. must_change_password
stays set, so the login page still forces a change and the bootstrap
shutdown timer still arms; only the plaintext-on-disk copy is removed.
Recovery is via a terminal-attached run or reset-password. Backend gate
and CLI warnings updated to match.
* Studio: commit the seeded admin before headless public re-exec
The headless-warn path deletes the seeded .bootstrap_password so a
re-exec'd child cannot inject it, but _ensure_cli_default_admin's INSERT
was never committed and rolled back on conn.close(). On a fresh
STUDIO_HOME an old studio-venv child then found no admin, regenerated a
fresh bootstrap password + file, and injected THAT into the public page,
defeating the deletion.
Commit the seeded admin right after _ensure_cli_default_admin so any
re-exec'd child sees the existing account and does not regenerate.
Regression tests cover both re-exec paths on a fresh (unseeded) DB.
* Studio: fail closed when the bootstrap password file cannot be removed
On the headless public path, deleting .bootstrap_password is the
protection against an old re-exec'd child injecting the seeded
credential. If unlink fails (locked file, read-only auth dir) the file
is still on disk, so warning and proceeding would still leak it for the
bootstrap-timeout window. Abort with a clear error instead. Regression
test covers the unlink-failure fail-closed path.
* Studio: hold no-echo for the whole password line, not per keystroke
The POSIX masked reader set cbreak/no-echo inside _getch_posix and restored
the terminal to echo-on in a finally after every single keystroke, because
_read_password calls _getch once per character. Between one char returning and
the next call re-entering cbreak, ECHO was on, so a keystroke arriving in that
window echoed the password in cleartext.
Move the terminal mode into a _prompt_raw_mode context that _read_password
holds around the entire line (mirroring unsloth_cli/commands/_password_prompt.py,
which already did this), restoring once when the line completes or aborts.
_getch_posix now only reads, since the mode is held by the caller. The context
is a no-op when stdin is not a real terminal, keeping the _getch test seam.
Add a regression test asserting the raw-mode context wraps the read exactly
once and every keystroke is read while it is active.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: strip the seeded bootstrap password when the auth DB check fails
The pre-exposure gate returned early on two auth-DB inspection failures and
proceeded to re-exec without removing the seeded .bootstrap_password:
- _connect_auth_db() failure: a seeded credential from a prior run may still
be on disk.
- the must_change_password read-back failure: worse, _ensure_cli_default_admin
had already seeded the admin and the code committed it (writing
.bootstrap_password) right before the failing SELECT.
In the mixed-version case (a new outer CLI re-execing an old studio-venv child
that predates the pre-bind gate), that child would read the file back and
inject the default admin credential into the public Cloudflare page. The
sibling headless branch already deletes the file for exactly this reason, so
these returns were an inconsistent gap.
Factor the delete-or-fail-closed logic into
_strip_seeded_bootstrap_password_or_exit and call it on both inspection
failures (and reuse it in the headless branch): strip the seeded file first
(version-independent protection), failing closed if the removal itself fails.
must_change_password stays set, so the login page still forces a change and the
bootstrap shutdown timer still arms.
Add tests for both new paths (connect failure and post-commit read-back
failure strip the file and proceed; a failed strip fails closed).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fail closed when the seeded admin cannot be committed before exposure
The pre-exposure gate wrapped _ensure_cli_default_admin (the INSERT), its
conn.commit(), and the must_change_password read-back in one try, and the
except recovered by stripping .bootstrap_password and proceeding to re-exec on
the assumption the admin was already committed. That assumption only holds when
the failing statement is the SELECT. When the INSERT or the commit itself fails
(e.g. a write lock held past the busy timeout on a fresh install), no admin row
is committed: it rolls back on conn.close(), and a re-exec'd old studio-venv
child (no pre-bind gate) then finds no admin, regenerates a fresh bootstrap
password + file, and serves that default credential on the public Cloudflare
page. Stripping the file cannot stop a regeneration.
Split the seed+commit into its own try that fails closed (refuse the public
launch, best-effort removing any half-written seed file) since we cannot prove a
committed admin; keep the separate read-back failure on the strip-and-proceed
path, where the admin is committed so an old child finds it and will not
regenerate. Add a test for the seed-commit-failure path.
* Studio: decode the CLI masked password reader with errors="replace"
The CLI reader read keystrokes with text-mode sys.stdin.read(1), which raises
UnicodeDecodeError on a pasted non-UTF-8 password (e.g. Latin-1 bytes), or under
PYTHONUTF8 yields a lone surrogate that later crashes the pbkdf2 encode -- either
aborts the launch with a traceback. The backend mirror (terminal_prompt.py)
already reads raw bytes through an incremental decoder with errors="replace".
Mirror that here: read with os.read and an incremental decoder so invalid bytes
map to U+FFFD, iterating over each emitted char (one byte can complete a
replacement plus the next char).
* Studio: resolve the child launcher before the pre-exposure gate
The gate strips the seeded .bootstrap_password on a headless public launch, and
it ran before the re-exec launchability check (studio venv / run.py / console
script present). So a headless launch with an incomplete studio setup would seed
the admin, delete the bootstrap password, then abort because the child could not
be found, leaving the admin at must_change_password=1 with no password ever
shown or injectable: locked out until `unsloth studio reset-password`.
Resolve and validate the child launcher first, in both `studio` (studio_default)
and `studio run`, and only then run the gate, so an unlaunchable setup exits
before anything is stripped. Add a regression test that a missing venv exits
without removing the seeded file.
* Studio: fail closed when the auth DB cannot be opened before exposure
The connect-failure branch of the pre-exposure gate stripped .bootstrap_password
and proceeded, on the assumption a committed admin from a prior run made an old
child find it and not regenerate. But on a fresh public launch whose
_connect_auth_db() itself fails (transient lock during the schema/seed step, or
an unwritable home), no admin is committed, so a mixed-version re-exec child that
predates the backend gate can find no user, generate a fresh bootstrap password,
and serve it on the public Cloudflare page. Stripping a file we cannot vouch for
cannot stop a regeneration.
Make this branch fail closed like the seed/commit failure path: we only continue
past the DB inspection once a committed admin is confirmed. The existing file is
left untouched so a retry (after a transient lock clears) can still prompt.
Update the connect-failure test to assert fail-closed, and give the in-venv
--secure flag test a real STUDIO_HOME with an already-changed admin so the gate
is a no-op rather than relying on a DB-open failure.
* Studio: invalidate seeded bootstrap files before deleting auth.db on reset
reset-password deleted auth.db first, then best-effort unlinked the seeded
.bootstrap_password and desktop secret. unlink() only ignores
FileNotFoundError, so a locked or read-only file (Windows AV, read-only auth
dir) survived while auth.db was gone. The next server start then re-seeded
from that stale plaintext and re-validated the exact credential the reset was
meant to revoke.
Invalidate the credential files first, truncating any that cannot be
unlinked, then delete the DB, so a surviving file can never carry a reusable
secret. clear_bootstrap_password now truncates on unlink failure for the same
reason, and its warning says the contents were cleared rather than claiming
the stale password is already invalid.
* Studio: require a servable frontend before the pre-exposure gate can strip the seeded password
A headless public launch strips the seeded .bootstrap_password before the
re-exec'd child starts. If the child then cannot serve the login page (the only
in-band way to change the seeded password) the admin is locked out
(must_change_password=1, no file, no UI) until reset-password.
Add _require_servable_frontend_or_exit and call it before the gate on both
`unsloth studio` and `unsloth studio run` public launches: fail closed if a
non-api-only public launch has no built frontend dist, before anything is
stripped. A user-supplied --frontend is validated to contain index.html so a
bad path cannot silently bypass the check; an auto-resolved dist is trusted
(_find_frontend_dist already requires index.html) and forwarded to the child.
Model-load aborts on `studio run` remain a residual: the parent must strip for
mixed-version safety (an old studio-venv child has no pre-bind gate) and model
loadability cannot be proven before exec, so that path stays recoverable via
reset-password.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden reset-password ordering and validate the in-venv backend before the strip
Three follow-ups to the pre-exposure hardening:
reset-password now deletes auth.db FIRST and proves it is gone before touching
the seeded credential files. If the DB cannot be removed (a running Studio or
Windows holds it open, or a read-only auth dir) it aborts with the credential
files untouched, so a forgotten-password reset is not left half-done with the
recovery credentials deleted while an un-resettable must_change_password=1 DB
survives. After the DB is gone it invalidates the stale credential files
(unlink, else truncate) and fails closed if a file can be neither removed nor
truncated, since a surviving plaintext would be re-seeded by
generate_bootstrap_password() and re-validate the revoked password.
The in-venv (in-process) launch path had no analogue of the re-exec launcher
check: a headless public launch would seed the admin and strip the seeded
.bootstrap_password in the gate before _load_run_module() later failed on a
broken/partial venv, leaving must_change_password=1 with no password to log in.
Add _validate_inproc_backend_before_strip, called on the in-venv path (both
`unsloth studio` and `unsloth studio run`) before the gate on the headless
public path, so a broken backend fails cleanly before anything is stripped. It
is scoped to the headless path so an interactive prompt is not delayed behind a
full backend import.
* Studio: validate the frontend and tunnel before the strip on every public path
Five follow-ups closing the remaining pre-exposure-strip lockouts:
The in-venv (in-process) paths of both `unsloth studio` and `unsloth studio
run` validated the backend but not the frontend before the gate, so a headless
public launch with a missing/bad dist would strip the seeded .bootstrap_password
and then abort in run_server() during frontend setup, leaving
must_change_password=1 with no login page. Both now validate a servable frontend
before the strip (cheap check first, backend import after) and serve the
resolved dist in-process.
The `studio run` re-exec discarded the dist that satisfied the pre-strip check
and only forwarded a user-supplied --frontend. In a shadowed install where the
parent finds a built dist the child cannot, it stripped and exec'd without the
path, and the child aborted during frontend setup. It now forwards the resolved
dist, matching `unsloth studio`.
On a headless --secure launch the bind is loopback, so the Cloudflare tunnel is
the only public exposure. If cloudflared is provably unavailable (found nowhere
and undownloadable) the tunnel cannot start, so stripping the recovery
credential would just lock the user out with no public URL ever served. Add
_tunnel_binary_confirmed_unavailable and, on --secure only, refuse the launch
with the credential preserved rather than strip. Wildcard --cloudflare binds
0.0.0.0 publicly regardless of the tunnel, so it still strips; any uncertainty
(helper not loadable) also still strips, since a possible credential leak
outweighs a recoverable lockout.
clear_bootstrap_password no longer claims it cleared the file's contents when
both unlink and truncate failed; it now reports the stale password is still on
disk and asks the user to remove it manually.
* Studio: fix cloudflared probe path and skip the bootstrap strip for a self-suppressing child
Two follow-ups to the --secure pre-exposure hardening:
The cloudflared availability probe loaded cloudflare_tunnel by file path but not
its backend deps: ensure_cloudflared() -> _cache_path() lazily imports
utils.paths.storage_roots, which only resolves when studio/backend is on
sys.path. From the outer CLI it is not, so the probe saw ensure_cloudflared()
return None (cache unresolvable) and wrongly treated the tunnel as unavailable,
refusing --secure even when cloudflared was cached or downloadable. Add the
backend dir to sys.path for the probe (and remove it after) so the cache path
resolves as it will in the child.
A headless --secure launch stripped the seeded .bootstrap_password before the
child proved the tunnel could actually connect, so a cloudflared that is present
but cannot establish the tunnel (blocked connectivity, Cloudflare outage) left
must_change_password=1 with no recovery credential. But the strip is only needed
when the re-exec'd child is an OLD studio-venv backend with no pre-bind
suppression: this install's own run.py sets app.state.suppress_bootstrap_injection
before binding and never serves the seeded credential publicly. Add
_child_self_suppresses (true in-process, or when the re-exec target is this
install's own run.py by path identity) and skip the strip in that case, keeping
.bootstrap_password as a local recovery credential; the strip stays fully in
force for the studio-venv console-script path and any venv-fallback run.py, where
an old child is actually possible.
* Studio: reword the pre-exposure terminal password prompt
* Studio: warn when -H is overridden by --secure; align pre-exposure prompt wording
- --secure/--secure run: emit a Note (not an error) when -H is a non-loopback
host, since --secure forces the loopback bind and would otherwise discard -H
silently.
- Reword the pre-exposure terminal prompt to 'exposed on the public internet'
in both the backend gate and the CLI mirror.
- Align the CLI success line with the backend ("Password updated for '<user>'.").
- Tests for the new -H warning (present when overridden, absent on loopback).
* Studio: add non-interactive --password to set the initial admin password
Headless hosts (CI, containers, systemd units) have no TTY, so the forced
first-exposure password change could not be completed unattended. Add a
non-interactive way to set the INITIAL admin password before the server binds:
- --password <value>, the UNSLOTH_STUDIO_PASSWORD env var, or --password -
(read one line from stdin). Off by default; unset falls back to the normal
interactive terminal prompt / browser setup.
- Applies on any launch (public --secure/--cloudflare or a headless -H 0.0.0.0
bind), only when the account still has its seeded bootstrap password. An
already-set password is a hard error, never an override; an invalid value
(too short, or equal to the bootstrap) fails closed before bind.
- The CLI applies the change in the parent, never forwards --password to the
re-exec child, and strips UNSLOTH_STUDIO_PASSWORD from the child env so the
secret never crosses to the child. run.py does the same on the direct path and
strips the env var so spawned subprocesses (cloudflared, llama-server, tools)
cannot inherit it.
Mirrors resolve_supplied_password across the CLI and backend, documents the
option in the README (including the argv-visibility caveat), and covers all
flows (env/stdin/literal, fail-closed cases, no-forward, env-strip,
reset-password roundtrip) in the CLI, backend, and unit suites.
* Studio: truncate the stale bootstrap file when unlink fails on a CLI password change
The post-change cleanup in _cli_update_password only warned when
.bootstrap_password could not be unlinked but was still writable (locked file,
read-only auth dir), leaving the old plaintext on disk. If auth.db is later
reset or removed, generate_bootstrap_password() reads that file back and
re-validates the revoked bootstrap password. Truncate the file on unlink
failure so its stale plaintext cannot be re-seeded, mirroring the backend
clear_bootstrap_password(); the password change is already committed, so this
never rolls it back. The warning now states truthfully whether the contents
were cleared or the file must be removed manually.
* Studio: tighten comments
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* unsloth start: add --resume to persist and reopen agent sessions
`unsloth start <agent>` launches a coding agent whose home is a throwaway
temp dir wiped on exit, so codex/openclaw/hermes/pi (which relocate their
whole home there) cannot resume a conversation after you quit. opencode and
claude keep their session data in a fixed user dir, so they already resume.
Add an opt-in --resume/--no-resume flag: it routes the launch to the stable
Unsloth agents dir (the same one --no-launch already uses) so the session
survives the exit, never touching the user's own ~/.<agent>. A bare --resume
also reopens the last conversation via the agent's native flag (codex
`resume --last`, opencode/claude/pi `--continue`). The default is unchanged:
a plain launch still uses a temp dir and persists nothing.
Add a dispatch-only `resume` job to the Local Agent Guides CI that drives the
real launch path and asserts the split: codex/pi are wiped without --resume
and persist with it, while opencode/claude persist either way.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* unsloth start: rename --resume to --persist
The session flag collided with agents' own resume flags. `unsloth start
claude --resume <id>` used to forward `--resume <id>` straight to Claude
(which keeps its history in ~/.claude regardless), so a boolean --resume on
unsloth start would have swallowed the session id and turned it into a stray
prompt. Name the persistence flag --persist instead, so every agent's native
resume flag (claude --resume <id>, codex resume, opencode --continue, ...)
still passes through untouched. Behavior is otherwise identical: --persist
keeps a launched agent's session under the Unsloth agents dir, and a bare
--persist reopens the last conversation.
Add a regression test that `--resume <id>` passes through verbatim, and in the
CI resume experiment skip the redundant second pass for opencode/claude (they
persist either way, and a second CPU turn only risks a timeout).
* unsloth start: correct --persist help and drop the buggy auto-resume
Reword the --persist help to be accurate: claude and opencode keep sessions in
the user's own stores and resume regardless, so --persist only stabilizes the
otherwise-ephemeral relocated home of codex/openclaw/hermes/pi. Drop the
bare-launch auto-append of native resume tokens: it errored on a first launch
with no prior session, and was inconsistent between launch and no-launch.
--persist now only keeps the session dir; resume via the agent's own command
(e.g. `unsloth start codex --persist resume`), which now finds it.
In the CI resume experiment, fail the pass when the launched turn exits
non-zero, so a write-then-error is not misread as PERSISTED.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
When a coding agent is missing, `unsloth start <agent>` offers to run the
vendor's own installer (curl | bash, irm | iex, or npm) after an interactive
confirm. Those installers execute with the user's privileges and there is no
signature or hash check on the fetched content, so a blind "yes" is a
supply-chain risk if the delivery path is compromised.
Keep the auto-install convenience but make consent informed: before the prompt,
name the exact remote source the installer fetches (or the command it runs for a
package installer) and state that nothing verifies a signature or hash. Behavior
is otherwise unchanged: non-interactive stdin still never executes anything, and
the confirm still defaults to no.
* fix: launch OpenClaw local TUI by default
* Fix/adjust OpenClaw launch paths for PR #6937
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Default OpenClaw to the local TUI only on a bare invocation
The first-arg startswith('-') branch rewrote passthrough globals into a broken
command: OpenClaw's grammar is openclaw [--dev] [--profile <name>] <command>, so
'unsloth start openclaw --profile test' became 'openclaw tui --local --profile
test', but tui does not accept --profile (or --dev), so the invocation failed.
A leading '--flag value' is ambiguous between a global (--profile test) and a tui
option (--message hi), so it cannot be reinterpreted safely. Default to the local
TUI only when no passthrough args are given, and forward everything else verbatim
so OpenClaw parses it under its own grammar. The bare-launch default (the point
of this change) is preserved; explicit subcommands and global flags pass through.
---------
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* feat(cli): detect MLX distributed launch context
* feat(mlx): wire distributed inference backend
* feat(cli): broadcast MLX distributed chat turns
* fix(cli): wait indefinitely for distributed chat turns
* fix(cli): report MLX distributed load errors cleanly
* fix(mlx): route distributed vlm through loader
* fix(cli): detect inline MLX host JSON
* fix(studio): harden distributed object sharing
* fix(studio): select JACCL distributed backend
* fix(cli): abort distributed error paths
* Distinguish real stream errors from model text via GenStreamError in distributed CLI
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fail loud when MLX distributed init returns a singleton group
The worker only reaches this block when distributed was explicitly
requested. A singleton (size 1) group means the launch failed to form a
real group (MLX built without distributed support, or an invalid launch
env/hostfile); silently continuing leaves nonzero ranks looping forever
on share_distributed_object. Raise instead so the surrounding handler
returns a clear load error.
* Tighten MLX distributed inference comments
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat(studio): route CLI trainer to MLX backend
* fix(studio): harden MLX trainer routing
* fix(studio): harden MLX trainer adapter routing
* test(studio): assert MLX CLI activation order
* fix(studio): address MLX CLI review feedback
* feat(cli): support MLX in legacy script
* fix(cli): adapt MLX tokenizer for raw text
* fix(cli): omit unsupported MLX eval batch arg
* fix(cli): feed raw text to MLX trainer
* Fix CLI MLX routing and Python 3.9 annotations
Route the MLX backend through create_mlx_trainer_adapter so the torch-free
Apple Silicon path never imports trainer.py (torch/unsloth/trl). Replace
from __future__ import annotations with typing.Optional/Union so the CLI
annotations stay Python 3.9 compatible without the unused-import lint hit.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strip return_tensors from MLX raw-text tokenizer proxy
On a torch-free MLX install, RawTextDataLoader calls the tokenizer with
return_tensors='pt'; the callable proxy forwarded that to the HF
tokenizer, which tried to build torch tensors and failed before
training. Drop return_tensors so the MLX path returns plain token ids.
* Tighten CLI MLX-backend comments
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: handle case-variant GGUF cache hits for unsloth start
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* gguf cache: keep split shards co-located and isolate cache tests properly
When a cached main shard was reused from an older snapshot, the extra shards
were resolved independently and could come from a different snapshot dir (or a
fresh download into the current ref), leaving llama.cpp unable to load a
multi-shard GGUF whose pieces are split across directories. Only reuse a cached
main shard when every sibling shard sits in the same snapshot; otherwise fetch
the whole set together so they stay co-located.
Also patch huggingface_hub.constants.HF_HUB_CACHE (not just the HF_HUB_CACHE env
var) in the two cache tests that seeded a temp cache: the snapshot lookup reads
the module constant, so the env-only override let the real cache leak in and
skip an asserted download.
* Do not let a companion-only cache snapshot shadow real GGUF variants
When listing GGUF variants from the local HF cache, a newer snapshot may
contain only a companion file (for example a vision projector fetched on
demand) while the actual quant files live in an older snapshot. The prior
scan returned the first snapshot whose vision flag was set, yielding an
empty variant list and hiding the real quants. Keep scanning older
snapshots for actual variants and carry the vision flag across snapshots.
Also record the disk-space fallback variant's size in expected_sizes so
the later cache-reuse probe can size-verify the fallback main shard
instead of only checking for its existence.
* Propagate cached repo casing to companions and preflight split co-location
Two fixes to the case-variant GGUF cache reuse:
- Resolve the requested repo id to its cached canonical casing once in
load_model, up front, and pass it to the main GGUF and its companions
(mmproj / MTP drafter). Previously only _download_gguf resolved the
casing internally, so a case-variant request loaded the main file from
the canonical cache dir while the companions kept the requested casing
and missed the cached vision projector / drafter offline. Extracted the
resolution into a shared _resolve_repo_id_casing helper.
- Apply the split-shard co-location check in the disk-space preflight. When
a split GGUF's shards are cached across different snapshots the whole set
is refetched later, so counting them as cached made the preflight read 0
bytes to download, skip the smaller-variant fallback, and then fail the
full download on a low-disk machine.
* Reuse a co-located split GGUF snapshot and fix split fallback size probe
- When reusing a cached split GGUF, scan snapshots for one that holds the
whole set co-located instead of taking the newest snapshot's first shard.
A newer snapshot with only the first shard no longer shadows an older
complete snapshot, so an already-cached split model is reused rather than
refetched (which would fail offline).
- The disk-space fallback records its size in expected_sizes only for a
single-file fallback. _find_smallest_fitting_variant returns the whole
variant size, so using it as the first shard's expected size rejected a
valid cached first shard of a split fallback and forced a re-download.
* Scan for a complete split snapshot in the preflight; require a loaded catalog hit
- The disk-space preflight now uses the same co-located snapshot scan as the
download path (_cached_colocated_split_main) instead of the newest-snapshot
probe, so a newer snapshot holding only the first shard no longer masks an
older complete one and trips the smaller-variant fallback for a fully cached
split model.
- _resolve_model only attaches to a /v1/models entry that is actually loaded
(loaded != False). /v1/models also lists cached-but-unloaded catalog entries,
and matching one by case skipped /api/inference/load and left the agent
pointed at a model that is not resident.
* Restrict cross-snapshot GGUF cache reuse to offline
Reusing a same-name blob from an older or case-variant snapshot bypasses the
Hub revision/etag check, so a repo that updates a GGUF in place could serve
stale weights online. Gate the cross-snapshot and case-variant reuse (both the
disk-space preflight accounting and the download path) on HF_HUB_OFFLINE.
Online, hf_hub_download fetches the current revision and resumes a partial
download, so the reuse is unnecessary there; offline it remains the resilience
fallback. Marked the two reuse regression tests as the offline scenarios they
represent and added an online test asserting a fresh fetch.
* Harden offline cache reuse and hub-id detection
Three follow-ups on the case-variant GGUF cache path:
- Honor every truthy HF_HUB_OFFLINE spelling (1/true/yes/on), not just "1", when
gating the cross-snapshot and case-variant cache reuse. With HF_HUB_OFFLINE=true
the Hub calls are already offline, so the reuse must trigger or the cached GGUF
fails to load; route both the preflight accounting and the download path through
the same offline parse the rest of the backend uses.
- Resolve mmproj/MTP companions from the actual cached snapshot when offline.
resolve_cached_repo_id_case can keep a partial lower-case spelling when any dir
exists under the requested casing, so an hf_hub_download on that casing misses the
canonical companion; scan every case-variant snapshot and return the cached path.
- Restrict the case-insensitive model-id match to syntactically valid hub ids
(a single namespace/name over the HF charset). A server-side relative path such
as models/Llama/Foo.gguf is no longer treated as a hub id, so it cannot
casefold-match a differently cased path on a case-sensitive filesystem. This is
host independent, unlike the local-existence probe which cannot see a server path.
* Only casefold-match model ids against a loopback Studio
A two-segment string like Models/Foo is indistinguishable from a hub id, and the
local Path.exists() probe in _is_hub_model_id cannot see a path that exists only
on a remote Studio host. So against a remote server, casefolding could attach to
a distinct server-side path (Models/Foo vs models/foo) on a case-sensitive
filesystem. Gate the case-insensitive match on is_loopback_url(base): only a
local Studio, where the existence probe is authoritative, casefolds. For a remote
Studio the match is exact and a case-mismatched request falls through to
/api/inference/load, whose already-loaded dedup resolves it correctly.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* fix: use Windows Hermes installer from unsloth start
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip the Hermes setup wizard during unattended start-install
unsloth start hermes auto-installs Hermes and then writes its own
session-scoped Hermes config. The install commands, as written, drop into
the installer's interactive setup wizard (hermes setup), which prompts for
global API keys and model choice and points the user at a different global
provider than the one Unsloth just configured, blocking the launch.
Pass the installer's skip flag on both platforms: the PowerShell scriptblock
form with -SkipSetup, and bash -s -- --skip-setup for the piped POSIX
installer.
* Refresh PATH from the registry after a Windows agent install
A Windows installer persists the agent's directory to the User/Machine PATH
in the registry and updates only its own process, so the current process
keeps a stale PATH until it restarts (the installers print 'restart your
terminal'). The post-install shutil.which then misses the just-installed
agent and unsloth start fails with 'installed but isn't on PATH yet',
forcing a re-run in a new shell.
Merge the registry PATH hives back into the process before re-resolving so a
freshly installed agent launches in the same invocation. No-op off Windows
and on any read error; only ever augments PATH.
* Fix/adjust PATH refresh for PR #6903
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* fix: force Unsloth provider selection for opencode
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* opencode: pin the model without clobbering the user's disabled providers
The session overlay wrote disabled_providers unconditionally and the inline
OPENCODE_CONFIG_CONTENT set disabled_providers to an empty list. Since that
inline layer outranks the user's global and project config and opencode
replaces the array rather than merging it, every provider the user had
disabled was silently re-enabled for the session. Only strip 'unsloth' from an
existing disable list, and drop disabled_providers from the inline config.
Also insert --model only on a bare launch: it is a global flag for the TUI, so
placing it before a passthrough subcommand (serve/run) breaks arg parsing; a
subcommand takes the model from the pinned config instead. Parse the printed
OPENCODE_CONFIG_CONTENT with shlex.split in the test so it round-trips under
POSIX shell quoting.
* Re-enable a globally disabled opencode unsloth provider for the session
A fresh OPENCODE_CONFIG overlay omits disabled_providers, and opencode
replaces that array across config layers only when a higher layer sets the
key, so a user's global disabled_providers of ['unsloth', ...] survived the
merge and left the session provider disabled even though the overlay defines
provider.unsloth and pins the model.
Consult the user's global opencode config (XDG_CONFIG_HOME/opencode, or
%APPDATA%/opencode on Windows) when the overlay has no list of its own, and
when the effective list disables unsloth write it back to the overlay minus
unsloth. The provider loads while the user's other disabled providers stay
disabled. Best-effort read: a missing or unparseable global config is a
no-op.
* Override opencode disabled_providers in the inline layer; keep model flag for TUI flags
Re-enabling a disabled unsloth provider now rides in the inline
OPENCODE_CONFIG_CONTENT layer instead of the session overlay. The overlay
sits below a project opencode.json, which could re-disable the provider; the
inline layer outranks both global and project configs and is recomputed each
run, so no-launch reruns never reuse a stale generated list. The effective
disabled list is read from the project config if the repo sets one, else the
global config, across config.json/opencode.json/opencode.jsonc (JSONC
tolerated), and written back minus unsloth only when unsloth is disabled.
Also keep the pinned --model when the opencode passthrough starts with a
top-level TUI flag such as --dir or --continue; only a real subcommand
(serve/run/...) takes the model from config, so a leading '-' now still gets
--model injected.
* Discover the opencode project config by walking up from the cwd
opencode finds a project config by searching ancestor directories, not just
the cwd. Walk from the cwd up to the filesystem root and use the nearest
directory that sets disabled_providers, so running unsloth start opencode
from a subdirectory of a repo whose root config disables unsloth still gets
the inline override.
* Only inject opencode --model on a bare launch; rely on the inline model pin
Injecting --model whenever the passthrough started with a flag could place it
before a subcommand (e.g. opencode --print-logs serve), which opencode can
misparse. --model is unnecessary for any passthrough because the inline
OPENCODE_CONFIG_CONTENT pins the model in the highest-priority layer, so the
session model is forced without the flag. Restrict --model to the bare launch
and pass any other invocation through untouched.
* Register the session provider under a dedicated OpenCode id
Selecting the Unsloth model reliably required the wrapper to re-enable a
user-disabled unsloth provider, which meant reconstructing OpenCode's full
disabled_providers resolution (global, OPENCODE_CONFIG overlay, project config
discovered via --dir or an ancestor walk, .opencode directories,
OPENCODE_CONFIG_DIR, config.json/opencode.json/opencode.jsonc precedence, and
{env:} variable substitution) and overriding it in the inline layer. That is
unbounded and cannot be kept correct.
Register the session provider under a dedicated id (unsloth-studio) instead. A
user's disabled_providers list would never target it, so the session model is
always selectable and the overlay no longer reads or writes disabled_providers
at all: the user's own disables, in whatever config layer, are left exactly as
they are. This removes the JSONC parser, the config-directory scan, and the
ancestor/global resolution helpers, and the tests that exercised them.
* Scope the opencode session to the Studio provider
opencode filters every provider, including a config-defined custom one, through
its enabled_providers allowlist and disabled_providers denylist, and pinning the
model does not bypass that gate (a filtered provider resolves to a not-found
error). The provider arrays are also replaced, not merged, across config layers.
So a user with an enabled_providers allowlist that omits the session provider
would still have the Studio model filtered out.
Set enabled_providers to just the session provider and clear disabled_providers
in the inline OPENCODE_CONFIG_CONTENT overlay (the highest-priority layer, which
replaces these arrays). This guarantees the Studio model loads regardless of the
user's provider filters, without reading or reconstructing their multi-layer
config. It is session-only: the overlay lives in the env for this launch and
never touches the user's config files, so their normal opencode is unchanged and
only this session is limited to the Studio provider.
Also drop the redundant --model on --no-launch so the printed command stays
append-safe for drivers that append a subcommand (the inline pin forces the
model), and parse both POSIX and PowerShell no-launch output in the opencode
tests so they are not shell-specific.
* Pin opencode small_model to the session provider
The session allowlists only the Studio provider, but opencode's separate
small_model (used for lightweight tasks) could still point at another provider
from the user or project config; under the allowlist that provider is filtered,
so the lightweight task would resolve a not-found error mid-session even with the
main model pinned. Pin small_model to the session model in the same inline
overlay so every model use stays on the enabled provider. The session serves one
model, so it is the only valid target, and this stays session-only like the rest
of the overlay.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* Clear stale yolo approval state on no-launch reruns
The no-launch session config dir is deliberately reused across runs, but
the config writers only ever added the --yolo auto-approval settings and
never removed them. After one --yolo --no-launch run, every later run
without --yolo kept OpenClaw's tools.exec security=full/ask=off policy
plus exec-approvals.json, and OpenCode's permission allow block, so tool
execution stayed silently pre-approved.
Non-yolo runs now reset that state: OpenClaw drops the exec policy keys
and the yolo defaults in exec-approvals.json (approvals OpenClaw itself
recorded are kept; the file is removed when only the yolo payload is
left), and OpenCode drops the permission block. Launch mode is untouched
since it already uses an ephemeral temp dir.
* Strip only yolo-written values on non-yolo cleanup
Match each field against the exact value the yolo path writes before
removing it, so a stricter exec policy, approvals defaults set by the
user or the OpenClaw UI, and deny/ask OpenCode permission entries all
survive a plain no-launch rerun. An unparseable exec-approvals.json is
left in place, matching how an unparseable config is handled.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Write a prompting policy on non-yolo instead of deleting to a permissive default
OpenClaw and OpenCode both treat an omitted policy as permissive: OpenClaw's
effective exec policy for an unset tools.exec is security=full/ask=off on the
gateway host, and OpenCode defaults an unset permission to allow. So clearing
the yolo values on a non-yolo run did not restore prompting, it fell back to
those permissive defaults and left tool execution auto-approved.
A non-yolo run now writes an explicit prompting policy: OpenClaw gets
security=allowlist/ask=on-miss (verified to prompt even with the approvals file
removed, since the stricter of config and approvals wins), and OpenCode gets
edit/bash/webfetch=ask. Only a permissive/yolo value is tightened; a stricter
deny (or an ask the user set) is preserved, and the yolo approvals defaults are
still stripped. The file-edit CI path opts opencode/openclaw into --yolo, since
those agents now prompt by default and the headless test needs auto-approval.
* Respect existing exec mode, sandbox/node host, and global permission rules on non-yolo reset
The non-yolo reset for openclaw/opencode assumed an omitted policy was the
permissive yolo default and rewrote it, which corrupted or weakened stricter
setups it should have preserved:
- OpenClaw tools.exec.mode is the normalized policy knob and cannot be combined
with explicit security/ask (OpenClaw rejects the whole config), so writing
security+ask alongside a mode:deny/ask policy both broke the config and
relaxed it. Leave a mode-based policy untouched.
- host=sandbox defaults to security=deny and host=node routes to a paired node;
neither is written by --yolo (which only writes host=gateway). Treating the
missing security as full and popping host broadened those into gateway/auto
exec. Only rewrite a gateway-routed permissive policy, and never pop a
non-gateway host.
- OpenCode permission can be a string ("deny") or a {"*": ...} catch-all.
The old code dropped a string form and overrode a catch-all by writing
per-tool ask, weakening a stricter user rule. Now a string is left in place,
a catch-all governs absent tools, and only an effective allow is tightened.
- The non-yolo ask policy only lived in OPENCODE_CONFIG, which loads below
project opencode.json, so a project config allowing edit/bash/webfetch still
auto-approved. Carry the ask policy in OPENCODE_CONFIG_CONTENT (above project
config) too, symmetric to how yolo carries its allow.
Also harden the openclaw path against a malformed non-dict tools value.
Adds tests for mode/sandbox/node hosts, string and catch-all permissions, and
the inline ask policy over a project config.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope non-yolo resets to the exact yolo fingerprint and preserve granular denies
OpenClaw: reset only the exact host=gateway/security=full/ask=off policy --yolo
writes, so an omitted or host=auto/sandbox/node policy (which can resolve to a
sandbox security=deny default) is no longer broadened to allowlist/on-miss, and a
deliberate tools.exec.mode is left alone (OpenClaw never migrates our security/ask
write into a mode).
OpenCode: carry a granular object or a deny inline verbatim so a per-tool user rule
is not collapsed to a blanket ask, but floor any object that grants allow anywhere to
the string ask (which fully replaces a project object) so no inline allow pattern can
leak through into a silent auto-approve on a non-yolo session.
* Stop overriding project config on non-yolo; require full approvals fingerprint
The non-yolo OpenCode reset carried a session permission in
OPENCODE_CONFIG_CONTENT, which outranks the project opencode.json we
cannot read. That inline override could not correctly reflect the project:
it weakened a project deny to a prompt, mishandled global string rules,
leaked through a granular object's permissive default when no catch-all
was present, collapsed an object with an allow (losing its deny), and
missed per-agent permissions. All of these stem from forcing a value over
an unknown project config.
A non-yolo run now only undoes what --yolo wrote: it flips our own
explicit per-tool allow back to ask in our config file and carries no
permission inline, so the project's own permissions are honored as
written. Clearing our persisted yolo state is the actual fix; --yolo still
carries its allow inline so it works over a project config.
OpenClaw approvals cleanup now strips the yolo defaults only when the full
fingerprint (security=full, ask=off, askFallback=full) is present, so a
mixed user policy that merely shares askFallback=full (whose omitted
default is deny) is kept intact.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* replaced connect with start
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix
* Studio: build the coding-agent command from the selected server
The API keys panel showed a hardcoded `unsloth start claude`. `unsloth start`
defaults to 127.0.0.1:8888 and only mints a key for a loopback server, so a
non-default port or a tunnel/remote base would target the wrong server or fail
to mint. Build the command from the panel base/key (and emit a key for
non-loopback), matching the other snippets in the panel.
* CLI: keep `unsloth connect` as a hidden alias for `unsloth start`
Avoids breaking existing scripts and docs that still call `unsloth connect`.
* Tests: stub _unstarted_cleanup in same-task disconnect test
The test builds _SameTaskStreamingResponse via __new__, so set the attribute
that __call__ now reads.
* Match coding-agent command loopback check to the CLI 127.0.0.0/8 rule (#6613)
* Keep unsloth_cli.commands.connect importable as a deprecated shim (#6613)
* Format the new coding-agents panel strings and import per biome (#6613)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop the unsloth connect alias and shim; unsloth start is the only command (#6613)
* Route unsloth connect to unsloth start as a hidden backward-compatible alias (#6613)
* Forward unsloth run model-load flags to unsloth start (gguf-variant, context-length, load-in-4bit, tensor-parallel) (#6613)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Session-scope coding agent config in unsloth start
Configure each agent for the current session instead of writing the Studio endpoint, key, and default model into the user's own config. Codex, OpenCode, OpenClaw, and Hermes get a private config relocated through their config-path env vars (CODEX_HOME, OPENCODE_CONFIG overlay, OPENCLAW_CONFIG_PATH plus OPENCLAW_STATE_DIR, HERMES_HOME). Claude Code suppresses the attribution header for the session via the CLAUDE_CODE_ATTRIBUTION_HEADER env var plus a --settings overlay, with no ~/.claude write. --launch uses an ephemeral temp dir removed after the agent exits; --no-launch uses a stable Unsloth-owned dir and prints the matching export lines.
* Read relocated agent session config in Local Agent Guides CI
The contract crosscheck and the openclaw/hermes patch helpers now read each agent's config from the relocated path printed by unsloth start --no-launch (CODEX_HOME, OPENCODE_CONFIG, OPENCLAW_CONFIG_PATH, HERMES_HOME) instead of fixed home paths. The Claude attribution A/B toggles the header for the session only (shipped-config HIT vs vanilla MISS) instead of editing ~/.claude/settings.json.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip the POSIX-only --no-launch parser test on Windows
test_no_launch_output_is_parseable mirrors the #6547 bash CI parser, which greps export/unset lines and only runs on Linux/macOS runners. On Windows --no-launch prints PowerShell ($env: / Remove-Item), so the export-line assertion does not apply there. Cross-OS staging CI surfaced this.
* Size Claude Code's auto-compact window to the loaded model's context
Claude Code auto-compacts against its native (~600k token) window, so against a smaller local model it overflows the server's context (silent truncation) long before it compacts. Set CLAUDE_CODE_AUTO_COMPACT_WINDOW to the loaded model's real context length (the value codex/openclaw already get via model_context_window / contextWindow). Omitted when the model reports no context length.
* Pin OpenCode/Hermes context window and set 90% compaction across agents
Feed every agent the server-determined sequence length (the value /v1/models reports from runtime_context_length) and a ~90% compaction threshold. OpenCode: a custom-provider model with no limit defaults to context 0, which silently disables auto-compaction, so set limit.context/output and scale the compaction buffer to 10% of the window. Hermes: pin model.context_length (it otherwise falls back to a 256k default when the server's /v1/models omits the field) and set compression.threshold 0.9. Claude: add CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=90 alongside the window. Codex (model_context_window) and OpenClaw (contextWindow) already carried the window and auto-manage off it.
* Add `unsloth start pi` recipe
Pi was the only agent without a built-in recipe, so the agent-guides CI
hand-wrote ~/.pi/agent/models.json. Add a first-class `pi` command mirroring
the others:
- write_pi_config writes the session-scoped OpenAI-compatible provider config
(key in the config, like openclaw/opencode).
- pi() launches `pi --provider unsloth --model <id>` (Pi defaults to the google
provider, so the provider/model are pinned on the command line) with HOME
relocated for the session. Pi has no config-dir env var and resolves ~/.pi off
$HOME, so HOME-scoping keeps the user's ~/.pi untouched.
Migrate the agent-guides CI off the hand-written config onto the
`unsloth start pi --no-launch` path (connection + file-edit), with a crosscheck
for the provider api, so the documented recipe is exercised.
* Harden unsloth start for Windows and WSL agent launches
Address the Codex review on PR 6613:
- write_pi_config now pins the loaded contextWindow and a sane maxTokens so Pi
compacts instead of overflowing a small Studio context (it otherwise assumes
its 128000 default), matching the other agents.
- pi() sets USERPROFILE (and HOMEDRIVE/HOMEPATH when present) alongside HOME on
native Windows, where Node resolves ~/.pi via USERPROFILE rather than HOME, so
the session no longer reads or writes the user's real ~/.pi.
- The WSLENV bridge flags path-valued vars with /p so a Windows npm shim under
/mnt receives translated paths, while scalar vars (the numeric context window)
pass through untranslated. WSLENV is deduped on the bare name.
- _print_env prints the launch command with PowerShell-safe quoting so the inline
--settings JSON survives copy-paste on native Windows --no-launch.
Add tests for the WSLENV path flagging, PowerShell quoting, the Pi context
window, and the Pi USERPROFILE relocation.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Set CLAUDE_CODE_NO_FLICKER for the Claude session
A local server streams in bursts, so Claude Code's full-screen TUI redraw
flickers between tokens. Disable it for the session via CLAUDE_CODE_NO_FLICKER,
alongside the other CLAUDE_CODE_* session env knobs.
* Add a normalized --yolo flag routed to each agent's auto-approve mode
It is easy to forget which agent spells "run tools without prompting" which way,
so `unsloth start` now accepts all three spellings as one option (--yolo,
--dangerously-skip-permissions, --dangerously-bypass-approvals-and-sandbox) and
routes to the agent's own mechanism:
- claude: --dangerously-skip-permissions
- codex: --dangerously-bypass-approvals-and-sandbox
- hermes: --yolo
- pi: --approve (Pi's only approval gate is project trust)
- opencode: a permission allow block in opencode.json (no CLI flag exists)
- openclaw: tools.exec security=full / ask=off / host=gateway (no CLI flag exists)
Because the option is parsed by `unsloth start`, the "wrong" spelling for an
agent still routes correctly instead of leaking through to the agent and erroring.
IS_SANDBOX is deliberately left unset for Claude so its root/sandbox safety gate
still applies. Adds routing, cross-routing, and per-config tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix review findings: IPv6 loopback command, pi USERPROFILE under WSL, yolo guard
From a 10-reviewer pass over the PR:
- studio/frontend agent-command.ts: normalize bracketed IPv6 hosts. URL.hostname
returns "[::1]" for http://[::1]:8888, which never matched the "::1" loopback
checks, so the copied command embedded the placeholder API key for a local IPv6
server instead of the bare auto-minting command. Now [::1] is treated as loopback
like the CLI's is_loopback_url, so the command matches the CLI contract.
- pi(): also relocate USERPROFILE (and HOMEDRIVE/HOMEPATH) when running under WSL
against a /mnt Windows shim, not just on native Windows. Windows Node resolves
~/.pi via USERPROFILE, and the WSLENV bridge translates the path, so pi no longer
falls back to the user's real ~/.pi in that case.
- _yolo_command_flags: use .get so a config-based agent (or a typo) yields no flag
instead of a latent KeyError.
Adds tests for the WSL pi USERPROFILE relocation, the yolo unmapped-agent guard,
and that opencode/openclaw --yolo stays config-only (no argv flag).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix round-2 review findings: WSLENV /p upgrade, agent help text
- _merge_wslenv now upgrades a user's pre-existing unflagged WSLENV entry (e.g. a
bare HOME or USERPROFILE) to the path-translated form (HOME/p) instead of leaving
it as-is, so a Windows agent shim under WSL receives the translated session path
rather than the raw Linux path.
- Generalize the `unsloth start` registration help to list all six agents (was only
"Claude Code, Codex").
Adds a test for the WSLENV unflagged-entry upgrade.
* Fix round-3 review findings: complete openclaw --yolo, refresh stale copy
- openclaw --yolo now also writes the host approvals file (exec-approvals.json with
defaults security=full / ask=off / askFallback=full) alongside the tools.exec
config. OpenClaw gates tool execution on both layers (the stricter wins), so the
config alone could still leave it prompting or denying. Mirrors `openclaw
exec-policy preset yolo`. ask=off means nothing is ever prompted, so the runtime
socket block is unnecessary.
- Studio API panel copy: clarify that a local server auto-mints the key while a
remote one embeds it in the command, and add pi to the swap hint.
- Local Agent Guides CI: drop the stale "pi has no start.py recipe" note now that
all six agents are driven via `unsloth start <agent> --no-launch`.
Adds the openclaw approvals-file assertions and a no-yolo openclaw test.
* start: parse claude --version with a regex so a format change does not drop optimization flags
* start: offer to install a missing agent (prompt then run its install command)
* start: auto-start a Studio server for --model when none is running, and stop it on exit
* inference: surface an actionable message when llama-server cannot compile a tool grammar
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix review findings: kill the auto-started server tree on Windows; apply the tool-grammar message to the OpenAI passthrough too
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* start: split --model org/repo:variant so a running session is not evicted
`unsloth start <agent> --model org/repo:QUANT` failed against an already-running
Studio server and, worse, killed whatever model another session had loaded.
/v1/models lists a loaded GGUF under its bare repo id (e.g. unsloth/Qwen3-1.7B-GGUF),
so _resolve_model never matched the `:QUANT`-suffixed request. It then POSTed
/api/inference/load with model_path=org/repo:QUANT, which (a) Hugging Face rejects
("Repo id must use alphanumeric chars, '-', '_' or '.'") and (b) evicts the model the
other session was using, so a second 'unsloth start' in a new tmux/terminal tore down
the first. Re-running the command then attached to the now-empty server, which is why
it 'worked the second time'.
Mirror the org/repo:QUANT -> org/repo + --gguf-variant QUANT shorthand that
'unsloth run' and llama.cpp already accept, splitting it in _connect before we match or
serve. Matching now resolves against the loaded bare repo id (no spurious reload, no
eviction), and any real load uses a valid repo id plus gguf_variant. An explicit
--gguf-variant still wins; local paths and Windows drive letters pass through untouched.
The auto-serve path likewise spawns 'unsloth run --model org/repo --gguf-variant QUANT'.
* start: harden auth-key handling, codex teardown, and CI transcript redaction
Three review findings:
1. CI could leak a live key. agent-guides-drive.sh printed the raw
'unsloth start --no-launch' transcript (which carries export UNSLOTH_API_KEY /
ANTHROPIC_AUTH_TOKEN lines) to the Actions log on both the failure path and the
success path before redact() ran. Add cat_redacted() and use it for those two
prints, so the key is scrubbed on the way to the log while the on-disk file stays
intact for the env parsing that follows.
2. Outages masqueraded as bad keys. _key_accepted caught a broad Exception and
returned False, so a 5xx or timeout while checking a cached key looked like a
rejection: it discarded a good key and minted extra ones (local) or reported 'no
saved key' (remote). Only treat HTTP 401/403 as a rejection; let other errors
propagate so a real outage surfaces.
3. Codex preflight could leave the auto-started server up. _require_gguf_for_codex
runs after _connect may have auto-started Studio but before _run installs its
teardown finally, so a preflight rejection (e.g. a transformers-backend model) left
the server holding the port/GPU until the atexit backstop. Tear it down explicitly
at the point of failure.
Tests: a 5xx on a saved key surfaces without minting; a non-GGUF codex preflight
tears down the auto-served server.
* start: fix IPv6/portless studio URLs, Pi config-dir isolation, and Pi install recipe
Four review findings:
1. Pi ignored the session config when PI_CODING_AGENT_DIR was already set. Pi's
getAgentDir() reads process.env.PI_CODING_AGENT_DIR before falling back to
$HOME/.pi/agent, so a value inherited from the user's shell sent Pi to their real
config and skipped our provider/key (the HOME relocation alone was not enough). Pin
PI_CODING_AGENT_DIR at the session's .pi/agent dir; it is path-valued so the WSL
bridge translates it automatically.
2. Pi install hint dropped Pi's documented --ignore-scripts. Pi's README installs with
'npm install -g --ignore-scripts @earendil-works/pi-coding-agent' and notes it needs
no install scripts, so accepting the prompt now follows that safe recipe.
3. Auto-start ignored a portless UNSLOTH_STUDIO_URL. unsloth run binds to
'parsed.port or 8888', so http://127.0.0.1 launched the child on 8888 but the health
poll (and the returned base) still used port 80, stalling until the startup timeout.
Normalize the base to host:8888 (IPv6-safe) before starting and polling.
4. API-panel command mistook IPv6 loopback for the bare default. The bare 'unsloth
start' only probes 127.0.0.1:8888 on the IPv4 stack, so http://[::1]:8888 must carry
an explicit UNSLOTH_STUDIO_URL. Drop ::1 from the bare-default host set while keeping
it a loopback host (URL emitted, no key needed).
Tests: PI_CODING_AGENT_DIR is set to the session dir; _effective_base normalizes
portless/IPv6 bases; a portless UNSLOTH_STUDIO_URL auto-serves on :8888.
* start: apply fresh-review findings across CLI, CI, and the API-panel command
From a fresh multi-reviewer pass over the merged head plus the latest Codex bot review:
1. Load knobs now always consult the server. _resolve_model matched on model id alone,
so --gguf-variant / --context-length / --no-load-in-4bit / --tensor-parallel were
silently ignored whenever the id was already loaded (asking for UD-Q4_K_XL kept a
Q8_0 serving). With any explicit knob the CLI defers to /api/inference/load, whose
already-loaded dedup answers without reloading when variant and settings match, so a
second session running the same command still attaches without evicting the first.
2. OpenCode --yolo and the session model pin now ride in OPENCODE_CONFIG_CONTENT. A
project's own opencode.json outranks OPENCODE_CONFIG, so a repo config could silently
override the session model and the --yolo permission block; OPENCODE_CONFIG_CONTENT
outranks project config. The API key stays in the private file, never in printed env.
3. The --no-launch recipe's last line is a self-contained one-liner (inline VAR=value
assignments before the command, conflicting vars blanked). People copy just the last
line, and a bare codex/claude there ran against the user's real ~/.codex or Anthropic
credentials with zero isolation, e.g. inheriting a pre-existing damaged ~/.codex
state DB and blaming the recipe. The CI drive script scrubs the key from the one
'invoking:' echo this adds.
4. The auto-serve log is 0600 and the parent handle is closed. It sat world-readable in
the shared tempdir under a predictable name while carrying the minted sk-unsloth-
key from the unsloth run banner.
5. _key_accepted fails with a clean message on outages. Non-auth errors (5xx, network,
timeout) surfaced as a raw traceback; 401/403 still mean a rejected key.
6. _effective_base strips URL paths, and https loopback targets never auto-serve.
http://127.0.0.1:8888/studio polled /studio/api/health (404) and https://127.0.0.1
polled the wrong scheme, both spinning until the 15-minute startup timeout.
7. API-panel command: only literal 127.0.0.1:8888 earns the bare command. localhost can
resolve to ::1, which the bare CLI never probes, so it keeps UNSLOTH_STUDIO_URL.
8. CI artifact sweep covers redacted-configs/ and agent-workdir/, not just logs/.
Tests: 125 CLI tests pass (new coverage for each fix), 156 backend tests pass, ruff
clean. Adds an unsloth connect alias regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* start: hand Pi a clean screen at launch
Pi paints inline from wherever the cursor sits: its first render assumes a
clean screen instead of clearing or entering the alternate screen itself
(current Pi never emits a clear at startup). Launched under unsloth start,
that left the session starting mid-scroll beneath the connection output.
Clear the screen (click.clear, cross-platform, no-op without a TTY) right
before the Studio banner so Pi opens exactly one line down on a clean
viewport. Launch path only: --no-launch recipes and piped output are never
wiped, and alternate-screen agents are left alone.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* start: auto-override hermes' 64K context floor for small model windows
Hermes refuses to initialize when the served model's context window is
under 64,000 tokens, and a second copy of the same check rejects the
compression model mid-session. write_hermes_config previously pinned the
real window, so any small local model (e.g. 40,960) failed at startup
with manual config.yaml instructions.
For windows below the floor the recipe now claims 65,536 in
model.context_length, scales compression.threshold so compaction still
fires at 90% of the real window, and sets
auxiliary.compression.context_length to cover the mid-session check.
Windows at or above the floor keep the exact previous behavior.
* ci: install pi with --ignore-scripts, matching the start.py hint
The pi cell predates the pi recipe in start.py and still installed the
package with lifecycle scripts enabled, so CI stopped exercising the
exact command users are prompted to run. npm_retry now passes extra
flags through, the pi branch mirrors the install hint verbatim, and the
stale no-recipe comment is refreshed.
* ci: fail loudly when a relocation var is missing from connect output
The empty-string guards ran after appending /config.toml or /config.yaml,
so they could never fire: crosscheck_contract silently skipped its
contract checks and patch_hermes_tools died on the root path with a bare
traceback. Check the raw variable first and guide_fail with the real
cause.
* staging: 6613 round 6 (https elision, no-launch home reuse, auto-start key fallback)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
Extracted and narrowed from unslothai/unsloth#6543 by @TheJagStudio.
This keeps the startup/banner and text file encoding hardening separate from the already-merged Python code-exec UTF-8 fix in #6548.
Co-authored-by: Jagrat Patel <81472856+TheJagStudio@users.noreply.github.com>
* studio: announce Cloudflare tunnel state and warn about public exposure on startup
The startup banner only printed a line when a tunnel URL was up, so a plain
`unsloth studio -H 0.0.0.0` launch silently created a public trycloudflare.com
URL with no indication that Studio had become reachable from the internet. The
only hint at the tunnel was the CLI help, shown when an invalid command was typed.
Make the banner always state the tunnel state for wildcard binds:
- ON: the public URL plus a warning that anyone with it can reach Studio from
outside the network, and that --no-cloudflare keeps it local-only.
- FAILED: requested but did not start (local network only).
- OFF: --no-cloudflare was passed (local network only).
Secure mode keeps its existing wording (the authenticated tunnel is intended and
--no-cloudflare is not valid there). Clarify the --cloudflare help text in both
the argparse and typer definitions. Default behavior is unchanged.
Also surface the state on the `unsloth studio run` banner, which runs the server
with silent=True and prints its own banner: it now calls _print_cloudflare_line
too, so the ON/OFF/FAILED notice and public-exposure warning are no longer
skipped on that path (previously it only echoed the URL when a tunnel was up).
For the OFF and FAILED notices, do not claim "local network only" when the
reachability probe just confirmed the raw port is reachable from the public
internet: --no-cloudflare and a failed tunnel disable only the Cloudflare link,
not the wildcard bind, so the message is reworded to flag the public raw port.
* Fix/adjust Cloudflare banner warnings for PR #6515
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust Cloudflare banner comments for PR #6515
* Fix/adjust IPv6 Cloudflare tunnel gate for PR #6515
* Fix/adjust Cloudflare review comments for PR #6515
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix silent run Cloudflare notice
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: start the Cloudflare tunnel for --secure even in --api-only, and add --api-only to `unsloth studio run`
--secure exposes ONLY the Cloudflare link (it forces a loopback bind), but
_cloudflare_tunnel_should_start gated the tunnel on `not api_only`, so
`run.py --secure --api-only` started no tunnel and then fail-closed with
"A secure Cloudflare link is not allowed". That blocked the natural headless
use: serve just the API (no web UI) over the authenticated tunnel.
Make --secure start the tunnel regardless of api_only (the non-secure path is
unchanged: tunnel only a 0.0.0.0 bind, never api-only Tauri or Colab). Then
expose --api-only on `unsloth studio run` and forward it through both the
re-exec args and the in-venv run_server call, so
`unsloth studio run --secure --api-only --model ...` is a one-liner secure API
server.
Verified end to end: `run.py --secure --api-only` now brings up the tunnel and
serves /api/health over it (200), with / returning 404 (no UI).
Tests: update the tunnel-gate truth table (secure+api-only now tunnels;
secure+colab still does not) and add --api-only registration + re-exec/in-venv
forwarding coverage to the run CLI tests.
* Trim comments to be succinct (no behavior change)
* studio: address review on parent --api-only and secure api-only CORS
- Reject --api-only on the parent `unsloth studio` group when a subcommand
is invoked, with the same redirect guidance used for --parallel/--secure;
otherwise the flag was silently dropped and the UI served anyway.
- Keep CORS any-origin for secure api-only serving: that mode publishes the
API over Cloudflare for remote browser clients, so the Tauri-only lockdown
(still applied to plain local api-only) would break preflight. Factored the
decision into cors_origins_for_mode() and gate it on api_only and not secure;
run_server exports UNSLOTH_SECURE before importing main.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: suppress TAURI_PORT and de-dup test for headless run --api-only
- run_server gains emit_tauri_port (default True, unchanged for the Tauri/
desktop path). The new headless `run --api-only` path passes False so the
Tauri-only TAURI_PORT= line no longer prepends the documented URL/API key
banner (it ran even under --silent and could break one-liner parsers).
- Remove a duplicate test_reexec_forwards_api_only that shadowed the
parametrized one; fold the --secure --api-only case into it so the secure
headless path is actually collected.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Installer: respect a declined Studio auto-start and keep Ctrl+C shutdown logs ordered
The `curl | sh` Studio auto-start prompt had two issues on Linux/macOS/WSL
(install.sh). install.ps1 already gates on input redirection, so Windows is
unaffected.
1. Typing n, or any closed/EOF /dev/tty, still launched Studio. The read
fallbacks defaulted to "y" (read failure, and the no-tty branch), so any
answer other than a cleanly delivered y/n line auto-started a blocking
foreground server. Default those to "n"; a real Enter still counts as yes
via ${_reply:-y}.
2. On Ctrl+C the shell prompt printed in the middle of Studio's shutdown logs.
The non-interactive installer shell took the default SIGINT action and died
before the child finished its graceful shutdown, so the prompt raced ahead
of "All subprocesses cleaned up". trap '' INT in the installer shell so it
waits for Studio's own graceful shutdown.
* Studio: wait for the uvicorn thread before the terminal returns on Ctrl+C
Builds on #6565 by @Imagineer99. The studio server runs uvicorn in a daemon
thread, so on Ctrl+C the process could return to the shell while that thread
was still writing its shutdown logs, interleaving them with the prompt.
Retain the uvicorn thread and join it (flushing stdout/stderr) before terminal
entrypoints return, from run.py's main shutdown path and the CLI shutdown paths.
Refinements over #6565:
- Bound the join at 5s (_SERVER_SHUTDOWN_JOIN_TIMEOUT, matching the existing
_graceful_shutdown subprocess timeouts) so a stalled uvicorn shutdown cannot
hang the terminal; the timeout warning branch is now reachable.
- Restore SIG_DFL for SIGINT/SIGTERM at the start of the signal handler so a
second Ctrl+C force-quits, and drop the redundant in-handler wait (the
post-loop wait already covers the signal path).
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: keep child Ctrl+C working and restore SIGBREAK
- install.sh: run studio in a subshell that resets INT to default
(trap - INT; exec ...) so the foreground child does not inherit the
installer shell's ignored SIGINT, which would otherwise swallow the
studio process's own Ctrl+C and graceful shutdown.
- run.py: also restore SIGBREAK to SIG_DFL in the signal handler so a
second Ctrl+Break force-quits on Windows, matching SIGINT/SIGTERM.
* install.sh: capture studio exit with || under set -e so the migration hint still prints
* Trim shutdown-fix comments to be terser (comments only, no code change)
* Dedup CLI shutdown-wait into finally blocks (review follow-up)
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: accept --not-secure as a back-compat alias for --no-secure
PR #6560 renamed the negative secure flag from --not-secure to --no-secure
to match argparse.BooleanOptionalAction. Re-add --not-secure as a hidden,
deprecated alias at both CLI layers so existing scripts and muscle memory
keep working, while --no-secure stays the documented spelling.
- studio/backend/run.py: extract the CLI parser into _build_arg_parser() so
the flag wiring is unit-testable, and register --not-secure as a hidden
store_false alias for --no-secure. Last flag wins, matching
BooleanOptionalAction semantics.
- unsloth_cli/commands/studio.py: add a hidden --not-secure option to
`unsloth studio` and `unsloth studio run`; it forces secure off and
forwards the canonical --no-secure to the backend.
- Tests at both layers for the alias and its polarity.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address review on --not-secure alias
- run.py: use argparse.SUPPRESS for the --not-secure default so the alias
never contributes a namespace default (the canonical --secure owns it).
- studio.py: resolve --not-secure last-wins from argv via _resolve_secure()
so `--not-secure --secure` keeps secure on, matching the backend's
BooleanOptionalAction and how --secure/--no-secure already behave.
- Add a CLI last-wins test covering both flag orders.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
studio.backend.run.__main__ adds "--secure" argument via argparse.BooleanOptionalAction, which automatically creates negative --no-secure, that is with **NO** prefix, instead of **NOT**.
* CLI: stop `unsloth connect` from leaking Studio credentials to unverified servers
`unsloth connect` (and `unsloth chat`) discovered a Studio base URL from
UNSLOTH_STUDIO_URL or the default localhost port after only an unauthenticated
/api/health probe, then sent credentials to it:
- keyless connect iterated every cached API key and sent each as a bearer token
to {base}/v1/models, so a malicious or port-preempting endpoint could harvest
all of them;
- with no cached key it self-issued a Studio JWT and POSTed it to
{base}/api/auth/api-keys;
- unsloth chat sent the same self-issued JWT to the discovered base.
The key cache was a flat, global list with no binding to a server identity, so a
key minted for one Studio could be replayed to any other.
Changes:
- Scope the agent key cache per base URL so a key is only ever replayed to the
exact server it was minted for. Pre-scoping flat caches are ignored rather
than replayed (at most one extra local mint on the next launch).
- Gate every automatic credential flow to loopback bases. A non-loopback
UNSLOTH_STUDIO_URL now requires an explicit --api-key and nothing is sent
automatically. SSH-tunnelled Studios that land on 127.0.0.1 keep working.
- Mint the API key locally against the Studio auth DB instead of POSTing a
self-issued JWT over the network, so no bearer token leaves the process on the
local path.
- Apply the same loopback gate to connect_studio_server (used by unsloth chat).
Fully closing same-host loopback-port preemption needs a signed /api/health
handshake so the client can verify the server identity before sending anything;
that is tracked as a server-side follow-up.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* CLI: verify Studio server identity before auto-sending credentials
Adds a challenge-response so `unsloth connect` and `unsloth chat` can confirm a
discovered loopback endpoint is really this install's Studio (not a process
that preempted the port) before sending it a cached or freshly minted
credential. This closes the same-host loopback-preemption gap left open by the
previous commit, which could only limit the blast radius.
Server:
- storage.get_or_create_identity_secret(): a dedicated server-wide secret in
app_secrets (kept separate from the per-user JWT secret), readable only by
the same OS user.
- storage.compute_identity_proof(nonce) = HMAC-SHA256(identity secret, nonce).
- GET /api/auth/identity?nonce=<base64url>: unauthenticated, returns the proof.
The nonce is opaque to the server and the proof reveals nothing about the
secret, so answering is safe.
Client:
- verify_studio_identity(base): sends a fresh 32-byte nonce, recomputes the
expected HMAC from the local same-user secret, and constant-time compares.
Fails closed on any error.
- connect._agent_api_key gates the loopback cached-key replay and the local
mint on it; connect_studio_server (used by unsloth chat) gates the
self-issued JWT on it.
A server that cannot read this install's secret (a different OS user, or a
remote/fake endpoint) cannot produce a matching proof, so the client refuses
and falls back to an explicit --api-key.
Tests: studio/backend/tests/test_identity.py (proof determinism, secret
persistence and caching, route response and nonce validation) and additions to
test_connect.py (the verify gate refuses when unverified, an explicit key skips
the check, and an end-to-end client plus server handshake against a stub HTTP
server).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* CLI: mint through the verified server instead of the local auth DB
CodeQL (py/clear-text-storage-sensitive-data) flagged the API key written to
the per-server cache and the agent config files once it was sourced from
storage.create_api_key(); the original HTTP-minted key did not trip the query.
Now that the identity handshake cryptographically confirms the loopback
responder really is this Studio before anything is sent, minting through the
server's /api/auth/api-keys endpoint with a self-issued JWT is safe again and
restores the original, CodeQL-clean data flow. The local-DB mint path is
removed.
The security properties are unchanged: discovery is still loopback-gated and
identity-verified, the key cache is still scoped per server, and a credential
reaches the server only after the handshake has proven its identity. The only
difference from the previous commit is that the self-issued JWT is sent to the
already-verified loopback server rather than the key being minted in-process.
Tests updated to mint through the fake server again.
* CLI: address review feedback on connect credential handling
- Reuse a saved per-server key before the loopback/identity gate. Keys are
scoped per base URL, so a key the user saved with --api-key for a remote or
SSH-tunnelled Studio (whose identity secret the local handshake can't match)
is replayed only to that exact server. The loopback + identity-handshake gate
now guards just auto-minting (self-issuing a JWT and creating a new key),
which is the path that needs a cryptographically verified local Studio. Fixes
keyless reuse being impossible for remote/tunnelled Studios the user had
saved a key for.
- connect_studio_server (unsloth chat / inference): when the user explicitly set
UNSLOTH_STUDIO_URL but the server can't be safely attached (non-loopback, or
identity unverifiable), fail with a clear message instead of silently loading
the model locally. Opportunistic discovery of the local default still falls
back to a local load.
- Harden cache parsing: tolerate a corrupt or hand-edited cache where a base
maps to a non-list (which would otherwise iterate a string into
single-character "keys"), and read the cache as UTF-8.
Tests updated and added: saved-key replay without the handshake for both local
and remote bases, keyless mint still refused when the loopback server is
unverified, and connect_studio_server erroring on an explicit remote while
falling back locally on default discovery.
* CLI: harden connect handshake against relay and gate cached minted keys
Addresses review feedback on the credential handshake:
- Refuse HTTP redirects on credential-bearing requests (the identity handshake,
/v1/models, key minting, and the chat HTTP backend). A process squatting the
discovered port could 302 /api/auth/identity to the real Studio and relay its
valid proof, or bounce a bearer-token request to another base, and urllib
follows redirects by default. A shared no-redirect opener now treats any 3xx
as an error.
- Give cached keys provenance. Keys the user supplied with --api-key are "saved"
and replay without the handshake (needed for remote or SSH-tunnelled Studios
whose secret the local handshake can't match). Keys we auto-mint are "minted"
and replay only after the identity handshake, so a port squatter can't collect
a previously minted localhost key just by answering the health check. New cache
shape: servers[base] = {"saved": [...], "minted": [...]}.
Known residual: a different-OS-user process that squats the port and can also
reach a genuine same-secret Studio elsewhere on loopback can still manually relay
the identity challenge. Fully closing that needs the proof bound to the server's
real listening port, or OS-level peer-credential checks; tracked as follow-up. A
same-user attacker is out of scope, since it can already read the 0600 key cache.
Tests: redirect rejection in the handshake, minted-cache requiring the handshake
while saved-cache bypasses it, and the existing suites updated for the new cache
shape. unsloth_cli (206) and test_identity.py (5) pass.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* CLI: keep urllib imports function-local in the no-redirect opener
The repo's import-hoist safety linter (scripts/verify_import_hoist.py) flags
hoisting urllib to module level because it re-points the 'urllib' name in the
pre-existing HttpChatBackend._request scope. Build the no-redirect opener lazily
with function-local urllib imports instead, matching this module's convention,
and restore the local 'import urllib.request' in _request and
verify_studio_identity.
* test(identity): skip route tests when routes.auth import chain is unavailable
The identity route tests build a TestClient from routes.auth, which pulls the
whole routes package (routes/__init__ -> inference -> llama_cpp, ...). In a
minimal test matrix without the heavy backend deps, or when another test in the
same process has already broken that import chain, importing it raised and the
two route tests hard-failed. Skip in that case instead: the proof crypto is
covered by the storage-level tests, and the full backend CI still exercises the
route. No behaviour change where the deps are present (5 passed in isolation).
* test(connect): make connect tests pass on native Windows
unsloth connect supports Windows: --no-launch prints PowerShell ($env:X =
"v" / Remove-Item Env:X) instead of POSIX (export/unset), and the launch
path bridges env into a Windows agent .exe over WSLENV. The tests hardcoded
the POSIX shell forms, so on a real windows-latest runner 12 of them failed on
the assertion string even though every command exited 0.
Add OS-aware assertion helpers (_assert_env_set / _assert_env_unset) that check
the right shell syntax for the host OS, and skip the two WSL-from-Linux shim
tests on native Windows (os.name is 'posix' inside WSL, so that path can't run
there). No change on Linux/macOS (57 passed); the connect command's behaviour
is untouched. Validated on a windows-latest staging runner.
* style(connect): tighten comments in the credential-leak fix
Condense the verbose explanatory comments and multi-line docstrings added by
this PR to one or two lines each, drop a few that just restated the code, and
keep the security rationale where it is load-bearing. Comment/whitespace only;
verified with unslothai/scripts comment_tools.py (check --strip-docstrings:
6/6 'code unchanged'). Tests unchanged: connect 57 passed, identity 5 passed.
* CLI/Studio: harden the identity handshake (review round)
Addresses the latest Codex/Gemini review of the handshake:
- Store the identity secret privately. sqlite3.connect created the auth DB
world-readable under a 022 umask, so another OS user could read app_secrets
and forge proofs, defeating the same-user assumption the handshake rests on.
The auth dir and DB are now restricted to owner-only (0700/0600); the JWT
secret and password hashes there get the same protection.
- Bind the proof to the server's listening port. The stateless HMAC(secret,
nonce) was relayable: a process squatting the discovered port could proxy the
challenge to the real Studio on another port and pass it back. The proof now
covers the port the server actually listens on (from the socket, never the
Host header) and the client checks it against the port it connected to, so a
relayed proof from a different port no longer matches. Closes the manual-relay
residual left after the redirect fix.
- Cap the identity response read (the server is still unverified at that point)
and serve the identity route from a sync def so its first-call SQLite read
runs in the threadpool instead of the event loop.
Tests: port-bound proof + relayed-proof rejection added; identity (5) and
unsloth_cli (58) suites pass. Verified end to end against a real backend
(auth DB owner-only, handshake + mint still succeed).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* CLI/Studio: bind the identity proof to the connection address, not just port
Follow-up to the port binding from the last review. Binding only to the port
left a cross-address relay: a squatter on a different loopback address but the
same port (for example localhost resolving to a squatter on ::1 while the real
Studio is on 127.0.0.1) could proxy the nonce to the real Studio and pass back
a proof that still matched, since both share the port.
The proof now covers the address and the port the connection landed on:
- Server: takes the address+port from request.scope, which uvicorn populates
from getsockname, so it is the real local address the client reached even
when Studio is bound to 0.0.0.0 (verified empirically), never the
client-controlled Host header.
- Client: resolves the base host to one concrete IP, talks to exactly that IP,
and binds the proof to (IP, port). A proof relayed from a Studio on a
different address or port was computed for that other endpoint and no longer
matches the one the client dialed.
Both sides normalise the address through ipaddress so equivalent forms compare
equal. Together with the private-secret and redirect fixes, this closes the
cross-user loopback relay an attacker can mount without reading the secret.
Tests: proof now bound to host+port; relayed-proof rejection retained; identity
(5) and unsloth_cli (58) suites pass. Verified end to end against a real backend
(handshake + mint still succeed).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* CLI: pick the loopback address at discovery so localhost does not regress
find_studio_server now resolves a bare localhost base to its concrete loopback
addresses and returns the first that answers /api/health, IPv4 127.0.0.1 first
(where unsloth studio binds by default). The whole flow (health probe, identity
check, credential send) then targets that one address instead of racing
IPv4/IPv6 resolution, where localhost could resolve ::1-first and hide a Studio
bound to 127.0.0.1. A literal IP or remote name is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Keep server-side tools enabled under --secure and on every bind
--secure binds loopback and exposes Studio only through an authenticated
Cloudflare HTTPS tunnel, but it was grouped with a raw 0.0.0.0 bind and
force-disabled all server-side tools (web search, Python, terminal). The
process tool policy overrode the client's enable_tools request, so the
model was never told the tools existed and answered in plain text. The
plain 'unsloth studio' command had no way to re-enable and printed nothing.
Tools now default on for every bind. The bind host and --secure no longer
change the tool policy; only an explicit --enable-tools/--disable-tools
forces it on or off. Both 'unsloth studio' and 'unsloth studio run' accept
the flags and the startup banner states the resolved policy.
- run.py: replace _apply_default_tool_policy(host, secure) with
_apply_cli_tool_policy(enable_tools); add an enable_tools kwarg to
run_server and --enable-tools/--disable-tools to the argparse.
- _tool_policy.py: resolve_tool_policy defaults to on for every host and
no longer prompts on a network bind.
- studio.py: drop the secure-as-public tool gating, add the flags to the
plain command, and reword the startup banner.
- Update and extend the secure-flag and tool-policy tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add tool-policy notice to plain server banner and refresh run --help
Follow-up to PR review:
- run.py: the plain 'unsloth studio' / --secure / direct run.py path went
through _emit_startup_output without any tool-policy line, so a
network-reachable launch was silent about code execution now that tools
default on. Thread enable_tools through _emit_startup_output /
_emit_secure_startup_output and print a one-line policy notice, followed by
a single stop hint.
- studio.py: the 'unsloth studio run' --enable-tools/--disable-tools and --yes
help still described the removed loopback-on/network-off default and the
confirmation prompt; reword to match the new policy.
- Add tests for the banner notice and the refreshed help text.
* Update CI tool-policy resolver tests for default-on behavior
tests/python/test_unsloth_run_tool_policy_resolver.py still asserted the
removed network-bind policy (0.0.0.0 and LAN IP default off, explicit enable
prompts and aborts on a declined prompt), so it failed the Python CI jobs.
Rewrite the truth table: every bind defaults on, explicit on/off always wins,
and the resolver never prompts (yes/silent/prompt kept for compatibility).
* Trim comments for the tool-policy change
Shorten the verbose docstrings and block comments added for --secure tool
handling; keep the security-relevant intent. Verified comment-only via an AST
diff (code unchanged).
* Add deterministic test that server-side tools execute under --secure
Drive the GGUF agentic tool loop with a fake llama-server stream and let the
real execute_tool run: python counts 1..100, terminal returns a UTC datetime,
and web_search runs through real _web_search with only the ddgs network
boundary mocked. A policy assertion pins that the post-fix --secure path
(policy None + per-request enable_tools) is what keeps these executions
reachable. No model, GPU, or live network; runs in the existing backend CI.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align _emit_startup_output banner test with the moved stop hint
The tool-policy notice now prints between the access banner and the stop
hint, so the stop hint is emitted once at the end instead of inline in the
banner (include_stop_hint is False and print_studio_stop_hint runs once).
Update the plain-localhost case to match; the mismatch and wildcard cases
already asserted this wiring.
---------
Co-authored-by: Michael Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* unsloth connect
* harden error paths, fix codex oss_provider routing, tighten key cache perms
* Increase timeout for studio server lookup and enhance key caching logic
* openclaw/opencode/hermes to connect
* improvements
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* error handling for requested models not loaded
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix claude connect env under WSL
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* Studio: trim serving-log noise and surface llama-server engine stats
Studio prints one structured line per HTTP request, so the SPA's polling and
per-invalidation fan-out bury the lines that matter.
- Dedup identical successful GETs within a short window (default 300ms,
UNSLOTH_STUDIO_ACCESS_LOG_DEDUP_MS) so a burst logs once. The dedup key
includes the query string, so distinct query-driven GETs are not collapsed.
Runs after the response is sent, so it adds no request latency; mutations,
non-2xx, and loading polls are untouched.
- Collapse pure-liveness polls (/api/health, /api/auth/status,
/api/inference/status, /api/inference/monitor) to a longer heartbeat
(default 10s, UNSLOTH_STUDIO_ACCESS_LOG_POLL_DEDUP_MS). The API monitor
console polls /monitor every 1.5s while open.
- Translate llama-server's Prometheus /metrics into a periodic vLLM-style
engine_stats line (generation/prompt throughput and requests in flight) from
a daemon poller, gated on UNSLOTH_STUDIO_ENGINE_STATS. Throughput uses
llama-server's predicted_tokens_seconds / prompt_tokens_seconds gauges, with
a tokens_predicted_total / prompt_tokens_total counter-delta fallback; it does
not use n_decode_total (which counts llama_decode() calls, not tokens). No KV
field is emitted, since llama.cpp does not expose kv_cache_usage_ratio.
--metrics is added only when probe_server_capabilities reports the binary
supports it, so older/custom binaries still load. The poller keeps retrying
through transient scrape failures (stop() drives shutdown) and a malformed
sample cannot crash its thread.
- api_monitor.append_reply: once the preview cap is reached, skip the per-chunk
re-concat (avoids O(n^2) on long generations) while still recording the "..."
truncation marker for a reply that lands exactly on the cap.
- unsloth studio --verbose and unsloth studio run --verbose both restore every
per-request log; --verbose before a subcommand is rejected with guidance
(matching --secure / --parallel). run --verbose still forwards --log-verbose
to llama-server, preserving the pre-existing pass-through verbosity.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add --secure Cloudflare-only mode and revamp API usage examples
--secure / --not-secure on `unsloth studio` and `unsloth studio run`:
- --secure binds 127.0.0.1, requires the Cloudflare tunnel, and advertises only
the Cloudflare link. cloudflared reaches the server over localhost, so the raw
port is never exposed on a public interface.
- If the tunnel cannot start, fail closed with a clear message instead of
silently leaving a raw 0.0.0.0 link.
- Default stays not-secure (no behavior change); coexists with the existing
--cloudflare/--no-cloudflare flag. Host defaults are unchanged.
- /api/health (authed) now reports the live tunnel URL.
API usage examples (Profile > API):
- Example tabs for curl, Python, curl + tools, Python + tools, plus an OS row
(Linux/macOS/WSL vs Windows) auto-detected from the platform.
- Windows curl passes the JSON body via a file so PowerShell does not strip the
quotes when calling curl.exe.
- Python + tools forwards enable_tools/enabled_tools through extra_body and
guards chunk.choices, since tool-lifecycle events carry no choices.
- Shows the loaded model name and the real API key while it is still revealed.
- A Cloudflare Tunnel toggle (default on) shows the public tunnel URL and uses
it as the base_url in the examples when a tunnel is running.
Tests cover the tunnel start gate and the --secure flag on both commands.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: gate --secure tools on public exposure and harden API examples
In secure mode the server binds loopback but is reachable via the public
Cloudflare tunnel, so resolve the tool policy against the public exposure
(0.0.0.0) rather than the loopback bind. This keeps server-side tools off by
default and prompts before enabling them, instead of inheriting the loopback
default of on. The startup tool notice now names the public surface.
Also reject --secure with --no-cloudflare directly in run_server and the
run.py argparse (not only the CLI), JSON-encode interpolated model names so
Windows paths and quotes cannot produce invalid JSON or broken snippets, and
force-refresh /api/health on the API panel so a tunnel that starts after the
first health read still surfaces its URL.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: API examples show direct host when tunnel toggle is off; move Copy onto code
The Cloudflare Tunnel toggle had no visible effect when Studio was opened
through the tunnel: the off state fell back to window.location.origin, which
equals the tunnel URL in that case. /api/health now reports the direct
host:port (server_url), and the API panel uses it for the off state so it shows
the real non-tunnel base. Also move the Copy button out of the tab row and onto
the code block.
* Studio: highlight API examples, add advanced tabs, fix tunnel toggle row
Syntax-highlight the curl/PowerShell/Python snippets with the app's shared
shiki plugin (bash/powershell/python). Add 'curl + advanced' and
'Python + advanced' tabs that set temperature/top_p/top_k/min_p/
repetition_penalty/max_tokens, enable thinking, and turn on all tools.
The Cloudflare Tunnel row no longer shifts the code block: the tunnel URL is
always rendered (dimmed when off) so toggling keeps the row height constant.
Key the highlighted block on its content so it remounts when only the base URL
changes (the renderer's block memo otherwise kept a stale URL).
* Studio: rename API tunnel toggle to Secure HTTPS, hint --secure when exposed
Rename the API examples toggle from Cloudflare Tunnel to Secure HTTPS. When the
server was not launched with --secure, show an info tooltip noting the raw
0.0.0.0 port is still globally reachable and pointing at --secure. /api/health
now reports whether --secure was used so the hint is hidden in secure mode.
* Studio: force tools off for plain network/secure launches
The plain 'unsloth studio --secure' (and '-H 0.0.0.0') launcher re-execs run.py
and never installed a tool policy, so the process default (honor per-request
enable_tools) let any API-key holder run Python/terminal tools over the public
endpoint. Force the policy off at the run.py entrypoint when network-reachable
(0.0.0.0 or --secure); 'unsloth studio run' still installs its own resolved
policy and does not go through this path.
* Studio: apply default tool policy in run_server, not the run.py entrypoint
The plain launcher runs from the studio venv and calls run_server directly, so
it never hit the run.py __main__ guard. Move the network/secure default-off tool
policy into run_server so every launch path (plain, --secure, direct run.py)
gets it; the run subcommand still overrides it with its resolved policy.
* Studio: clarify --secure help text on the network exposure tradeoff
Spell out in --help (both unsloth studio and unsloth studio run, plus the
run.py argparse) that --not-secure also serves the raw 0.0.0.0 port reachable
from anywhere on the network, matching the API panel's Secure HTTPS hint.
* Studio: cache API-key PBKDF2 derivation to cut per-request /v1 auth overhead
validate_api_key re-ran the 100k-round PBKDF2 on every authenticated
request, adding ~15ms to each /v1 call made with an sk-unsloth- key.
Benchmarked against the bare llama-server it proxies to, API-key requests
carried ~22ms of fixed overhead vs ~7ms for the JWT path; the gap was
entirely this redundant key derivation (Pydantic validation measured
0.005ms, so it is not a factor).
The raw-key to hash mapping is a pure deterministic function of the fixed
server salt, so memoize it per process, keyed by a salted HMAC of the key
(never the key or a recoverable digest). The cached value equals what is
already stored at rest. Revocation and expiry remain enforced by the
SQLite read on every call, so a cache hit only skips the KDF, never the
active or expiry checks. Only keys that exist in the DB are cached, so
unknown-key spam cannot grow it.
After the change the API-key /v1 overhead drops to ~8ms, at parity with
JWT, while the at-rest PBKDF2 hashing is unchanged.
Adds test_api_key_expiry.py covering API-key and JWT expiry enforcement
and the new cache: it skips the KDF on repeat and still rejects revoked
or expired keys.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten comments across the secure-tunnel and API-key changes
Condense multi-line comments and docstrings to one or two lines, drop the
ones that restate obvious code, and remove an orphaned test section header.
Comment-only: verified with comment_tools.py check (9/9 code unchanged), the
auth/secure-tunnel/CLI test suites, and a clean frontend typecheck and build.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: enable stdio MCP servers on a loopback bind
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address codex review on stdio MCP loopback gate
* Studio: fix banner URL and preserve stdio MCP env opt-in on network binds
* Studio: scope loopback to exact aliases and honor force-disable on run_server reuse
* Studio: cover force-disable across a public re-bind and fix a stale test comment
* Studio: keep stdio MCP off on Colab loopback launches
* Studio: set tool policy before server startup
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* Studio: Add Tensor-Parallel llama.cpp support
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden Tensor-Parallel fallback and GPU selection
* Studio: reconcile split-mode extras and harden tensor-split planning
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reconcile split-mode extras in backend duplicate-load guard
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: preserve inherited non-tensor split modes on reload
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: honor cancellation in tensor fallback, preserve tensor mode on rollback, and don't raise an explicit small context
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reconcile split-mode in reload check and strip it on tensor downgrade
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strip --tensor-split alongside --split-mode so inherited ratios don't override the tensor planner
An inherited or stale --tensor-split in llama_extra_args was appended after
Studio's computed --tensor-split and won last in llama.cpp, re-introducing the
asymmetric-GPU OOM tensor mode is meant to prevent. Group -ts/--tensor-split
into the split-mode shadow set so it is stripped on inherit and on the layer
fallback; parse_split_mode_override still keys on the mode value only.
* Drop quantized KV for the tensor attempt and report native max context
Tensor mode aborts on a quantized KV cache, so a user with q8_0/q4_1 etc. who
enabled Tensor Parallelism silently fell back to layer split. Clear the cache
type (and strip inherited/explicit --cache-type) for the tensor attempt only;
the layer fallback re-runs with tensor off and keeps the user's choice.
Also report max_available_ctx from the native context, not an explicit small
-c, so the context slider no longer warns too early in tensor mode.
* Reconcile inherited split-mode extras in the already-loaded check
When a same-model load omitted llama_extra_args, the tensor comparison resolved
the raw (None) request and treated an inherited --split-mode tensor server as a
mismatch, forcing a needless reload. Compare using the stored extras stripped
the same way the reload strips them.
* Pass tensor_parallel through compare-mode loads
The generalized compare path loaded each GGUF without tensor_parallel, so
compare ran layer split even with the toggle on and left the settings sheet
stale. Send the toggle and hydrate the loaded state from the response, matching
the main chat and recipe load paths.
* Add --tensor-parallel flag to unsloth studio run
The headless one-liner could only reach tensor mode by passing --split-mode
tensor as a raw llama.cpp extra. Add a first-class --tensor-parallel/
--no-tensor-parallel option that sets the tensor_parallel field on the
/api/inference/load payload, forwarded through the studio-venv re-exec like the
other polarity flags. Matches the web UI toggle and the API field.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* fix(studio): load run.py by path for editable installs
`studio update` can leave a partial site-packages/studio/backend/ tree
(plugin build artefacts only). That shadowed tree wins over an editable
install and breaks `from studio.backend.run import ...`. Loading run.py
by file path via importlib sidesteps the conflict.
The module is cached in _RUN_MODULE so repeated calls are cheap.
If exec_module fails, the module is removed from sys.modules before
re-raising so a subsequent retry starts clean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle None __file__ when checking cached run module for PR #5909
* Harden _load_backend_auth_storage against None __file__ and resolve cache-key path (PR #5909)
* Adapt studio run/cloudflare in-venv tests to _load_run_module loader (PR #5909)
---------
Co-authored-by: Jim Dawdy <jimdawdy@Jims-MacBook-Pro.local>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: add `unsloth chat` CLI command
Interactive chat REPL on the shared Studio backend: trained-model picker
when no model is given, /think and /compare toggles (adapter toggle on
CUDA, side-by-side base-model load on MLX), markdown streaming, and
connect-if-running Studio server mode so models stay warm across
sessions and are shared with the UI.
* fix settings
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix error handling and compare base precision
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix chat CLI backend imports and GGUF drafter loading
* Hide split thinking tags in chat CLI streams
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* Studio: auto Cloudflare tunnel for 0.0.0.0 launches
Binding Studio to 0.0.0.0 for remote access often leaves the raw
http://<ip>:<port> URL unreachable (https-vs-http, blocked high ports,
closed cloud security groups). On a wildcard bind, auto-start a free
cloudflared quick tunnel and show its https://*.trycloudflare.com URL in
the startup banner:
Secure link access via Cloudflare: https://<random>.trycloudflare.com
- new studio/backend/cloudflare_tunnel.py: find or download+cache the
cloudflared binary (per-OS/arch GitHub release, safe .tgz extract),
start the tunnel, parse the URL, tear it down. Stdlib only; best-effort
and non-fatal throughout (a missing binary or offline box never blocks
or slows startup).
- run_server starts the tunnel for 0.0.0.0 only (skips loopback, api-only
and Colab), prints the line in the banner, and _graceful_shutdown stops
the child so it never orphans.
- --cloudflare/--no-cloudflare flag (default on) on `unsloth studio` and
`unsloth studio run`, forwarded through the re-exec into run_server.
- tests for the helper, the CLI flag forwarding, and the run.py defaults.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio cloudflare: send a User-Agent on the cloudflared download
GitHub's CDN can 403 the default Python-urllib User-Agent on release asset
downloads. Set an explicit UA and pin it with a test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio cloudflare: address review (opt-out for subcommands, tunnel teardown)
- reject --no-cloudflare placed before a subcommand (it would not reach the
subcommand), mirroring the --parallel guard
- register the tunnel before waiting for its URL so a shutdown during the wait
stops cloudflared instead of orphaning it
- tear the server + children down if `unsloth studio run` startup aborts
(health timeout, model-load error, Ctrl+C) before the wait loop
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* -Added catch for unsloth running inside of system32 (for windows)
- changed import statement to import os as _os instead of only os.path as _osp
- replaced __osp mention with __os.path
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Adressed geminis codereview. Did not change Error message as "System32" is precise enough and the _system32 variable is lowercase only
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* Fix Windows installer winget msstore certificate failure
`winget install` was invoked without `--source winget`, so winget also
queried the msstore source. When msstore fails certificate pinning
(error 0x8a15005e, "The server certificate did not match any of the
expected values") winget aborts and demands `--source`, so the Python
(and uv) install fails even though the package exists in the winget
source.
- Pass `--source winget` to all winget install calls (Python x2, uv).
Both packages live in the winget source, so this is strictly correct
and skips the failing msstore round-trip entirely.
- Add a python.org fallback (Install-PythonFromPythonOrg) that downloads
the official installer and runs it silently per-user (no admin/UAC)
when winget is unavailable or fails for any reason. Mirrors the
existing uv -> astral.sh fallback so Python installs without manual
steps. Resolves the latest 3.13.x from python.org with a pinned
fallback, and selects the amd64/arm64/x86 installer per architecture.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Pin remaining setup.ps1 winget calls to --source winget
Two winget invocations in studio/setup.ps1 still queried all sources and
could hit the same msstore certificate-pinning failure (0x8a15005e) that
broke the Python install in install.ps1:
- `winget show Nvidia.CUDA --versions` (CUDA Toolkit version probe)
- `winget install ... ShiningLight.OpenSSL.Dev` (OpenSSL dev for llama-server)
Every other winget call in this file already passes `--source winget`
(Git, CMake, VS Build Tools, CUDA install, Node.js, and setup.ps1's own
Python 3.12 install), so these two were stragglers. Both packages live in
the winget source; pinning it makes setup robust to an unhealthy msstore
source, matching the rest of the file.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Stop amd-smi GPU probe from popping a DiskPart UAC prompt
On Windows, AMD GPU detection in install.ps1 and studio/setup.ps1 runs
`amd-smi list` / `static --asic` / `version`. amd-smi (shipped in
System32 by the Adrenalin driver) auto-elevates to read GPU/APU memory
details, surfacing a confusing DiskPart UAC prompt mid-install. The
Studio backend already documents and circuit-breaks on this in
studio/backend/utils/hardware/amd.py, but the installers did not.
Add an Invoke-AmdSmiNoElevate helper (both scripts) that runs amd-smi via
Start-Process under __COMPAT_LAYER=RunAsInvoker so it cannot auto-elevate
(no prompt), with a 30s timeout (matching amd.py) so a flaky amd-smi
cannot stall the install for minutes. On failure/timeout the existing WMI
name -> gfx fallback still resolves the arch, so detection is unchanged on
working hosts.
Verified on a Strix Halo (Radeon 8060S / gfx1151) box: the prompt is gone
and the probe is bounded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add experimental ROCm-on-WSL setup helper for Strix Halo (gfx1151)
install.sh already routes gfx1151 (Radeon 8060S / Strix Halo) to the
repo.amd.com/rocm/whl/gfx1151 wheels once a ROCm runtime is present, but
it does not install AMD's driver/ROCm stack -- a large, admin-gated
prerequisite. scripts/install_rocm_wsl_strixhalo.sh automates the Linux
side on a dedicated Ubuntu 24.04 WSL2 distro: ROCm 7.2 (wsl usecase), the
rocr4wsl HSA runtime, a librocdxg build, env setup, and a PyTorch gfx1151
GPU smoke test. A hard preflight refuses to run until the Adrenalin
>=26.3.1 driver is actually present, so it cannot half-install.
Procedure adapted from AMD's ROCm-on-WSL docs and community gfx1151 notes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Detect AMD GPUs by name so native Windows gets a GPU llama.cpp
The gfx-arch inference from the WMI GPU name was gated behind $HasROCm,
which the hipinfo/amd-smi probe leaves false on the common Windows case
(Adrenalin driver only, no HIP SDK -- and amd-smi often cannot read the
arch without elevation). So an AMD GPU was detected by name but never
mapped to a gfx target, --rocm-gfx was not forwarded, and studio setup
fell back to a CPU llama.cpp build.
Un-gate the inference (install.ps1 + studio/setup.ps1) so it runs whenever
an AMD GPU name is available. The inferred gfx is forwarded as --rocm-gfx,
which makes install_llama_prebuilt.py download the matching lemonade-sdk
ROCm prebuilt (e.g. llama-bNNNN-windows-rocm-gfx1151-x64.zip) -- a
GPU-accelerated llama.cpp that bundles its own ROCm runtime, so it runs
with just the Adrenalin driver. PyTorch's ROCm wheels still require a
confirmed HIP SDK ($HasROCm), so this only affects llama.cpp / inference
and never pulls broken ROCm torch.
Also broaden the name->arch table to every family lemonade ships Windows
assets for: gfx120X (RDNA 4), gfx110X (RDNA 3), gfx1151/gfx1150
(RDNA 3.5), and gfx103X (RDNA 2). Unknown names still fall back to CPU.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Suppress amd-smi DiskPart UAC prompt in the Python install/runtime paths
The earlier PowerShell guard covered install.ps1 / setup.ps1, but the
Python installer (install_llama_prebuilt.py detect_host,
install_python_stack.py ROCm probes) and the Studio backend monitor
(amd.py) also shell out to amd-smi on Windows, where it auto-elevates and
pops the same DiskPart UAC prompt mid-install / at runtime.
Inject __COMPAT_LAYER=RunAsInvoker into the amd-smi subprocess env on
Windows so it runs un-elevated (no prompt). Callers already tolerate an
empty/failed result and fall back to WMI / name detection (installer) or
the existing circuit breaker (amd.py). Gated to Windows so Linux/macOS
amd-smi behaviour is unchanged.
- install_llama_prebuilt.py: handled centrally in run_capture (covers
detect_host's `amd-smi list` and the version probe).
- install_python_stack.py: new _amd_smi_env() helper on its 3 raw
subprocess.run amd-smi calls.
- amd.py: merge RunAsInvoker into the existing child env.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Tighten AMD GPU name->arch patterns to avoid mismatches
The W9[0-9]{3} and RX 90[0-9]{2} patterns added for RDNA 4 were
speculative and over-broad: W9xxx would also match old GCN FirePro
W9100/W9000 cards (wrong gfx1201 -> a lemonade gfx120X download that
fails validation), and RX 90[0-9]{2} was redundant with the explicit
9070/9060 entries. Drop both; keep only confirmed RDNA 4 SKUs. Unmatched
AMD names still fall back cleanly to CPU.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fetch the llama.cpp validation model via huggingface_hub
The prebuilt validation downloads a tiny GGUF test model from huggingface
via bare urllib. On Windows / proxy setups where the server sends an
incomplete TLS chain, urllib cannot complete the Amazon CA chain (it does
no AIA intermediate fetching) and fails with CERTIFICATE_VERIFY_FAILED, so
a perfectly good GPU prebuilt is rejected and the installer falls back to a
CPU source build.
Route the validation-model download through huggingface_hub
(hf_hub_download) -- the same mechanism Studio uses for model downloads,
which completes the chain where urllib cannot -- keeping the direct URL as
a fallback. This lets the lemonade ROCm prebuilt validate and install on
cert-restricted machines (verified: hf_hub_download succeeds where urllib
returns CERTIFICATE_VERIFY_FAILED).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Guard the remaining raw amd-smi version probe via run_capture
A ROCm-version detector in install_llama_prebuilt.py called amd-smi version through a raw subprocess.run that bypassed run_capture's Windows RunAsInvoker guard, so it still triggered the DiskPart UAC prompt during setup. Route it through run_capture like the other amd-smi calls.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Forward --rocm-gfx even when the ROCm runtime is unconfirmed
setup.ps1 forwarded --rocm-gfx (and picked the windows-hip llama.cpp
prebuilt) only inside `if ($HasROCm)`. On Adrenalin-only hosts (amd-smi
present but no HIP SDK, so $HasROCm stays false) the gfx arch was
name-inferred but never forwarded, so install_llama_prebuilt.py saw
has_rocm=False and installed the CPU build -- even though the lemonade
gfx1151 GPU prebuilt runs fine there (it bundles its own ROCm runtime;
verified: llama-cli --list-devices -> ROCm0: AMD Radeon 8060S, 69 GB).
Forward --rocm-gfx whenever a gfx arch is known (it is authoritative and
implies ROCm in install_llama_prebuilt.py), and treat a known gfx arch as
windows-hip in the existing-install mismatch check. --has-rocm stays gated
on the confirmed-runtime signal.
Verified on Radeon 8060S / gfx1151: the installer now selects, validates,
and installs llama-b1286-windows-rocm-gfx1151-x64.zip (ROCm DLLs present)
instead of the CPU build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Install AMD ROCm PyTorch on name-inferred gfx hosts (enables Train/Export)
setup.ps1 picked the AMD ROCm PyTorch wheels only inside `if ($HasROCm ...)`.
On Adrenalin-only hosts (amd-smi present but no HIP SDK, so $HasROCm is
false) the gfx arch was name-inferred but the ROCm-wheel branch never ran,
so the host got torch+cpu. With CPU torch, torch.cuda.is_available() is
False, so the Studio backend sets CHAT_ONLY=True and hides Train/Export.
Un-gate the ROCm PyTorch index resolution on a known gfx arch (mirrors the
llama.cpp --rocm-gfx fix). AMD's per-arch Windows wheels
(repo.amd.com/rocm/whl/<gfx>) bundle the ROCm runtime, so they work without
a HIP SDK; a failed install still falls back to CPU.
Verified on Radeon 8060S / gfx1151: torch 2.11.0+rocm7.13.0 installs and
torch.cuda.is_available() -> True, device "AMD Radeon(TM) 8060S Graphics",
GPU matmul OK -> CHAT_ONLY=False -> Train/Export enabled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Force amd-smi un-elevated process-wide in the Python installers
Guarding individual amd-smi call sites kept missing some (install_python_stack.py's probe loop and its Windows GPU re-check), so the DiskPart UAC prompt kept reappearing. Set __COMPAT_LAYER=RunAsInvoker process-wide at the top of install_python_stack.py and install_llama_prebuilt.py on Windows so every amd-smi subprocess (current and future) runs un-elevated with no per-call guard. Safe: these scripts only spawn amd-smi/rocminfo/hipinfo probes and pip/uv. setup.ps1 keeps per-call guards because it also spawns winget installers that need elevation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix Invoke-AmdSmiNoElevate exit code on PS 5.1 + RX 7700S arch match
Start-Process -PassThru leaves the returned process object's .ExitCode
$null after WaitForExit on Windows PowerShell 5.1, so the helper set
$LASTEXITCODE to $null and every caller's `if ($LASTEXITCODE -eq 0 ...)`
was always false -- the amd-smi GPU / gfx-token / ROCm-version detection
branch was effectively dead (masked only because the un-gated WMI
name->gfx inference still ran). Reproduced on PS 5.1.26100.
Rewrite the helper to use [System.Diagnostics.Process]::Start with a
ProcessStartInfo (UseShellExecute=false), whose .ExitCode is reliable,
with async stream reads (ReadToEndAsync) to avoid a pipe-buffer deadlock
and WaitForExit(timeout) to bound a flaky amd-smi. __COMPAT_LAYER=
RunAsInvoker (inherited via the process env) still suppresses the
auto-elevation / DiskPart prompt. Also drops the temp files and the
empty-ArgumentList edge case. Verified: exit code propagates
(7 -> $LASTEXITCODE=7), output captured, env restored.
Also fix the gfx1100 name pattern `RX 7700(?! S)` -> `RX 7700(?!S)` so the
spaceless retail name "RX 7700S" is correctly excluded (it belongs to the
gfx1102 row). Both found by PR review.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Address PR review follow-ups (install.sh table, update path, tests, WSL)
From the multi-agent PR review:
- install.sh: sync the AMD name->arch table with install.ps1 / setup.ps1
(the bash table had drifted to the old narrow patterns). Adds RDNA 2
(gfx103X), workstation PRO W SKUs, and more Strix Halo/Point names, and
orders gfx1102 before gfx1100 so the spaceless retail name "RX 7700S"
resolves correctly (bash case has no negative lookahead). AMD-ROCm-only:
the name inference stays gated behind _has_amd_rocm_gpu(), so NVIDIA /
CPU / macOS are unaffected.
- setup.ps1: the "dependencies up to date" fast path skipped the torch
reinstall, so an existing user who had CPU torch (installed before
ROCm-wheel support) stayed stuck in CHAT_ONLY. Now, when an AMD gfx arch
is known AND the installed torch is CPU-only, don't skip -- force the
dependency pass so the ROCm wheels install.
- scripts/install_rocm_wsl_strixhalo.sh: resolve the real /opt/rocm dir
instead of hardcoding ROCM_VER for LD_LIBRARY_PATH / the librocdxg
symlink (breaks if amdgpu-install lays ROCm under a patch-version dir);
add a LIBROCDXG_REF pin knob and a "verified against" freshness header.
- tests/studio/install/test_pr5940_followups.py: cover _hf_resolve_url_parts,
_fetch_validation_model_bytes (hf path + urllib fallback), run_capture's
Windows-only amd-smi RunAsInvoker injection, and install.ps1 vs setup.ps1
name-table parity (catches future drift). 14 tests, all passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix DiskPart UAC prompt: skip amd-smi on Windows without a HIP SDK
On Windows, amd-smi re-initialises the ROCm runtime on every invocation
(even `amd-smi version`) and, on hosts without a working HIP runtime
(consumer APUs/dGPUs with only the Adrenalin driver), elevates a child
process at runtime -- popping a UAC/DiskPart prompt. amd-smi's own
manifest is asInvoker, so __COMPAT_LAYER=RunAsInvoker cannot suppress
that runtime elevation (verified: even `amd-smi version` hangs and
times out with RunAsInvoker set).
Replace the ineffective RunAsInvoker-only approach with a real gate:
only spawn amd-smi on Windows when a HIP SDK is detectable (hipinfo
present, so amd-smi runs un-elevated) or the user opts in with
UNSLOTH_ENABLE_AMD_SMI=1. The gfx arch is already resolved from WMI
name inference (forwarded via --rocm-gfx), so ROCm wheel + lemonade
llama.cpp selection is unaffected. Linux/macOS amd-smi never elevates
and is untouched (no regression). RunAsInvoker is kept as harmless
belt-and-suspenders for tools that DO use manifest elevation.
Applied consistently across:
- studio/backend/utils/hardware/amd.py (runtime GPU polling)
- install.ps1, studio/setup.ps1 (install-time detection)
- studio/install_llama_prebuilt.py (prebuilt arch probe + version)
- studio/install_python_stack.py (ROCm version + arch probe)
Verified live on AMD Radeon 8060S (gfx1151), native Windows: fresh
install detects the GPU, installs ROCm torch (torch.cuda.is_available()
True), launches Studio with no DiskPart prompt, and inference, tool
calling, web search, LoRA finetuning, and GGUF export all run on the GPU.
Tests: add 6 _amd_smi_allowed() gating tests + PowerShell-installer gate
assertions; update the three amd-smi monitoring tests to opt in (they
mock amd-smi as available). Full suite: 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: helpful WSL message when the GPU isn't exposed to ROCm
In WSL, an AMD GPU's ROCm-on-WSL runtime is only available with a recent
Adrenalin driver AND a distro AMD supports (currently Ubuntu 24.04). When
neither is in place, GPU detection (rocminfo/_has_amd_rocm_gpu) finds
nothing and we silently fall back to CPU.
Add an actionable hint in the CPU-fallback path, shown only on WSL and
only AFTER detection has already failed -- so it is forward-compatible:
the moment a driver/distro DOES expose the GPU (e.g. if AMD later adds
Ubuntu 26.04 support), detection succeeds and the hint never fires. The
message:
- notes a GPU is plumbed in (/dev/dxg) but no ROCm runtime is exposed,
- lists the two prerequisites (Adrenalin driver + Ubuntu 24.04),
- if the distro is not 24.04, says AMD may not support it yet,
- tells the user to `wsl --install Ubuntu-24.04` and re-run,
- links AMD's ROCm-on-WSL guide + the experimental Strix Halo helper.
Verified live: on Ubuntu-24.04 the hint shows (version-warning omitted)
and the CPU install completes; on Ubuntu-26.04 the extra "this distro may
not be supported" line appears and points to 24.04.
Also fix the experimental scripts/install_rocm_wsl_strixhalo.sh: AMD's
repo.radeon.com/amdgpu-install/ is indexed by unified installer version
(30.30, 31.30, ...), NOT ROCm version, so the hard-coded
amdgpu-install/7.2.0/ path 404'd. Scan the installer dirs newest-first
for a noble .deb matching the target ROCm major.minor (ROCm 7.2 ->
30.30.x/amdgpu-install_7.2.x), falling back to the newest available.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* WSL: fix shortcut collision + pin ROCm-on-WSL driver reqs from AMD docs
Two WSL-related fixes informed by AMD's official ROCm-on-WSL docs and
field reports for Strix Halo / Ryzen AI Max+ (Radeon 8060S, gfx1151):
1. Shortcut collision (real bug). install.sh's WSL branch wrote
"Unsloth Studio.lnk" to the SAME Desktop / Start Menu folder as the
native-Windows installer (install.ps1 New-StudioShortcuts). Running
install.sh in WSL therefore silently retargeted the native shortcut at
the WSL launcher (wt.exe -> wsl.exe), so the desktop/start-menu icon
stopped launching native GPU Studio. Now the WSL shortcut uses a
DISTINCT name -- "Unsloth Studio (WSL - <distro>).lnk" -- and fetches
the Unsloth .ico to %LOCALAPPDATA%\Unsloth Studio so it shows the
proper icon. Native and WSL shortcuts now coexist.
2. Precise ROCm-on-WSL prerequisites. Research (AMD radeon-ryzen WSL
compatibility matrix, gianni.rosagallina.com Feb-2026 guide,
ROCm/ROCm#4952/#5509/#6022) confirms WSL GPU on Strix Halo requires
AMD Adrenalin Edition >= 26.1.1 (26.2.2+ is the first production
ROCDXG/WSL release) + ROCm 7.2.1 + Ubuntu 24.04; an older driver does
not inject the ROCm/DXG runtime into /usr/lib/wsl/lib, so rocminfo sees
only the CPU. install.sh's WSL hint and the experimental
install_rocm_wsl_strixhalo.sh header/preflight now state the exact
driver version (was a guessed ">=26.3.1"), bump ROCM_VER to 7.2.1, link
AMD's radeon-ryzen docs, and document the known librocdxg caveat that
usable VRAM is currently capped at the .wslconfig memory setting.
bash -n clean; install test suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: hint when the AMD driver is too old for ROCm-on-WSL
Adds a detect-and-guide hook for the optional WSL-GPU path. An AMD GPU on
native Windows can also be used inside WSL2, but only with AMD Adrenalin
Edition >= 26.2.2 (the first production ROCDXG/WSL release). Native Windows
GPU works with any recent driver, so this is purely about enabling the WSL
path.
We intentionally do NOT auto-install the driver: AMD referrer-gates driver
downloads (scripted curl/Invoke-WebRequest are blocked) and does not publish
Adrenalin via winget, so no installer can reliably fetch it -- and silently
swapping a live display driver is risky. Instead we point the user at AMD's
official download page (one click), after which the existing WSL detection
lights up automatically.
- install.ps1: new Show-AmdWslDriverHint -- when an AMD GPU is present and the
installed driver predates the 26.2.2 release (DriverDate < 2026-02-01),
print a concise tip with the AMD download URL. Handles DriverDate as either
a CIM DateTime or a WMI string. Suppress with UNSLOTH_SKIP_AMD_DRIVER_HINT=1.
- install.sh (WSL hint): add the direct Adrenalin 26.2.2 download URL and note
that AMD downloads are referrer-gated (open in a browser).
Verified: hint fires on a Sept-2025 driver, auto-suppresses on >= 2026-02-01;
install.ps1 parses; install.sh bash -n clean; suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.ps1: refresh shell icon cache after creating the shortcut
After writing the Desktop / Start Menu .lnk, nudge Explorer to refresh
its icon (ie4uinit.exe -show). Without this, a stale icon cache can show
a blank shortcut icon until the next explorer restart -- most visible
when a shortcut of the same name was rewritten (e.g. a native install
followed by a WSL install, which previously shared the name; now they use
distinct names, but the cache nudge makes the icon appear immediately
regardless). Best-effort and wrapped in try/catch so it never fails the
install. The bundled unsloth.ico itself is valid (verified it renders).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* setup.ps1: don't silently CPU-build llama.cpp on an AMD GPU
For AMD, GPU acceleration comes from the lemonade ROCm prebuilt (it bundles
the ROCm runtime, no HIP SDK needed) and is the preferred/default path. The
source-build fallback is CPU-only -- a HIP/ROCm *source* build would need the
full HIP SDK + ROCm clang toolchain, which the prebuilt exists to avoid.
Previously, if an AMD-GPU host ever fell through to the source build (e.g. the
prebuilt could not be downloaded), it printed "building llama.cpp (CPU-only,
no NVIDIA GPU detected)" and quietly produced a CPU binary -- masking the lost
GPU acceleration. Now that case emits a loud [WARN] explaining the GPU prebuilt
is the AMD path and how to restore it (re-run / check network / set
UNSLOTH_LLAMA_RELEASE_TAG), so AMD never silently degrades to CPU.
No behavior change on the happy path: AMD still gets the GPU prebuilt (verified
on gfx1151: ggml-hip.dll bundled, ~80% GPU compute during inference). NVIDIA
(CUDA source build) and CPU-only hosts are unchanged.
setup.ps1 parses; install suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: remove shared llama.cpp build, kill lock-holders, match WSL shortcut
Three gaps found by running a real uninstall on a native-Windows + WSL host;
all fixes are scoped to Unsloth-owned paths and no-op on the other pathways
(env/custom-root, NVIDIA/AMD/CPU, Mac) so nothing else regresses.
uninstall.ps1:
- Remove the default-mode SHARED llama.cpp build + cache. setup.ps1 installs
them at ~/.unsloth/llama.cpp and ~/.unsloth/.cache -- SIBLINGS of studio,
not under it -- so deleting <studio> left hundreds of MB behind. Now removed
explicitly, then ~/.unsloth is dropped ONLY if empty (never nukes unrelated
content). No-op in env/custom mode (llama.cpp nests under the custom root,
removed already) and when absent. UNSLOTH_LLAMA_CPP_PATH (user-owned) is kept.
- New _StopProcessesLockingRoots: _StopStudioProcesses only matched the venv
unsloth/python/studio exe, so it missed (a) llama-server.exe under llama.cpp
and (b) an orphaned multiprocessing python fork that ran from the SYSTEM
python but loaded a venv DLL (bitsandbytes) -- on Windows an open DLL handle
blocks the directory delete, leaving a half-removed install. The new helper
kills any process whose image path OR loaded module is under a target root
(module scan scoped to python/unsloth/llama-server names; vendor-agnostic).
- _RemovePath now retries (transient post-kill handle release).
uninstall.sh:
- Remove the default-mode ~/.unsloth/llama.cpp + ~/.unsloth/.cache; rmdir
~/.unsloth only if empty.
- WSL Windows-side shortcut cleanup now matches by TARGET (any
"Unsloth Studio*.lnk" whose target launches wsl.exe), covering both the
legacy "Unsloth Studio.lnk" and the new "Unsloth Studio (WSL - <distro>).lnk"
-- and never removes a native-Windows shortcut (which launches wscript.exe).
uninstall.ps1 parses; uninstall.sh passes sh -n and bash -n.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.ps1: invalidate Win11 Start Menu tile cache after creating shortcut
The Start Menu shortcut kept showing a blank/generic icon even after the
Explorer icon-cache rebuild, because Windows 11's StartMenuExperienceHost
keeps its OWN pre-rendered tile-icon cache
(%LOCALAPPDATA%\Packages\Microsoft.Windows.StartMenuExperienceHost_cw5n1h2txyewy\
TempState\TileCache_*.bin + StartUnifiedTileModelCache.dat), separate from
Explorer's iconcache_*.db. ie4uinit and an explorer.exe restart do not touch
it, and they don't recycle the host -- so a rewritten same-name shortcut keeps
showing the first-rendered (often the generic wscript ">") tile until the host
restarts on its own.
Fix: after creating the shortcut, drop only the Start Menu RENDER caches
(TileCache_* + StartUnifiedTileModelCache.dat) and stop StartMenuExperienceHost
(Windows auto-relaunches it), so the tile re-resolves the real icon via the
shell image factory. start2.bin (the user's pinned layout) is deliberately
preserved. Guarded by Test-Path (Windows 10 has no such host -> skipped) and
wrapped in try/catch so it can never fail the install. Windows-only
(install.ps1); no effect on Linux/macOS/Studio.
Verified live: rendering the shortcut via IShellItemImageFactory::GetImage (the
API StartMenuExperienceHost uses) returns the Unsloth sloth icon, color-matched,
after this invalidation -- previously it returned the generic script tile.
install.ps1 parses; install suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ROCm-on-WSL for AMD Strix Halo (gfx1151): auto-setup + runtime enablement
Make Unsloth Studio set up ROCm-on-WSL automatically for AMD Strix Halo
(Radeon 8060S / gfx1151) and use the GPU at runtime, validated end-to-end
on a Ryzen AI Max+ PRO 395 (ROCm 7.2.1 + librocdxg + Adrenalin Apr-2026):
rocminfo enumerates gfx1151, torch.cuda True, ~85.8 GB UMA pool.
Every change is a strict no-op for all other configs (NVIDIA/CUDA,
discrete + native-Linux AMD ROCm, macOS/MLX, Windows, CPU-only, non-Strix
WSL) and can never abort the installer.
- scripts/install_rocm_wsl_strixhalo.sh: rewrite to the validated recipe.
Fixes that would have broken a working box: drop the /usr/lib/wsl/lib
preflight (a working ROCDXG host has only d3d12/dxcore there); remove the
obsolete rocr4wsl step (gone from the 7.2.1 repo; would hard-fail and also
rips out the standard hsa-rocr ROCDXG needs); dynamic librocdxg soname
(was hardcoded 1.1.0; build is 1.2.0); direct apt-repo install; Windows
SDK auto-discovery; persist env to /etc/profile.d + ~/.bashrc; idempotent.
- install.sh: _maybe_bootstrap_rocm_wsl auto-offers/runs the helper when it
detects a Strix Halo APU in WSL (/dev/dxg) with no ROCm runtime, then
loads the env so detection routes to the gfx1151 wheels. Fast-path when
already configured. Fix an inaccurate WSL hint line.
- studio/backend/main.py + worker.py: set HSA_ENABLE_DXG_DETECTION=1
in-process before torch (gated on /dev/dxg AND librocdxg.so), so the
worker uses the GPU even when launched outside a login shell. Mirrors the
existing BNB_ROCM_VERSION injection.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: clean up ROCm-on-WSL artifacts + Start Menu tile cache
- uninstall.sh: remove the ROCm-on-WSL helper artifacts -- the librocdxg
build clone (~/.unsloth/librocdxg, which otherwise blocks the empty-dir
rmdir of ~/.unsloth), the throwaway smoke-test venv, the persisted env
(/etc/profile.d/unsloth-rocm-wsl.sh) and the ~/.bashrc block. The system
ROCm userspace is a shared prereq like CUDA and is kept by default;
UNSLOTH_UNINSTALL_ROCM=1 removes it too. No-ops on macOS / non-Strix Linux.
- uninstall.ps1: invalidate the Win11 Start Menu tile cache after removing
the shortcut so its tile disappears promptly (mirrors install.ps1),
preserving start2.bin.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: accurate AMD ROCm messaging (HIP SDK optional, not required)
The Windows installer printed "HIP SDK not found - GPU-accelerated training
unavailable" / "ROCm wheels require the HIP SDK" whenever the HIP SDK was
absent. That is misleading: for a detected AMD GPU arch (gfx1151 etc.),
setup.ps1 installs AMD's bundled-runtime ROCm PyTorch wheels (repo.amd.com)
which ship their own ROCm runtime and do NOT need the HIP SDK -- verified
end-to-end (torch 2.11.0+rocm7.13.0, cuda True, QLoRA training on GPU) on a
Radeon 8060S with no HIP SDK installed.
Gate the GPU-detection + rocm-step messages on a detected gfx arch: when one
is known, state that GPU PyTorch uses bundled-runtime wheels and the HIP SDK
is optional; only when the arch is unknown fall back to the HIP-SDK hint.
Behavior (torch routing) is unchanged; this is messaging only. No-op for
NVIDIA/CUDA, HIP-SDK-present, and CPU paths (they hit earlier branches).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: fix /opt/rocm data-loss + make WSL shortcut create/remove interop-robust
Two fixes from the 3-reviewer regression audit + live testing on a
systemd-enabled WSL distro (interop disabled):
F1 (data-loss, install_rocm_wsl_strixhalo.sh): the /opt/rocm symlink-repair
could force-delete a pre-existing REAL ROCm install. The guard only checked
that /opt/rocm is a real directory, not that it is the stray librocdxg stub.
Now it only touches /opt/rocm when it is NOT a real install (no bin/rocminfo,
bin/hipcc, or .info/version present), and MOVES it aside (rocm.unsloth-stub-bak)
instead of deleting it, so a wrong guess can never lose data.
WSL interop robustness (install.sh + uninstall.sh): both relied on
`command -v powershell.exe`, which is true even when WSL interop cannot EXECUTE
it (on systemd distros powershell.exe fails with "Exec format error"). Result:
the WSL shortcut silently failed to create (install) and to remove (uninstall).
- uninstall.sh: test that powershell.exe actually runs; if not, remove the
"Unsloth Studio (WSL...).lnk" files directly via drvfs (/mnt/<drive>), which
works without interop. The name is WSL-install-specific, so a native install's
"Unsloth Studio.lnk" is never touched.
- install.sh: when the shortcut cannot be created, warn with the manual launch
command + how to re-enable interop, instead of failing silently.
No behavior change on the interop-on path. The regression audit otherwise found
no regressions on Linux/Mac/Windows/CPU/NVIDIA install paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.sh: fast-path fully restores ROCm-on-WSL env when the drop-in is gone
Reinstall regression found by uninstall->reinstall testing: after a Studio
uninstall that removed /etc/profile.d/unsloth-rocm-wsl.sh but KEPT the shared
ROCm (the default), a non-login reinstall hit the bootstrap fast-path
(librocdxg present) and its else-branch only set HSA_ENABLE_DXG_DETECTION --
NOT PATH/LD_LIBRARY_PATH. So rocminfo was not on PATH, GPU detection failed,
and the installer fell back to CPU-only PyTorch.
Fix: when librocdxg is present but the env drop-in is missing, restore the
FULL env inline (HSA + TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL + PATH +
LD_LIBRARY_PATH) so rocminfo is found and detection routes to the GPU, and
recreate /etc/profile.d/unsloth-rocm-wsl.sh so future shells and the Studio
worker get it too. No change to the env-present fast-path or any other host.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: clear Explorer icon cache so shortcut icons aren't blank
Root cause of the persistent blank Desktop + Start Menu icons: Explorer caches
each shortcut's icon in iconcache_*.db and does NOT re-read the .ico when a
same-name .lnk is recreated across reinstalls. The .ico and .lnk are correct
(the shell renders them non-blank via IShellItemImageFactory; the .ico has real
image data at 16/32/48/128 px), but the stale cache entry wins. The previous
fix only ran a weak `ie4uinit -show` + the Start Menu tile-cache clear -- it
never invalidated Explorer's icon cache, so the desktop icon stayed blank.
Fix (native install.ps1 New-StudioShortcuts AND the WSL shortcut path in
install.sh):
- ie4uinit -ClearIconCache (thorough; replaces -show as the primary refresh)
- SHChangeNotify(SHCNE_ASSOCCHANGED) to force a live desktop/taskbar refresh
WITHOUT restarting explorer
- keep the Win11 Start Menu tile-cache invalidation (and add it to the WSL
shortcut path too, preserving start2.bin)
Non-disruptive (no explorer restart). install.ps1 parses clean; install.sh
passes bash -n + dash -n; the heredoc-generated WSL PowerShell parses clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: per-item SHChangeNotify(UPDATEITEM) reliably fixes blank icons
The blank Desktop/Start Menu shortcut icons are a stale Explorer PER-ITEM icon
cache: when a same-name .lnk is recreated across reinstalls, Explorer caches the
previously-resolved (often generic "white page") icon for that item and won't
re-extract the .ico on its own. The .ico and the .lnk's IconLocation are correct
(every icon API renders the sloth) -- only Explorer's cached display is stale.
The previous refresh (ie4uinit -ClearIconCache + a GLOBAL SHCNE_ASSOCCHANGED
broadcast) does NOT recover a stale item -- confirmed by reproduction. The
reliable, NON-disruptive fix (no explorer restart) is a PER-ITEM
SHChangeNotify(SHCNE_UPDATEITEM, SHCNF_PATHW, <lnk path>) for each created
shortcut, which forces Explorer to re-read that exact item's icon.
Verified end-to-end: deliberately staled a shortcut to the generic icon, ran the
installer's exact new refresh code, and the sloth icon recovered with NO explorer
restart (confirmed by capturing the live desktop via PrintWindow).
Applied to both native install.ps1 (New-StudioShortcuts) and the WSL shortcut
path in install.sh. Still clears the on-disk icon cache (ie4uinit) and the Win11
Start Menu tile cache (preserving start2.bin).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: remove leftover llama.cpp .staging root so ~/.unsloth is cleaned
The llama.cpp atomic-install staging root (install_llama_prebuilt.py
INSTALL_STAGING_ROOT_NAME=.staging) is a sibling of the llama.cpp install
dir (~/.unsloth/.staging in default mode). It is normally pruned after a
successful activate, but an interrupted or retained build can leave a
<name>.staging-XXXX tree behind. The uninstallers removed llama.cpp and
.cache but not .staging, so the final empty-dir cleanup of ~/.unsloth failed
and the directory lingered. Reproduced on WSL (Ubuntu-24.04) where an empty
llama.cpp.staging-XXXX dir kept ~/.unsloth alive after uninstall.
Remove ~/.unsloth/.staging in both uninstall.sh and uninstall.ps1. No-op in
env/custom mode (staging nests under the custom root removed already) and
when absent. Cross-platform fix (the staging logic is platform-agnostic).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: WSL-absent hint + fix here-string lint false positive
install.ps1: in the AMD WSL-ROCm driver hint, detect when wsl.exe is absent
and add a one-line "wsl --install -d Ubuntu-24.04" pointer so a Strix Halo
user with no WSL yet gets an actionable next step (the hint previously assumed
an Ubuntu-24.04 distro already existed). Best-effort, informational only.
test_rocm_support.py: test_no_here_strings did a crude substring check that
false-positived on the conda-style block marker
printf '# <<< Unsloth ROCm-on-WSL (gfx1151) <<<' -- a string literal written
into the /etc/profile.d drop-in, also used as a sed delimiter pair by
uninstall.sh, not a here-string. Strip quoted spans before the check so the
lint still catches a real here-string operator but ignores quoted literals.
install.sh remains POSIX-clean (sh -n / dash -n / bash -n all pass).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer: address PR review comments (gfx1150 mapping, amd-smi opt-out, WSL bootstrap, SDK path, make)
Apply the valid bot review findings on #5940; reject the ones that don't hold.
Fixed:
- AMD name->gfx table (setup.ps1 + install.ps1): Radeon 890M and Ryzen AI 9 HX
370/375 are Strix POINT (gfx1150), not Strix Halo (gfx1151). Move 890M / HX 37x
/ AI 9 HX to the gfx1150 row and drop the bogus HX 38x pattern (no such Strix
Halo SKU). Matches the runtime classifier in worker.py (890M/880M -> gfx1150;
8060S/8050S -> gfx1151). Prevents Strix Point hosts from getting the wrong ROCm
prebuilt/wheels.
- amd-smi opt-out (setup.ps1 + install.ps1): an explicit UNSLOTH_ENABLE_AMD_SMI=
0/false/no/off now wins over the HIP-SDK heuristic, so a host with a HIP SDK
binary but a broken runtime no longer gets the DiskPart/UAC prompt the opt-out
exists to avoid.
- amd-smi warning probes (install_python_stack.py): _has_rocm_gpu and
_detect_amd_gfx_codes now gate amd-smi behind _amd_smi_allowed() (and pass
_amd_smi_env()), closing the last unguarded amd-smi spawn on Windows.
- WSL ROCm bootstrap (install.sh): the "already-usable ROCm?" early return now
requires rocminfo to enumerate the real gfx1151 agent instead of the generic
_has_amd_rocm_gpu (whose broad gfx[1-9][0-9] match accepts a fallback
"gfx11-generic" ISA), so a Strix Halo box missing the ROCDXG bridge is no longer
skipped. The shared helper is untouched (no gfx90a regression).
- install_rocm_wsl_strixhalo.sh:
* Quote-safe Windows SDK discovery: the old for-in-$(ls -d "...Program Files
(x86)/...") word-split on the space and never matched; use find + read loop.
* Add `make` to apt prereqs (cmake only recommends it; minimal images lacked it
and the librocdxg `make -j` build failed).
* Verification requires gfx1151 exactly (not gfx1[0-9]) so a generic ISA or an
unrelated RDNA GPU can't pass while the real GPU is absent.
Reviewed but NOT changed:
- "Forward inferred ROCm arch without HasROCm" (setup.ps1): already correct --
--rocm-gfx is forwarded under `if ($script:ROCmGfxArch)`, not `if ($HasROCm)`.
- "Route inferred arch into install.ps1 torch path": not a bug -- install.ps1
installs CPU torch as a base by design and setup.ps1 swaps in the ROCm wheel for
the inferred arch (gate `($HasROCm -or $ROCmGfxArch) -and cpu`); verified live
the native install ends on torch 2.11.0+rocm7.13.0.
- "$p null guard after Start-Process" (install.ps1/setup.ps1): redundant -- the
amd-smi runner uses [Process]::Start wrapped in try/catch, so a null process
already returns "" with LASTEXITCODE=1 (no uncaught exception).
- "ls -> find for /usr/lib/wsl/lib" (gemini): stale -- that heuristic was removed;
only a comment about it remains.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer(rocm-wsl): auto-install the Windows 11 SDK via winget (fewer manual steps)
librocdxg's build needs the Windows SDK 'shared' headers on the Windows host.
Previously the helper just die()d with "install the Windows 11 SDK and re-run" if
they were missing -- a manual prerequisite that broke the otherwise-seamless
`curl ... install.sh | sh` one-liner on Strix Halo.
Now, when the headers aren't found, the helper installs the Windows 11 SDK on the
Windows host from inside WSL via winget (powershell.exe interop), then
re-discovers them. The SDK installer elevates -> ONE UAC prompt on the Windows
desktop; the headers appear under /mnt/c immediately (drvfs is live, no reboot).
The user already consented to the ROCm-on-WSL setup, so no extra prompt is added
beyond the OS UAC gate.
- New _find_win_sdk (space-safe find of the newest installed SDK 'shared' dir)
and _install_windows_sdk_via_winget helpers.
- winget IDs tried newest-stable first: Microsoft.WindowsSDK.10.0.26100, then
.22621. The presence of the headers (re-check) is the source of truth, not
winget's exit code. </dev/null so winget never consumes a piped `curl|sh` stdin.
- Best-effort + non-fatal: interop-off / no-winget / declined-UAC all fall
through to the existing clear manual-install die(). Opt out with
UNSLOTH_SKIP_WIN_SDK_INSTALL=1.
Removes the last avoidable manual step from the WSL Strix Halo path; only the AMD
Adrenalin driver (AMD referrer-gates the download) remains manual. Verified
_find_win_sdk resolves the spaced "Program Files (x86)" path; bash -n clean; all
winget flags validated against `winget install --help`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer(amd): gate install-time amd-smi probe to fix DiskPart UAC prompt
install_python_stack.py's Windows "AMD GPU detected but ROCm torch missing"
warning probe ran `amd-smi list` whenever amd-smi was on PATH -- and amd-smi
ships in C:\Windows\System32 with the AMD Adrenalin driver -- without the
_amd_smi_allowed() gate that every other amd-smi call site in the file uses.
On Adrenalin-only hosts (no HIP SDK) amd-smi elevates a child at runtime and
pops a UAC/DiskPart prompt that __COMPAT_LAYER=RunAsInvoker cannot suppress
(amd-smi's manifest is asInvoker). The probe also ran before the
ROCm-torch-installed check, so it fired on every Windows AMD install.
Gate it behind _amd_smi_allowed() and pass _amd_smi_env(), matching
_has_rocm_gpu()/_detect_amd_gfx_codes(). When skipped, the only loss is the
best-effort "AMD GPU detected" note on HIP-SDK-less hosts.
Adds a per-function AST regression test asserting every function in
install_python_stack.py that names the amd-smi command and spawns a subprocess
also references _amd_smi_allowed() (flags the pre-fix code; passes after).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* studio(cli): fix `unsloth studio stop` crashing on Windows
`stop` used the POSIX `os.kill(pid, 0)` liveness probe, but on Windows
CPython raises OSError (WinError 87, "The parameter is incorrect") for
*every* pid -- alive or dead. `stop` only catches ProcessLookupError /
PermissionError, so the OSError propagated and the command crashed with
a traceback before ever reaching its (correct) `taskkill /F` path.
Add a cross-platform `_pid_alive(pid)` helper (tasklist on Windows,
signal-0 elsewhere) and use it for both the pre-check and the post-kill
wait loop. The actual kill path is unchanged.
Verified on Windows (Python 3.13): os.kill(pid,0) raises WinError 87 for
both a live and a dead pid; `_pid_alive` returns True/False correctly and
the full stop() flow (alive -> taskkill -> dead -> "stopped") passes
end-to-end against a throwaway process.
Adds tests/studio/test_cli_studio_stop_windows.py (AST guard against a
bare os.kill(pid,0) liveness probe + mock-only _pid_alive behaviour for
the win32 tasklist branch and the POSIX signal-0 branch).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer(amd): fix install.sh name->arch table misrouting Strix Point to gfx1151
The bash name->arch inference table in install.sh placed Strix Point
identifiers (Radeon 890M, "Ryzen AI 9 HX 370/375", "AI 9 HX") in the
gfx1151 (Strix Halo) row, diverging from the install.ps1 / setup.ps1
PowerShell tables which correctly map them to gfx1150. It also carried a
stray "HX 38" token absent from the PowerShell source-of-truth.
Align install.sh with the PowerShell tables:
gfx1151 row: 8060S|8050S|8040S|Strix Halo|Ryzen AI Max|AI Max
gfx1150 row: 890M|880M|860M|840M|Strix Point|Krackan|HX 37|AI 9 HX|...
Impact is low (the bash table only feeds the display label _gpu_disp_gfx
and the "set UNSLOTH_ROCM_GFX_ARCH=..." hint; wheel selection is driven
by the detected ROCm version, not this name string) but a Strix Point
user would otherwise see/copy the wrong gfx arch.
Add a parity test (test_install_sh_name_arch_agrees_with_ps_for_strix_and_non_amd)
that parses install.sh's case table and asserts Strix Halo->gfx1151,
Strix Point->gfx1150, RX 7700S->gfx1102, and NVIDIA/Intel->no match,
cross-checking against install.ps1 (the previous parity test only
compared install.ps1 <-> setup.ps1, missing install.sh).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.ps1: keep prebuilt-llama ownership guard within the test's block window
The AMD additions to the prebuilt-llama.cpp block (the windows-hip vs
windows-cpu existing-install kind validation) pushed the
install_llama_prebuilt.py invocation to ~1999 chars after the
"installing prebuilt llama.cpp bundle (preferred path)" anchor, right at
the edge of the 2000-char window that
test_setup_ps1_prebuilt_llama_cpp_has_ownership_guard slices -- so the
helper string was truncated and the test failed with "substring not
found" (CI: Repo tests (CPU)).
The ownership-guard invariant (Assert-StudioOwnedOrAbsent precedes the
install_llama_prebuilt.py call) was already satisfied; only the proximity
to the anchor regressed. Move the "installing prebuilt..." substep to
immediately before the install (after the existing-install pre-cleanup),
which also reads better (validate/clean existing -> then "installing"),
shrinking anchor->helper from 1999 to 413 chars. Behaviour is unchanged
(console message ordering only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.sh: auto-run Strix Halo ROCm-on-WSL setup by default
`curl -fsSL https://unsloth.ai/install.sh | sh` should make a Strix Halo
(gfx1151) GPU usable inside WSL with no extra commands. Previously the
ROCm-on-WSL bootstrap was opt-in: it required UNSLOTH_ROCM_WSL_AUTO=1 or an
interactive [Y/n] at a TTY, and silently skipped under a pipe (no /dev/tty),
so the piped one-liner never set the GPU up automatically.
Flip it to auto-by-default for the single narrow case the existing guards
allow (WSL + Strix Halo + /dev/dxg + no usable ROCm yet) -- exactly the GPU
setup the user ran the installer for. Opt out with
UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The Tauri desktop app keeps its own consent UI
(only auto-runs when it passes UNSLOTH_ROCM_WSL_AUTO=1). All hardware/OS
guards are unchanged, so non-Strix / non-WSL / NVIDIA / native-Linux / macOS /
CPU paths are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* PR comments: condense to be succinct (comments/docstrings only)
Shorten the verbose explanatory comments and docstrings this PR added across
the installer, scripts, backend shims, CLI, and tests -- tighter, fewer lines,
while preserving every non-obvious "why" (os.kill WinError 87, amd-smi
RunAsInvoker/UAC, /dev/dxg + librocdxg gating, the ROCm-on-WSL bootstrap guard
chain, ownership guards, etc.). No executable code, string literals, messages,
or behavior changed.
Verified comments-only: docstring-normalized AST equality (Python, 9 files),
non-comment token equality (PowerShell, 3 files), comment-stripped diff +
sh -n / bash -n (shell, 3 files). Behavior re-confirmed: get_torch_index_url +
gfx name->arch table 44/44 under dash & bash; rocm_support / pr5940_followups /
cli_studio_stop tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Installer: address PR review (amd-smi opt-out, pipefail, multi-distro, non-root)
Fixes valid findings from the Codex/Gemini PR review:
- install.ps1 / setup.ps1: gate the `amd-smi version` ROCm-version fallback with
$amdSmiAllowed so UNSLOTH_ENABLE_AMD_SMI=0 opt-out is honored (the device
probe was gated but this fallback wasn't), avoiding the DiskPart/UAC prompt.
- install_rocm_wsl_strixhalo.sh: make the post-verification rocminfo summary
best-effort (|| true) so head's early pipe-close under `set -o pipefail` can't
fail the bootstrap after gfx1151 was already enumerated; pin the Windows SDK
`winget install` to --source winget (matches the msstore-cert fix rationale).
- install.ps1: python.org fallback installs the py launcher per-user
(InstallLauncherAllUsers=0, avoids admin), and derives the fallback full
version from the requested minor so a non-default UNSLOTH_PYTHON (e.g. 3.12)
isn't silently replaced with 3.13 when the listing is unreachable.
- install.sh: recreate /etc/profile.d/unsloth-rocm-wsl.sh via `sudo tee` for a
non-root reinstall (a plain redirect failed silently, dropping the ROCm env).
- uninstall.sh: scope WSL Windows-side shortcut removal to the current
WSL_DISTRO_NAME (per-distro name or -d "<distro>" arg) so uninstalling one
distro no longer deletes other distros' launchers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Studio ROCm Windows: fix field-reported issues from Strix Halo testers
Four fixes from PR #5940 field reports (Win11 native, gfx1151):
1. bitsandbytes arch-probe spam: bnb's get_rocm_gpu_arch() runs
hipinfo.exe via subprocess PATH at import; the AMD torch wheel ships
hipInfo.exe in the venv Scripts dir, which is only on PATH for
activated venvs. Every bnb import logged "Could not detect ROCm GPU
architecture: [WinError 2]" ERROR + WARNING (even with the HIP SDK
installed, whose bin dir is not on PATH either). Prepend the Scripts
dir to PATH before bnb imports in main.py, worker.py, and
install_python_stack.py, gated on the file existing (only AMD wheels
ship it). Verified on gfx1151: ROCM_GPU_ARCH now resolves to gfx1151
with zero errors.
2. OOM-guard double-tax on native Windows unified APUs: mem_get_info's
total is the WDDM budget the driver grants HIP (BIOS carve + ~half
of remaining RAM) -- the OS share is already outside it. The 0.80
unified cap on top denied loads that fit (field report: 48.49 GiB
budget -> "38.79 GiB allowed" OOM for a 47.29 GiB load with 48.08
free). Use 1.0 on win32 unified; Linux keeps 0.80, discrete 0.90.
3. "Missing VRAM" confusion: log the WDDM budget vs physical RAM with
the fix (BIOS UMA frame buffer / AMD Software Variable Graphics
Memory) when the grant is under 75% of RAM, so a 48 GiB cap on a
96 GiB box reads as policy, not a Studio bug.
4. llama-server fit-step crash (Qwen3.6-27B-MTP + mmproj, lemonade
gfx1151): --fit defaults to 'on' upstream, so the fit step runs even
when Studio already placed the model via -ngl -1, and aborts in
ggml-cuda.cu on some ROCm hosts. Retry the spawn once with --fit off
when the server crashes during startup and Studio's own VRAM math
had placed the model (never when use_fit or an explicit fit flag was
passed). Also keep the TAIL of crash output in the error log (the
diagnostic line prints last; head-truncation cut exactly that) and
reference the full on-disk log.
Verified live on Radeon 8060S: bnb import clean, Qwen3.5-4B-MTP loads
and generates through the new spawn loop, stub-crash retry appends
--fit off and recovers, fraction probes confirm WDDM overcommit and
sub-1.0-only enforcement on current AMD wheels.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio ROCm Windows: GPU-name fallbacks so nothing depends on amd-smi
amd-smi does not reliably exist on Windows: the HIP SDK never ships a
CLI, inbox Windows Update drivers do not, and only some full Adrenalin
packages drop amd-smi.exe into System32 (field report: fresh Win11 +
Adrenalin + HIP SDK, still no amd-smi anywhere). Make every consumer
work without it:
- install_python_stack._detect_windows_gfx_arch: two new probes after
hipinfo/amd-smi -- (2b) the venv Scripts hipInfo.exe shipped by AMD
torch wheels (drives `studio update` on driver-only hosts), and (4) a
last-resort GPU marketing-name -> gfx table via WMI
(Win32_VideoController), mirroring setup.ps1's $nameArchTable so a
standalone repair resolves the arch with zero AMD tooling installed.
- install_llama_prebuilt._resolve_exe: also probe the venv Scripts dir
so a standalone rerun finds hipInfo.exe without HIP_PATH.
- hardware/amd.py _run_amd_smi: which() guard before spawning --
absence now disables the poller in one step instead of burning the
3-strike circuit breaker on FileNotFoundError; corrected the stale
comment claiming Adrenalin ships amd-smi.
Simulated against the real detection functions on gfx1151: amd-smi
absent, present-but-crashing (exit 1), present-but-hanging (60s sleep
vs 5-10s probe timeouts), and hard opt-out -- all resolve gfx1151, no
exceptions, bounded time. Full adversarial install (broken amd-smi
stub first on PATH + UNSLOTH_ENABLE_AMD_SMI=1, fresh uninstall first):
exit 0, name-table arch inference, lemonade gfx1151 b1292 prebuilt,
torch 2.11.0+rocm7.13.0 cuda_avail=True on the 8060S, Studio boots
healthy and stops cleanly.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: per-attempt llama-server log names + amd-smi test portability
Found by cross-platform simulation of the --fit off retry (Windows +
Linux sandboxes, real load_model with stub servers):
- llama-server log filename now carries the spawn-attempt index. The
retry can respawn within the same epoch second; reusing the name
opened the same file with "w" and truncated the crash log the retry
warning had just pointed the user at (proven with a frozen
time.time: one file, crash evidence gone; with the suffix both
attempts keep their logs). Regression-pinned in
test_llama_cpp_wait_for_health.py.
- test_amd_primary_gpu_with_mock now mocks shutil.which alongside
subprocess.run: the amd-smi absence guard which()-checks before
spawning, so on hosts without a real amd-smi (Linux CI, driver-only
Windows) the subprocess mock was never reached and the test failed.
Surfaced by running the suite in a clean Linux sandbox.
Simulation coverage on both OSes: 67-case platform/edge matrix
(real shipped code blocks under win32/linux/darwin spoofs: OOM-guard
fractions + VGM-hint boundary, bnb PATH-prepend gates, retry
eligibility incl. equals-forms and decoy tokens, GPU-name table
adversarial set, WMI fallback without powershell, monitor absence
semantics), 6-scenario live retry matrix (crash-once/crash-always/
exit-zero/explicit-fit/hang/log-collision) against real llama-server
spawns on Windows and WSL (GPU success legs on the 8060S), and a
3-engine browser matrix (chromium/firefox/webkit) driving the live
backend's health + authed /v1 chat completion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: classify unified-memory via props.is_integrated first
Align the ROCm OOM-guard classifier with PR #5988's UMA gate: consult
hipDeviceProp_t.integrated (props.is_integrated) before the hardcoded
arch set. Strictly additive -- truthy upgrades to unified; 0/absent
falls through to the existing gfx1150/gfx1151 + device-name logic, so
wheels that omit or zero the field cannot downgrade the known APU set.
Extends correct unified-cap treatment to APUs outside that set (e.g.
gfx1103 Phoenix iGPUs) and keeps Studio's two unified-memory consumers
on one driver signal. Verified live on gfx1151 (is_integrated == 1 on
the AMD Windows wheel -> ('gfx1151', True) via the new path).
* AMD detection: probe rocminfo with HSA_ENABLE_DXG_DETECTION and sync setup.sh gfx table
Fleet validation on a Strix Halo WSL2 box showed the system rocminfo
(HSA 1.18, ROCm 7.2.1) only enumerates the GPU over /dev/dxg when
HSA_ENABLE_DXG_DETECTION=1, and that rocminfo can sit at /opt/rocm/bin
off PATH outside login shells. Detection probes that miss either of
these report no GPU on a working ROCDXG host and select the CPU build
even though the lemonade bundle offloads fine (95.7 tok/s measured vs
64.5 CPU on the same laptop). Seed the env (a no-op on bare metal) and
the PATH fallback in install.sh, studio/setup.sh, and the installer's
Linux rocm probe, mirroring what main.py/worker.py already do for the
runtime.
Also sync studio/setup.sh's name->gfx table with install.sh: 890M and
the HX 37/AI 9 HX SKUs are Strix Point (gfx1150, not gfx1151), RX 7700S
must match gfx1102 before the gfx1100 row, and the RDNA2/workstation
rows were missing. New parity test pins the two bash tables together so
they cannot drift again.
* Studio: persist server session logs + native-crash stacks to disk
Field report (Strix Halo, 96 GB UMA carve, WSL and native Windows):
"the studio just terminates without a warning". A native crash in the
GPU runtime kills the process with no Python traceback, and a desktop-
shortcut console closes before anything can be read. The server only
ever logged to the console, so there was nothing to send back.
run_server now tees stdout/stderr to
~/.unsloth/studio/logs/server/server-<ts>-pid<n>.log (console behavior
unchanged; file copy is best-effort), arms faulthandler at the same
file so access violations / SIGSEGV leave a stack trace on disk, and
exports PYTHONFAULTHANDLER=1 so training workers inherit crash dumps
on their captured stderr. Armed before `from main import app` so even
import-time failures leave evidence. Keeps the newest 20 session logs;
opt out with UNSLOTH_STUDIO_NO_FILE_LOG=1. Prints "Session log: <path>"
at startup so users know what to attach.
Verified on this box: a forced real segfault (faulthandler._sigsegv)
leaves the full session output plus "Fatal Python error: Segmentation
fault" and the thread stack in the file while the console shows
nothing; a normal server boot captures the startup banner and serves
health as before.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* AMD probe: honor a pre-set HSA_ENABLE_DXG_DETECTION value
Match the shell helpers, which use the parameter-default form: a user
who exports HSA_ENABLE_DXG_DETECTION=0 to deliberately hide the GPU
from DXG detection should not have the probe override it.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison.
alkinun