Comment-only follow-up to #7454. That change carried 523 comment lines, many of
them three and four line preambles where one line says the same thing. This
collapses them and drops the ones restating what the code already says, for a
net 77 lines.
Scope is limited to comments #7454 itself introduced. The files it touched hold
about 3,761 comments in total; the rest predate it and are untouched, verified
by checking that every removed line is one that commit added.
Nothing that records why a non-obvious decision was made was dropped, only
compressed. Still stated: the normcase-before-versus-after Windows separator
trap, the innermost-indexed-model rule for nested directories, an HTTPException
being a decision rather than a failure to decide, that only an explicit False is
anonymous to huggingface_hub while None borrows the server owner's login, the
fail-closed tri-state custom-code gate, and the regressions each test was
written for.
Code is provably unchanged: comment_tools.py check reports 17/17 files
comments-only. Backend CI command 10337 passed, 0 failed. tsc -b clean.
* Studio: say which model is missing instead of "No model loaded"
A /v1 request naming a model that is not downloaded returned the generic
"No model loaded. Call POST /inference/load first.", which cannot fix it.
Return 404 model_not_found naming the model and listing what can serve,
and make the API usage examples name a model the server actually has.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: page the API monitor, show model load/unload, pin the example quant
The monitor rendered all 50 retained entries in one scroller: page it 5 at a
time, freezing history while paged back so live traffic cannot reorder it.
Add model load/unload rows so the feed shows what the server is doing, and
stop the header rendering the loaded model as a raw host path. Advertise each
model's GGUF quant on /v1/models so the example pins repo:QUANT, and move the
auto-switch section above the monitor with shorter copy.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: optionally download a model named in an OpenAI API request
Auto-switch only ever loaded models already on disk, so naming one this
server does not have either 404s or, when something else is loaded, gets
quietly answered by the resident model.
Add openai_api_auto_download_model (off by default, gated on auto-switch).
When on, a /v1 request naming a GGUF repo that is not downloaded starts a
background fetch and returns 503 with Retry-After and a typed
model_downloading code. The resident model keeps serving in the meantime,
and the retry after the download completes is served by the new model
through the existing auto-switch path.
The download reuses the Hub manager's service layer, which already does
repo-id validation, casing, claim bookkeeping, disk preflight, resume and
cancel. The in-loader download is deliberately not used: it silently falls
back to a smaller quant under low disk, which is wrong when the caller
named an exact one.
Admission is narrow, since a request only needs an API key:
- namespace/name only, so gpt-4 and other foreign ids fall through to the
resident model exactly as before
- GGUF only, decided from the remote file list rather than the repo name
- anything declaring auto_map is refused, so trust_remote_code stays a
deliberate opt-in in the UI and can never be granted over the API
- a single download at a time, plus a free-disk reserve
- one model_info call answers existence, gating and the quant list, so a
missing repo, a gated repo and a wrong quant each get their own error
With the setting off every one of these paths is byte-identical to before.
Also:
- monitor rows for downloads, with a live percentage
- public_model_id resolves an HF cache snapshot to its repo id, so a
cache-loaded model is no longer labelled with a commit sha; this drops
the duplicate helper added for the monitor and fixes the same leak in
the inference status response
- the unedited sk-unsloth-YOUR_KEY from the copyable examples now says so
instead of "Invalid or expired API key"; every other bad key keeps the
generic message
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add an Unload button to the API monitor
The monitor names the loaded model but offered no way to free it. Idle
auto-unload is the only existing release path, and it needs a TTL and a
wait.
The button sits next to Refresh, appears only while a model is loaded and
is disabled mid-unload. /unload matches on the internal identifier, which
this response deliberately omits because it would be a host path, so the
click reads it from /api/inference/status the same way the chat runtime
does rather than widening the monitor payload.
Also stamp the manual unload row with the quant, read before the teardown
clears it, so it reads repo:QUANT like the load row it pairs with.
* Studio: keep the API monitor Unload button visible when idle
It only rendered while a model was loaded, which hid the one manual
release path at exactly the moment someone goes looking for it. Render it
always, disabled with a "No model is loaded" tooltip when there is nothing
to free.
* Studio: never answer a named model with a different one
Asking for a model this server is not serving returned 200 from whatever
was resident. Requesting gemma-4-E2B-it-GGUF:UD-Q6_K_XL while UD-Q4_K_XL
was loaded got a confident answer from the wrong quant, with nothing in
the response saying so.
A name carrying a namespace (org/model, optionally :QUANT) is a concrete
reference, so 404 instead, with the reason:
- wrong quant -> names the quants that are actually downloaded
- not on disk -> lists what is available
- on disk but auto-switch off -> says to turn it on
Ids without a namespace (gpt-4, claude-3, default) are foreign labels
rather than references, so they still fall through to the resident model
and drop-in clients are unaffected. A bare org/model is still satisfied by
any loaded quant of that repo; only an explicit :QUANT must match.
The check runs whatever the auto-switch and auto-download toggles are,
since serving the wrong weights is wrong in every configuration. It is
skipped when nothing is loaded, where the existing no-model-loaded error
already says the right thing, and when the model is on disk with
auto-switch on, where a failed swap should still fall back.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: use a simpler prompt in the API usage examples
"What is Unsloth Studio?" rather than "Can Unsloth Studio do API calling?".
One constant feeds all nine snippet tabs.
* Studio: only refuse a model reference meant for this server
A namespace alone was treated as a concrete model reference, so a /v1
request naming anthropic/claude-3.5-sonnet, openai/gpt-4o or any other
LiteLLM or OpenRouter style vendor/model id started returning 404 instead
of being answered by the resident model. Refuse only on evidence the
caller meant this server: an explicit GGUF quant label, or a repo that is
actually on disk here. gpt-4 and vendor/model alike fall through again,
while the wrong-quant and wrong-repo cases this PR exists for still
refuse.
Also from review:
- Release the single download slot by object identity, not repo id. A
stale watcher could clear a newer download of the same repo and let a
second multi-GB fetch start alongside it.
- Catch BaseException around admission: CancelledError is not an
Exception, so a cancelled request stranded the slot for the process
lifetime.
- Honour the download service's accepted=False, which it returns without
raising for a cross-variant conflict, instead of promising a download
that was never dispatched.
- Treat a failed status probe as unknown rather than idle, so a transient
read cannot fail the monitor row and free the slot under a live worker.
- Check gated repos with auth_check. The Hub serves metadata for a gated
repo without granting its files, so the licence gate was being reported
as the unrelated custom-code refusal.
- Size the disk reserve from the download plan, which includes the mmproj
and MTP companions the worker fetches with every quant.
- Never fetch under the server's own HF token. The repo is named by
whoever holds an API key, so the ambient token let that key pull the
owner's private repos.
- Refuse an explicit quant on a backend with no quant identity, gated on
the suffix really being a quant so Ollama style :latest tags still match.
- Raise instead of falling through when the diagnosis fails: the mismatch
is already established by then, only the wording is uncertain.
- Report a failed switch as 503 model_switch_failed rather than answering
as the resident model.
- Fail an open monitor row under the same lock as the check, so a finish
landing in between cannot stamp an error onto a completed row.
- Usage examples never emit a hardcoded model id: the catalog is tri-state
and the panel asks for a model to be loaded instead of printing one the
server cannot serve. It also refreshes when the loaded model changes.
- Keep the monitor pager reachable while frozen entries expire.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: scope the auto-download 404 cache to the caller's credentials
The Hub answers 404 for a private repo the caller cannot see, so caching
that verdict per repo alone let one anonymous request mark a private repo
unservable for everyone for the whole TTL. A later caller sending a valid
X-Unsloth-HF-Token skipped the probe and fell through to the resident
model instead of downloading what it asked for. Keyed on the repo id plus
a digest of the token now, so the token itself is never held.
Two more from the same review:
- Clear the chat runtime checkpoint after unloading from the API monitor,
as the chat eject flow already does. The store went on treating the
freed checkpoint as loaded and the usage examples kept naming it.
- Point gated and not-found callers at the X-Unsloth-HF-Token header.
Automatic download deliberately ignores the server's own Hugging Face
identity, so telling the user to add a token in Studio sent them round
the same 403 forever.
* Studio: tighten the comments added by this branch
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep API auto-download off the server's Hugging Face identity
Passing None for the caller's token was not anonymous. spawn_worker
substitutes the backend's HF_TOKEN for a falsy one, and HfApi(token=None)
falls back to a cached login, so a repo named by an API-key holder could
still be fetched under the owner's Hub identity and land in the shared
catalog. The metadata probe and auth_check now pass an explicit False,
and dispatch threads allow_ambient_token=False so the worker stays
anonymous too. The flag defaults to True, so the UI download path keeps
the ambient fallback that private repos rely on.
Three more from the same review:
- Require an exact hf_variant match only when the suffix is really a
quant. The llama.cpp branch still compared Ollama style :latest and :8b
against the loaded quant and refused the resident model, which is the
opposite of what looks_like_quant classifies them as.
- Decode an HF cache repo id only when the models-- component is followed
by snapshots. An ordinary directory whose name merely starts with
models-- was being read as an encoded repo id.
- Return the probing response before consulting the job registry when an
adopted claim has no variant yet. A stale error on the whole-repo key
could otherwise release the slot the first request's probe still holds,
letting a second large download start beside it.
* Studio: stop treating a namespace as what decides model intent
The rule refused a reference only when it carried a namespace, which was
wrong in both directions. vendor/model is how LiteLLM and OpenRouter name
every provider, and a standalone or custom-folder GGUF is advertised
without one, so asking for a path-free local id such as model-Q4_K_M was
answered by whatever else happened to be resident. The slashless early
return is gone and the same evidence test now applies to every id: an
explicit quant, or a model that actually resolves here. gpt-4 and default
still fall through because they are not local, not because of their shape.
Also:
- Recognise bits-per-weight quant labels. _extract_quant_label emits
IQ4_XS-3.53bpw and the resolver and downloader both accept it, but
_GGUF_KNOWN_QUANT_RE has no bpw group, so looks_like_quant rejected a
reference the rest of the machinery understands.
- Upper-case the synthetic names handed to _pick_best_gguf. Its preference
tokens are upper case and matched case-sensitively, so a repo with
lower-case filenames skipped the preference and took the first entry,
which can be F16.
- Only offer a downloaded but unloaded model as a runnable example when
auto-switch is on. It is off by default, so the copied snippet hit the
no-model-loaded error, which is the failure this branch exists to fix.
The tool-passthrough cancel test stubbed asyncio.to_thread module-wide, so
it cancelled at the first thread hop rather than the generation hop it
means to test. Model resolution runs off the loop before the monitor row
opens, so that stub now passes the resolver through.
* Studio: tighten the comments added since the last pass
* Studio: match a resident model through its resolver alias
A manual load stores the model by its on-disk path while the resolver and
/v1/models advertise it as publisher/model, so _loaded_satisfies could not
recognise the alias. Reducing the resolution to a boolean then threw away
the load path that would have proved the match, and the request was
refused with 404 for a model the server was serving at that moment. Common
for LM Studio models and custom-folder aliases. The resolved path is
compared against the resident backend before anything is refused.
Also:
- Size disk admission on what is left to fetch. expected_bytes is the whole
plan, so a resumed quant or a companion already pulled in by another
quant was charged for twice and could 507 a download that fits. Cached
blobs are subtracted through existing_blob_bytes, the same accounting the
worker's own preflight does, and it falls open to the full size when no
blob hashes are available.
- Report a cancelled download as cancelled. The catch-all sent every state
other than complete or idle through fail_open, so a deliberate cancel
rendered as a download failure rather than the monitor's cancelled state.
- Keep polling the servable ids while nothing is loaded. The poll settled
as soon as auto-switch was on, so turning it back off left the examples
naming an unloaded model until something else remounted the panel.
* Studio: shorten the comments added in the last pass
* Studio: keep the FLA fast-path tests hermetic across transformers versions
_discover_fla_model_types scans the *installed* transformers for modeling
files importing `from fla.`, so `models/qwen3_5/` only exists from
transformers 5.x. The backend supports transformers>=4.51, and on a 4.x
install the Qwen3.5 gate returns False, so 14 tests in
test_training_worker_flash_attn.py silently exercised a no-op instead of the
install path and failed their call-count assertions.
Pin the discovered model_type set in those 14 tests, the same way
test_hook_does_not_install_tilelang_for_model_outside_allowlist already pins
it against newly added FLA model_types. Test-only change: the production
gate and the _discover_fla_model_types unit tests are untouched.
* Studio: keep the /v1 admission check off the model-scanning path
The admission check added here runs on every /v1 request, including with
auto-switch off, where the route used to return straight away. It called
resolve_local_gguf, whose index is cached for 5s and otherwise rebuilt by
walking ./models and every HF cache root, under a lock the next caller waits
on. On an install with a large cache that scan measured 6.1s, longer than the
TTL that is meant to amortise it, so steady traffic would keep rebuilding it.
Answer from the last built index instead and never rebuild from the request
path: a stale answer is fine here, since what is on disk barely moves and a
finished download already invalidates the index. The first request, before any
scan has completed, warms the index on a background thread and skips the check
rather than blocking on it. That also makes the lookup a dict read, so it no
longer needs handing to a thread.
Cold resolve on this box goes from 6152495us to 0.4us, and the whole hook now
costs the same for a foreign label as for the resident model.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix the admission hook's cold, stale and contended index paths
Five review items, four of them on the admission hook added here.
Skipping the check until the first scan lands also skipped explicit quant
mismatches, so the first request after startup could ask for :Q8_0 while
Q4_K_M was resident and be answered by it. The early return was redundant as
well: with an empty index resolved is None and here is False, so the gate below
already lets a bare name through and refuses an explicit quant, which is what
the except branch has always concluded. Dropped it and index_is_built with it.
index_is_built took _lock, which _index holds for the whole scan, so once a
warm was running every later request blocked on the event loop for exactly as
long as the scan it was there to avoid. The warm now has its own lock and reads
the timestamp unlocked, which is safe because _scan is only ever rebound.
Warming only when the index had never been built left a model fetched in the
Hub UI, or dropped into a scan folder, invisible for the life of the process,
since only the auto-download watcher calls invalidate_index. Warm on staleness
too, and unconditionally, so it refreshes within a TTL without a scan on the
request path. Rescanning is capped at a tenth of the scan's own duration: a big
install takes longer to scan than the TTL, and warming on the TTL alone would
keep a thread scanning continuously.
An Ollama-style tag names no quant, so the resolver misses it and auto-download
saw a model the resident one already answers to, then 404'd it for having no
such quant. Return early when the loaded model satisfies the reference.
Frontend: a cancelled download said "Model download failed", because the label
collapsed everything non-completed into failure.
The backend tests get an autouse fixture that stops the warm from walking the
developer's real HF caches; that scan starved the loop under the timing
sensitive streaming tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make /v1/models and the admission hook agree on what is local
Three review items, all on the seam between the catalog scan and the resolver
index, which run on separate schedules.
/v1/models can advertise a local GGUF the resolver has not indexed yet. A bare
id carries no quant to refuse on, so a client asking for one it had just been
handed was answered by the resident model instead. The hook now reads the
catalog cache as evidence too, never scanning it. It takes the path rather than
a yes/no because the converse also happens: the catalog can list the resident
weights under an alias the loaded entry does not answer to, and those must stay
served.
That alias was also emitted twice by /v1/models, once as the loaded basename a
manual load records and once as publisher/model marked unloaded, because the
dedup only compared ids. Compare the path as well.
A directly loaded standalone .gguf takes its quant from the filename, but the
resolver stores such files with no quants, so the advertised <stem>:<quant>
stopped resolving as soon as anything else loaded. Advertise a quant only when
that reference resolves, and downgrade only on a definite answer so a cold
index leaves the metadata alone.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten the comments this branch adds
Collapse the multi-line notes in the auto-download path, the /v1 admission
hook and their tests to one line each, keeping the reason and dropping the
restatement. No behaviour change.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: four admission and catalog fixes from review
Lowercasing paths in _resolves_to_resident made /srv/models/Foo and
/srv/models/foo the same weights on any case-sensitive filesystem, so a request
for one could be answered by the other and /v1/models could mark the wrong
entry loaded. That helper now backs residency as well as admission, so use
os.path.normcase, which folds case only where the filesystem does.
Advertising a quant whenever the resolver could not disprove it kept the bug it
was meant to fix: a standalone .gguf loaded before the first scan still got
<stem>:<quant> published, and the usage examples persist that. No proof is not
proof, so omit it and warm the index instead.
A 401 from an expired or invalid X-Unsloth-HF-Token skipped the 403 and 404
branches and surfaced as "could not reach Hugging Face, retry shortly". It now
says to replace the token, kept apart from the gated refusal since a rejected
credential is not an unaccepted licence.
An image request naming an undownloaded text-only GGUF started the whole
download and only then hit the capability guard, which never sees a remote
target, so every retry 400d and the bytes were wasted. Thread require_vision
into admission and check it against the mmproj companions the disk preflight
already asks build_gguf_variant_plans for.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make the Hub error fixture carry a status on both hub majors
The 401 test built HfHubHTTPError directly, which works on 0.x and fails on 1.x
where response is required and keyword-only, so all four Python jobs failed
while the same test passed locally.
_hub_error already handled both constructors, but the 0.x branch left the
exception with no response at all, and hf_error_status reads the status off it
for the types that do not encode it in their name. So it could only produce a
usable error on 1.x, which is why the test bypassed it. Attach the status when
the constructed exception lacks it, and use the helper.
Cover the helper itself against stand-ins for both constructor shapes, since
whichever hub is installed only ever exercises one of them.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: invalidate on every download, resolve bare tags, keep polling
Three review items.
Only the API auto-download watcher dropped the resolver cache, so a GGUF
fetched in the Hub UI stayed absent to the cache-only request path and the
request was answered by whatever was resident. finalize_worker_exit is the one
point every download worker exits through, so invalidate there. That closes the
window without leaning on the TTL, which the scan-duration throttle can stretch
past 5s on an install where the scan itself takes longer than that.
A downloaded but unloaded GGUF asked for as org/model:latest missed the
resolver, since the suffix was always treated as an exact quant. With
auto-download on that probed the Hub and returned a 404 for a quant that was
never a quant; with it off it refused without switching. Fall back to the base
entry when the suffix is not quant-shaped, and keep exact matching for real
quants so a swap can never serve the wrong weights under the right name.
The usage examples stopped polling once a model was resident, but idle unload
frees one without touching the store, so nothing re-ran the effect and the
examples kept naming a model that could no longer be reloaded. Slow the poll to
60s instead of stopping it.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: hold the download slot while it is in use, and keep quants to llama.cpp
_loaded_satisfies refuses a quant reference against the Transformers backend by
name, but the path match did not carry that rule. A Transformers model active
from a directory that also holds GGUF exports therefore matched a request for
one of those quants and answered it with the safetensors weights. Only
llama.cpp has a quant identity, so admission now passes llama_only whenever the
reference is quant-qualified. A bare name still matches either backend, and
/v1/models residency keeps the default so a loaded Transformers model is still
reported loaded.
The 24 hour watch window was bounding ownership of the single-flight slot when
it should only have been bounding progress reporting, so a legitimately slow
download had its slot handed back while the worker was still writing, admitting
a second multi-gigabyte download beside it. Resolve the row on the clock, but
keep the slot on a slower poll until the job is actually terminal. Past the
deadline an unknown state does release it, since it means the worker cannot be
probed and holding it on that forever would wedge auto-download.
* Studio: keep what the resolver already knew when a download lands
Invalidating cleared the index to empty. The request path reads that cache
without scanning, so from a completed download until the rebuild landed it had
no evidence about any local model, not just the new one, and a bare request for
any of them was answered by whatever was resident. Wiring the hook into the
shared completion path in the last commit widened that from auto-download to
every download.
Mark the scan stale and keep the entries instead. Both _index and
warm_index_soon rebuild on a zero stamp, while the request path still sees
everything it knew a moment ago. Only a completed download invalidates, and
that only ever adds models, so nothing retained goes false.
Warm from the completion hook too, so the rebuild starts when the download
lands rather than when the next request happens to need it.
* Studio: match the quant, not just the directory, and default-select bare tags
Two quants of one repo share a directory, so the path match could not tell them
apart and an explicit :Q8_0 was answered by a resident Q4_K_M that
_loaded_satisfies had already refused by name. The llama_only fix in the last
commit only ruled out the wrong backend, not the wrong quant on the right one.
Both path matches now require the resident hf_variant to equal the requested
quant whenever the reference is quantified; a bare name still matches on the
path alone, since it claims nothing about the weights.
The local resolver already treated a tag that names no quant as meaning the
repo, but remote admission still looked for a quant literally called "latest",
so the same reference resolved locally and 404d remotely. Branch on
looks_like_quant there too. A real quant the repo does not have is still a 404
and never a substitution, which is what separates this from the loader's
low-disk fallback.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: one quant preference, and stop trusting a stale checkpoint
list_local_gguf_variants sorts by descending size, so the head of variants was
the biggest quant, often F16, while remote admission and a plain load both rank
through _pick_best_gguf. A bare id therefore meant a different quant depending
on which side answered it, and the local answer was the one that could evict a
working model and then fail or OOM starting an F16 next to a usable Q4.
/v1/models advertised that same head for pinning. Pull the ranking into one
preferred_quant helper and have both sides use it.
The usage examples returned a stored checkpoint without ever consulting
/v1/models, and the polling added last round was gated on not having one, so
for a stored checkpoint it never ran. An idle unload then left the panel
showing a snippet that could not run. Poll whenever mounted, and prefer the
checkpoint only while the catalog still backs it or switching can reload it. A
catalog that has not answered yet is not evidence against it.
The static contract pinned the old dependency array, so it now asserts the
intent it documents: a finished load re-runs the fetch, and the effect is not
gated on having no checkpoint.
* Studio: fix the Windows path compare, and advertise a label the worker knows
The case fix normalized the separator to "/" and then called os.path.normcase,
which on Windows folds case and rewrites the separator back to a backslash, so
the descendant checks compared against a "/" the path no longer had. A manually
loaded GGUF reached through an alias then read as a different model, giving a
false 404 and an alias marked unloaded. Run normcase first and normalize the
separator after it.
There are two quant-label extractors and they only agree while a recognized
quant token is present. With none, _extract_quant_label takes the last
hyphenated segment, "7b" of llama-7b, while build_gguf_variant_plans and the
worker key the whole stem: the plan lookup missed and the job exited on a
variant it had no shards for. Use the canonical extractor for the unrecognized
case only. Checked across real filenames first, the two match on every
recognized quant and part on bpw-qualified labels, which _extract_quant_label
keeps apart on purpose so byteshape's IQ4_XS at 3.53, 3.97 and 4.19 bpw stay
separate variants.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: a stored checkpoint needs catalog evidence, not just the switch setting
Preferring it whenever switching was on short-circuited the catalog check, so a
checkpoint the store still held after the model was deleted or moved kept being
named even though /v1/models had already proved it absent, and the snippets 404d
instead of falling back to a model that is actually there.
A lookup rather than a disjunction, which settles the whole matrix in one place:
no answer yet keeps the checkpoint, since that is not evidence against it; listed
and resident keeps it; listed but unloaded keeps it only when switching can
reload it; absent falls back whatever the setting says.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: normalize the quote style pre-commit would have rewritten
* Studio: cover the model that just landed, and pin the quant the catalog has
Retaining the index on invalidation protects what was already scanned and by
construction cannot contain the model that just finished downloading, so a bare
request for it in the window before the rebuild was still answered by the
resident model. Record the repo at the completion hook and treat that as
admission evidence alongside the resolver and the catalog; the next completed
scan clears the notes, since the index then covers them. Publishing a rebuilt
index before completion becomes observable would have closed it too, but that
blocks the download worker for the length of the scan.
Catalog membership proves the repo, not the saved quant, and the examples then
pinned the stored one. A quant deleted while another quant of the same repo
remained produced repo:deleted-quant, a missing-quant 404 with a runnable
alternative listed right beside it. Pin what the catalog advertises: for a
resident entry that is the resident quant, for an unloaded one it is a quant
actually on disk. The store is only consulted before /v1/models has answered.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: apply three rules everywhere they belong, not only where reported
The trust probe was the last credential handoff still passing a raw token.
huggingface_hub reads None as "use the cached login", so a caller-named repo
was read with this server's Hugging Face identity whenever the caller sent
none, which is exactly the isolation the metadata probe and the worker already
keep. It takes _hub_token now. Enumerated the rest of that path while there:
auth_check, model_info and spawn_worker were already correct.
finalize_worker_exit is shared with dataset downloads, so the resolver hook
fired for every completed dataset, scanning the model directories for nothing
and recording the dataset id as local-model evidence, which turns a bare /v1
request naming that id into a refusal instead of a foreign-id fallthrough.
Gated on repo_type.
_already_serving decided "bare" on the presence of a colon while
_loaded_satisfies and the resolver decide it on whether the suffix names a
quant, so org/model:latest against a serving Q8_0 read as a mismatch and
swapped in the preferred Q4_K_M for a request either one answers. That rule now
lives in four places, each fixed in its own round, so this time I looked for
the rest and found a fifth: describe_local_miss splits on the bare colon and
its docstring claims it splits like the resolver. It no longer did, and would
report a missing quant named "latest". Fixed here too, unreported.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: probe before refusing busy, and scan once when the index is cold
The busy refusal fired before anything established the requested label was a
model at all, so any namespaced id a drop-in client sends was told to wait out
an unrelated download for as long as it ran. Probe first and refuse only a
label the Hub actually serves as GGUF; anything else falls through to the
resident model as before. A probe failure answers "not downloadable", since
stranding ordinary traffic costs more than missing a busy refusal.
Treating an unbuilt index as "nothing here" let the first request after startup
be answered by the resident model under another model's name. That was a
deliberate trade to keep the scan off the request path, and it was the wrong
one. Cold, the scan now runs once on a thread, bounded so a pathological
install falls through rather than hanging the request. Built, the request path
still never scans, so the latency fix stands.
The watcher freed the slot the moment it saw an error, while Retry-After is
thirty times the poll interval, so the client came back to an empty slot and
restarted the same failing download instead of being told. Hold the failure on
the slot until a retry surfaces it, and let another repo take it after three
retry intervals so a client that never returns cannot keep it.
The watcher also invalidated on completion, which now lands after
finalize_worker_exit's warm and marks that fresh scan stale, pushing a
synchronous rescan onto the retry. Removed.
_loaded_satisfies lowercased paths as well as aliases, so it returned satisfied
before the case-preserving compare below could run. Both now go through one
helper: paths compare with normcase, aliases stay case-insensitive.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: an unfinished scan is not absence, and a decided refusal is not a failure
Bounding the cold scan then reading the bound as "not here" left the same hole
one branch over. A timeout now answers 503 model_indexing with a Retry-After
and leaves the warm running. A foreign label sent inside that window is asked
to retry rather than falling through, which is a real cost, but the window is
one request on an install whose scan exceeds ten seconds and it clears itself,
where answering with the wrong weights does not.
That uncovered a worse one. Every check here runs inside a broad except whose
job is "could not verify, so fall through", so an HTTPException raised in the
block was logged as a verification failure and the request was answered by the
resident model. Any refusal decided in there was being swallowed. Re-raise it
ahead of that handler.
Canonicalizing generic labels made them real variant keys, but the matcher
still decided on shape, so repo:llama-13b fell past an exact match and fetched
llama-7b. Match exactly first, whatever the shape; a quant-shaped suffix that
matches nothing is still a miss and never a swap.
Marking a catalog alias loaded while publishing the preferred on-disk quant
claimed alias:Q4 was loaded while Q8 was serving, and requiring the resident
quant to match then made pinning it a 404. Advertise the resident variant when
the entry resolves to the resident model.
* Studio: keep the asyncio.timeout fallback tests runnable on Python 3.10
Both tests deleted asyncio.timeout to force _wall_clock_timeout down its
pre-3.11 branch, but monkeypatch.delattr raises when the attribute is already
absent. On Python 3.10, the one version the fallback exists for, there is
nothing to delete, so the two tests errored with AttributeError before reaching
the code they cover. Passing raising=False makes the deletion a no-op there and
leaves the assertions running against the same branch on every version.
Every other delattr in the repo already passes raising=False for exactly this
reason. Verified with asyncio.timeout removed from the interpreter: the two
tests fail with the CI AttributeError before this change and pass after, and
the file still runs 89 passed on 3.13 where the deletion is real.
* Studio: decide GGUF residency, servability and variant keys by one rule each
Four admission and catalog fixes, each closing a gap between two places that
were answering the same question differently.
The /v1/models catalog asked _resolves_to_resident without llama_only, so a
Transformers model live from a directory that also holds GGUF exports marked a
GGUF alias loaded and gave it a GGUF quant. The usage examples then pinned
alias:quant that nothing could serve with switching off. Every entry in that
loop is advertised as GGUF, so residency there is llama.cpp residency.
The busy probe accepted any .gguf sibling while admission excludes mmproj, MTP
drafters and big-endian builds. A repo holding only companions is not
downloadable, so it was held at model_download_busy for the length of an
unrelated download instead of falling through to the resident model as it does
when no download is running. It now reuses _gguf_variants, the same filter.
split_model_ref refused any slash-bearing suffix, but an unrecognized GGUF below
a subdirectory keys on its path (build/llama-13b), which is_valid_gguf_variant
allows and the catalog advertises. Pinning such a variant could not parse, so
only the default-ranked one was reachable. A slash-bearing suffix is now a
variant exactly when a real Hub repo precedes it, which still leaves
C:/models/x.gguf a path rather than a quant.
The usage examples treated a downloaded-but-unloaded model as runnable only
under auto-switch, but a standalone UNSLOTH_MODEL_IDLE_TTL reloads exactly what
it freed on the next request. The panel hid runnable examples after an idle
unload. Tracked apart from auto-switch, because the stash restores the stored
checkpoint only and never an arbitrary catalog entry.
Also stub the index walk in the three cold-index tests that missed it: a real
multi-root scan inside the cold-wait budget made them time out into a 503 under
load rather than assert what they are there for. One of them flaked locally.
Verified each fix is load-bearing by reverting it and watching its test fail.
Backend CI command: 10195 passed, 0 failed. tsc -b and the frontend build clean.
* Studio: bound the Hub admission probes and stop guessing at nested model paths
Three review fixes plus a test-isolation one.
_resolves_to_resident matched on a path prefix, so two separately indexed models
that nest (/models/A alongside /models/A/sub/B) both satisfied it: loading B
made a request for A resident and answered it with B's weights, and the catalog
marked A loaded. A prefix match now counts only when no catalog entry sits
deeper, which is the innermost indexed model that actually owns the file. With
nothing indexed there is no nesting to tell apart, so the directory-to-weights
match this exists for is unchanged.
auth_check and hf_hub_download take no timeout of their own, and both ran while
the provisional single-flight slot was held, so an unresponsive Hub stalled the
request far past the metadata budget and reported every other model busy for the
duration. Both are bounded now. Each default errs the safe way: an unchecked
repo is not a cleared one, so the custom-code probe refuses on timeout, while a
slow gated-repo check stays inconclusive because the download's own auth is the
real gate.
The usage examples caught a failed refresh into an empty catalog and a disabled
auto-switch, which made a transient error authoritative and blanked every
example while the model was still servable. The catalog is deliberately
tri-state; a failure now keeps the last answer and retries.
Also start the backend tests from a built, empty model index. Stubbing only the
background warm still left the cold path walking real caches synchronously
inside the admission wait, so on a large install a test asserted against a 503
"still indexing" instead of its subject. _build_index is untouched, so the tests
that call it directly still exercise the real walk.
Verified each fix is load-bearing by reverting it and watching its test fail.
tsc -b clean. Backend CI command green apart from two failures reproduced only
on this box (a real model-dir scan and an orphan-process cleanup), neither
touched by this PR; staging CI is the gate for those.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Let a decode failure degrade instead of escaping a fail-closed helper
Pinning utf-8 makes a read that used to return mojibake on Windows raise
instead. 33 of those reads sit under a handler catching OSError or
json.JSONDecodeError but not UnicodeDecodeError, which subclasses
ValueError, so a corrupt file would now escape a helper written to return
a default. Adds UnicodeDecodeError to those tuples only.
* Treat an undecodable install lock as stale instead of retrying forever
* Pin utf-8 on shipping-code text I/O instead of the operator locale
113 read_text/write_text/open call sites across unsloth, studio and
unsloth_cli let locale.getencoding() decide the encoding. That is utf-8 on
the Linux and macOS runners and cp1252 on a stock Windows install, so the
same file decodes differently for a Windows user and silently produces
mojibake or raises UnicodeDecodeError.
Adds tests/test_runtime_text_encoding.py to keep it that way. It resolves
openers through each file's own imports rather than a fixed list of module
names, so an aliased tarfile.open or a local from PIL.Image import open is
not asked for an encoding it does not take.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scan tracked files only and resolve the unbound Path calling forms
* Honour PEP 263 when scanning sources and migrate a legacy JSONL before appending
* Scope guard imports lexically and only migrate a legacy file when it round-trips
* Leave a legacy JSONL untouched and resolve path aliases in the foreign-opener check
* Tighten comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add Voice settings tab (dictation, dictionary, read aloud)
New Voice tab in Settings, placed just before About:
- Dictation: microphone picker, browser STT engine, recognition language,
and an inline mic test with a live transcript
- Dictation dictionary: entries rewrite matching speech to their exact
spelling and casing, applied in both dictation paths
- Recent dictations: last 20 final transcripts with copy and clear, so
text can be recovered if it lands in the wrong place
- Read aloud: optional button on assistant responses with two engines,
curated system voices (novelty and legacy voices filtered, quality
ranked, capped at 20) or the TTS audio model loaded in Unsloth via
/audio/generate (e.g. Orpheus), plus speed, pitch, volume and preview
Settings persist in localStorage (unsloth_voice_settings) and are read
at call time so changes apply without reloading the runtime. Adds en
keys plus the tab label for ja, zh-CN and pt-BR.
* Studio: drop the single option STT engine select, rename TTS option
The STT engine dropdown only had one entry, so it added noise without
giving a real choice. The engine row can come back once local STT
models land. Also renames the TTS engine option Unsloth TTS model to
Load TTS model to make the action clearer.
* Studio: harden Voice settings against edge cases found in simulation
Simulated the feature across Chromium, Firefox and WebKit plus node
level unit runs and backend contract checks. Fixes from the findings:
- Dictionary rewrite used a replacement string, so entries containing
dollar patterns corrupted transcripts (A$$AP became A$AP, $& injected
the match). Switched to the callback form of String.replace
- Persisted voice settings now validate types on hydration: non string
micDeviceId, dictationLanguage and ttsVoiceURI, and non boolean
ttsEnabled fall back to defaults instead of flowing into the UI
- Dictionary entries are trimmed, capped at 120 chars and re-sanitized
on hydration
- The Test dictation panel now falls back to the default microphone
when the saved device is unplugged, matching the composer adapter
Test coverage: 46 unit assertions (dictionary regex edge cases across
unicode, word boundaries and injection, voice curation for simulated
macOS, Windows and Linux voice inventories, corrupt storage merge),
13 backend contract checks against /audio/generate on an isolated
instance, and 60 browser assertions across the three engines covering
rendering, degradation without SpeechRecognition, curation in a real
DOM, dictionary persistence with unicode and dollar entries, the
no-model preview error path and corrupt localStorage recovery.
* Studio: address Voice settings review feedback
Verified each review comment before acting. Confirmed and fixed:
- Editing a dictionary entry was broken in two ways: the store trimmed
on every keystroke so spaces could not be typed, and clearing the
field deleted the entry and unmounted the input mid edit. Updates now
keep the raw value and a blur commit trims or removes the entry
- The unplugged mic fallback checked instanceof DOMException, but a
cross browser probe showed Firefox and WebKit throw
OverconstrainedError objects that are not DOMExceptions, so the
fallback never fired there. Matching on the error name now
- When the browser ended a dictation test on its own (silence timeout),
the mic stream stayed open. All recognition end paths now stop the
tracks and save the transcript through a single finalize path
- The studio TTS audio element now releases its WAV data URL as soon as
playback ends, fails or is cancelled
- Allow microphone now reports insecure contexts (no mediaDevices)
accurately instead of claiming access was blocked
- Voice tab copy moved into i18n keys per src/i18n/AGENTS.md, so locale
overlays can translate it; en is the baseline and parity passes
- unsloth_voice_settings added to the Reset all local preferences key
list so voice preferences obey the reset
- Non default microphones note that the system default is used when the
browser speech engine cannot bind a specific device, since browsers
without the start(track) overload ignore the argument silently
Re-ran the full simulation set after the changes: 46 unit assertions,
13 backend contract checks and 60 browser assertions across Chromium,
Firefox and WebKit all pass, plus a dedicated browser probe for the
dictionary editing behavior.
* Studio: use the chat mic icon in Voice settings for consistency
The Voice tab and its buttons used the hugeicons Mic02 glyph while the
chat composer uses a custom filled mic. Extract that composer icon into
a shared lib/mic-icon component, drop the duplicate inline copies in
thread.tsx and shared-composer.tsx, and use it for the Voice tab icon
and the tab's mic buttons so the microphone looks the same everywhere.
* Studio: address second round of Voice settings review feedback
Verified each new comment against the current code first. One item was
already fixed in the previous round (recording transcripts when the
browser ends a dictation test on its own). Confirmed and fixed:
- The microphone row showed a picker with generic names when browsers
enumerate unlabeled devices before permission, leaving no way to
grant access from the row. It now branches on whether labels are
visible and shows Allow microphone otherwise
- Compare chat dictation ignored the selected microphone. It now opens
the chosen device with the same fallback rules as the main adapter,
passes the track to recognition where supported and releases the
stream when recognition ends
- Closing the Voice tab cancelled the shared speechSynthesis even when
read aloud was playing a chat message. Cleanup now only cancels when
the tab owns an active preview
- Double clicking Start test could race two recognizers and leak the
first stream. A starting flag set before the getUserMedia await makes
start reentrancy safe
- Turning off the read aloud setting mid playback removed the only stop
control. The stop button now renders whenever a message is speaking
- When an engine lacks the start(track) overload, both dictation paths
now release the selected device stream before retrying with the
default microphone instead of holding it open
- Read aloud support no longer requires Web Speech synthesis: the
Unsloth TTS engine only needs audio playback, so it stays available
in WebViews without speechSynthesis, with a clear error if the system
engine is chosen there
Not addressed here: cancelling in flight backend TTS generation on
stop. The route runs generation in a worker thread without a
cancellation path, which is shared pre existing behavior with audio
chat generation and belongs in a backend change.
All suites re-run green: 46 unit, 13 backend contract and 60 browser
matrix assertions across Chromium, Firefox and WebKit, plus probes for
the unlabeled device branch and the double click race.
* Studio: drop empty and duplicate voiceURIs so the Voice tab never renders a crashing Select item
* Studio: guard dictation mic lifecycle in Voice test and Compare composer
Release a microphone opened after the component unmounts, and stop Compare
dictation on a permission or security failure instead of silently recording
from the default device, matching the main chat adapter.
* Studio: fix dictation and read-aloud lifecycle edge cases in Voice settings
- Join final dictation chunks with a space so recorded transcripts do not merge words
- Ignore a stale recognizer onend so a quick stop then restart is not torn down
- Use previewingRef so a double click on TTS preview does not orphan the first request
- Keep the read-aloud stop control visible when a new run starts while a message is spoken
- Stop the dictionary remove button from deleting an adjacent entry on a blur then click race
* Studio: trim redundant Voice settings comments
* Studio: fix Voice preview and Compare dictation edge cases
- Only cancel the shared speechSynthesis for a system-voice preview, so stopping
a Studio preview no longer stops an unrelated chat read-aloud
- Release the Studio preview audio and its WAV data URL on normal completion
- Iterate every finalized result in Compare dictation so batched phrases are kept
- Cap persisted recent dictations to the last 20 on hydration
* Studio: use clipboard fallback for recents and release failed preview audio
- Copy recent dictations via the copyToClipboard helper so the execCommand
fallback works in Safari and insecure http LAN contexts
- Release the Studio preview audio when play() rejects, not just on ended/error
* Studio: add local speech-to-text dictation engine
Add an offline dictation engine that transcribes with a local faster-whisper
model, alongside the existing browser (Web Speech) engine. The browser engine
streams audio to Apple or Google speech services and needs internet; the new
engine runs on the server, works offline, and drives any chat model without
evicting it (it loads in the backend process, separate from the model
subprocess). It also gives Firefox dictation, which has no Web Speech support.
Backend: a lazily-loaded, kept-warm faster-whisper sidecar and three routes
under /api/inference/audio (stt/status, stt/load, transcribe). faster-whisper
is torch-free, so this does not disturb the existing model stack.
Frontend: a Dictation engine setting (browser or local model), a curated model
picker with sizes, and MediaRecorder capture posted to the transcribe route.
The model warms automatically when the engine is selected, with live status.
* Studio: stream local STT transcription as you speak
Local dictation showed nothing until you stopped, because the whole clip was
transcribed once on stop. Now the growing recording is re-transcribed on a
fast pass every second and emitted as live interim text, with an accurate
final pass on stop. Partial recordings decode fine, and the model refines
earlier words as more audio arrives.
Adds an interim flag to the transcribe route (beam 1, no VAD) for the fast
preview pass; the final stop uses the accurate path.
* Studio: make local dictation stop instant and reliable
Stopping local dictation waited for a final network transcription before the
session ended, so the stop button did not flip and a second click ended the
session early and dropped the text. Now stop commits the live transcript
immediately, releases the mic at once, and ignores a second stop while
finalizing. Previews run more often so the committed text is current.
* Studio: record local dictation in short clips for reliable streaming
Re-transcribing a growing buffer every second got slower as it grew, flooded
the backend, showed stale words, and could leave the stop button stuck waiting
on a backlog. Record short independent clips instead and transcribe each once,
appending the text as you speak. Work per clip is bounded, so stopping is
prompt (with a hard timeout as a safety net) and long dictations stay smooth.
* Studio: dictate then transcribe once on stop, ChatGPT style
Local STT dictation streamed by re-transcribing the growing clip, which
was quadratic and saturated the backend (multi-second lag), and stop only
halted the recorder without releasing the mic, so it kept recording. Record
the microphone continuously, release it the instant the user stops, and
transcribe the whole clip once. Stopping is immediate and the transcript
lands in about a second. Also add the tiny model for the fastest option.
* Studio: surface dictation and read-aloud failures instead of failing silently
- Compare dictation reports microphone and speech-recognition errors via toast,
reusing the main chat adapter's describeMediaError and describeSpeechError
- Read-aloud toasts genuine model or synthesis failures while ignoring cancellations
* Studio: ChatGPT-style recording bar for dictation
Clicking the mic now drops the composer into a dedicated recording bar
with a live waveform, a discard (X) and a confirm (tick), instead of a
plain stop button. The tick stops recording and transcribes the clip;
the X throws the recording away and keeps whatever text was already in
the composer. The model adapter taps the mic with an analyser to drive
the waveform, and the router tracks the live session so the X can cancel
it without transcribing.
* Studio: transcribe dictation while speaking, ChatGPT layout
Match ChatGPT's recording layout: the bar now renders in place of the
input with the left plus button kept, the waveform in the middle, and
the discard and confirm buttons together on the right.
Cut the post-confirm delay by transcribing in the background as the user
talks. The audio is split at natural pauses (voice-activity detection off
the same analyser that drives the waveform) and each clip is transcribed
as it is cut, so confirming only has to finish the short final tail. The
model is also warmed when recording starts so the first run never pays a
cold load.
* Studio: ChatGPT waveform, hide tools while dictating, faster STT
Make the recording UI read like ChatGPT: the waveform is now a dense row
of round dots that rise into thin centered bars, and while dictating only
the plus button shows, with the mode badge and tool toggles hidden so the
bar is just the waveform and controls.
Speed up transcription: decode greedily (beam_size=1), which is several
times faster on CPU with negligible accuracy loss on short dictation
clips, and cap background segments at 6s so the final tail after confirm
stays short.
* Studio: finish ChatGPT voice bar and low-latency STT
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: full-width waveform with a timer that freezes on stop
Use the full-width waveform for the recording bar: brighter, bigger bars
that advance on a fixed cadence (keeping peaks between advances) so they
glide instead of racing by, inset from the composer edges. Keep a visible
timer and the green confirm button, matching the ChatGPT reference, and
freeze the timer and waveform the moment the user confirms.
* Studio: fix multilingual local dictation
* Studio: speed up dictation and release local STT
* Studio: harden dictation finalization and STT decoding
* Studio: restore Firefox dictation fallback
* Studio: add dictation history manager
* Studio: manage speech model downloads
* Studio: remove em dash from voice model label
* Studio: move dictation history into Voice
* Studio: source local STT from Unsloth Whisper models
Point the dictation STT sidecar and its Model Hub download entries at
Unsloth's Hugging Face Whisper repos (small, large-v3-turbo, large-v3)
and run them through Transformers, so Studio only ever downloads
Unsloth-uploaded weights. Drop faster-whisper and the Systran/mobiuslabs
repos; keep the Model Hub as the only download path via local_files_only,
and keep PyAV for audio decoding.
Device selection uses float16 on CUDA and float32 on MPS and CPU, since
Whisper's decoder is unstable in float16 on MPS and repeats tokens.
Shorten the model picker labels to name plus download size and update the
STT tests for the new backend.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: smooth dictation waveform and keep pill height
* Studio: align STT model dropdown width and tidy voice copy
* Studio: guide to local engine when browser dictation is offline
* Studio: clarify voice section and STT model copy
* Studio: keep STT warm with training-aware eviction
* Harden STT lifecycle and browser compatibility
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model discovery test lint
* Harden cross-browser microphone errors
* Harden cross-browser microphone errors
* Surface voice test recognition errors and fall back to Studio TTS
- Voice test now toasts non-abort speech-recognition failures instead of
ending silently, matching the main and Compare dictation paths.
- Read-aloud routes to the backend model when the runtime lacks Web Speech
synthesis (audio-only WebView), so it no longer errors immediately.
* Fix reviewed STT lifecycle races
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix read-aloud fallback controls
* Guard read-aloud stop when deleting a non-speaking message
aui.message().stopSpeaking() throws unless this message is the one being
read aloud, so calling it unconditionally rejected the delete handler before
the message was removed. Only stop speech when this message is speaking.
* Cap recent dictation transcript length before persisting
Recent dictations only limited entry count, so a long transcript stored the
full text in the persisted voice settings and a few could exceed the
localStorage quota, throwing synchronously from the uncaught dictation cleanup
path. Truncate each entry on save and on hydration, matching the dictionary cap.
* Studio: keep dictation mic clickable and guide to local model
Register the dictation adapter unconditionally so the mic stays enabled
for any engine and starts working right after switching to the local
model on an already-open thread.
When the browser engine cannot run (Firefox, Brave, non-secure origins),
clicking the mic shows a toast that points to the local speech-to-text
model instead of leaving a disabled button. The toast stacks its action
below the text with a fully rounded button.
* Studio: add bottom padding below the dictation guidance toast button
* Studio: increase bottom padding under the dictation toast button
* Studio: add bottom padding inside the dictation toast button
* Studio: add five Whisper defaults and custom model search
Add private UnslothAI Tiny and Base mirrors to the curated local STT choices while keeping Small as the default. Let users search or paste a Transformers-compatible Whisper repository and validate it end to end.
Keep short dictations in one clip to avoid repeated padded encoder work, then split longer recordings near Whisper's 30-second boundary.
Update hidden model filters and tests, including the CPU-only CI runtime stub for PyAV.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: use public Unsloth Whisper repositories
Point the Tiny and Base dictation defaults to the public unsloth repositories and remove the private mirror references from model filtering and tests.
* Studio: update Whisper download sizes
Reflect the cleaned public Tiny and Base repositories in the curated model labels.
* Studio: right-align STT model size, fix dropdown wheel scroll, refresh sizes
- Show the download size on the right of each model row so long names
like Whisper Large v3 Turbo no longer hide it
- Update curated Whisper sizes to the safetensors weights actually
downloaded: Tiny 151 MB, Base 290 MB, Small 967 MB
- Drive the model list scroll from a wheel handler so the mouse wheel
scrolls it inside the Settings dialog, not just the scrollbar
- Add a search icon and shorten the placeholder to Search model
* Studio: do not search when a dictation model is picked, shrink repo label
- Treat the filled-in model text as a selection, not a query, so choosing
a model no longer kicks off a Hugging Face search
- Make the repository line under each model name smaller
* Studio: tighten dictation model and local engine descriptions
* Studio: keep model display on pick instead of the query, shrink row text
- Guard the combobox input so selecting a model shows its name and does
not echo the typed query back or start a search
- Map the item label to the friendly display so picks fill the field
- Reduce the model name and size text in each row
* Studio: show only the model name in the dictation field, shrink size label
- Drop the download size from the search field; the name alone is shown
once a model is selected, with sizes kept in the dropdown list
- Reduce the size label text in each row
* Studio: clarify the dictation model description
* Studio: drop Hugging Face from the dictation model description
* Studio: move the dictation dictionary to its own Manage subpage
- Replace the inline entry list with a Manage row, matching Dictation
history, so a long dictionary no longer crowds Voice settings
- Add a DictationDictionaryView subpage that holds the entry editor
* Studio: match STT field font, use best voice for System default
- Bump the dictation model field text to text-sm so it matches the
engine dropdown next to it
- Resolve the System default read-aloud voice to the top curated voice
instead of the browser default, which is a robotic legacy voice on macOS
* Studio: rerank read-aloud voices and drop duplicate voice entries
- Rank by vendor quality, then the user's locale, then a preferred list of
natural voices, so the best voice leads instead of the first alphabetically
- Collapse voices that macOS reports twice under one name and language
* Studio: fold dictionary and recents into the dictation section
- Drop the separate Dictation dictionary and Recent dictations headings;
their Manage rows now sit under Dictation, split by the row divider
- Shorten the custom spellings description
* Studio: add search and sort to dictation history
- Filter saved dictations by text with a search field
- Sort by newest, oldest, or A to Z; show a no-matches message
- Keep Clear all available regardless of the current filter
* Studio: settle cancelled STT loads before training and fix dictation review items
Wait for a cancelled STT load to exit and release its memory before
reporting it freed for training, so the loader cannot still be inside
from_pretrained()/.to(device) holding VRAM when the training subprocess
starts. A load that finishes before observing the cancel now gets
unloaded so the memory is actually reclaimed.
Clear the accelerator cache before the CPU fallback in load() so a failed
CUDA/MPS load does not strand reserved VRAM once the sidecar is marked
CPU-resident.
Send the saved Hugging Face token when polling STT download progress so a
gated or private repo resolves and shows the correct Load/Downloaded
state instead of reporting missing.
Mark the composer Dictate button as type="button" so clicking it does not
also submit the draft when the composer already has text or attachments.
* Studio: pin dictation settings per session and close STT startup races
Capture the STT model and language when a dictation session starts and
pass them to every queued segment and the warm-up load, so changing the
model or language mid-recording no longer transcribes the same clip with
the wrong model or a model that is not downloaded.
Check the local runtime at the top of transcribe(), before the model
cache lookup and the bounded audio decode, so a server missing PyTorch or
Transformers returns 501 up front instead of decoding a long clip first.
Treat the training startup window as active for STT device selection.
start_training frees VRAM in before_spawn but only assigns _proc later, so
a concurrent STT load could take the GPU that was just cleared. A startup
flag now reports training active from the free until the process is live,
forcing those loads to CPU; a finally clears it on every exit.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: stub the STT runtime check in transcribe orchestration tests
transcribe() now verifies the local runtime up front, so the unit tests
that exercise transcription orchestration must treat the runtime as
present to keep passing where PyTorch, Transformers, and PyAV are not
installed. Stub ensure_stt_available in the shared fixture and restore
the real check in the availability and load-rejection tests.
* Harden custom Whisper dictation models
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add whisper.cpp dictation engine with per-engine downloads and history rework
Engines
- New GGML STT sidecar that runs a managed whisper-server subprocess with
idle unload, plus a pinned static build script (scripts/build_whisper_cpp.sh)
- Dictation engine picker now offers Browser, Local transcription
(whisper.cpp), and Local transcription (Transformers)
- Both local engines serve the same five curated Whisper models and download
them directly with byte-level progress reported by /audio/stt/status
- Models auto load on selection and when their download finishes
- Unload and training admission account for both engines
Benchmarks (Apple Silicon, greedy, warm, same checkpoints)
- whisper.cpp transcribes 2.4x to 5x faster than Transformers and loads in
about 0.45s vs 0.86s for Whisper Small
- whisper.cpp GGUF path is unchanged by the Transformers addition
(load 0.445s -> 0.444s, short clip 0.391s -> 0.347s, long 1.197s -> 1.129s)
Voice settings UI
- Plain curated model select replaces the searchable combobox
- Single download progress bar with transfer rate for both engines
- Dictation history now stores every dictation with Show more pagination,
a top Clear history action, and links back to the chat it was spoken into
- Archived chats dialog gets the same pagination
- Delete dialog offers deleting a dictation together with its chat
Tests: 88 backend STT tests pass, including new snapshot download coverage.
Frontend typecheck, lint, i18n parity, and production build pass.
* Merge local engines into one option and source GGML models from unslothai
Engine selection
- The dictation engine dropdown is back to two choices: Browser and Local
transcription. The selected model decides the backend: curated ids run
GGML checkpoints through whisper.cpp, searched Hugging Face repositories
run safetensors through Transformers
- Model picker lists the curated models and searches Hugging Face for other
Whisper repositories, validating them before selection. The trigger is a
plain button so the selection never renders inside a text input
- /audio/stt/status accepts a model query param so downloaded state works
for custom repositories; the engine param on load, transcribe, and
download routes is derived from the model everywhere
Model source
- Curated GGML checkpoints now download from the Unsloth-hosted
unslothai/whisper-*-GGUF repositories (one repo per model) instead of
ggerganov/whisper.cpp; cache lookups, progress totals, and in-flight blob
tracking are per-model
Fixes
- Voice settings and dictation history were not persisting: the quota-safe
localStorage wrapper was declared after the store that uses it, so the
persist storage factory failed silently. Every settings write also threw
mid-click, which kept the model picker popover from closing on selection
- is_model_downloaded now verifies config, preprocessor config, and real
weight files instead of trusting an offline snapshot lookup, so a partial
download left by an aborted fetch shows the Download button instead of
failing to load
- Removed whisper.cpp mentions from user-facing text: the ready status
shows Loaded instead of the runtime name, picker rows show the source
repository, and runtime error messages say local transcription runtime
Verified with automated browser sessions and live API checks: selection
closes the picker with no page errors, persisted settings hydrate on
reload, a stale partial snapshot triggers download then loads on MPS and
transcribes, and curated models download from the unslothai repos. 88
backend STT tests, typecheck, lint, i18n parity, and build pass.
* Skip the duplicate source line for custom models in the STT picker
A custom repository's display name is its id, so search results and the
appended current selection rendered the same string twice. The source
line now only renders when it differs from the name; curated rows keep
their name, unslothai source repository, and download size.
* Verify every shard of a sharded checkpoint in the downloaded check
A snapshot holding one of N shards (or a corrupt shard index) passed the
downloaded check and then failed at load. When model.safetensors.index.json
exists, every shard in its weight map must now be present. Found by
simulation; covered by a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Rename stale _starting references in the pump resilience tests
The startup flag on TrainingBackend was renamed to _spawn_in_progress but
two tests added alongside it still asserted on the old name, failing the
Python 3.11 to 3.13 CI jobs.
* Make the selected model row clearly highlighted in the STT picker
The current selection was a faint background tint. It now uses the accent
background with a medium weight name. Two line rows use a small corner
radius; single line custom repo rows keep the pill shape.
* Address review feedback on STT snapshot checks, VRAM release, and dictation UX
Verify snapshot completeness in the load preflight so a partial download
fails before the audio is decoded, for curated and custom repos alike.
Drop the failed accelerator traceback before the CPU retry so the cache
clear can actually release that memory. Keep unloading the GGUF sidecar
after cancelling an in-flight Transformers load; both engines can hold
memory at once. Allow Auto language with English-only .en checkpoints,
matching the backend which sends no forced language. Keep the discard
button usable while a transcription is pending so a slow or hung request
cannot trap the composer in dictation mode. Stop linking Compare and
settings test dictations to the unrelated active single chat thread.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move the CPU retry out of the exception handler
On Python 3.10 the interpreter exception state keeps its own reference
to the traceback, so dropping it from the caught exception was not
enough to release the failed accelerator load during the retry. Leaving
the handler before clearing the cache works on every supported version.
* Address review feedback on session handoff, chat pinning, and server lifetime
Starting a dictation from a second entry point now cancels the session
it replaces, so the old recording cannot keep the microphone open or
save a transcript with no discard button pointing at it. The linked
chat is pinned when recording starts, so switching threads while a
transcription finalizes cannot relink the transcript to the newly
opened chat. whisper-server is now bound to Studio's lifetime like the
other long-lived children: PDEATHSIG on Linux, the parent job object on
Windows, and pid adoption so the shutdown sweep reaps it; before this
it survived a Ctrl+C exit as an orphan still holding the model.
* Remove the dictation mic test from Voice settings
The composer dictate button covers the same check, so the test row, its
transcript panel, the unsupported fallback row, and their strings and
search entry are gone.
* Studio STT: gate GGUF whisper-server on training and fix dictation retry and dictionary edits
GGUF (whisper.cpp) sidecar:
- Launch whisper-server with --no-gpu while training is active, mirroring the Transformers sidecar's CPU device choice, so a mid-training dictation cannot reclaim the VRAM training just freed.
- Report is_loading() during whisper-server startup so training VRAM admission accounts for the accelerator memory it is about to bind.
- Require PyAV in is_available() so /audio/stt/status reports the engine unavailable when uploads cannot be decoded, instead of loading fine and then 501ing at transcription.
- Reject a missing model before decoding audio, matching the Transformers download preflight.
Voice settings:
- The download Retry button now restarts the download; the sidecar error is sticky until a new start(), so re-polling alone never cleared it.
Dictation dictionary:
- Tabbing from an emptied entry to its remove button no longer commit-splices the row first, which shifted indices and deleted the wrong entry.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio STT: fix curated GGUF whisper filenames to match hosted repos
The unslothai/whisper-*-GGUF repos host the checkpoint as whisper-<id>.bin,
not ggml-<id>.bin, so every curated dictation download and cached-path
lookup 404'd and the whisper.cpp engine could never load a model. Point
GGML_STT_MODELS at the real filenames and guard the naming with a test.
* Studio STT: validate a custom dictation repo before downloading it
The Transformers STT engine accepts an arbitrary owner/model repo, but the
download route handed it straight to snapshot_download, pulling a possibly large
non-Whisper repository into the shared HF cache. Confirm the repo is a Whisper
checkpoint first with the existing metadata-only validate_remote_model (no
weights); curated ids short-circuit and the GGUF engine (curated-only) is
unaffected. A non-Whisper repo now 422s before any download.
* Studio STT: preempt a still-loading GGUF server for training admission
A whisper-server still in its startup window binds accelerator memory but has no
loaded_model yet, so training admission could miss it and launch into an OOM.
Make the GGUF startup cancellable (cancel_pending_load signals an abort event and
terminates the starting process without the load lock; _wait_for_server observes
it and raises SttLoadCancelledError; wait_for_load_to_settle blocks on the lock
until the killed server is reaped), and always fold the GGUF sidecar into the
resident-STT summary so a resident Transformers model cannot mask a loading GGUF
server. free_stt_model_for_training now cancels an in-flight load and waits for it
to settle before training claims the memory.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio STT: fall back to Transformers when whisper-server is absent
A curated dictation model (including the default small) hard-pinned the GGUF
engine, but standard installs do not ship whisper-server, so every recording
501'd instead of using the Transformers engine that serves the same checkpoint
-- the GGUF sidecar's own documented contract. Add _resolve_serving_stt_engine:
a GGUF request for a curated id (the only ids GGUF accepts, all Transformers-
servable) downgrades to Transformers when whisper-server is unavailable, applied
consistently to download, load and transcribe (not unload, which targets a
specific engine). The Voice tab likewise falls back to the Transformers status so
the model is not shown unavailable and download is not blocked.
* Studio STT: hide custom Whisper caches from the legacy model pickers
The legacy /cached-models (and /cached-gguf) routes called is_hidden_model with
only the owner/model id, which cannot reach the config-based Whisper check, so a
downloaded custom (non-curated) Whisper checkpoint was still offered as a chat
model. Pass the cached snapshot path so _path_is_whisper_model inspects the repo
config and hides it, matching the discovery route.
* Studio STT: hide GGUF dictation repos, lock-free status, unload fallback, split training eviction
- Hide the curated GGUF dictation repos (unslothai/whisper-*-GGUF) from the chat
model inventory and pickers, backend and frontend. Only their Transformers
safetensors companions were hidden; the GGUF repos use a different org and a
-GGUF suffix and carry a raw .bin with no whisper config.json, so they leaked
into chat pickers.
- Make the GGUF sidecar loaded_model/device accessors lock-free, mirroring the
Transformers sidecar. transcribe() holds self._lock across the whole inference
call, so /audio/stt status polls and training admission previously blocked
behind an in-flight transcription.
- stt_unload resolves through the serving resolver: a "gguf" pick on a host
without whisper-server is served by the Transformers fallback, so unload must
target that engine or the resident model is never freed. Unload also attempts
every engine even if one raises, so a failure freeing one backend no longer
skips the other.
- free_stt_model_for_training frees the Transformers and GGUF sidecars under
independent exception boundaries so a failure unloading one no longer skips
the other before training claims the memory.
Adds tests/test_stt_review_fixes.py covering all four.
* Studio STT: resolve Auto dictation language for the model engine + snapshot process liveness
- The model dictation adapter sent the raw setting (the literal "auto") to the
backend, while the browser engine resolves Auto via resolveDictationLanguage.
A batch of non-English voice notes came back mostly English on Auto. Add
resolveModelDictationLanguage: only the literal "auto" is resolved to a
concrete locale, gated so it becomes a language the model AND Whisper can
honor (mirroring the backend's known-whisper-languages set); an explicit
language, or a locale Whisper cannot honor, stays unchanged/auto-detect. Wire
it into both adapter call sites.
- GgmlSttSidecar._process_alive() read self._process twice; a concurrent
unload() nulls it under the lock while loaded_model/device read lock-free, so
a null between the two reads called None.poll(). Snapshot once. Adds a
deterministic regression test.
* studio: tighten comments and docstrings in the dictation modules
* studio: harden dictation model downloads, GGML readiness, and recording paths
Address review findings on the STT dictation feature:
- build_whisper_cpp.sh refuses to delete a whisper.cpp tree under a custom
Studio home unless it carries the Studio ownership marker, matching the
setup.sh policy, and marks trees it creates
- _snapshot_is_complete validates every shard of a sharded PyTorch
(pytorch_model.bin.index.json) checkpoint like the safetensors path, and
requires tokenizer assets (tokenizer.json or vocab.json + merges.txt)
- custom-repo downloads pin the revision resolved at validation time and
restrict snapshot_download to the model/tokenizer/config/preprocessor file
classes Studio loads
- the GGML sidecar holds its port reservation until just before spawning
whisper-server and only accepts readiness from a responder that both looks
like whisper.cpp's server and belongs to the still-running managed child,
probing twice, so mic audio cannot be posted to a foreign local process
- the recording adapter transcribes every non-empty segment; the RMS meter
only shapes segment boundaries and can no longer discard quiet speech
- Compare-pane dictation can cancel a pending transcription on second click,
with the button relabeled while finalizing
- localStorage quota recovery halves the dictation history until the save
fits, so small histories shrink too
- the System default TTS voice resolves to the platform default voice
- new dictation UI imports go through the chat and hub feature barrels
Regression tests cover the build-script gate, sharded PyTorch and tokenizer
completeness, revision pinning and allow patterns, and the whisper-server
readiness probe.
* Fix STT download and voice picker follow-ups
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add dictation button regression coverage
* Studio: prebuilt whisper.cpp via the shared llama.cpp install core, slim bundles paired to the llama prebuilt (#7294)
* Studio STT: add prebuilt whisper.cpp (whisper-server) installer
New install_whisper_prebuilt.py downloads a per-platform whisper-server
bundle published by the unslothai/whisper.cpp prebuilt CI into the managed
whisper.cpp dir (build/bin/whisper-server) so local dictation needs no
compiler. Mirrors install_node_prebuilt.py / install_llama_prebuilt.py:
host + backend detection, sha256 pins (whisper_prebuilt_pins.json) as the
trust anchor, staging + install lock + atomic swap, traversal-safe extract,
co-located shared libs (RUNPATH=$ORIGIN), an UNSLOTH_WHISPER_PREBUILT_INFO.json
marker with idempotent "already matches", and exit codes 0/1/2/3. Not wired
into setup yet; the pins ship empty so every asset fails closed until the
first fork release is published and its digests are reviewed in.
* Studio STT: install prebuilt whisper.cpp during setup and update
Add a fail-open whisper.cpp block to setup.sh after the llama.cpp section so
`unsloth studio update` (and a fresh install) fetch the prebuilt whisper-server
into the managed whisper.cpp dir the sidecar discovers. It skips a user-set
WHISPER_SERVER_PATH/UNSLOTH_WHISPER_CPP_PATH, honors UNSLOTH_SKIP_WHISPER_INSTALL,
forwards the resolved ROCm gfx, and never aborts setup: a busy install keeps the
existing runtime, and an unavailable prebuilt stays quiet (source build is opt-in
via UNSLOTH_WHISPER_FORCE_COMPILE) since Transformers STT and browser dictation
remain. Register UNSLOTH_WHISPER_PREBUILT_INFO.json as Studio-owned evidence.
* Studio STT: harden whisper-server child env + WSL ROCm detection
- Sidecar spawns whisper-server with a scrubbed child env that prepends the
binary dir (co-located GPU libs) to the loader path, and on WSL2 ROCm loads
the system HIP first (HSA_ENABLE_DXG_DETECTION=1) so a bundle's bare-metal HIP
does not segfault on /dev/dxg. Secret-bearing vars are dropped from the child.
- find_whisper_server_binary now requires an executable, not just a file.
- Installer rocm probe passes HSA_ENABLE_DXG_DETECTION and falls back to
/opt/rocm/bin/rocminfo so a WSL ROCm host is not misdetected as CPU-only;
gfx parsing skips the gfx000 CPU agent and generic ISA lines.
- Tests for the child env (secret scrub, lib dir, WSL HIP precedence), the
executable check, and the WSL rocm detection.
* Studio STT: in-app whisper.cpp prebuilt update stack + ship pins in the wheel
Mirror the llama.cpp update stack for the whisper.cpp prebuilt so Studio can
detect and install a newer whisper-server release from inside the app:
- backend/utils/whisper_cpp_freshness.py: read UNSLOTH_WHISPER_PREBUILT_INFO.json
and compare the installed release against the newest unslothai/whisper.cpp
release. Whisper tags are v<upstream>-unsloth.<N>, so is_behind compares a
(major, minor, patch, serial) key with a strict downgrade guard; 24h cache;
fail-open.
- backend/utils/whisper_cpp_update.py: run install_whisper_prebuilt.py to fetch
and atomically swap the newest bundle, unloading the warm GGUF sidecar first.
- backend/routes/whisper.py mounted at /api/whisper (update-status + update).
- pyproject: add whisper_prebuilt_pins.json to studio package-data so the
installer's trust anchor ships in the wheel (it is a data file, not a .py
module, so package discovery alone does not include it; node_prebuilt_pins.json
is listed for the same reason). Without this a pip-installed wheel had no pins
and the prebuilt install aborted to Transformers STT.
Adds test_whisper_cpp_freshness.py (version parser, is_behind matrix + downgrade
guard, marker layouts, stale decision, fail-open).
* Studio STT: verify whisper prebuilts via the release checksum index, like llama.cpp
Re-align the whisper.cpp prebuilt installer to install_llama_prebuilt.py's trust
model: instead of a committed whisper_prebuilt_pins.json, verify every download
against the release's own whisper-prebuilt-sha256.json checksum index, fetched
from the same GitHub release.
- parse_release_checksums / fetch_release_checksums / expected_sha256_for replace
the pins layer. The index is validated for schema/component and that its
release_tag matches the resolved release; an asset absent from it, a release
that does not publish it, or a manifest sha256 that disagrees with it all fail
closed to a source build.
- resolve_release_tag now resolves the newest published release at runtime (or an
explicit --published-release-tag), matching llama and the freshness check;
removed the pinned-default and the UNSLOTH_WHISPER_ALLOW_UNVERIFIED opt-in.
- Delete studio/whisper_prebuilt_pins.json and drop its pyproject package-data
entry (nothing to ship now, same as llama which has no committed pins).
- Adds test_install_whisper_prebuilt_checksums.py (index parser, fail-closed on
uncovered asset, tampered-manifest guard, newest-release resolution).
This is a same-origin checksum (integrity, not authenticity), identical to the
llama.cpp installer; pair releases with GitHub artifact attestations for provenance.
* Resolve whisper prebuilt release via the download host (no GitHub API)
Mirror install_llama_prebuilt.py's fast path: resolve the release tag from
the releases/latest redirect and fetch the manifest + checksum index from
constructed releases/download URLs, so the common install path makes zero
api.github.com calls (unauthenticated api.github.com is capped at 60 req/hour
per IP; the download host is not). Fall back to the GitHub API only on a 404,
malformed asset, or tag mismatch.
* Studio STT: coverage-aware whisper prebuilt selection via a shared core
whisper's select_artifact returned the first os/arch/backend manifest match and
ignored the SM-coverage fields the release manifest already carries, so a
Blackwell B200 (sm_100) was served cuda12-legacy (sms 50-61) -- runnable only via
forward PTX JIT. install_llama_prebuilt.py on the same host correctly picks
cuda13-newer.
Extract the coverage-aware selection into a shared, component-agnostic core under
studio/backend/utils/prebuilt/ (selection + GPU host-capability detection), lifted
from llama's linux_cuda_choice_from_release / _artifact_covers_sms / _sm_range and
generalised over a normalised artifact. whisper's HostInfo now records the GPU
compute caps + driver CUDA version (honoring CUDA_VISIBLE_DEVICES), and
select_artifact routes CUDA/ROCm through the shared selector: every visible SM
must be covered, the tightest-covering profile wins (Blackwell-aware runtime-line
ordering), ROCm matches the gfx target exactly, and an uncovered GPU falls back to
the CPU bundle. CPU/Metal/Vulkan keep first-match. The resolver JSON, exit codes,
and "already matches" contract are unchanged.
On the B200 the installer now resolves cuda13-newer, matching llama.
* Studio STT: gate whisper CUDA selection on the on-disk runtime, like llama
The prebuilt CUDA bundles are dynamically linked and intentionally do NOT ship
libcudart/libcublas -- they load the same runtime the host already has. So the
driver's advertised CUDA version is only an upper bound: a cuda13 bundle still
needs cuda13 runtime libraries present on disk. Port llama's on-disk runtime
scan (detected_linux_runtime_lines / detected_windows_runtime_lines) into the
shared core and intersect it with the driver-compatible lines in
select_cuda_attempts. A host with a cuda13 driver but only cuda12 runtime (e.g.
torch-cuda12) now correctly gets a cuda12 bundle instead of an unloadable cuda13
one; a host with no CUDA runtime at all falls back to CPU.
Fixes a glob bug in the port (any(Path(d).glob(p) for d in dirs) tests generator
truthiness, not a match) that made every major report present; add a real
filesystem test that exercises the scan.
* studio: harden shared prebuilt core to full llama parity
Apply the review findings on the shared coverage-aware prebuilt-consumer
core so whisper.cpp selection is exactly equivalent to the llama.cpp path.
hosts.py: port llama's CUDA_VISIBLE_DEVICES handling. A GPU hidden by an
index/UUID selector now reports has_usable_nvidia False instead of staying
usable, via supports_explicit_visible_device_matching plus the physical /
explicit-match branches, and _select_visible_rows now matches rows the way
llama does (index or UUID, gpu- prefix optional) and skips unmatched tokens
rather than keeping all rows. Adds the Linux /proc/driver/nvidia/gpus
fallback and has_physical_nvidia. Adds parse_macos_version.
runtime_libs.py: the Linux on-disk scan now requires the exact libcudart /
libcublas SONAME (libcudart.so.13), not a libcudart.so.13* glob, so a bare
versioned file without the SONAME symlink no longer counts as loadable.
Hardens the ldconfig parse against an empty left-hand side.
selection.py: fix the Blackwell/torch reordering so it keys on the covering
runtime lines (falls through to the torch preference when the covering lines
were filtered out), matching linux_cuda_choice_from_release. Corrects the
compatible_runtime_lines_for_driver docstring: the bundles do not ship the
CUDA runtime, so the driver version is only an upper bound and the caller
must intersect with the on-disk scan.
install_whisper_prebuilt.py: enforce a macOS artifact's min_os (new
HostInfo.macos_version) so a bundle that cannot load on the host OS version
is dropped. Keep resolver stdout to only the JSON line by leaving logs on
stderr in --resolve-prebuilt mode, and map an unexpected probe failure to
prebuilt_available False instead of a traceback.
Tests: new host-probe suite for the visible-device logic, exact-SONAME
runtime-scan cases, macOS min_os filtering, resolver stdout-only-JSON,
exit-code mapping, and the repo key.
* studio: fix whisper prebuilt selection + launch parity gaps from review
A parallel review surfaced integration defects where the whisper path could
select or launch a bundle that cannot run on a concrete host. Each is fixed to
match install_llama_prebuilt.py.
macOS min_os: the manifest labels macOS requirements as macos-<version>
(e.g. macos-14.0), which the version parser could not read, so the guard was a
no-op and a macOS-13 host would install the macos-14 Metal bundle. Strip the
platform prefix before parsing.
ROCm gfx detection: _detect_rocm_gfx returned the first gfx token and ignored
HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES. Since exact
ROCm matching treats that token as the active GPU, a mixed APU + dGPU host
(gfx1151 + gfx1100) with HIP_VISIBLE_DEVICES=1 installed the wrong archive. Route
through a shared pick_rocm_gfx_target (lifted from llama) that parses per-GPU
sections and honors the visibility vars (empty / -1 -> no AMD GPU).
--rocm-gfx override: recording the arch without setting has_rocm left the host on
its CUDA/CPU path so the ROCm bundle was never picked. --rocm-gfx now implies
has_rocm and clears NVIDIA state, like llama's _apply_host_overrides.
CUDA launch env: a CUDA bundle ships the ggml CUDA backend but not
libcudart/libcublas, and the sidecar launch env exposed only the bundle dir, so
on a host whose CUDA runtime lives only in the PyTorch wheels the selection would
gate cuda usable but the server could not load it. Add the CUDA-from-PyTorch
runtime dirs to the child loader path for CUDA bundles (bundle dir still first),
mirroring binary_env.
Also normalize a manifest artifact's supported_sms defensively (parity with
llama's parser) and document that blackwell_min_toolkit_for_caps is retained for
the Phase B llama Windows path.
Not changed (verified parity, not defects): Linux/Windows min_os is enforced
nowhere in llama (macOS only); the resolver is optimistic about the checksum
index and the install path verifies.
* studio: tighten prebuilt-core code comments
* studio: lift shared prebuilt installer core out of the whisper installer
* studio: reuse the llama.cpp prebuilt installer machinery for whisper
* studio: unify llama and whisper prebuilt installers on a shared descriptor core
* studio: consolidate prebuilt installer tests into the shared core suite
Grow tests/studio/install/test_prebuilt_core.py from 62 to 164 tests so every
component-agnostic behavior runs against both descriptors: the full seven
profile CUDA release matrix (multi-GPU, on-disk runtime gating, shuffle
stability, missing SM metadata, dotted SM normalization, no-driver fallback
policy), the ROCm gfx family matrix, macOS min_os gating and its helper,
backend resolution incl. cpu-fallback precedence and Intel-mac auto detect,
checksum-index non-object and plain-lookup cases, the tar symlink/hardlink
extraction guards moved from the llama suite, and the compute-cap, visible
device, runtime-line and Blackwell helper value tables moved verbatim from
the llama characterization suites.
Delete only tests whose exact behavior the master now asserts for the same
component: 40 pure-alias helper cases in test_selection_logic.py (replaced by
value-identical master tables plus an alias-identity pin), 6 extraction moves
and the master-absorbed zip-symlink case in the llama logic suite, 3 routing
twins in test_rocm_support.py already pinned byte-for-byte in
test_selection_logic.py, the 2 Blackwell helper tables in the backend resolve
suite, 28 whisper logic tests and 10 whisper checksum tests re-asserted by
the master whisper parameterization. Wrapper wiring pins, the llama release
plan dialect, fingerprints and every llama-only behavior stay untouched.
* studio: dedupe sidecar and update helpers into the backend prebuilt package
* studio: chain whisper.cpp prebuilt updates onto the llama.cpp update flow
* studio: consume paired slim whisper prebuilts via the llama ggml runtime
* studio: serve every whisper backend from slim prebuilts
* studio: drop the whisper fat per-accelerator selection chain
unslothai/whisper.cpp releases are slim-only from v1.9.1-unsloth.2: one
ggml-less bundle per os/arch, paired to the llama.cpp prebuilt that provides
every ggml backend. Delete the whisper-side fat CUDA/ROCm/metal/vulkan
selection glue; keep slim selection + pairing, link_ggml_runtime, and one
legacy shape, the published fat CPU bundle of an explicitly pinned pre-slim
release. Exit 2 now reads as prebuilt unavailable (whisper never source
builds); setup already treats it that way.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Wire libomp runtime DLL alongside ggml in slim whisper installs
llama's clang-built windows-arm64 ggml-base.dll imports
libomp140.aarch64.dll, shipped in the llama bundle but not a system DLL.
Without it next to whisper-server.exe the loader fails with
STATUS_DLL_NOT_FOUND before main. MSVC x64 links vcomp140.dll from
System32 and Linux ggml uses system libgomp.so.1, so only windows-arm64
was affected. The empty-runtime guard still requires a real ggml
library; libomp alone is not a pairing.
* studio: drop whisper-side fat-selection support structure
Slim whisper bundles are selected per os/arch only; all accelerator
capability comes from the installed llama.cpp prebuilt, whose installer
already did the coverage-aware selection. Remove the machinery that only
existed to pick among fat per-accelerator whisper bundles:
- prebuilt_core: delete the generic CUDA/ROCm coverage selection
(select_cuda_artifact, select_rocm_artifact, ArtifactView adapters,
detected_cuda_runtime_lines, the exact-SONAME linux probe) that no
shipped component routes through; llama keeps its own selection chain
and whisper shadows select_artifact with the slim-only version.
select_artifact is now a plain os/arch/backend first-match.
- install_whisper_prebuilt: drop the HostInfo CUDA fields
(compute_caps, driver_cuda_version, torch_runtime_line) and the torch
runtime probe that populated them; nothing reachable reads them, and
the resolver payload sources runtime_line from the artifact.
- whisper_cpp_update: delete the standalone start_update job worker;
whisper applies only run as the chained phase of the combined
llama+whisper update. The status payload keeps its job field (idle).
- routes/whisper: drop the progress logger that could never fire.
- tests: remove tests of the deleted paths and tests duplicating the
descriptor-parameterized core suite or the llama freshness suite.
Contracts unchanged: resolver JSON keys, exit codes, marker fields,
pairing logs, and the pinned pre-slim fat CPU escape hatch.
* Address review feedback on the whisper prebuilt update and install paths
- Pin the chained whisper phase to the release the freshness check
offered, so the download-host latest pointer cannot reinstall an
older build in a loop
- Wire the whisper prebuilt install into setup.ps1 (Windows setup
previously skipped it entirely)
- Treat a non-executable server or missing wired ggml libraries as a
broken install instead of reporting already matches
- Keep whisper sidecar reloads out of the job-level reload flag and
resync chat state after a partial chained update that unloaded llama
- Repoint home and profile vars for the whisper-server subprocess at a
managed scratch dir and drop credential-store pointers
- Clear the prebuilt marker before the opt-in source build overwrite
- Write the prebuilt marker with explicit utf-8 encoding
* Tighten comments in the whisper prebuilt consumer
* Harden the Windows whisper setup phase and the chained update edges
- setup.ps1: honor WHISPER_SERVER_PATH / UNSLOTH_WHISPER_CPP_PATH /
UNSLOTH_SKIP_WHISPER_INSTALL, run the custom-home ownership guard
before the atomic install, and forward the release-tag pin and ROCm
hints like setup.sh
- sidecar: a cpu-selected install launches whisper-server with --no-gpu
(slim wiring links every llama backend, so the flag is what keeps a
deliberate CPU choice off the GPU)
- chained update: leave whisper unpinned on macOS (the llama phase can
walk back there, and a newest-tag pin could be an impossible pairing
on every retry) and treat installer exit 2 as kept-existing-runtime
instead of failing the combined job
- job.to_tag now comes only from the llama phase, so a whisper-only
round cannot report a llama update that never ran
* Fix slim whisper runtime follow-ups
* Address remaining whisper update reviews
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address remaining prebuilt update reviews
* Fix remaining chained update reviews
* Fix remaining whisper runtime review edges
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
---------
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
Adds a configurable Hugging Face model download cache location to Unsloth Studio, selectable from Settings, with per-cache download manifests, scoped deletion, and read-only inventory of previously selected caches.
* refactor(studio): move chat model picker into features/model-picker
Relocate model-selector + its support files from components/assistant-ui
into a self-contained features/model-picker feature (own barrel), mirroring
the modular Hub layout. Pure move + import repoint; no behaviour change.
* feat(model-picker): add per-model config persistence layer
Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType,
specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted
to localStorage (unsloth_model_configs) with schema versioning + LRU budget.
KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple).
Reuses features/hub/lib/model-identity for normalization; adds storage-key layer
and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted).
* feat(picker): modular backend for chat-template validate + default fetch
New studio/backend/picker package (schemas/service/routes) mounted at /api/picker:
- POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives)
- GET /api/picker/chat-template/{model_name} (default template from tokenizer_config.json,
reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec)
Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend
changes to the existing inference load route (per-model load fields already supported).
* feat(model-picker): bind picker on-device list to shared hub inventory
Picker now sources cached + local models from useHubInventory (the Hub's shared
store) via a thin adapter, replacing its own /api/models/* fetchers + module
caches. Hub, download manager, and picker now share one source of truth, so
completed downloads reflect in the picker automatically. Partial/live-download
rows are filtered from the cached lists (unchanged rendering). Local naming/search
preserved via additive LocalInventoryRow modelId/displayName. Variant expander,
scan-folder management, recommended-fit, search, external providers untouched.
Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical
(hub cached rows carry no mtime); default 'recent' (load-time) sort preserved.
* feat(model-picker): per-model config step inside the picker
Picking a (non-external) model now opens an in-picker config view built from
main's current load controls (context length, KV cache dtype, speculative
decoding, draft tokens, tensor parallel) plus a chat-template editor backed by
the picker validate/default endpoints. 'Remember for this model' persists the
config per model+variant; Run forwards the config to the existing load flow via
meta.config. External models bypass the step. Two-view orchestration lives in
model-selector (single interception point); pickers.tsx call sites untouched.
trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent.
* feat(chat): apply/persist per-model config through the load flow
handleCheckpointChange threads meta.config into the selection; stageOrLoad and
the autoload/Hub-run paths now apply the picker config (explicit pick or saved
remembered config) via applyPerModelConfigToRuntime before staging/loading, with
keepSpeculative set so a remembered speculative mode survives the model switch.
Replaces the old remembered-load-settings seeding (resolveInitialConfig now the
single source). SelectedModelInput carries config.
* refactor(chat): remove per-model load config from the right sidebar
The load knobs (context, KV cache, speculative, draft tokens, tensor parallel)
and the chat-template editor now live only in the picker config step. The sheet's
Model section keeps the staged Load/Cancel flow (config is applied at pick time);
sampling params, system prompt, and RAG are unchanged. Deletes the superseded
remembered-load-settings module + the store's applyRememberedLoadSettings action,
removes the now-dead sheet state/imports, and points the settings reset at
unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped).
* fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section
The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight
into the right-sidebar Run-settings flow -- the old 'configure before load' path
now fully replaced by the in-picker config step. Removed the gear + its component.
Also gate the sheet's 'Model' section to staged picks only (pendingSelection):
after the load-knob strip its content is staged-only, so it was rendering an
empty section header whenever a model was merely loaded.
* chore(chat): remove dead per-model-config setters + modelControlsDisabled
After the load-config UI moved into the picker, the store's per-model setters
(setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/
setCustomContextLength/setChatTemplateOverride) had zero callers
(applyPerModelConfigToRuntime writes via setState), and the sheet's
modelControlsDisabled was unreferenced. Verified dead across the whole tree.
* fix(chat): config-step Load actually loads (ignore Load-on-selection)
Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config
step's load went down the deferred-staging path -- opening the right sidebar with
'<model> is staged, not loaded yet / Choose Load model'. The in-picker config step
IS the deliberate load action, so its Load now loads immediately (or downloads +
auto-loads when not cached) regardless of the toggle. Renamed the button
'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle.
* refactor(chat,hub): retire 'Load on selection' — config step is the only load flow
The in-picker config step (and the Hub Run button) now fully supersede the old
stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere:
- chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when
not cached (the previous default behaviour, now universal).
- hub Run: drops the stage branch; downloaded GGUFs load directly with their saved
per-model config (no collision with the chat config step — both end at selectModel).
- store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and
its settings-reset entry removed.
- staged sidebar section is now a download-progress view (auto-loads on completion).
No manual staging remains; stageModel is used only for background auto-load downloads.
* feat(model-picker): default chat template from GGUF + thread variant through config flow
Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template
in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through
the picker service, /api/picker/chat-template route, frontend templates API, and
use-model-defaults so the right variant's template is fetched.
Also refine the picker config-page/model-selector wiring, drop the dead
ggufNativeContextLength runtime path, and add the per-model-config storage keys to
the settings prefs export.
* feat(model-picker): read safetensors chat template + hide editor where it has no effect
Resolve the default chat template for safetensors models: prefer the modern
chat_template.jinja, fall back to the tokenizer_config.json chat_template field,
then chat_template.json (multimodal processor), then the GGUF embedded template.
Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch.
Hide the chat-template editor in the picker for safetensors models — the override
is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors
currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays
for when the safetensors apply path is wired up in a later branch.
* fix(model-picker): set legacy-migration flag only after the write succeeds
Set unsloth_model_configs_migrated only once writeMap confirms the migrated
map persisted, so a quota/storage failure no longer marks migration done and
silently drops the user's pre-existing remembered settings — the next load retries.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* MVP model picker fixes
* MVP picker config fix
* MVP safetensors config
* MVP max seq config
* MVP max seq fix
* Fix static max tokens cap ignoring model context
* Fix picker GGUF scan parity
* fix(studio): harden model picker config loading
Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload.
* Fix model picker config flow
* Fix model picker config loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Avoid recursive per-model config migration reads
* Apply the displayed context length when loading a GGUF
* Fix template validation, cached template lookup, and failed load rollback
- Validate chat templates with the loopcontrols extension so templates
that use break or continue tags pass the picker validator, matching the
inference renderer that already accepts them.
- Read the default chat template from the newest cache snapshot rather than
an arbitrary iterdir order, so an older cached revision no longer prefills
a stale template.
- Capture the runtime per-model config before a load and reapply it when the
load fails, so a failed switch leaves the active model context, KV cache,
template, and speculative settings as they were.
* Make chat template view only for safetensors models
Custom chat template overrides are applied at inference only for GGUF
models, which pass the template to llama-server. The safetensors backend
renders with the model built-in template and ignores the override, so
editing it would save a value that never loads. For safetensors the
config page now opens the template as a read-only preview with a note
that editing is not available yet. This can become editable once
inference support for custom safetensors templates lands in main.
* Fix model picker config edge cases
- Restore prior runtime config when a load no-ops for the active model
- Cap the picker validator request body via the protected prefixes
- Keep the GGUF context slider max above the loaded context
- Fetch subfolder chat templates for uncached Hub repos
- Show the compare side config when reopening the picker
* Keep saved GGUF context above the fallback ceiling
* Show the model config in the run settings sidebar
* Fix model config sidebar reset and context slider
- Stack the remember toggle and action buttons in the sidebar
- Reset the config to defaults instead of the loaded values
- Fetch the native context so the slider max is not the loaded value
* Fix model picker config and download regressions
- Run picker chat template routes off the event loop
- Depth and root guard local template directory scans
- Restore download manager flow for uncached hub picks
- Apply per model context length on reload
- Import model picker symbols from the feature barrel
* Fix model picker config and cached download sorting
- Restore load settings when a Hub run is rejected mid load
- Reuse one NumericValueInput instead of a duplicate copy
- Fix double decode of the model name in the template route
- Remove the unused reset-to-loaded settings action
- Fix cached model download sorting
* Fix model picker per-model config edge cases
Honor a saved or typed max seq length above the model's native context so
RoPE extended values are no longer clamped and silently overwritten. Allow
typing past native while the slider keeps native as a soft ceiling.
Guard the fetch success paths in use-model-defaults against an aborted
signal, and refetch when the HF token changes.
Hash the chat template content in the sidebar remount key instead of its
length. Enable reset for a GGUF whose native context is unknown, and floor
the context slider max so it can never fall below the min.
* Fix GGUF context auto-fit and gated model config token
Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit.
Send the HF token as a query param so gated safetensors models resolve their max position embeddings.
Derive model default state during render to drop the set-state-in-effect calls.
* Fix native GGUF context ceiling and guard picker template reads
Restore the native context store field so the sidebar slider keeps the
full ceiling for drag and drop GGUFs. Limit local chat template reads to
the browse allowlist, skip malformed repo ids, and drop unused model
picker exports.
* Fix model picker lint boundaries
* Fix model picker review findings
Chat template editor never seeded its draft. Radix only calls onOpenChange
from internal events, so the seed in the nextOpen branch was dead and a model
with a saved override opened empty. Saving then cleared the override. Drop the
dead branch, treat draft as an untouched sentinel, and reset it on every close.
Uncached Hub picks could auto load a model after the user left the chat. Main
detached the staged pick on route exit and on chat context change. Carry the
context key on the pending pick and skip the load when it no longer matches.
Also clear configTarget when the picker closes, restore the onUpdated ref so
variant rows stop resubscribing on every parent render, skip the LRU write when
the entry is already most recent, import NumericValueInput relatively, and drop
the unused ModelUpdateAction barrel export.
* Preserve GGUF context on active reload
* Fix model picker per-model config regressions
- Stop reloading the already loaded model on re-pick
- Hide infra models from the chat picker
- Detect vision support on cached GGUF repos
- Honor saved maxSeqLength on auto load
- Restore default chat template for local GGUFs
- Warn on save failure and revert config on cancel
- Refetch picker inventory on open
- Persist read only per model config safely
* Fix stale model auto load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker numeric input sizing and constraints
Size value inputs to their content so long context lengths are not clipped,
restrict them to numeric characters, and stop the speculative decoding label
from truncating in the sidebar.
* Fix picker CI tests and harden chat template resolution for PR #6647
- tests: point the descender guard at the moved model-selector.tsx path
- tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate
- picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged)
- compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Protect future-schema per-model configs from deletion for PR #6647
savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast.
* Protect future-schema per-model configs from quota eviction for PR #6647
The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them.
* Fix GGUF context persistence, compare context, and rollback settings for PR #6647
Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (f4838782cb reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override.
shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context.
use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults.
* Preserve native path token when reloading the active model for PR #6647
handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly.
* Make picker template validation resilient and accept HF generation tags for PR #6647
Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag.
* Honor remembered compare config and parse processor chat_template.json for PR #6647
* Fix failed-load rollback context and processor template map fallback for PR #6647
* Restore speculative decoding config on failed-switch rollback
When a model switch fails after the previous model was unloaded, the
rollback reload restored tensor_parallel, KV cache dtype and the chat
template override, but omitted speculative_type and spec_draft_n_max and
cleared their loaded shadows to null. The previous model therefore came
back running at backend defaults (speculation off) while the UI still
showed it enabled, and the status resync confirmed the off state. Resend
the previous model's speculative settings in the rollback load and keep
the store's active and loaded speculative fields in sync with them.
* Reset max sequence length when a model has no saved config
applyPerModelConfigToRuntime reset every per-model field except
maxSeqLength, which it only wrote when the incoming config had one.
maxSeqLength is the sole field carried on store.params, so selecting a
model with no remembered config left the previous model's value in place
and later loaded the new model at that leaked length. Fall back to the
standing default so an unremembered model loads at its own default.
* Surface a message when a variant update cannot start
startManagedUpdate handled the conflict and error start outcomes but let
busy fall through as if the update began, so the confirm dialog closed
with no job created and the cached variant stayed stale. Show an info
message when the repo is busy with a sibling transfer so the click is
not silently dropped.
* Keep per-model speculative choices out of the global default
A staged load with a per-model or one-off config sets keepSpeculative,
which already skips reading the global speculative preference. The
matching save still ran unconditionally, so the model-specific choice was
written to the global unsloth_chat_speculative_type and a later model with
no saved config started from it instead of Auto. Skip saveSpeculativeType
when keepSpeculative so the per-model choice stays isolated.
* Seed non-active model settings from the app default max length
The Run settings page captured initialMaxSeqLength from the loaded
model's runtime params and fell back to it for a model with no saved
config. Opening settings for a different, unloaded model and clicking
Load then sent the active model's context (for example 64k) instead of
the 4096 default, risking validation failures or OOMs. Seed the default
for non-active models and keep the runtime value only for the active one.
* Prefer sidecar tokenizer chat template over the GGUF copy for variants
_chat_template_from_dir returned the embedded GGUF template first when a
variant was selected, reversing the tokenizer-first precedence of the
no-variant path. A model whose chat_template.jinja or tokenizer_config.json
supersedes a stale embedded template then got the wrong template on
variant selection. Keep tokenizer files first regardless of variant; the
variant only picks which GGUF is the fallback. Adds regression tests for
both the tokenizer-wins and gguf-fallback cases.
* Keep per-model speculative choices load-local in autoload and compare
The interactive load path treats a per-model speculative choice as
load-local and skips writing it to the global default. Autoload and
generalized compare still called saveSpeculativeType unconditionally, so a
remembered off or ngram setting leaked into unsloth_chat_speculative_type
and later models with no saved config inherited it. Persist the global
preference only when the value came from the global settings.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context
* Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it
* Studio: drop the merge's orphaned staged-model store helpers and unused alert imports
The main merge left isPendingGguf and pendingSelectionMatches referencing the
removed PendingModelSelection type, and the alert-dialog/alert imports unused
after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed.
* Studio: cache a null default chat template so the viewer stops re-fetching it
A model with no sidecar or embedded template resolves to a terminal null, but
that result was never cached, so reopening the template viewer re-ran the
backend and Hugging Face lookup every time.
* Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context
A GGUF loaded from a local file or custom folder has no variant label, so the
run-settings panel treated it as non-GGUF and clamped Max Tokens to the session
max_seq_length instead of the loaded GGUF context. Detect it via the reported
GGUF context and the .gguf checkpoint suffix, matching the chat page.
* Studio: prompt to re-select a local model file when its lease expired before reload
A file-picked GGUF is reachable only through a native path token that the
desktop host prunes after a TTL. Reloading reused that token blindly, so a
reload long after the initial load failed with an opaque error. Track the
token's expiry and, when it has passed, ask the user to re-select the file
instead of attempting a doomed reload.
* Fix descender-clipping test to tolerate sidebar layout utilities
The sidebar account-block div carries layout utilities (min-w-0, flex-1)
between 'flex' and 'flex-col', so the descender-clipping guard's regex,
which required 'flex' immediately followed by 'flex-col', no longer matched
and the test failed to locate the account-block div. Generalize the prefix
to allow intervening flex utilities while still capturing the leading-*
class before the collapsible visibility utility and asserting leading-tight,
so the guard against clipped glyph descenders is fully preserved.
* Harden picker chat-template resolution
Enforce the 64 KiB chat-template contract at the validate endpoint's request
model so a direct caller cannot submit a template far larger than the frontend
allows (MaxBodyMiddleware only bounds the whole request body, not this field);
oversized templates now return a clean 422.
Apply sidecar-over-GGUF template precedence globally across cached snapshots
instead of per snapshot. A repo with multiple cached revisions previously
returned the first snapshot's template, so a newer GGUF-only revision could
win over an older revision's maintained chat_template.jinja sidecar, which
contradicted the documented intent that sidecars supersede the embedded copy.
* Guard per-model config against future-schema and lossy migration
Two forward-compatibility gaps in the versioned per-model config store:
- The load/apply path returned and normalized a stored record without checking
its schema version, so a record written by a newer client was reinterpreted
under the current schema and applied to a live model load, even though save,
delete and eviction all refuse to touch future-schema records. Reject
future-schema records on load too.
- The one-time legacy migration enforced the storage budget without protecting
the entries it had just migrated and set the completion flag unconditionally.
When storage was already full of future-schema records (which are unevictable
by an older client), the migrated entries were the only evictable ones and
could be dropped while migration was still marked complete. Protect the
migrated keys during eviction and only mark migration complete when they
survive, so it retries once space frees up.
* Discard chat-template validation results after the dialog closes
Server-side template validation is async, but closing or cancelling the editor
did not abort it, so a late-arriving valid response still called onSave and
applied a template the user had already dismissed. Track a validation token
that is bumped on close and ignore any validation result whose token is stale.
* Record native lease expiry when loading a picked GGUF from the chip
The pending-native-model chip loaded via stageOrLoad directly, bypassing
loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a
chip-loaded file. A later reload then either skipped the lease-expiry guard
entirely (expiry left null) or compared against a previously loaded file's
stale expiry, so reload could reuse an already-pruned token or wrongly block a
still-valid one. Route the chip through loadNativeModelIntent, which builds the
same selection and records the expiry.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prefer sidecar template for a directly selected local GGUF file
A direct .gguf file path read its embedded chat template without checking the
parent directory for a maintained sidecar (chat_template.jinja /
tokenizer_config.json), while directory and variant selections already prefer
the sidecar. That let the config editor preview or save a stale embedded
template for the same model depending on how it was selected. Check the parent
directory sidecars first, then fall back to the embedded copy, and cover both
paths with tests.
* Resolve cached chat template per revision, newest first
The earlier change searched every cached snapshot for a sidecar before
considering any snapshot's embedded GGUF template, which let an obsolete sidecar
from an older revision override the newest revision's template. Restore
per-snapshot resolution (newest first): a revision's sidecar still supersedes
its own embedded GGUF copy, but a newer revision is no longer overridden by an
older revision's sidecar.
* Preserve autoload transport conflicts and surface background busy downloads
- When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound
instead of clearing it. Clearing it re-keyed the download surface and its
cleanup cancelled the conflict the toast tells the user to resolve, so the
Hub resume affordance was gone the moment it appeared. Return early on
conflict, mirroring the started branch, so resolving it from the Hub still
auto-loads on completion.
- The background-download branch handled started and conflict but silently
dropped a busy outcome, leaving the user with no feedback when a peer variant
of the same repo was already downloading. Surface the same busy toast the
autoload path uses.
* Fix context length, GGUF template, fetch state and lease expiry bugs
Keep explicit context length values instead of collapsing to null at
native. The collapse made the slider jump back at the native maximum
and made Reload load the previous context instead of the chosen one.
Prefer the first split when resolving a GGUF without a variant. Later
splits carry no chat template metadata, so picking the largest file
could return no template for a sharded model.
Clear stale fetch state when template and metadata lookups retry, so
a previous terminal error is not shown while a new fetch is running.
Record native path lease expiry together with the token when a load
commits. The expiry was written by only one load path and even when
the load did not start, so a reload could be blocked with an expired
file message for a still valid token.
* fix(model-picker): resolve review findings across config, inventory, and templates
- Apply remembered per-model config in the training-compare chat handoff so a
prior model's customContextLength no longer leaks into the next load
- Match GGUF variant labels with the inventory extractor too, so cached
no-quant-token files resolve their default chat template
- Show "Auto" instead of a fabricated 32768 when native context is unknown
- Reuse the identical staged auto-load object on same-pick so a re-pick during
download pre-flight no longer disarms auto-load via "busy"
- Union supports_vision when deduping cross-cache inventory rows
- Serve hidden-model needles from a new GET /api/hub/hidden-models endpoint and
merge them client-side, covering runtime-configured RAG embedders
- Clamp GET chat templates to MAX_CHAT_TEMPLATE_BYTES (route + jinja sidecar),
matching the validate endpoint's contract
- Lower-clamp stored customContextLength to shared CONTEXT_LENGTH_MIN
- Wipe unsloth_chat_load_on_selection in Settings "Reset all"
- Drop stale pendingHasContext comment describing deleted staging machinery
* Fix stale defaults cache, token in query string and rounded up context ceiling
Refresh cached chat template and max position data when a model update
completes. Send the HF token for model config requests in the dedicated
header instead of the URL. Snap the native sequence length ceiling down
to the nearest step so the slider cannot exceed the declared maximum.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix compare pane reverting active checkpoint on non-GGUF load
Re-read runtime params after setCheckpoint so the fresh checkpoint is
kept instead of being overwritten by the pre-setCheckpoint snapshot.
* Send the HF token via header for the vision and embedding checks
checkVisionModel and checkEmbeddingModel still passed the HuggingFace
token as a ?hf_token= query parameter, so it landed in server access
logs, proxy logs, and browser history. Move them to the
X-Unsloth-HF-Token header like getModelConfig already does, and accept
the header on the check-vision and check-embedding routes with the
existing query parameter kept as a fallback for older clients.
* Cap the chat template on the model load path
The load endpoint accepted an unbounded chat_template_override, so a
direct caller could hand llama.cpp an arbitrarily large Jinja template
even though the frontend, the validate endpoint, and the read paths all
enforce the 64 KiB limit. Reuse MAX_CHAT_TEMPLATE_BYTES in the
LoadRequest validator, rejecting oversized templates with a fast
character-count check before the exact UTF-8 byte check.
* Protect existing per-model configs during legacy migration
When the one-time legacy import pushes the store over budget, eviction
now protects the entries the user already has and drops only the
just-migrated legacy entries, so importing old load settings can never
discard a newer per-model config.
* Reset clears the context override instead of pinning the native value
Reset wrote the discovered native context into customContextLength for
GGUF models, but isDefaultConfig treats any non-null customContextLength
as an explicit pin, so Reset with Remember enabled persisted a fixed
context and future loads stopped using the native auto context. Reset
now restores the full default (customContextLength null); the native
value is still shown through the existing display fallback.
* Bound chat-template sidecar reads to a size limit
The chat_template.json, tokenizer_config.json, and Hub-downloaded sidecar
readers decoded and json-parsed the whole file before the extracted
template hit the 64 KiB response cap, so an oversized metadata file could
exhaust memory. Read them through a bounded reader (4 MiB envelope) that
returns None when the file is larger, matching the existing chat_template.jinja
size guard. Adds tests for oversized tokenizer_config.json and chat_template.json.
* Keep the native-path token and lease expiry in sync
Rollback after a failed reload restored the previous token but left the
failed load's expiry in the store, so a later reload could be falsely
blocked as expired (token A paired with load B's lease). Restore the
previous lease alongside the token, and clear the expiry wherever the
token is cleared on a non-GGUF transition, so the two never diverge.
* Clear the native file lease on compare-pane loads
* Studio: add regression tests for the model-picker per-model-config
Guard the specific regressions that reverted the predecessor change:
- backend pytest (studio/backend/tests/test_model_picker_regression.py):
infra-model hiding, HF token via header with query fallback, and the
chat-template byte caps.
- source contracts (tests/studio/test_model_picker_contracts.py): the token
stays out of the URL, the context ceiling is floored, the native lease is
cleared on compare-load and restored on rollback, the default caches key on
the inventory version, and the hidden needles stay present.
- Playwright E2E (tests/studio/playwright_model_config.py) wired into
studio-ui-smoke.yml on port 18898: Context Length persists across a reload,
Reset clears the stored override, and infra models are absent from the picker.
- optional GPU-gated inference smoke (tests/studio/test_gpu_inference_smoke.py)
that auto-skips on GPU-less CI and stays short on a GPU.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: model pinning, row menus, hub inference settings, and inventory filters
Pinning
- Add a pinned models store (localStorage) with repo and per-quant pins
- Pinned section in the model selector's On Device list and the hub inventory,
with newest pins first so Pin to top lands on top
- Deleting a repo drops its pins
Row menus
- Replace loose row icons with a shared 3-dots menu (pin, reveal in file
manager, copy identifier, copy path, delete) on picker rows, hub quant rows,
the hub run bar, and on-device inventory rows
- Menus only render for models actually on disk; platform-aware reveal labels
- Backend: cached-model-path and reveal-cached-model endpoints resolving
managed HF-cache repos only
Hub inference settings
- Gear in the GGUF run bar opens an Inference settings dialog reusing the chat
page's controls: model config (context length, KV cache, speculative
decoding, chat template), system prompt, reasoning, sampling, tools and
retrieval
Inventory
- Model-type filter (text, vision, embedding, STT, TTS, diffusion) beside the
sort pill, both with a sort icon, capped widths and truncation so the
On device heading never wraps
- Unsloth-owned repos without an upstream provider logo fall back to the
Unsloth mascot avatar
- Discover / On Device tabs widened; hub search bar narrowed to match
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: revert the Unsloth mascot avatar fallback
Unsloth-owned repos without an upstream provider match go back to the
colored-initial tile, and unslothai is no longer a relabeled owner.
* Studio: run-bar options on single models, and aligned type/capability filters
- Give single-model (non-GGUF) run bars the same 3-dots options menu and
settings gear as GGUF, at repo level
- Drop Pin to top from the run-bar menus; pinning stays in the On Device list
- Add an Image to text (diffusion) capability with detection, and surface it
in both the hub Discover capability filter and the On Device type filter
- Align the On Device type filter with the Discover capability options and
share the same detection so both dropdowns match
* Studio: apply hub inference config on reload, eject action, and run-bar polish
- Fix inference settings not applying: the hub dialog now writes the config to
the runtime before reload, matching the chat page (selectModel reads runtime
state, not the selection)
- Order the settings gear before the 3-dots menu in the run bars
- Replace the loaded-model run-bar action (New Chat) with Eject, wired through
the inspector to the hub's ejectModel
- Truncate the results heading so a long search query clips instead of
overlapping the header pills in split view
- Use a plain magnifying-glass icon for the no-results empty state
* Studio: fix GPU settings loss, load guards, pins, filters, and cached paths
Reloading a model from the chat sidebar or the hub gear dialog rebuilt the
per-model config without the GPU memory fields, so manual GPU layers, MoE
placement, and the GPU pick were reset on every reload and could be saved
over a remembered config. The active config now comes from a shared
useActiveModelConfig hook that carries the GPU fields for GGUF models, and
the sidebar remount signature tracks them through a shared gpuFieldsSignature
helper.
The in-flight load guard lived in a ref inside each useChatModelRuntime
instance, so the chat page, hub page, and gear dialog could not see each
other's loads. A load started from the gear dialog left the hub page free to
eject the model mid-reload or start a second concurrent load. The runtime
store now records the loading pick, selectModel checks it across instances,
and ejectModel refuses with a toast while any load is in flight.
The cached-model-path endpoint matched GGUF files by basename and excluded
only mmproj, so Copy path and Reveal could return an MTP drafter for a quant
and returned 404 for directory layouts like BF16/model-00001.gguf. Variant
files are now resolved from snapshot-relative paths with the same drafter,
mmproj, and big-endian exclusions as the load path, shared through a new
_main_variant_gguf_label helper.
Hub and picker fixes:
- rename the diffusion capability label from "Image to text" to
"Image generation", since it detects image generators
- validate pinned quants through the cached variant listing, keep the last
verified set while revalidating, and drop deleted quants immediately
- pass a measured scroll margin to the on-device virtual list so rows past
the overscan stay visible below the pinned block
- keep the delete menu for stopped partial safetensors downloads
- give the inventory type filter a reset in Clear filters, a truthful empty
state with a Show all types action, and hide it on the datasets view
- order picker pinned rows by pin recency, include pinned matches in the
empty-state check, and sync pins across browser tabs
- count only the visible rows in the On device list header
Tests: contract checks for each fix in test_model_picker_contracts.py and a
backend test for the variant label selection.
* Studio: reveal cached models in Windows Explorer under WSL
The reveal endpoint only branched on macOS, Windows, and generic Linux.
Under WSL the Linux branch spawned xdg-open, which is missing on a stock
distro without a Linux desktop, so the request failed with a 500 and the
UI showed a failed to open file manager error.
WSL is now detected with the existing helper and the path is converted
with wslpath before opening explorer.exe, selecting the file the same
way native Windows does. Directories open directly. When interop is
unavailable the old xdg-open fallback still runs. The macOS, native
Windows, and native Linux branches are unchanged, and the Tauri app is
covered since its hub reveal calls this same local endpoint.
Tests: platform guards for the WSL reveal, the interop fallback, and
the unchanged native Linux behavior in tests/studio/test_reveal_file_manager.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Adjust model picker row spacing and cogwheel hover consistency
* Studio: exact hidden model ids and newest revision cached paths
A custom RAG embedder repo was published to the frontend as a basename
substring needle, so a generic name like org/model could hide unrelated
models in the pickers. The hidden-models endpoint now sends full repo ids
that are matched exactly.
Copy path and Reveal picked a GGUF variant from an arbitrary cache
revision when the same file existed in more than one. The newest revision
now wins, matching the whole repo lookup.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker GPU config, metadata, and cache selection
Load each compare model with its saved GPU memory mode, GPU layers, CPU MoE layers, and selected GPU IDs. Reconcile saved GPU IDs with the current hardware. Include the active native GGUF path token in metadata checks. Search all Hugging Face cache roots when resolving cached models and select the largest visible cache entry. Remove obsolete barrel exports and the staging-only GPU memory helper.
* Studio: hide hub inference settings gear for now
The cogwheel in the hub download cards is out of scope for this PR. The
dialog component stays in place and a TODO marks where the button
returns in a future PR.
* Refresh hidden model matchers
* Fix GGUF detection, compare context pin, and picker delete staleness
Treat any pick with a GGUF variant as GGUF in selectModel so the first
load after downloading an uncached quant validates and sizes with the
right GPU settings instead of unloading the current model on a wrong
preflight. Variant picks now also set isGguf on their selection meta.
Stop compare panes from inheriting the active model's context pin when
their own saved config says Auto. Null context in a remembered config
now means no pin, matching how the pane settings are shown.
Route picker deletes through the hub inventory client, which
invalidates the HF cache scan and the variants cache. The legacy
delete route left the scan cache warm, so deleted models reappeared
in the picker until the TTL expired. Removed the now unused legacy
delete client and updated the contract test to match.
* Studio: fix stale GGUF load-marker ordering test
The load-in-flight marker still precedes the hub-download guard and the
unload, but the llama_extra_args inheritance that used to sit between the
marker and the guard now runs ahead of the GGUF branch, so it is no
longer a landmark inside the sliced source. Drop it from the ordering
assertion and keep the marker -> guard -> unload invariant.
* Studio: fix per-model config edge cases in compare loads and saved defaults
- chat-settings-sheet: gate the MTP fallback note and context/VRAM warning on
the broader isGguf (variant, loaded gguf context, or .gguf suffix) instead of
isLoadedGguf, so direct-file and custom-folder GGUF loads still surface
those diagnostics.
- shared-composer: a compare pane's context now comes from its own config only
(a saved pin, else null for Auto/native). It no longer inherits the active
model's shared snapshot, which resolveFitMaxSeqLength treated as an explicit
pin and could load a pane at another model's context (VRAM/OOM), matching the
single-model load path.
- model-config-page: when an auto-fit GGUF is saved with fixed GPU layers
(Manual) and Remember, pin the displayed fitted context so a later fresh load
keeps the placement instead of sending native/0 and recreating the OOM.
- per-model-config: treat Auto GPU memory mode and Auto/default speculative type
as follow-global defaults; do not persist them as per-model overrides so later
global preference changes keep applying.
* Studio: gate vision capability on GGUF projectors and bound remote template downloads
- cache_inventory: only mark a cached repo vision-capable when it holds an actual
GGUF mmproj projector, not any file whose name merely contains "mmproj" (e.g.
mmproj_config.json), matching the runtime's GGUF-only projector detection.
- picker/service: pre-check the remote file size before downloading an uncached
repo's chat template / tokenizer config, so a maliciously large sidecar is
skipped instead of fetched and retained in full, mirroring the size gate the
local-file path already applies.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add source-contract guards for the per-model-config edge-case fixes
Guard the four per-model-config fixes against silent regression in CI:
- local GGUF diagnostics gate on the broad isGguf, not the variant-only isLoadedGguf
- fixed-layer GGUF saves pin the displayed context
- Auto GPU mode and Auto/default speculative are not persisted as per-model overrides
- a compare pane's context comes from its own config, not the active model's snapshot
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: clear manual GPU knobs on Default and resolve local embedders before repo-id
- model-config-page: switching GPU Memory back to Default now clears the Manual-only
knobs (gpuLayers/nCpuMoe/selectedGpuIds); otherwise a remembered config kept stale
pins that a later load re-applied when the global GPU preference was Manual, despite
the page showing Default.
- routes/models hidden_model_matchers: resolve an existing local path before the repo-id
regex, mirroring is_hidden_model, so a local embedder shaped like "models/embedder" is
hidden by exact path instead of leaking as a chat model.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add _is_mtp_drafter to the model_config stub in the export-paths test
routes/models.py imports _is_mtp_drafter from utils.models.model_config at module
load, but the lightweight stub in test_export_absolute_paths.py did not provide it,
so loading the module under the stub raised ImportError on Backend CI. Add the stub.
* Studio: read a picked GGUF's chat template through the native path lease
The picker chat-template GET has no native-path-lease plumbing, so a
desktop-picked (drag-drop) GGUF could not show its default chat template
in Run Settings until the model was loaded: the endpoint only receives
the display label, not the leased file path.
Read the embedded template through the existing lease-aware
/api/inference/validate probe instead. A new include_chat_template flag
resolves the granted canonical path and returns the GGUF's own embedded
template, never a sibling sidecar (the grant authorizes just that one
file); it skips the training guard like include_context_length and is
bounded by MAX_CHAT_TEMPLATE_BYTES. The frontend fetch mints a one-shot
validate-model lease when a native token is present and keeps the plain
GET path for HF and allowlisted local models.
Adds backend and source-contract regression tests.
* Studio: call worker.direct_wheel_url in the ROCm wheel-url test
The ROCm Mamba/SSM test referenced worker.py's private _direct_wheel_url,
but the worker imports the wheel helper under its public name
direct_wheel_url (utils.wheel_utils). When the worker module loads (its
imports resolve in CI), worker_mod._direct_wheel_url raised AttributeError;
the test only masked it by skipping when the worker could not be imported.
Call the name that actually exists so the assertion runs; it still returns
None for an empty cuda_major (ROCm).
* Studio: reset max sequence length to the app default, not the loaded value
For a non-GGUF active model, the per-model config seeds maxSeqLength from
the loaded runtime value so the panel opens showing the running context.
Reset set config.maxSeqLength to null, but the null fallback resolved back
to that captured runtime value, so the field kept showing the old custom
length and the config saved/reloaded it again. A remembered or active
max-length override therefore could not be cleared from Run settings.
Fall the null/default case back to the app default (clamped to the model's
native ceiling) instead of the active runtime snapshot, so Reset actually
clears the override. The initial view is unaffected: an active model's
config.maxSeqLength is already non-null, so it still shows the loaded value.
Adds a source-contract regression guard.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: persist default max length, refresh deleted quants, hide non-chat locals
Three follow-up fixes from review of the per-model-config picker:
- Max sequence length: the persisted per-model record now keeps config's
maxSeqLength (null after Reset) so isDefaultConfig can clear a remembered
override; the resolved app-default is substituted only into the load
request, never the saved record. Previously Reset saved the concrete
default and left the model pinned/remembered.
- GGUF variant expander: deleting a downloaded quant from a repo that still
has other cached quants now bumps the expander refresh key, so the removed
quant stops showing as downloaded and clickable (which would try to reload
the deleted file) until the repo is collapsed and reopened.
- Local picker rows: require capabilities.canChat before listing a local
models-folder / LM Studio row. A weightless folder (only config.json) is
classified non-chat, and toLocalModelInfo drops capabilities, so selecting
such a row would try to load a path the inventory already marked non-chat.
Adds source-contract regression guards for all three.
* Fix compare-pane and Reset context defaults in model picker
Two related per-model-config default regressions:
- A non-GGUF compare pane with no saved maxSeqLength fell back to the
active model's shared runtime snapshot, so comparing a saved 128K model
against an unconfigured pane loaded the latter at 128K and could OOM. It
now falls back to the shared app default (DEFAULT_MAX_SEQ_LENGTH), the
same fallback the single-model config path uses.
- contextAtDefault treated an explicit customContextLength equal to the
native ceiling as a default, which wedged the Reset button disabled for
a deliberate pin-to-native. It now counts as default only when there is
no override at all.
DEFAULT_MAX_SEQ_LENGTH becomes a single exported constant in
per-model-config.ts so the single-model config and the compare path share
one source of truth. Adds source-contract guards for both fixes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip over-cap remote Jinja templates so the tokenizer template wins
The remote chat-template resolver bounded raw chat_template.jinja downloads
only by MAX_TEMPLATE_METADATA_BYTES (4 MiB), then returned the first
non-empty Jinja unconditionally. The picker route drops any template larger
than MAX_CHAT_TEMPLATE_BYTES (64 KiB), so an uncached repo whose
chat_template.jinja sits between 64 KiB and 4 MiB returned no template at
all, even when a valid smaller tokenizer_config.json template existed. The
local path already skips oversized .jinja files and falls through.
Gate the extracted Jinja on MAX_CHAT_TEMPLATE_BYTES and continue searching
when it exceeds the cap, matching _chat_template_from_jinja_file. The 4 MiB
download bound stays for JSON files that merely embed a small template. Adds
a regression test that a big Jinja plus a valid tokenizer config resolves to
the tokenizer template.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard legacy per-model-config migration idempotency
The v1->v2 localStorage migration (unsloth_load_settings ->
unsloth_model_configs) runs on every store read, so it must migrate exactly
once and never re-run, duplicate, or clobber a newer per-model config on a
reload or restart. That was covered only by a manual proof, so add durable
guards:
- Source-contract test pinning the three idempotency layers (the in-memory
legacyMigrationChecked guard, the persistent unsloth_model_configs_migrated
flag set in every terminal branch, and the non-overwriting Object.hasOwn
merge-skip) plus the readMap invocation. Reddens if any layer is dropped.
- Playwright model-config E2E: promote the legacy-migration step to a gating
check (soft_fail, which gates under the CI STUDIO_UI_STRICT=1) that the
migrated value is preserved and the flag is set, then reload again with a
fresh legacy seed present and assert the stored key set is unchanged, so a
second reload cannot re-migrate, duplicate, or clobber.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Note the migration E2E now gates idempotency under STUDIO_UI_STRICT
* Tighten model-picker per-model-config code comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
* Studio: validate Hugging Face tokens before use
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep token validation failures non-blocking
* Studio: harden Hugging Face token preflight
* Studio: make token validation effect lint-safe
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Studio: hide infra models from the hub cached inventory
The hub inventory scans behind /api/hub/cached-gguf and /api/hub/cached-models
returned the llama.cpp install validation probe (ggml-org/models) and the RAG
embedder (unsloth/bge-small-en-v1.5[-GGUF]) as on-device models. Share the
hidden-model check from routes/models.py via utils/models/hidden_models.py and
apply it in both scans. A GGUF infra repo stays visible when the user
explicitly downloaded a variant through the Hub, since variant manifests only
exist for user-initiated downloads.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make On Device trust the hub inventory, match repo ids exactly, lighten the hidden-model import
Follow-up on the hub cached-inventory hidden-model change, addressing the review.
On Device now trusts the Hub inventory API for cached rows. The backend already
hides the RAG embedder and the llama.cpp probe and re-includes a GGUF infra repo
once the user downloads a variant through the Hub, but the frontend was
re-hiding it by repo id, so the user-downloaded variant never appeared in the On
Device list or the count. isVisibleInventoryRow now short-circuits cached rows
(kind === "cache") to visible and keeps client-side needle hiding only for local
filesystem rows and Discover.
is_hidden_model matches Hub repo ids exactly (case-insensitive) against the probe
plus the effective embedder and its GGUF companion, instead of substring
matching the configured-embedder basename. A custom embedder with a generic
basename like org/model no longer hides unrelated cached repos such as
user/model-chat or org/model-instruct. The probe filename and local-path
embedders keep exact matching.
The helper moves to utils/hidden_models.py and is imported at module scope in the
hub cache scanner, so it no longer pulls in utils/models/__init__ (the eager
model-config/checkpoint stack) and a broken import fails at startup instead of
being swallowed per-repo and silently emptying the inventory. routes.models
keeps the _is_hidden_model and _safe_resolve aliases and drops the unused
_HF_REPO_ID_RE re-export that was failing source lint.
Tests: exact repo-id matching with a custom embedder, the cached-models scan
keeping an unrelated repo, and a clean-interpreter check that the helper imports
without the model-config stack.
* Studio: match the llama.cpp probe filename on both path separators
The hidden-model check compared the probe's on-disk filename with
Path(value).name, which on a POSIX interpreter does not split a Windows-style
path ("...\stories260K.gguf") and would let the probe through. Split on both
separators so the probe is matched regardless of which OS produced the path,
matching the tolerance of the previous substring check. Adds a Windows-path
assertion to the probe test.
* Studio: harden hidden infra model handling
* Fix hidden cache row confirmation
* Fix hidden local rows and confirmed hint merges
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle snapshot-configured hidden models
* Hide basename-only default embedders
* Fix dynamic embedder inventory filtering
* Studio: hide the configured RAG embedder from Discover and feed rows
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Replace standalone Studio wording with Unsloth
Replace the single word Studio with Unsloth wherever it is used as
shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n
locales, workflow display names, comments and docstrings.
Kept unchanged: the full name Unsloth Studio, third party product
names (LM Studio, Visual Studio, Mac Studio), feature names
(Recipe Studio, Fine-tuning Studio and its translations), and all
identifiers such as env vars, commands, paths and filenames.
* Address review feedback on the Studio wording rename
Use "an" before Unsloth where the rename left the article as "a".
Restore the split brand where Unsloth and Studio render as two halves
of the full product name: the onboarding sidebar subtitle and the
IPv6 localhost warning. Scope two messages to the full name Unsloth
Studio where plain Unsloth was misleading: the AMD README bullet and
the CLI studio setup error.
* fix(studio): recover stalled Hub downloads over HTTP
* fix(studio): preserve retry generation and progress baseline
* fix(studio): keep XET retry handoff nonterminal
* fix(studio): preserve retry cancellation on claim failure
* fix(studio): make retry failure cancellation atomic
* fix(studio): close skipped retry state gaps
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Stabilize chat-only export gate detection on Windows
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Retrigger CI on a user-authored head
* fix(studio): serialize XET HTTP retry handoff
* List XET to HTTP retries that are briefly released from the repo guard as active downloads for PR #6858
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Settle no-process active downloads on shutdown so a parked XET retry cannot spawn after cleanup for PR #6858
* Settle exited-error and no-process downloads on shutdown and persist their cancel markers for PR #6858
* Keep terminal HTTP failures uncancelled and block companion deletion for released retry peers for PR #6858
* Trim download lifecycle test coverage
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* refactor(studio): move chat model picker into features/model-picker
Relocate model-selector + its support files from components/assistant-ui
into a self-contained features/model-picker feature (own barrel), mirroring
the modular Hub layout. Pure move + import repoint; no behaviour change.
* feat(model-picker): add per-model config persistence layer
Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType,
specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted
to localStorage (unsloth_model_configs) with schema versioning + LRU budget.
KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple).
Reuses features/hub/lib/model-identity for normalization; adds storage-key layer
and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted).
* feat(picker): modular backend for chat-template validate + default fetch
New studio/backend/picker package (schemas/service/routes) mounted at /api/picker:
- POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives)
- GET /api/picker/chat-template/{model_name} (default template from tokenizer_config.json,
reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec)
Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend
changes to the existing inference load route (per-model load fields already supported).
* feat(model-picker): bind picker on-device list to shared hub inventory
Picker now sources cached + local models from useHubInventory (the Hub's shared
store) via a thin adapter, replacing its own /api/models/* fetchers + module
caches. Hub, download manager, and picker now share one source of truth, so
completed downloads reflect in the picker automatically. Partial/live-download
rows are filtered from the cached lists (unchanged rendering). Local naming/search
preserved via additive LocalInventoryRow modelId/displayName. Variant expander,
scan-folder management, recommended-fit, search, external providers untouched.
Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical
(hub cached rows carry no mtime); default 'recent' (load-time) sort preserved.
* feat(model-picker): per-model config step inside the picker
Picking a (non-external) model now opens an in-picker config view built from
main's current load controls (context length, KV cache dtype, speculative
decoding, draft tokens, tensor parallel) plus a chat-template editor backed by
the picker validate/default endpoints. 'Remember for this model' persists the
config per model+variant; Run forwards the config to the existing load flow via
meta.config. External models bypass the step. Two-view orchestration lives in
model-selector (single interception point); pickers.tsx call sites untouched.
trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent.
* feat(chat): apply/persist per-model config through the load flow
handleCheckpointChange threads meta.config into the selection; stageOrLoad and
the autoload/Hub-run paths now apply the picker config (explicit pick or saved
remembered config) via applyPerModelConfigToRuntime before staging/loading, with
keepSpeculative set so a remembered speculative mode survives the model switch.
Replaces the old remembered-load-settings seeding (resolveInitialConfig now the
single source). SelectedModelInput carries config.
* refactor(chat): remove per-model load config from the right sidebar
The load knobs (context, KV cache, speculative, draft tokens, tensor parallel)
and the chat-template editor now live only in the picker config step. The sheet's
Model section keeps the staged Load/Cancel flow (config is applied at pick time);
sampling params, system prompt, and RAG are unchanged. Deletes the superseded
remembered-load-settings module + the store's applyRememberedLoadSettings action,
removes the now-dead sheet state/imports, and points the settings reset at
unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped).
* fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section
The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight
into the right-sidebar Run-settings flow -- the old 'configure before load' path
now fully replaced by the in-picker config step. Removed the gear + its component.
Also gate the sheet's 'Model' section to staged picks only (pendingSelection):
after the load-knob strip its content is staged-only, so it was rendering an
empty section header whenever a model was merely loaded.
* chore(chat): remove dead per-model-config setters + modelControlsDisabled
After the load-config UI moved into the picker, the store's per-model setters
(setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/
setCustomContextLength/setChatTemplateOverride) had zero callers
(applyPerModelConfigToRuntime writes via setState), and the sheet's
modelControlsDisabled was unreferenced. Verified dead across the whole tree.
* fix(chat): config-step Load actually loads (ignore Load-on-selection)
Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config
step's load went down the deferred-staging path -- opening the right sidebar with
'<model> is staged, not loaded yet / Choose Load model'. The in-picker config step
IS the deliberate load action, so its Load now loads immediately (or downloads +
auto-loads when not cached) regardless of the toggle. Renamed the button
'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle.
* refactor(chat,hub): retire 'Load on selection' — config step is the only load flow
The in-picker config step (and the Hub Run button) now fully supersede the old
stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere:
- chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when
not cached (the previous default behaviour, now universal).
- hub Run: drops the stage branch; downloaded GGUFs load directly with their saved
per-model config (no collision with the chat config step — both end at selectModel).
- store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and
its settings-reset entry removed.
- staged sidebar section is now a download-progress view (auto-loads on completion).
No manual staging remains; stageModel is used only for background auto-load downloads.
* feat(model-picker): default chat template from GGUF + thread variant through config flow
Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template
in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through
the picker service, /api/picker/chat-template route, frontend templates API, and
use-model-defaults so the right variant's template is fetched.
Also refine the picker config-page/model-selector wiring, drop the dead
ggufNativeContextLength runtime path, and add the per-model-config storage keys to
the settings prefs export.
* feat(model-picker): read safetensors chat template + hide editor where it has no effect
Resolve the default chat template for safetensors models: prefer the modern
chat_template.jinja, fall back to the tokenizer_config.json chat_template field,
then chat_template.json (multimodal processor), then the GGUF embedded template.
Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch.
Hide the chat-template editor in the picker for safetensors models — the override
is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors
currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays
for when the safetensors apply path is wired up in a later branch.
* fix(model-picker): set legacy-migration flag only after the write succeeds
Set unsloth_model_configs_migrated only once writeMap confirms the migrated
map persisted, so a quota/storage failure no longer marks migration done and
silently drops the user's pre-existing remembered settings — the next load retries.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* MVP model picker fixes
* MVP picker config fix
* MVP safetensors config
* MVP max seq config
* MVP max seq fix
* Fix static max tokens cap ignoring model context
* Fix picker GGUF scan parity
* fix(studio): harden model picker config loading
Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload.
* Fix model picker config flow
* Fix model picker config loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Avoid recursive per-model config migration reads
* Apply the displayed context length when loading a GGUF
* Fix template validation, cached template lookup, and failed load rollback
- Validate chat templates with the loopcontrols extension so templates
that use break or continue tags pass the picker validator, matching the
inference renderer that already accepts them.
- Read the default chat template from the newest cache snapshot rather than
an arbitrary iterdir order, so an older cached revision no longer prefills
a stale template.
- Capture the runtime per-model config before a load and reapply it when the
load fails, so a failed switch leaves the active model context, KV cache,
template, and speculative settings as they were.
* Make chat template view only for safetensors models
Custom chat template overrides are applied at inference only for GGUF
models, which pass the template to llama-server. The safetensors backend
renders with the model built-in template and ignores the override, so
editing it would save a value that never loads. For safetensors the
config page now opens the template as a read-only preview with a note
that editing is not available yet. This can become editable once
inference support for custom safetensors templates lands in main.
* Fix model picker config edge cases
- Restore prior runtime config when a load no-ops for the active model
- Cap the picker validator request body via the protected prefixes
- Keep the GGUF context slider max above the loaded context
- Fetch subfolder chat templates for uncached Hub repos
- Show the compare side config when reopening the picker
* Keep saved GGUF context above the fallback ceiling
* Show the model config in the run settings sidebar
* Fix model config sidebar reset and context slider
- Stack the remember toggle and action buttons in the sidebar
- Reset the config to defaults instead of the loaded values
- Fetch the native context so the slider max is not the loaded value
* Fix model picker config and download regressions
- Run picker chat template routes off the event loop
- Depth and root guard local template directory scans
- Restore download manager flow for uncached hub picks
- Apply per model context length on reload
- Import model picker symbols from the feature barrel
* Fix model picker config and cached download sorting
- Restore load settings when a Hub run is rejected mid load
- Reuse one NumericValueInput instead of a duplicate copy
- Fix double decode of the model name in the template route
- Remove the unused reset-to-loaded settings action
- Fix cached model download sorting
* Fix model picker per-model config edge cases
Honor a saved or typed max seq length above the model's native context so
RoPE extended values are no longer clamped and silently overwritten. Allow
typing past native while the slider keeps native as a soft ceiling.
Guard the fetch success paths in use-model-defaults against an aborted
signal, and refetch when the HF token changes.
Hash the chat template content in the sidebar remount key instead of its
length. Enable reset for a GGUF whose native context is unknown, and floor
the context slider max so it can never fall below the min.
* Fix GGUF context auto-fit and gated model config token
Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit.
Send the HF token as a query param so gated safetensors models resolve their max position embeddings.
Derive model default state during render to drop the set-state-in-effect calls.
* Fix native GGUF context ceiling and guard picker template reads
Restore the native context store field so the sidebar slider keeps the
full ceiling for drag and drop GGUFs. Limit local chat template reads to
the browse allowlist, skip malformed repo ids, and drop unused model
picker exports.
* Fix model picker lint boundaries
* Fix model picker review findings
Chat template editor never seeded its draft. Radix only calls onOpenChange
from internal events, so the seed in the nextOpen branch was dead and a model
with a saved override opened empty. Saving then cleared the override. Drop the
dead branch, treat draft as an untouched sentinel, and reset it on every close.
Uncached Hub picks could auto load a model after the user left the chat. Main
detached the staged pick on route exit and on chat context change. Carry the
context key on the pending pick and skip the load when it no longer matches.
Also clear configTarget when the picker closes, restore the onUpdated ref so
variant rows stop resubscribing on every parent render, skip the LRU write when
the entry is already most recent, import NumericValueInput relatively, and drop
the unused ModelUpdateAction barrel export.
* Preserve GGUF context on active reload
* Fix model picker per-model config regressions
- Stop reloading the already loaded model on re-pick
- Hide infra models from the chat picker
- Detect vision support on cached GGUF repos
- Honor saved maxSeqLength on auto load
- Restore default chat template for local GGUFs
- Warn on save failure and revert config on cancel
- Refetch picker inventory on open
- Persist read only per model config safely
* Fix stale model auto load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker numeric input sizing and constraints
Size value inputs to their content so long context lengths are not clipped,
restrict them to numeric characters, and stop the speculative decoding label
from truncating in the sidebar.
* Fix picker CI tests and harden chat template resolution for PR #6647
- tests: point the descender guard at the moved model-selector.tsx path
- tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate
- picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged)
- compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Protect future-schema per-model configs from deletion for PR #6647
savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast.
* Protect future-schema per-model configs from quota eviction for PR #6647
The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them.
* Fix GGUF context persistence, compare context, and rollback settings for PR #6647
Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (f4838782cb reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override.
shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context.
use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults.
* Preserve native path token when reloading the active model for PR #6647
handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly.
* Make picker template validation resilient and accept HF generation tags for PR #6647
Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag.
* Honor remembered compare config and parse processor chat_template.json for PR #6647
* Fix failed-load rollback context and processor template map fallback for PR #6647
* Restore speculative decoding config on failed-switch rollback
When a model switch fails after the previous model was unloaded, the
rollback reload restored tensor_parallel, KV cache dtype and the chat
template override, but omitted speculative_type and spec_draft_n_max and
cleared their loaded shadows to null. The previous model therefore came
back running at backend defaults (speculation off) while the UI still
showed it enabled, and the status resync confirmed the off state. Resend
the previous model's speculative settings in the rollback load and keep
the store's active and loaded speculative fields in sync with them.
* Reset max sequence length when a model has no saved config
applyPerModelConfigToRuntime reset every per-model field except
maxSeqLength, which it only wrote when the incoming config had one.
maxSeqLength is the sole field carried on store.params, so selecting a
model with no remembered config left the previous model's value in place
and later loaded the new model at that leaked length. Fall back to the
standing default so an unremembered model loads at its own default.
* Surface a message when a variant update cannot start
startManagedUpdate handled the conflict and error start outcomes but let
busy fall through as if the update began, so the confirm dialog closed
with no job created and the cached variant stayed stale. Show an info
message when the repo is busy with a sibling transfer so the click is
not silently dropped.
* Keep per-model speculative choices out of the global default
A staged load with a per-model or one-off config sets keepSpeculative,
which already skips reading the global speculative preference. The
matching save still ran unconditionally, so the model-specific choice was
written to the global unsloth_chat_speculative_type and a later model with
no saved config started from it instead of Auto. Skip saveSpeculativeType
when keepSpeculative so the per-model choice stays isolated.
* Seed non-active model settings from the app default max length
The Run settings page captured initialMaxSeqLength from the loaded
model's runtime params and fell back to it for a model with no saved
config. Opening settings for a different, unloaded model and clicking
Load then sent the active model's context (for example 64k) instead of
the 4096 default, risking validation failures or OOMs. Seed the default
for non-active models and keep the runtime value only for the active one.
* Prefer sidecar tokenizer chat template over the GGUF copy for variants
_chat_template_from_dir returned the embedded GGUF template first when a
variant was selected, reversing the tokenizer-first precedence of the
no-variant path. A model whose chat_template.jinja or tokenizer_config.json
supersedes a stale embedded template then got the wrong template on
variant selection. Keep tokenizer files first regardless of variant; the
variant only picks which GGUF is the fallback. Adds regression tests for
both the tokenizer-wins and gguf-fallback cases.
* Keep per-model speculative choices load-local in autoload and compare
The interactive load path treats a per-model speculative choice as
load-local and skips writing it to the global default. Autoload and
generalized compare still called saveSpeculativeType unconditionally, so a
remembered off or ngram setting leaked into unsloth_chat_speculative_type
and later models with no saved config inherited it. Persist the global
preference only when the value came from the global settings.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context
* Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it
* Studio: drop the merge's orphaned staged-model store helpers and unused alert imports
The main merge left isPendingGguf and pendingSelectionMatches referencing the
removed PendingModelSelection type, and the alert-dialog/alert imports unused
after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed.
* Studio: cache a null default chat template so the viewer stops re-fetching it
A model with no sidecar or embedded template resolves to a terminal null, but
that result was never cached, so reopening the template viewer re-ran the
backend and Hugging Face lookup every time.
* Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context
A GGUF loaded from a local file or custom folder has no variant label, so the
run-settings panel treated it as non-GGUF and clamped Max Tokens to the session
max_seq_length instead of the loaded GGUF context. Detect it via the reported
GGUF context and the .gguf checkpoint suffix, matching the chat page.
* Studio: prompt to re-select a local model file when its lease expired before reload
A file-picked GGUF is reachable only through a native path token that the
desktop host prunes after a TTL. Reloading reused that token blindly, so a
reload long after the initial load failed with an opaque error. Track the
token's expiry and, when it has passed, ask the user to re-select the file
instead of attempting a doomed reload.
* Fix descender-clipping test to tolerate sidebar layout utilities
The sidebar account-block div carries layout utilities (min-w-0, flex-1)
between 'flex' and 'flex-col', so the descender-clipping guard's regex,
which required 'flex' immediately followed by 'flex-col', no longer matched
and the test failed to locate the account-block div. Generalize the prefix
to allow intervening flex utilities while still capturing the leading-*
class before the collapsible visibility utility and asserting leading-tight,
so the guard against clipped glyph descenders is fully preserved.
* Harden picker chat-template resolution
Enforce the 64 KiB chat-template contract at the validate endpoint's request
model so a direct caller cannot submit a template far larger than the frontend
allows (MaxBodyMiddleware only bounds the whole request body, not this field);
oversized templates now return a clean 422.
Apply sidecar-over-GGUF template precedence globally across cached snapshots
instead of per snapshot. A repo with multiple cached revisions previously
returned the first snapshot's template, so a newer GGUF-only revision could
win over an older revision's maintained chat_template.jinja sidecar, which
contradicted the documented intent that sidecars supersede the embedded copy.
* Guard per-model config against future-schema and lossy migration
Two forward-compatibility gaps in the versioned per-model config store:
- The load/apply path returned and normalized a stored record without checking
its schema version, so a record written by a newer client was reinterpreted
under the current schema and applied to a live model load, even though save,
delete and eviction all refuse to touch future-schema records. Reject
future-schema records on load too.
- The one-time legacy migration enforced the storage budget without protecting
the entries it had just migrated and set the completion flag unconditionally.
When storage was already full of future-schema records (which are unevictable
by an older client), the migrated entries were the only evictable ones and
could be dropped while migration was still marked complete. Protect the
migrated keys during eviction and only mark migration complete when they
survive, so it retries once space frees up.
* Discard chat-template validation results after the dialog closes
Server-side template validation is async, but closing or cancelling the editor
did not abort it, so a late-arriving valid response still called onSave and
applied a template the user had already dismissed. Track a validation token
that is bumped on close and ignore any validation result whose token is stale.
* Record native lease expiry when loading a picked GGUF from the chip
The pending-native-model chip loaded via stageOrLoad directly, bypassing
loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a
chip-loaded file. A later reload then either skipped the lease-expiry guard
entirely (expiry left null) or compared against a previously loaded file's
stale expiry, so reload could reuse an already-pruned token or wrongly block a
still-valid one. Route the chip through loadNativeModelIntent, which builds the
same selection and records the expiry.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prefer sidecar template for a directly selected local GGUF file
A direct .gguf file path read its embedded chat template without checking the
parent directory for a maintained sidecar (chat_template.jinja /
tokenizer_config.json), while directory and variant selections already prefer
the sidecar. That let the config editor preview or save a stale embedded
template for the same model depending on how it was selected. Check the parent
directory sidecars first, then fall back to the embedded copy, and cover both
paths with tests.
* Resolve cached chat template per revision, newest first
The earlier change searched every cached snapshot for a sidecar before
considering any snapshot's embedded GGUF template, which let an obsolete sidecar
from an older revision override the newest revision's template. Restore
per-snapshot resolution (newest first): a revision's sidecar still supersedes
its own embedded GGUF copy, but a newer revision is no longer overridden by an
older revision's sidecar.
* Preserve autoload transport conflicts and surface background busy downloads
- When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound
instead of clearing it. Clearing it re-keyed the download surface and its
cleanup cancelled the conflict the toast tells the user to resolve, so the
Hub resume affordance was gone the moment it appeared. Return early on
conflict, mirroring the started branch, so resolving it from the Hub still
auto-loads on completion.
- The background-download branch handled started and conflict but silently
dropped a busy outcome, leaving the user with no feedback when a peer variant
of the same repo was already downloading. Surface the same busy toast the
autoload path uses.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio: Inkling support fixes (context sizing, tool-call healing, reasoning effort, audio icon)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: exclude /api/export/status from request access logs
The frontend polls /api/export/status every 5s to detect export start, so it
fires continuously even when idle. Each poll emitted an info request_completed
access line, making up most of the server access logs. Add it to _EXCLUDED_PATHS
alongside /api/train/status. The endpoint is unchanged; export state is still
logged by the export modules and streamed over SSE, so no signal is lost.
* Studio: collapse hub download-progress polls in the access log
download-status and gguf-download-progress (plus the dataset equivalents)
are polled about twice a second for the whole download, so each emitted an
info request_completed line. Add them to _QUIET_POLL_PATHS so they collapse
to one heartbeat line per 10s instead of one per poll.
* Studio: log hub download progress at 10% steps
The access log carried no real progress, only poll pings. Emit one
hub_download_progress line per 10% step from the shared snapshot progress
reader, so an active download shows actual percentage without a line per
poll. Throttled per job and resynced if the same download restarts.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: drop successful chat thread/project CRUD from the access log
A single chat turn fans out about twenty requests under /api/chat/threads
and /api/chat/projects (list, fetch, per-message forks, and the message
writes) that only reflect the UI re-rendering. Suppress their 2xx access
line so the log keeps the signal (generation, tool calls, code execution,
engine stats) and errors. Non-2xx on these paths still log.
* Studio: silence transformers torch_dtype deprecation warning
transformers logs "`torch_dtype` is deprecated! Use `dtype` instead!" once at
model-config load via logger.warning_once (logging, not warnings), so a warnings
filter cannot catch it. Attach a small logging.Filter in setup_logging, which
runs before any model config is parsed, to drop that record on the transformers
loggers that emit it.
* Studio: quiet inference load-progress polls and log throttled load progress
The frontend polls /api/inference/load-progress about twice a second for the
whole model load, so each emitted a request_completed line. Add it to
_QUIET_POLL_PATHS (heartbeat) and emit one inference_load_progress line per 10%
step from the load-progress route, so a load shows real percentage instead of a
line per poll.
* Studio: fully suppress download/load progress poll access lines
The download-status, download-progress, gguf-download-progress, active-downloads
and transport-status polls (model and dataset), plus inference load-progress,
fire ~2x/s for the whole download or load. Their progress is now reported by the
hub_download_progress / inference_load_progress events (and the viewer's progress
line), so the per-poll access line adds nothing. Drop it on 2xx and keep it on
errors, instead of the prior 10s heartbeat. Chat CRUD suppression is folded into
the same _is_quiet_success helper.
* Studio: suppress training-tab model/dataset download-progress polls
The training tab polls /api/models/download-progress and
/api/datasets/download-progress about twice a second for the whole prep phase.
These are separate routes from the /api/hub equivalents and only scan the cache,
so their 2xx access line adds nothing (on Windows they always read 0 since the
bytes live in snapshots/, not blobs/). Suppress the 2xx line and keep errors,
alongside /api/models/gguf-download-progress.
* Studio: drop transient pre-auth 401 on chat thread/project polls
On first load the SPA fires chat thread/project GETs before the initial token
refresh, so they 401 until /api/auth/refresh runs and the retries succeed. That
pre-auth 401 is a bootstrap artifact, not an error; suppress it alongside the
already-quiet 2xx line. Genuine 4xx/5xx on these paths, the download/load poll
401s, and all /api/auth/* still log.
* Studio: quiet tab-switch list polls and per-poll scan/reconnect logs
Switching between the Train, Export, and Chat tabs refetches list endpoints on a
timer, and each hit re-logs internal detail. Heartbeat /api/train/runs,
/api/models/checkpoints, /api/models/local and /api/rag/knowledge-bases (10s
window, first hit and errors still log), and downgrade two per-poll INFO lines to
debug: the checkpoints scan summary ("Found N training runs") and the
per-reconnect SSE resume line. The meaningful "replayed N missed steps" line,
logged only when steps were actually replayed, stays at info.
* Studio: enable tokenizer parallelism for dataset prep on Windows/macOS
TOKENIZERS_PARALLELISM was forced off everywhere to stop datasets' forked map()
workers from deadlocking, but that fork only happens on Linux. On spawn platforms
(Windows/macOS) dataset.map() runs in-process (dataset_map_num_proc returns None),
so disabling tokenizer parallelism leaves the fast tokenizer single-threaded and
dataset prep runs serially on one core. Keep it off on Linux (fork safety) and on
for spawn platforms, where there is no fork to deadlock. Measured ~7x faster
tokenization (12.5s -> 1.7s for 20k rows on a 32-core Windows box).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: log throttled training status to the server log
Training step/loss/epoch only went to the UI via SSE, so the server log showed
inference engine_stats and train/runs heartbeats but nothing about the actual
run. Emit one throttled training_progress line (step/total, percent, loss, epoch,
eta) from the CUDA event pump: the first step, then at most every 30s, plus the
final step, resyncing when a new run restarts the counter. Per-step UI streaming
is unchanged.
* Studio: quiet llama.cpp update-status polls and log throttled update progress
The prebuilt llama.cpp update polls /api/llama/update-status about twice a second
for the whole download and install. Suppress its 2xx access line (errors still
log) and emit one throttled llama_update_progress line per 10% step from the
status route, so the update shows progress without a line per poll. The existing
"llama update: installing" and "llama update: success" events still bracket it.
* Studio: quiet the export log-tail poll
The Export tab polls /api/export/logs about once a second to stream the export
subprocess output into the UI panel. Suppress its 2xx access line; the real
progress is already logged as event-driven "Export subprocess status: <phase>"
lines plus the subprocess start and checkpoint-loaded events, and errors still log.
* studio: keep errors and mutations visible in access-log suppression
Make the quiet-success access-log suppression GET-only so chat thread/project
mutations (POST/PUT/DELETE) still log; only their list-poll 2xx and the
transient pre-auth 401 are dropped.
Suppress /api/export/status 2xx only (move it out of the all-status exclude
set) so a 401/403/500 on it stays visible.
Legacy /api/models and /api/datasets download-progress polls emit no
hub_download_progress events, so heartbeat them via the 10s quiet-poll window
instead of suppressing outright, keeping download visibility (notably on
Linux). The event-emitting /api/hub download polls stay fully suppressed.
Update and extend the middleware tests to cover GET-only suppression, the
export-status error path, and the legacy download heartbeat.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tighten access-log and training-progress comments
Comment-only pass: collapse the multi-line explanations in the logging
middleware and the throttled training-progress logger to fewer lines while
keeping the rationale. No behavior change.
* studio: log structured export_progress phases
Emit a structured export_progress event per phase (consolidated in the server
log like training and download progress) instead of a plain status string, and
add a phase milestone at the start of the heavy export step so the
merge/save/convert is visible in the server log, not only in the forwarded
stdout panel.
* Studio: reset training-progress log throttle on each new run
start_training rebuilds the per-run progress state but left _last_progress_log_ts/_last_progress_log_step at their prior values. A run started within 30s of a previous one whose last logged step matched the new run's first step would hit the step == prev short-circuit and drop the promised first training_progress line, then stay suppressed until the old 30s window expired. Reset both fields when a new job is accepted.
* Studio: keep post-bootstrap chat 401s visible in the access log
The chat thread/project 401 suppression dropped every GET 401 on those prefixes, so a genuine expired-session 401 vanished alongside the transient pre-auth race. Gate the 401 drop on a per-middleware bootstrap latch that flips once /api/auth/refresh first succeeds: before that the 401s are the pre-refresh race and are suppressed; after it any chat 401 is a real failure and logs. Add a test for the post-refresh case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: limit chat access-log suppression to the exact list polls
The chat thread/project suppression matched by startswith, so it also dropped the 2xx access line for detail and message reads (/threads/{id}, /threads/{id}/messages, /threads/{id}/messages/{id}, /projects/{id}) that are not the high-frequency list polls, losing their access and latency logging. Match the two list paths exactly instead, so only the intended list polls (and their pre-auth 401 race) are suppressed while detail and message reads keep their access line. Add a regression test.
* Studio: reset inference load-progress throttle for each load
The load-progress throttle (_last_load_progress_step) is a module global that persisted across loads, so a cached or small load whose first sampled /api/inference/load-progress response already reported fraction=1.0 hit step == prev (10) from a prior completed load and emitted no inference_load_progress line, while that endpoint's access log is suppressed, leaving the new load with no progress signal. Arm the throttle at load initiation in _load_model_impl so each load's first step always logs. Add a regression test.
* Studio: tighten logging comments
Collapse a few verbose comments (tokenizer-parallelism note, torch_dtype filter, legacy download-poll heartbeat, chat list-path suppression) to fewer lines without changing intent or code.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: expose Windows drive roots in the folder browser
The model-selection folder browser bounds navigation to the roots returned
by _build_browse_allowlist(), which exposed Linux removable-media mounts via
linux_run_media_mount_roots() but had no Windows analog. As a result a user
on C: could not browse to D:/E: to pick a model directory.
Add windows_drive_roots(), a Windows-only companion to
linux_run_media_mount_roots() that lists readable logical drive roots, and
wire it into both browse-allowlist builders and their suggestion chips so
other drives are both navigable and offered as quick-picks. The helper is a
no-op on Linux/macOS, so existing platforms are unaffected.
Closes#6368
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: cover the Windows drive-root browse wiring with an integration test
Add an allowlist integration test mirroring the Linux side's
test_legacy_browse_allowlist_includes_linux_run_media_mounts: it extracts
_build_browse_allowlist from routes/models.py, stubs external_media so
windows_drive_roots() yields a fake drive root, and asserts that root becomes
browsable through the built allowlist. Proves the wiring, not just the helper.
* Studio: skip inactive drives via GetLogicalDrives before probing
Resolve active logical drives from GetLogicalDrives() before probing each
letter with os.path.isdir. Probing a drive letter mapped to a disconnected
network share can otherwise block the async backend for tens of seconds per
letter. The call degrades gracefully (falls back to probing all letters) when
ctypes/windll is unavailable, so behavior is unchanged on Linux/macOS. Tests
override the bitmask source to stay deterministic on real Windows hosts.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: allow browsing descendants of a drive-root allowlist entry
routes/models.py _is_path_inside_allowlist() checked descendants with
startswith(root_real + os.sep). A drive root ("D:\") already ends in a
separator, so the prefix became "D:\\" and a child like "D:\models" was
rejected with 403 after the browser opened the drive root. Only append a
separator when the root does not already end in one. folder_browser.py already
uses commonpath and was unaffected. Adds a regression test covering the
separator-terminated-root descendant case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: enforce the system-directory denylist during folder browsing
Exposing whole Windows drive roots (and any legacy-registered filesystem root)
widened the browse allowlist above system directories, but the browse
resolvers only re-applied the credential/config denylist, not the
_denied_path_prefixes() system-dir denylist that scan-folder registration
enforces. That let browse-folders enumerate C:\Windows, C:\Program Files,
/etc and /proc.
- Add is_denied_system_path() to both storage modules and enforce it in both
browse resolvers (legacy routes/models.py and hub folder_browser.py), on each
resolved child and on the final target, keeping the /run/media carve-out.
- Rework the legacy _is_path_inside_allowlist to use splitdrive + commonpath so
a Windows drive root authorizes its descendants while a bare POSIX / does not,
and to compare case-insensitively like the hub browser.
- Reject the filesystem root in the legacy add_scan_folder, matching the hub.
- Hide denied system dirs from browse listings and suggestion chips.
- Add tests/test_browse_denylist.py and update the external-media path tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make browse-denylist tests OS-portable
The browse-time denylist tests used real /etc and tmp_path locations; on macOS
tmp lives under the (legitimately denied) /private/var and /etc resolves to
/private/etc, so three tests failed there. Pin the platform / use a tmp-based
denied prefix so they assert the same behavior on Linux, macOS and Windows.
* Studio: apply the bare POSIX-root guard to the hub folder browser too
The _is_path_inside_allowlist guard that stops a legacy-registered '/' scan
folder from authorizing every absolute path lived only in the legacy browser.
The hub browser used commonpath without it, so a stale '/' row let it descend
into /var, /root, /home -- which the system-directory denylist (/proc /sys /dev
/etc /boot /run) does not cover, while the legacy browser blocked them. Mirror
the legacy guard so both browsers treat '/' identically.
Also resolve each directory entry before the denylist check in both listing
loops, so a symlink or junction pointing into a denied dir is hidden instead of
rendered as a row that 403s on descent. Adds legacy-vs-hub parity tests.
* Studio: bound Windows drive probing so a disconnected mapping can't stall the browser
GetLogicalDrives includes mapped network drives, so a disconnected but still
mapped drive (e.g. Z: -> \\nas\share) stays set in the bitmask and reaches
os.path.isdir, which can block for tens of seconds while Windows tries to
reconnect. Because windows_drive_roots() runs synchronously while building both
folder-browser responses, one stale mapping stalled every browse request.
Probe each surviving drive in a daemon thread bounded by a short timeout and
skip it if it does not answer in time, so a hung mapping is dropped instead of
blocking the caller. Connected drives (local or network) still respond well
within the timeout, so drive discovery is unchanged. Corrects the
GetLogicalDrives docstring, which claimed the bitmask alone prevented the stall.
* Studio: probe drive/media roots once per browse request, not twice
Both folder browsers called windows_drive_roots() (and
linux_run_media_mount_roots()) twice per browse request: once to seed the
allowlist in _build_browse_allowlist() and again to build the suggestion chips.
With the bounded drive probe, a disconnected mapped network drive then paid the
timeout twice per folder click. Probe both once in the request handler and pass
the results into _build_browse_allowlist(), reusing them for the chips, in both
the legacy and hub browsers. Adds a test asserting the roots are reused, not
re-probed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: run the legacy browse endpoint in the threadpool, fix its stale test
Two follow-ups from review of the drive-probe changes:
- browse_folders was 'async def' but does only blocking filesystem I/O (the
timeout-bounded drive probe, iterdir, realpath). On the event loop a
disconnected mapped drive waiting out its probe timeout stalled every other
request. Declare it sync 'def' so FastAPI runs it in the threadpool, matching
the hub browse endpoint. No await was used in the body.
- test_browse_folders_hides_sensitive_dirs monkeypatched _build_browse_allowlist
with a zero-arg lambda; the once-per-request refactor now calls it with
(media_roots, drive_roots), so the lambda raised TypeError. Accept and ignore
the args.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: probe Windows drive roots concurrently so multiple dead mappings don't stack timeouts
windows_drive_roots() probed each candidate serially, so N disconnected-but-mapped network drives each paid the full per-drive timeout in turn (e.g. four stale mappings added ~8s to every folder-browser request). Collect the candidate roots first, then probe them all at once under a single overall deadline, so the added delay stays at ~one timeout regardless of how many drives are disconnected. _readable_dir_within stays as a thin single-path wrapper for its existing callers/tests.
* Studio: tighten comments in the folder-browser drive-root changes
Condense the comments and docstrings added by the Windows drive-root and
system-directory denylist work to be shorter and clearer while keeping the
security and correctness rationale intact. Comment and docstring text only;
no code changes.
* Studio: iterate the input, not the results dict, when collecting readable drive probes
_readable_dirs_within returned {path for path, ok in results.items()...}, but a probe thread that exceeded the join deadline is still alive and can insert its key into results during that iteration, raising 'dictionary changed size during iteration' -- reachable exactly in the disconnected-mapped-drive case the probe exists for. Iterate the fixed input list and read results.get(path) (an atomic read) instead.
* Studio: keep the browse-route containment tests denylist-inert so they pass on macOS
test_browse_folders_route.py exercises allowlist containment and the file-vs-directory guard, not the system-directory denylist. On macOS pytest tmp_path resolves under /private/var, a denied prefix, so _resolve_browse_target 403s the fixture dirs before the containment logic runs (4 failures). Add an autouse fixture that makes is_denied_system_path inert in this file; the denylist keeps its own coverage in test_browse_denylist.py.
* Studio: keep the hub browse tests denylist-inert so they pass on macOS
* Studio: register a UNC share root; only reject local filesystem roots
* Studio: reject device drive roots and browse a registered UNC share root
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: treat device-namespace volume GUID roots as local filesystem roots
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Studio: fix the permanent GGUF "update available" on no-symlink caches
Without the symlink privilege (the default on Windows with Developer Mode
OFF), hf_hub_download MOVES the downloaded blob into snapshots/ instead of
symlinking it out of blobs/, so blobs/ is left empty and scan_cache_dir
reports blob_path = the snapshot file itself. Path(blob_path).name is then
the GGUF FILENAME, not the file's etag.
_repo_gguf_blob_map recorded that filename as the file's local blob hash, so
_variant_update_available_from_requirement's `remote_sha256 in local_set`
test could never match and every cached GGUF reported "update available"
forever. Re-downloading could not clear it: the same file is rewritten, still
with no blob.
Only treat Path(blob_path).name as a hash when the file really lives in the
cache's blobs/ dir; otherwise record a size identity so the file still appears
in the map (dropping it would make the update check read it as absent and
report the same phantom update). The comparison falls back to the remote
ExpectedFile.size only when the cached file carries no blob hash, so the
blob-hash path is unchanged wherever HF does produce blobs.
A remote requant that keeps the byte size identical is not detected in that
layout; re-hashing multi-GB GGUFs on the inventory hot path is the only
stricter option.
Fixes#7060
* Studio: match GGUF update checks by manifest sha256, closing the equal-size requant blind spot (#7060)
On a no-symlink cache (Windows without Developer Mode) blobs/ is empty, so the update check falls back to comparing byte size. A Studio download records each file's sha256 in its manifest, so feed that into the local identity set: the check can then match by hash and detect an equal-size requant. The size fallback now applies only when no real hash is present, so a manifest hash that differs is still reported as a genuine update. Adds regression tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Revert the manifest-sha256 identity merge; keep the size-identity fallback (#7060)
The download manifest is written before the transfer with the expected remote hashes, so it records download intent, not verified on-disk content, and completion is checked by size only. Merging those hashes into the local identity set could clear the update badge for an interrupted equal-size update that left the old bytes on disk. The accompanying all-size-identity gate also suppressed the size fallback whenever an older revision contributed a real blob hash, which re-showed a false update on mixed hash and size caches. Restoring the plain size-identity fallback keeps the fix without those regressions.
* Studio: don't delete no-symlink GGUFs during stale-variant reclaim (#7060)
On a no-symlink cache (Windows without Developer Mode) the downloaded file is moved into snapshots/ and scan_cache_dir reports its blob_path as that snapshot file, whose name is the filename, not an etag. reclaim_replaced_gguf_variant treated that name as a blob hash, which never matches the current hashes to keep, so it unlinked the freshly downloaded file. Only extract a deletable hash when the blob path is a real cache blob under the repo blobs directory, and keep any file we cannot identify; stale no-symlink revisions leak rather than risk removing the current file. Adds a regression test.
* Studio: anchor GGUF blob-hash detection to the repo cache blobs dir (#7060)
The inventory update check and the stale-variant reclaim both decide whether a scanned file is a real cache blob (name is the etag) or a moved no-symlink snapshot file (name is the filename). Both now share one _is_real_cache_blob helper that anchors to the repo cache blobs directory instead of matching any parent folder named blobs, so a repo that ships GGUFs under its own blobs subdir is no longer misread as the cache blob store. Threads repo_path through _repo_gguf_blob_map. Adds a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten comments in the GGUF update-check helpers (#7060)
Post-review comment pass: shorten the internal blob-identity and size-fallback docstrings. Comments only, no behavior change.
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: resolve the repo-root MTP drafter after the MTP/ GGUF rename
The Gemma 4 QAT GGUF repos renamed the higher-precision MTP/ subdir
copies from gemma-4-...-<quant>-MTP.gguf to mtp-gemma-4-...-<quant>.gguf,
so their basenames now start with the same mtp- prefix as the small
repo-root drafter (mtp-gemma-4-E4B-it.gguf).
The drafter selectors filtered candidates by a mtp- basename prefix and
took the first in sort order. With the new names the MTP/ copies also
match, and because MTP/ (uppercase) sorts before the lowercase root file,
selection flipped to the large BF16 copy under MTP/ instead of the root
drafter both functions document they should pick.
Restrict both selectors, and the companion byte estimate, to root-level
mtp-*.gguf so the MTP/ copies stay explicit-selection only:
- core/inference/llama_cpp.py _pick_mtp (loader auto-download)
- hub/utils/gguf_plan.py preferred_mtp_sibling (Hub variant plans)
- routes/inference.py _remote_gguf_companion_bytes (VRAM headroom)
Also reuse a drafter already in the local cache before downloading, so a
device that already holds a copy on disk does not re-fetch it.
Old-scheme names keep working (they have no root-level mtp- sibling to
mis-select). Adds regression tests for the new naming, both selection
paths, and the on-disk reuse.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: gate MTP drafter cache reuse to offline mode
Reuse the cached drafter only when HF is offline. Online, route back
through _download_companion_gguf/hf_hub_download so the current revision
is checked (etag) and a changed drafter is refetched, matching the
offline-only cross-snapshot reuse already used for the main GGUF. This
avoids pairing freshly downloaded weights with a stale cached draft.
Make the reuse tests offline and add an online-skips-reuse test.
* Studio: prefer a root MTP drafter across all cached snapshots
Offline reuse scanned snapshots one at a time and returned the first
snapshot that held any drafter, only preferring root within it. A newer
partial snapshot with just the MTP/ copy could shadow the small root
drafter in an older snapshot. Collect drafters across all snapshots and
prefer any repo-root file before an MTP/ copy.
* Studio: keep newest-first snapshot order when reusing cached drafters
Collecting root candidates and sorting by absolute snapshot path could
pick a drafter from an older snapshot. _iter_hf_cache_snapshots yields
newest first and the main GGUF is resolved in that order, so preserve it
(root still preferred over MTP/ copies) to avoid pairing a fresh main
weight with a stale drafter revision.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(studio/hub): apply repo_id length limit per segment, not whole string
is_valid_repo_id() applied the 96-char limit to the full "namespace/repo_name"
string, so a repo with a valid (<=96 char) name but a long combined id was
falsely rejected. Match huggingface_hub.validate_repo_id by checking the length
per segment instead. Fixes#6946.
* Fix long repo id state filenames
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix external drive custom folder selection
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update studio/backend/tests/test_linux_external_media_paths.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep legacy media scan validation strict
* Apply sensitive-dir denylist to legacy folder browser for PR #6799
The legacy /api/models browse endpoint gained the new /run/media mount
roots in its allowlist but not the credential/config guard that scan-folder
registration and the Hub browser already enforce. Filter sensitive names
during enumeration and reject them in _resolve_browse_target so .ssh, .aws,
.config, etc. under allowlisted roots stay unbrowseable, matching the Hub
browser. Add a public contains_sensitive_path_component helper and cover the
legacy resolver with a regression test.
* Trim redundant comments in PR #6799 changes
* Skip sensitive Linux media roots
* Reject sensitive dirs at exact browse roots for PR #6799
Both _resolve_browse_target functions only checked contains_sensitive_path_component
while walking descendant parts, so requesting an allowlisted root itself (empty
relative path) returned it unchecked. A pre-existing scan-folder row under ~/.ssh,
~/.aws, ~/.config, etc. (registerable before the denylist was added) is re-added to
the allowlist on upgrade and could then be browsed. Check the resolved target once
before returning in both the legacy and Hub browsers, and cover the root case in
both test suites.
* fix: avoid unused path helper reexports
* fix: import sensitive path helpers directly
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* add models for /update endpoint
* add logic for identifying out of date hf models
* add endpoint for updating hf models
* add relevant field to GgufVariantDetail
* make exception handling better
* add update_available flag for cached_models, and moved /update endpoint from inference -> models
* hook up /update endpoint on the frontend
* implement update scenarios for the model picker
* fix bug where downloaded flag for an older revision was being wrongly set to false
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix import and make hf calls async
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* remove has_vision from UpdateRequest
* fix ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* clear cancel event before updating gguf variant
* set _cancel_event back if it was set initially
* add hf_token to get_paths_info
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: harden model update endpoint and update checks
- update_hf_model: pass snapshot_download local_dir (local_path is not a
valid kwarg and 500s when updating bicodec audio models)
- get_gguf_variants: wrap the remote update check so a network, rate-limit,
gated, or offline failure degrades to "no update info" instead of failing
the whole variant listing, matching list_cached_models
- add regression tests for both paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: HF model update detection and Update action for cached models
Surface an "Update available" cue and a managed Update action for cached
on-device models. /api/hub/update-status compares each cached main GGUF
file's local blobs against the remote main revision using set membership
across all cached revisions, so a repo that was already updated (and still
holds the old snapshot alongside the new one) is not falsely flagged.
The Update action re-downloads through the download manager so it shows in
the Downloads panel with progress and cancel. The frontend wires the Update
button into the GGUF, on-device, and model-selector cards and keeps the
quant label fully visible when the action buttons crowd the row.
Adds regression tests for the multi-revision update check.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: accept force_download kwarg in hf_xet_fallback test double
The download seam now passes force_download to the attempt callable; the _FakeAttempt mock did not accept it, failing 6 tests with TypeError. Add the keyword (default False) so the scripted-results double matches the seam.
* Fix Studio model update regressions
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address Studio update review feedback
* Address Studio update edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Share GGUF update status helper
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF update detection and cache cleanup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix cached GGUF update badges
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Studio: harden the data-recipe and inference consumer loops against pump death
Follow-up to #6643. The same single-unsupervised-consumer pattern the training
pump had lives in two sibling loops, with the same failure mode: one bad event
kills the only thread that updates the in-memory state every UI surface reads,
while the worker subprocess keeps running.
- data_recipe JobManager._pump_loop: a malformed worker log line that makes
parse_log_message raise no longer kills the pump. Guard _handle_event, the
queue read, and the worker-exit finalize, and broaden _drain_queue so a drain
error still finalizes the job instead of leaving it wedged "active" (which also
leaked the workflow-scoped API key until its 24h expiry).
- inference InferenceOrchestrator._dispatcher_loop: guard the routing body so a
malformed response or a mailbox put error can't kill the dispatcher and hang
every in-flight generation (callers key liveness on the subprocess, not on
this thread).
Adds regression tests for both.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: extend consumer-loop hardening to RAG, hub, auth, and stream-reader paths
Continuation of the data-recipe and inference pump hardening: the same
"background producer updates in-memory state that a single unsupervised
consumer surfaces to the UI" pattern shows up in several more Studio paths,
each able to silently freeze a UI surface while the worker keeps running.
RAG ingestion SSE (core/rag/ingestion.py):
- job_events polled the queue with a blocking get and never noticed client
disconnect or a dead worker, so a closed tab or a producer that died
without emitting a terminal event left the stream hanging. It now polls
with a timeout, emits heartbeats, ends on terminal job status, caps idle
time, and always pops the job registry in finally.
- Added _reap_finished_jobs() and call it from start_ingestion so finished
job state does not accumulate.
Startup reconcile (storage/rag_db.py, main.py):
- reconcile_orphaned_ingestion_jobs() marks ingestion jobs (and their
documents) that were left non-terminal by a previous crash as failed, so
the UI does not show jobs stuck "running" forever after a restart. Wired
in at startup next to cleanup_orphaned_runs().
Hub download watcher (hub/services/download_lifecycle.py):
- _watch() could leave a job pinned "running" if finalize raised. Body is
now guarded: on failure it logs and sets the job to error, and always
invalidates the hf cache scan in finally.
External provider stream (core/inference/external_provider.py):
- read timeout was None (no stall ceiling); set to 300s so a wedged
upstream surfaces as an error instead of an indefinitely hung stream.
Auth store (auth/storage.py):
- Enable WAL + busy_timeout on the auth DB so token validation (read on
every request) and login writes stop serialising on the rollback journal.
Matches studio_db / rag_db / providers_db.
Login rate limiter (routes/auth.py):
- _LOGIN_IP_BUCKETS could grow unbounded under spoofed-IP traffic; cap it
and prune stale buckets, mirroring the per-account bucket handling.
Training progress SSE (routes/training.py):
- Break promptly on client disconnect instead of waiting for the next
yield to fail on a closed socket, matching the export / data-recipe SSE
routes.
llama-server stdout drain (core/inference/llama_cpp.py):
- Broaden the drain guard so an unexpected decode/read error logs at debug
and stops the drainer cleanly instead of escaping the thread.
Frontend stream readers (chat-api.ts, rag-api.ts):
- Wrap the SSE read loops in try/finally + reader.cancel() so early return
([DONE]), thrown errors, and consumer aborts release the reader lock
instead of holding it until GC.
Tests:
- test_training_progress_stream_nan: fake request now implements the async
is_disconnected() the route polls, matching the other SSE route fakes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address Codex review feedback on the consumer-loop hardening
Four follow-ups from the automated review, all on code this PR introduced:
- Data-recipe pump (manager.py): a queue read that keeps raising an error
outside the read's narrow catch set (e.g. a broken queue pipe after the
child died) hit the `continue` guard and skipped the dead-worker finalize
below, spinning forever and leaving the job wedged "active" with its
workflow key unretired. On a read failure, fall through to finalize when
the worker is no longer alive. Added a regression test.
- RAG ingestion SSE (ingestion.py): the 5-minute idle cap could end the
stream while the job was still pending/running (a large document spends
minutes in embedding/storing with no per-batch progress event). The route
then sends [DONE], and the client treats a no-terminal-frame end as
completion, marking the document indexed mid-ingestion. Drop the idle cap:
while the worker is alive and non-terminal we keep heartbeating; the stream
ends only on terminal DB status, the None sentinel, or client disconnect.
- Login rate limiter (auth.py): the per-IP path pruned but then added the
new IP unconditionally, so a spoofed-source-IP spray kept _LOGIN_IP_BUCKETS
unbounded and made every new IP pay a full-dict prune scan. Gate the add on
the cap, mirroring the account path.
- Hub download watcher (download_lifecycle.py): if finalize raised before it
reaped (proc.wait) and dropped the worker (e.g. an I/O error draining
stderr), the crash path published a terminal state while the live Popen
stayed registered and kept writing the cache, and the terminal set_job let
claim() admit a retry on the same repo. Terminate + drop the worker before
setting the terminal state.
* Studio: keep login throttling working when the per-IP bucket dict saturates
Review follow-up. The previous cap fix skipped creating a bucket for a new IP
once _LOGIN_IP_BUCKETS was full, returning ip_fails=0. Under a sustained spray
that also fills the account dict, every failure from such an IP then looked
first-seen and _login_blocked had no bucket to enforce, so the cap effectively
disabled throttling once saturated.
Bound the dict with a FIFO eviction instead: if the IP is new and the dict is
full, reclaim expired buckets (rate-limited so a burst of distinct IPs can't
make each failure an O(n) sweep) and, if still full, evict the oldest-inserted
IP. The new IP always gets a real bucket, so a saturating (e.g. spoofed
X-Forwarded-For) spray stays throttled while memory stays bounded. Added a
regression test that saturates the dict and asserts a later IP is still blocked.
* Studio: address Codex review (RAG queue lifecycle, stream error, orphan chunks)
Three follow-ups on the Phase 6 changes:
- RAG ingestion SSE (ingestion.py): job_events removed the per-job queue in its
finally on ANY exit, including an early client disconnect while the worker is
still running. That dropped the worker's later events (the queue is the only
one _emit writes to) and made a reconnect find no queue and receive only
[DONE], which the client treats as completion. Only drop the queue on a
terminal exit (None sentinel / terminal DB status); leftover terminal queues
are still swept by _reap_finished_jobs. Added queue-lifecycle tests.
- External provider stream (routes/inference.py): once the 300s read timeout can
fire, the stream's except path failed the monitor but ended without an error
frame or [DONE], so the chat client saw a bare EOF and saved the timed-out
answer as a successful partial with no error. Emit an SSE error frame (and
[DONE]) on stream failure so the client surfaces it.
- RAG startup reconcile (storage/rag_db.py): marking a half-ingested document
failed left its chunks/fts/vec rows intact, and retrieval filters by scope not
status, so a failed document could still be retrieved and cited. Purge the
document's chunks when reconciling it to failed (the doc row stays for
re-ingest).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: release the remaining SSE stream readers (training, data-recipe, export)
reviewer.py follow-up. The chat and RAG SSE readers were wrapped in
try/finally + reader.cancel(), but the other three readers built on the same
response.body.getReader() pattern were left without it: streamTrainingProgress,
streamRecipeJobEvents, and streamExportLogs leak the ReadableStreamDefaultReader
lock (held until GC) when the consumer aborts, returns early, or a parse/callback
throws. Wrap each in try/finally + reader.cancel() (export already had a
try/catch, so it only needed the finally). All five frontend SSE readers now
release the reader symmetrically.
* Tighten resilience comments and docstrings
Condense the verbose explanatory comments and internal-helper docstrings added
in this branch to shorter, clearer forms. Comment/whitespace only; verified no
code changed via AST diff. No behaviour change.
* Studio: keep chunks for completed docs during ingestion reconcile
Startup reconciliation flips orphaned (non-terminal) ingestion jobs to failed and
purges the document's chunks so a failed source can't be retrieved. But it dropped
the chunks unconditionally, so a document the worker had already committed as
'completed' before the crash (only its job row left non-terminal) lost every chunk
while still reporting 'completed'. That leaves an empty source that retrieval can't
return and dedup (status != 'failed') blocks from re-ingest.
Only purge chunks when the document UPDATE actually transitions it to failed; an
already-completed document keeps its chunks. Adds reconcile regression tests for
both the completed-doc and genuine in-flight-orphan cases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: drop a finished RAG job's queue when the client disconnects
job_events kept the per-job queue until it consumed the None sentinel, so a UI
that stops on the terminal event (its reader.cancel aborts the stream before
[DONE]) left the queue registered until the next _reap_finished_jobs sweep; a
batch of uploads followed by idling retained them all.
_run writes the terminal DB status before emitting the terminal event, so on
generator exit, drop the queue when the job's DB row is already terminal (worker
done, nothing to resume) and keep it only while the worker is still running. Adds
a disconnect-after-terminal-event regression test.
* Remove stray async task output files committed by mistake
* Studio: harden login IP throttle and end progress stream on disconnect
Two Codex review items:
Login per-IP throttle: when the per-IP bucket dict saturated, FIFO eviction could
drop a still-hot (blocked) bucket, so an IP could flood the dict with distinct
(or spoofed) source IPs to push out its own bucket and retry as first-seen. Stop
evicting hot buckets; a new IP that can't fit now shares a bounded overflow
counter that still trips the per-IP threshold, so a saturating spray stays
throttled and no live counter is reset.
Progress SSE: on client disconnect the polling loop only broke and fell through
to the unconditional final 'complete' frame, so a buffered or proxying consumer
could read a still-active run as completed. Return from the generator instead.
Adds regression tests for both (spray cannot reset a hot bucket; disconnect while
active emits no complete frame).
* Studio: shard the login overflow counter and stop cancelling chat stream after [DONE]
Two Codex review items:
Login throttle overflow: the single shared overflow counter meant that once a
saturating spray pushed it past the per-IP threshold, _login_blocked returned 429
for every new unbucketed source IP, before credentials were checked -- a global
login denial. Shard the overflow into a fixed array of counters keyed by hash(ip),
so a hot shard only throttles the IPs that map to it while a single source's
repeated failures still concentrate in one shard and stay throttled. Memory stays
bounded and no live bucket is evicted. Adds a regression test that a hot overflow
shard does not block an unrelated IP.
Chat stream: the reader.cancel() in the SSE finally fired even after a natural
[DONE]/EOF. The backend finalizes its api-monitor entry right after yielding the
sentinel (the local pass-through finishes after the last yield), so a client
cancel there can be observed as a disconnect and mark a completed request as
cancelled. Track natural completion and only cancel on an early/abnormal exit.
(No frontend unit test: the Studio frontend has no test harness.)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: give prep-timeout test fakes an is_disconnected method
The progress stream now ends on client disconnect (await request.is_disconnected()
before falling through to the terminal frame). After merging that into the
prep-timeout tests added later on main, their _FakeRequest/_ReconnectRequest must
provide is_disconnected or the generator raises AttributeError under CI.
* Studio: keep the login overflow throttle when bucket capacity frees up
_login_blocked only consulted the per-IP overflow shard while the bucket dict was
still at capacity. If a slot freed before the 60s window expired (e.g. another
IP's successful login calls _clear_login_bucket), a source counted in a hot shard
stopped being blocked and its next failure got a fresh per-IP bucket, resetting
the throttle the overflow path exists to preserve. Always max in the IP's shard
(shards are empty outside saturation, so it is a no-op in the common case). Adds a
regression test that a hot source stays throttled after a bucket frees.
* Studio: clear a login IP's overflow throttle on successful login
_clear_login_bucket reset the per-IP and per-account buckets on a successful
login but not the overflow shard, so after the dict saturated and an IP was
counted in overflow, a later successful login left those entries behind and the
next failed attempt could immediately return 429.
Store overflow entries as (timestamp, ip) so a source is throttled by its own
count within the shard (also removing cross-IP collateral within a shard), and
drop just that IP's entries in _clear_login_bucket. Adds a regression test that a
successful login clears the overflow throttle.
* Studio: bound the login overflow shard memory under high-cardinality spray
The per-IP overflow tracked failures in a time-pruned deque of (timestamp, ip)
tuples, so a spoofed-X-Forwarded-For spray of distinct one-off IPs grew memory and
the per-check scan with request cardinality for the whole window -- undermining
the bucket cap that exists to bound memory. Replace each shard with a fixed-
capacity dict (ip -> [count, window_start]): O(1) lookups, and when a shard is
full a one-off IP evicts the lowest-count entry (Space-Saving) so memory is hard-
bounded while a persistent attacker keeps a high count and is never evicted. Adds
a regression test that shards stay within the per-shard cap under a 5000-IP spray.
* Studio: purge chunks for already-failed docs during ingestion reconcile
The reconcile chunk-purge was gated on the documents UPDATE actually flipping a
non-terminal doc to failed. A doc the worker had already marked 'failed' before
the crash (job row left non-terminal) was not re-flipped, so its committed chunks
were kept and stayed retrievable/citable, since retrieval filters by scope not
status. Purge chunks whenever the document is not 'completed' (failed, in-flight,
or gone), preserving the completed-doc carve-out. Adds a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: don't inherit an evicted IP's count onto a new overflow source
When a full overflow shard evicted the lowest-count entry, the new source
inherited that count (Space-Saving base + 1). If a shard was saturated with hot
entries, an unrelated new IP could land at/over the threshold and be 429'd after a
single attempt -- cross-IP collateral despite the per-source-isolation intent.
New entries now start clean at count 1; the only cost is that a heavy hitter that
is the lowest-count entry in a fully saturated shard can briefly reset, which is
preferable to blocking a bystander. Adds a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: carry overflow failures into a new IP bucket on transition
_login_blocked took max(per-IP bucket, overflow shard) rather than combining them,
so a source could log (threshold-1) failures in overflow during saturation and,
once a bucket slot freed, another (threshold-1) in a fresh bucket within the same
window -- roughly doubling the per-IP limit. When a saturated-era IP first gets a
real bucket, migrate its windowed overflow count into that bucket (and drop the
overflow entry) so the combined failures throttle at the intended limit. Adds a
regression test.
* Studio: reconcile a completed doc's orphaned job to completed, not failed
When a crash left an ingestion job non-terminal after its document was already
committed as completed, reconcile marked the job failed. After restart the upload
UI has no in-memory SSE queue and falls back to getJob(), which treats a failed
job as an indexing failure and removes/toasts a document that is actually
searchable. Mark the job completed (keeping its chunks) when its document is
completed. Extends the completed-doc reconcile test to assert the job status.
* Studio: clamp the overflow failure count migrated into a login bucket
A saturated source could accrue an unbounded overflow count, then materialize
one deque entry per recorded failure when a bucket slot freed, allocating an
arbitrarily large deque under the login lock. Only at-or-above the per-IP
threshold matters for blocking, so cap the count there at the record and take
sites; the migration is now bounded without weakening the limit.
* Studio: keep the RAG job stream alive on a transient status read
The heartbeat poll read the job row unguarded; a momentarily-locked DB would
raise out of job_events, which the SSE route turns into a terminal error frame,
and the UI drops a document whose worker is still running. Treat a failed status
read as non-terminal: heartbeat and retry, and keep the queue so a reconnect can
resume.
* Studio: set busy_timeout before journal_mode on the auth DB
Switching journal_mode needs a lock, so if a refresh-token write already holds
one, journal_mode=WAL raises SQLITE_BUSY and the shared try leaves the
connection on SQLite's default zero lock wait. Set busy_timeout first so the
switch waits instead of failing.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: clean up empty leftover quant folders so they can be deleted
An interrupted or cancelled split GGUF download leaves snapshots/<rev>/<quant>/
behind with no shards. Such a folder is neither a completed download nor a
tracked partial (no .incomplete blobs, no manifest), so it was invisible in the
variant list and a per-variant delete returned 404, leaving it on disk forever.
- list_empty_gguf_variant_dirs: detect quant folders that are empty in every
snapshot, excluding any quant that has shards in another snapshot.
- get_gguf_variants_response: surface those quants as partial (cleanable) so the
UI shows a delete affordance.
- _delete_gguf_variant_from_repos: remove the empty (or just-emptied) quant
subfolder and count it toward the result so the delete succeeds instead of 404.
Adds hub/tests/test_empty_variant_folder.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: simplify empty-dir check to any(iterdir())
* Studio: tighten comments on empty-quant-folder cleanup
* Studio: surface empty-folder removal failures and cleanables on local/offline paths
Address review feedback on the empty leftover quant folder cleanup:
- _remove_empty_variant_dirs now returns removal failures (read-only cache or a
locked dir), and the variant delete raises 409 instead of a misleading 404; a
concurrent download refilling the dir (ENOTEMPTY) is still treated as a skip.
- Empty leftover folders are surfaced as cleanable on every variant-listing path
(prefer_local_cache / offline / HF-fallback), not just a remote listing, via a
single post-process that flips a listed quant to partial or appends an
unlisted one.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: surface empty-folder cleanables even when metadata fetch fails
When the cache holds only an empty leftover snapshots/<rev>/<quant>/ folder
from an interrupted split download and the client is offline or the HF
metadata request fails, _compute() re-raised before cleanables were marked,
leaving the folder undeletable. Now fall back to marking cleanables against an
empty response and return them if any; otherwise re-raise the original error.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add an Open button to reveal the models folder in the file manager
* Studio: report models folder creation failures
---------
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* Studio: free chat model VRAM at training start only when the GPU is tight
The training start route unconditionally tore down the transformers/MLX
inference subprocess before training, and never stopped the llama.cpp GGUF
server at all, so a loaded GGUF chat model kept holding VRAM for the whole
run. Conversely the HF model was always unloaded even when there was plenty
of room to keep it.
Make the unload VRAM aware and cover every inference backend:
- Add routes/training_vram.py with summarize_resident_chat(),
can_keep_chat_during_training() and free_chat_models_for_training(). The
keep/unload decision reuses the same estimator and live per device free
VRAM reader the training GPU selection already uses (auto_select_gpu_ids,
estimate_required_model_memory_gb, get_visible_gpu_utilization), so the
probe agrees with the placement computed later in start_training.
- When a chat model is resident and training fits alongside it with a
conservative margin (required_gb * 1.15 + 4 GB), keep it loaded so the
user can train and chat at the same time; on a multi GPU box training
lands on a different GPU and both coexist. Otherwise unload the HF/MLX
orchestrator and the llama.cpp GGUF server before training starts.
- The export subprocess shutdown stays unconditional and now runs first so
its freed VRAM is reflected in the decision.
Default deny: non CUDA backends, unestimable models, or any probe error
fall back to the previous always unload behavior.
Adds tests/test_training_vram_coexistence.py and updates two existing route
tests in test_gpu_selection.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: per-GPU floor for explicit GPU lists + don't unload chat on invalid gpu_ids
Address review feedback on the chat coexistence probe:
- Explicit gpu_ids mode now enforces a per-GPU floor in addition to the
aggregate free-VRAM check, mirroring auto_select_gpu_ids' min_per_gpu_N.
Without it, an uneven split such as free [45, 10] for a 40 GB job passed
the aggregate threshold and kept chat loaded even though the 10 GB GPU
could not hold its training shard, risking an OOM.
- Invalid explicit gpu_ids (ids outside the visible set, or a UUID/MIG
mask) make resolve_requested_gpu_ids raise. That request is rejected with
a 400 before training starts, so leave the resident chat model untouched
instead of unloading it.
- Tighten the target_modules / gpu_ids type hints to List[str] / List[int].
Adds tests for the per-GPU floor (uneven split unloads, even split keeps)
and for invalid gpu_ids keeping the chat model loaded.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: only free chat VRAM once training will start; handle in-flight and CPU-only chat
Address the second review pass on the chat-coexistence path:
- Run the chat/export VRAM teardown as a before_spawn hook inside
TrainingBackend.start_training, fired only after the start guards pass.
Previously the route freed chat VRAM before calling start_training, so a
refused start (e.g. a lingering pump thread) would tear down the resident
chat model even though no training job began.
- Treat an in-flight HF chat load (loading_models set, no active model yet)
as not safely sizeable: free it rather than risk both OOMing as the load
keeps allocating after training starts.
- Do not count or tear down a GGUF llama-server confirmed to run entirely on
CPU (_gpu_offload_active is False): it holds no VRAM, so killing it cannot
help training fit.
Adds tests for the before_spawn hook (runs on start, skipped when a
subprocess is alive or a pump thread will not die, survives a hook error),
the in-flight load flag, and the CPU-only GGUF exclusion in both the resident
summary and the unload path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: treat any in-flight chat load (HF swap / mid-start GGUF) as unsafe to keep
Tighten the in-flight detection in summarize_resident_chat so the keep check
never sizes a load that is still allocating:
- Flag loading on ANY non-empty loading_models, not only when active_model_name
is empty. load_model adds the new model to loading_models before clearing the
old active_model_name, so a replacement load during a swap was previously
sized as a normal resident and could OOM as the new model finishes loading.
- Flag a GGUF server that is active but not yet healthy (is_loaded False) as
in-flight: it is still mmaping/offloading layers, so its final VRAM footprint
is unknown.
Consolidates the signal into a single resident["loading"] flag; the route frees
the chat model whenever it is set. Adds tests for the replacement HF load and
the mid-start GGUF cases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten comments in chat/training VRAM coexistence (comments only)
* Studio: run before_spawn VRAM hook only after GPU-selection validation
Reviewers found the before_spawn hook fired before prepare_gpu_selection
validated gpu_ids (and before config build), so a refused start (invalid
gpu_ids -> 400, or a bad grad-clip value) could still tear down chat/export
VRAM. Move the hook to immediately before proc.start(), once all synchronous
validation and process construction have passed. This also fixes the route's
in-flight-chat loading branch, since that teardown runs inside the same hook.
Add test_hook_skipped_when_gpu_selection_rejects.
* Studio: recompute GPU auto-selection after the before_spawn VRAM hook
Codex P2: with before_spawn moved after prepare_gpu_selection, placement was
frozen against the pre-teardown VRAM state while the hook freed export/chat
afterward. Auto-selection could pin training onto a GPU the hook then cleared
(or onto a kept chat model). Split validation from placement: explicit gpu_ids
are still validated before the hook (raise -> 400, no teardown; explicit
placement is VRAM-independent), but VRAM-dependent auto-selection now runs
after the hook so it sees the freed memory.
Add test_auto_placement_runs_after_hook and test_explicit_placement_validated_before_hook.
* Studio: allow chatting during training (lift sidebar gate + VRAM-aware load guard) (#6335)
* Studio: allow chatting during training (lift sidebar gate + VRAM-aware load guard)
The sidebar disabled New Chat, project, and home navigation while a training
run was active, so users could not chat during training even though the backend
serves inference fine alongside a run. This removes that gate and adds a backend
guard so the one genuinely risky operation, loading a new local chat model
mid-training, is refused with a clear 409 when it would not fit beside the run.
Frontend (app-sidebar.tsx): drop the chatDisabled = isTrainingRunning gate and
its consumers. Navigation triggers no model load on its own, so chat stays
usable during training.
Backend (routes/training_vram.py, routes/inference.py): add
can_load_chat_during_training plus a load/validate guard that sizes the same
effective load the loader performs (LoRA 4-bit to 16-bit resolved first, HF auto
placement via auto_select_gpu_ids, explicit multi-GPU per-GPU floor, GGUF sized
from on-disk shards and companions or the selected remote variant). It is a
no-op when training is inactive, never blocks external providers or
already-resident models, and default-denies only on a CUDA sizing failure so a
load can never OOM the run. Validate refuses early with the real settings so the
frontend does not unload the resident chat model for a load that would be
rejected.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address review feedback for chat-during-training load guard
- Run the load/validate VRAM guard via asyncio.to_thread so the sync
nvidia-smi + HF metadata work never blocks the event loop.
- Size the GGUF KV cache at the requested context (_estimate_gguf_kv_gb)
and add it to the local GGUF estimate so large-context picks are not
under-counted.
- Keep the requested quantization when adapter_config.json is malformed
(not a JSON object) instead of raising in _effective_load_in_4bit.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: size the training load guard at the launcher's effective GGUF context
The GGUF KV-cache estimate used max_seq_length only, but the llama.cpp
launcher honors a user --ctx-size/-c in llama_extra_args. A load such as
max_seq_length=4096 with --ctx-size 131072 was sized against a 4k cache
while the server allocates 131k, so the guard could approve a long-context
GGUF load that then OOMs training. Size the guard's KV at the larger of
max_seq_length and the parsed --ctx-size (reusing the launcher's own
parse_ctx_override), keeping the conservative f16 cache so the estimate is
never smaller than what the server allocates.
The chat model picker also validated with the raw max_seq_length while
/load sizes with resolveLoadMaxSeqLength, so validate could pass, unload
the current model, then have /load reject the native-context load. Validate
now uses the same effective context; the load path is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: size the GGUF training guard at the server parallel-slot count
The KV-cache estimate assumed a single slot, but llama-server allocates the
cache across --parallel slots (app.state.llama_parallel_slots). On a Studio
launched with --parallel N>1 the guard under-sized the cache N-fold and could
approve a GGUF chat load that then OOMs training. Thread the same slot count
the loader uses into the guard's KV estimate; default 1 leaves single-slot
setups unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments for chat-during-training guard
* Studio: keep chat generation alive across navigation; Train spinner + Return to Chat
Hoist the base chat runtime above the routed outlet so navigating to Train (or any tab) no longer aborts an in-flight generation; only an explicit Stop cancels. Add a Train sidebar spinner and swap New Chat to Return to Chat while a run is active, with a lightweight completion watch so the spinner clears from any tab. Also respawn a chat llama-server killed mid-session and guard unreadable HF cache dirs that 500'd the hub model list.
* Studio: show Return to Chat on the Train tab whenever a chat is live
Previously the top sidebar item only swapped to Return to Chat while training was running; on the Train tab with an idle/just-finished run it stayed New Chat, which started a fresh thread and cancelled an in-flight generation. Show Return to Chat (and navigate back, preserving the run) whenever a generation is running or its thread is still active, or training is in progress.
* Studio: keep a running chat alive when starting a New Chat
Starting a New Chat (or switching threads) while a generation was in flight
remounted the single-chat runtime provider, which detached the in-flight run
and cut the previous chat off (it showed up frozen / empty when reopened).
Key the single-chat view by project instead of by thread or new-chat nonce so
the provider stays mounted and assistant-ui switches to a fresh thread in place.
The previous generation keeps streaming in the background and autosaves on
completion, and returning to that thread reattaches the live run instead of
reloading a half-saved one.
Also:
- "Return to Chat" now lands on the thread that is still generating rather than
the empty new chat that became active after New Chat.
- Skip the explicit /inference/cancel POST when an abort comes from a runtime
detach (navigation / background switch) rather than an explicit Stop, so a
backgrounded generation is never cancelled behind the scenes.
* Studio: make model export non-blocking and inline
The Export tab opened a full-screen modal that trapped focus, could not be
closed or cancelled while running, and showed no progress. It also stopped
training and unloaded the chat model before loading, so export could not run
alongside them.
Export now mirrors the training runtime pattern:
- Inline panel embedded where the Export Model button was, with no modal or
backdrop, so the rest of the UI stays usable during an export.
- Global export runtime store plus an app-root lifecycle hook, so a run keeps
going and streaming across navigation and is reflected on the Export nav item
from any tab.
- The worker log stream now stays connected across the load to export phase
boundary instead of stranding on "Waiting for worker output".
- Progress bar driven by phase and quant index (quant N of M for GGUF), with
elapsed time and a working Cancel.
- load-checkpoint no longer stops training or unloads inference; export loads in
its own subprocess in parallel and surfaces out-of-memory as a clear error.
- Add POST /api/export/cancel and is_export_active on /api/export/status.
* Studio: show Return to Chat on the Export tab too
Extend the New Chat to Return to Chat swap to the Export route so leaving a
running chat for Export offers a way back to the live generation, matching the
Train tab.
* Studio: smooth out Export animations and polish the panel
- Drop the height-based reveal animations (source switch, run panel, quant
picker, hub fields) that caused flashing and reflow; use instant swaps and
quick opacity fades instead.
- Method and quant cards now transition colors only, with no transition-all or
hover lift, so selecting a method or quant is crisp instead of jumpy.
- Auto-scroll the export panel into view when it opens and add a scroll-to-bottom
button when its output is below the fold, like Chat.
- Show Return to Chat on the Export tab while an export is running, matching how
training drives it on the Train tab.
- Surface the current phase or stage in the live output before the first worker
line arrives so the panel never looks stuck while progress is advancing.
* Studio: show Return to Chat on every non-chat tab
Generalize the Return to Chat swap from just Train/Export to any non-chat route
(Recipes, Projects, Hub, ...) so a running or active chat is always one click
away, instead of showing New Chat there.
* Studio: stream export logs over the Cloudflare tunnel; drop janky export animations
Exporting over a --secure Cloudflare quick tunnel showed "connecting..." with no
logs while the progress bar advanced. Cloudflare buffers text/event-stream and
only flushes when the stream closes, so the SSE log stream never reached the
browser during the run (direct localhost is unaffected, which is why this only
showed up over the tunnel).
Add a tunnel-safe JSON poll fallback (GET /api/export/logs?since=) that the
runtime lifecycle hook polls while a run is active. Short JSON responses are not
buffered by the proxy, so logs show up in near real time over the tunnel. It
shares the orchestrator's monotonic seq cursor with the SSE stream and the store
de-dupes by seq, so the two transports run together (SSE on localhost, poll over
the tunnel) without double-printing. A successful poll marks the panel
"streaming" instead of leaving it stuck on "connecting...".
Also remove the framer-motion AnimatePresence reveals from the export config and
run panel (quant picker, hub fields, the inline run panel, and the live log
section). The expand/slide animations flashed and felt clunky; the sections now
render in place.
* Studio: recover export over the Cloudflare tunnel when the blocking POST times out (524)
A model export over a --secure Cloudflare quick tunnel showed "Request failed
(524)" even though the export succeeded on the backend (the GGUF was written).
Cloudflare returns 524 when a single request takes longer than ~100s to respond,
and a GGUF conversion routinely runs for minutes, so the blocking per-method
export POST is cut off while the backend keeps going.
Confirm completion via short status polls instead of relying on the long POST
response (the same approach that fixed log streaming):
- The orchestrator records each finished op's outcome (status / output_path /
error) with a monotonic seq, exposed on GET /api/export/status.
- parseJson now preserves the HTTP status; a 524/520/522/523/502/503 or a
status-less network drop is classified as a recoverable transport error.
- runExport wraps each phase (load, every export method, each GGUF quant): on a
recoverable failure it keeps the run alive (logs keep streaming, the panel
shows "reconnecting...") and polls status until the still-running op finishes,
then settles from the recorded result, recovering the output path for the
success banner. A real 4xx still fails immediately; localhost still uses the
fast POST response. applyBackendStatus also settles a reloaded run from the
last-op record.
Verified over the tunnel: a 3m14s gemma-4-E4B-it GGUF export now ends on the
success banner with the output path instead of 524.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep the export method + logs visible after navigating away mid-export
While an export was running, navigating to another tab and back to Export
remounted the page and reset the local form state (exportMethod, quant levels),
so the method card showed unselected and the run panel's log area was hidden
until the card was re-clicked. The run itself lives in the global store and was
unaffected.
Seed exportMethod / quantLevels from the active run's summary via lazy useState
initializers on (re)mount, and gate the panel's log area on the live run
(isExporting / logLines / the run's method) rather than only the local form
selection. The card stays selected and the logs/progress stay visible across
navigation; nothing changes when no run is active.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: address export/training review findings
- Export: guard Start against an empty GGUF quant selection so an inline-panel
run with no quant can't settle as success with no file produced.
- Export: thread the source HF token into the background load so gated/private
HF source exports (and gated bases) authenticate, matching the consent path.
- Export: only settle a recovered (non-owned) run as a finished export when the
last backend op was an export, not a standalone load_checkpoint.
- Training: free the export subprocess whenever an export is active, not only
once a checkpoint is loaded, so an in-flight export load can't race training
for VRAM (current_checkpoint is unset during the load phase).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio Hub: default downloads to Xet transport
Model and dataset downloads defaulted to HTTP; flip the default to Xet for faster parallel chunked transfers.
- Frontend: DEFAULT_TRANSPORT_MODE is now Xet, so a user with no saved preference starts on Xet. effectiveTransportMode() already downgrades to HTTP and warns when hf_xet is unavailable, so this degrades gracefully.
- Backend: DownloadModelRequest.use_xet and DownloadDatasetRequest.use_xet default to True, keeping the API in step with the UI. Set use_xet=False for sequential HTTP Range-resume.
- Align the internal _spawn_download_worker default so no caller silently falls back to HTTP.
Inference and training model loads were already Xet-first with an HTTP stall fallback, so this brings explicit downloads in line with the rest of Studio.
* Studio Hub: gracefully fall back to HTTP when Xet is unavailable
With Xet now the default, an omitted or explicit use_xet=True from a non-UI API caller would 400 on installs without hf_xet, since resolve_transport raises when the transport is unavailable.
Add resolve_effective_use_xet(), which downgrades a Xet request to HTTP (with a warning) when hf_xet is missing, mirroring the frontend's own downgrade. Both the model and dataset flows now derive a single effective use_xet and feed it to resolve_transport and spawn_worker, so the recorded transport and the worker env can never disagree. The UI is unaffected: it already resolves availability and passes use_xet explicitly.
* Add tests for resolve_effective_use_xet Xet to HTTP fallback
* Trim comments for PR #6433
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: add 'Load on selection' toggle to configure load options before loading
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: seed staged speculative decoding from the standing default
* Studio: address PR review for load-on-selection staging
* Studio: handle direct GGUF staging and stale-stage edge cases from load-on-selection review
* Studio: cancel replaced staged downloads and keep staged pick on load failure
* Studio: centralize staged-download cancel and guard staged-load restore
* fix: address staged GGUF load review
* fix: honor staged GGUF load metadata
* fix: clarify load-on-selection tooltip
Keep the load-on-selection hint visually anchored to the control and make the on/off behavior explicit without changing the broader deferred-load flow.
* Studio: reset orphaned staged knobs on abandon and cap Max Tokens to staged context
* Studio: remove dead code and cancel staged download when loading a different model
* fix: surface staged model in run settings before deferred load
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* fix(hub): stop demoting cached gguf variants on mmproj or filename mismatch
- bug: a quant with its bytes on disk was marked not fully downloaded when the API-preferred main filenames did not match or the mmproj adapter was absent
- fix: fall back to the on-disk quant byte signal, the same one inventory uses for on-device, so a present quant is no longer demoted
- broaden mmproj detection to accept any mmproj-looking cached file, not only the API-preferred name
* feat(hub): full-page redesign with trending feed, search, and persisted state
- convert the hub to a full-page view with a new layout, model cards, and a sortable models table
- add a trending/latest/finetune feed with model and section deep-link params validated on the hub route
- add recent searches and rework model and dataset search with pagination and infinite scroll
- persist feed and token state through a dedicated store and persist-storage layer
* fix(hub): browse sorting, deep-link presets, dataset URLs, and persistence
- sort dropdown: drive HF-wide browse across all repos by the chosen sort, respecting format and capability filters
- section deep-link: apply the section preset (format and sort) on refresh and deep-link, not only on click
- dataset detail URL: persist the resource kind so refresh and share resolve datasets correctly
- gguf card: show a Loading state for hub-cache dir-path repos via repo-id match
- active-model CTA: New Chat now actually opens a fresh chat
- persist-storage: fix throttle keying with a Map and dedupe a duplicated util
- transport-toggle: drop redundant controlled-tooltip state
* Studio: polish hub redesign UI and unify segmented tabs
Refinements on top of the hub full-page redesign:
- Unify every segmented control (Discover/On Device, models/datasets,
Unsloth/All, recent/name/size, settings tabs, train dataset source,
profile shape, theme, OS toggle) on one filled-pill design.
- Hub list now loads in larger batches with a shorter fetch interval so
results fill in fast instead of dripping one row at a time.
- Disable remote avatar fetches in list rows and brighten the colored
initial fallbacks so they read clearly without network calls.
- Add a split master-detail view for model lists and make it the default.
- Left-align the split "Showing GGUF models" header with the rows below.
- Round the "Load more" footer box and tidy On Device stats layout.
- Show recent trainings on Recipes and Export, falling back to recent
chats when there is no training history.
* Studio: address hub review comments (filter warning + scrollMargin)
- DiscoverFetchMoreFooter only shows the "results may be hidden by your
filters" note when a filter is actually active, instead of always.
- Use the destructured scrollMargin prop directly in the row transform
rather than reaching into virtualizer.options.scrollMargin.
* Fix/adjust Hub metadata and deep links for PR #6349
* Studio: drop avatar ring in hub split view
The split master-pane rows (discover + on device) added a ring-1 around
the owner avatar that read as a shadow. Remove it so split-view avatars
match the flat avatars elsewhere; grid cards and the full list keep theirs.
* Studio: hub sort + scope as dropdown pills beside view tabs
Recent/Name/Size and Unsloth/All were segmented controls that dropped to
their own row in the narrow split pane. Make each a compact dropdown pill
(HubOptionMenu) that sits in the header actions slot next to the view-mode
tabs in every layout, so split view no longer needs a separate row.
* Studio: align hub list header with the view tabs and rows
- Vertically center the "On device" / "Showing GGUF models" title with the
dropdown pill and view-mode tabs (items-center instead of items-end), so a
short title no longer sits low against the taller tab row.
- Nudge the back chevron 2px further left (-ml-2) so its glyph edge lines up
with the start of the row hover below it.
* Studio: align back chevron tip with the row hover edge
The arrow glyph is inset ~6px inside its centered icon box, so an
edge-aligned button left the visible chevron sitting in from the column.
Pull the button out (-ml-3.5) so the chevron tip lands on the row hover's
left edge instead of floating to its right.
* Studio: unify every bare tick on the shared check mark
Point all plain checkmarks at the canonical @/lib/tick-icon tick (the one
already used in the chat composer and menus), so there is a single tick
across the app:
- Hub: model-inspector, hub-option-menu, path-info-button were importing
the stock hugeicons Tick02Icon; switch them to the shared icon.
- Chat / assistant-ui: artifact-surface, prompt-storage-dialog, reasoning,
tool-ui-python, tool-ui-terminal, tool-ui-code-execution, and the
tool-fallback status map used lucide CheckIcon; render the shared tick
via HugeiconsIcon instead (tool-fallback wraps it to fit its icon map).
The circular CheckmarkCircle success badges are intentionally left as-is.
No bare CheckIcon/stock Tick02Icon references remain; verified the tick
renders in every converted spot via typecheck + build.
* Studio: nudge back chevron 2px right
-ml-3.5 pushed the chevron a touch too far left; -ml-3 sits it just
inside the row hover edge, aligned with the avatars below.
* Studio: search base-model chips across all publishers
Clicking a Base model chip searches the Hub for the upstream repo, which
lives under another publisher (google, meta, etc.). It left ownerScope at
the default "unsloth", so the search hard-restricted to the Unsloth org and
could never surface the base model. Switch the scope to "all" for this action.
* Studio: label the safetensors list header "Safetensors"
The focused list heading showed "Showing Checkpoint ... models" while the
format dropdown labels the same checkpoint filter value "Safetensors". Match
the dropdown so the header reads "Showing Safetensors ... models".
* Studio: simplify the focused list heading to "Models"
Drop the format/capability composition (e.g. "Showing Safetensors Reasoning
models") so the focused list heading just reads "Models" (or "Datasets").
Search keeps its "Results for ..." label.
* Studio: drop the header refresh button to the text baseline
The refresh button sat at the heading's vertical centre. Nudge it down so
it lines up with the bottom of the title text instead.
* Studio: hide redundant "Back to Hub" in the split detail pane
In split view on large screens the master list sits beside the detail, so
the back button is redundant. Hide it there (lg) and reclaim the top space.
It stays on the small-screen overlay and the full-page detail, where the
list is hidden and back is the only way out.
* Studio: match the readme scroll fade to the left column
The detail pane relied on the sticky back-bar's fade, which is now hidden in
split view. Add the same hub-scroll-fade overlay the master list uses so the
readme fades consistently at the top when scrolled. The back-bar, when shown,
sits above and covers it.
* Studio: align Hub refresh button to the heading text bottom
* Studio: nudge Hub refresh button up to the heading text
* Studio: optically centre the HF token shield in its circle
* Studio: preview the first visible on-device row in split view
* Studio: calm the on-device row colour and fix size tooltip contrast
* Studio: fix Hub reset tab and clear search when opening a section
* Studio: fix Hub feed defaults, filter sync, and GGUF vision download state
* Studio: condense Hub redesign code comments
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: shimmyshimmer <info@unsloth.ai>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio: fix Downloaded model list disappearing and order it by last download
The chat model picker scan for cached GGUF and safetensors models aborted
whenever an auxiliary Hugging Face cache dir (such as ~/.cache/huggingface/hub)
was unreadable, returning an empty list. That hid the Downloaded section and
let already downloaded models appear under Recommended. Isolate each cache
probe so an inaccessible directory is skipped instead of failing the scan.
Also order Downloaded newest-first using cached blob mtimes (multi-quant repos
group by their most recent quant), keep the section visible while searching,
and make the per-quant downloaded check per-snapshot and mmproj aware so a
Recommended quant is never falsely marked downloaded.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden gguf-variants scan and dedupe by newest timestamp
Guard f.stat() per file so a broken symlink or unreadable file in a
snapshot no longer aborts the downloaded check early, and match quant
labels case-insensitively. When the same repo is present in multiple
caches with equal size, keep the newest last_modified so Downloaded
ordering reflects the most recent copy.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: apply cache-scan guards to sibling endpoints found in review
Extend the inaccessible-cache guard and mmproj/stat hardening to the
parallel HF cache code paths flagged in review:
- list_local_models and the Hub inventory scan now skip an unreadable
auxiliary cache instead of returning 500.
- The GGUF download-progress endpoint excludes mmproj adapters and
guards f.stat() so one bad file does not zero a repo's progress.
- The offline snapshot scanner guards its is_dir() probes.
- The chat-only picker no longer renders a blank list when a search
matches only cached non-GGUF models.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: support separate-file MTP GGUF drafters (Gemma 4)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix review findings for separate-file MTP drafters
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: pair local MTP drafters by name and include them in reload dedup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: manage --model-draft in extras and reject MTP/ copies as models
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: training survives a non-writable HF datasets cache
A shared HF datasets cache can contain subtrees owned by another user
(for example populated by an earlier root-run job). datasets then dies
with "[Errno 13] Permission denied: ..._builder.lock" while locking
the cached builder and the training run fails. load_dataset in the
training worker and trainer now goes through a wrapper that catches the
EACCES and rebuilds the dataset in a Studio-owned cache under
cache_root()/hf-datasets, logging the fallback.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope the HF_DATASETS_CACHE override to the fallback load
* Route non-streaming dataset preview loads through the cache-safe wrapper
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Adds the Studio Hub and download manager: browse Hugging Face models and datasets, download GGUF and safetensors with live progress and cancellation, and manage on-device inventory. The Hub does not require a GPU, so it is available on chat-only hosts.
CI: all substantive checks pass, including the three Core jobs after unsloth-zoo#736. The two red checks are non-code flakes, a transient npm-registry DNS resolution failure in the package scan and one quantized vision-model output assertion whose sibling shards passed.
Daniel Han